6 Appendices

6.1 Appendix A: Software Coordination Issues raised Sept 19

Several issues surrounding the coordination of the atmosphere model development and interactions within the group and with collaborators was brought up at the Sept 19, 2001 CCSM programmers meeting. The point was that CCSM as a whole probably has to deal with many of these issues as well. A group discussion ensued. The items discussed included

• loss of control of model versions. Model changes and model versions are either not well enough documented or not well enough communicated within the community. There is a lack of understanding about which mods and which physics are in model versions and runs.
• poor understanding in the model development community about who is doing what.
• how to coordinate changes among several development groups including waccm, fv-dyn, etc.
• how to interact with outside developers including NEC and Fujitsu with regard to access to the repo, testing, verifying code changes, and merging code back into the repository.
• lack of control of a "frozen" version.
• model release frequency.
• concept of a change control or change review board to control changes and slow down and better coordinate the development process.
• inadequate testing standards. "A bug shut us down for 2 weeks".
• lack of policy on repository access.
• inability to secure branches of the repository from global access.
• increasing long-term planning and goals for model development.
• whether cgd has the software engineering expertise and resources to do both code development and manage infrastructure.
• how all this fits into software engineering training and consulting.
• how other model development centers and commercial software shops manage their projects.
• the possiblity that the project should hire a computer scientist to look into and plan for software engineering and infrastructure issues.
• what the software engineer new hires might do.
• how the atm model / working group differs from other model development group in terms of coordination and control.
• whether we need to "stop" to get the infrastructure together.
• the fact that there is no clear, well-defined management/coordination plan associated with scidac and that little resources are specifically allocated to deal with the software engineering issues. This is true of other collaborations as well. They focus on code development and not enough on infrastructure.

6.2 Appendix B: Construx Training Action Items

In December of 2001, Construx and NCAR's CCSM Software Engineering Group (CSEG) had a three-day training and consulting session at NCAR. The training was given by Steve Tockey and was based on Steve McConnell's book, "Software Project Survival Guide." The CSEG had previously read and discussed the book, chapter by chapter, and felt the general approach was reasonable and appropriate for improving CCSM's software development process.

The resulting identification of CCSM/CSEG project challenges, potential solutions, and a suggested action plan are summarized here.

Software Project Challenges:

• Uncontrolled requirements creep
• Small penalty for lateness/no incentive for early (Or are we overly minimizing the penalty for lateness, given the impact on respect and credibility?)
• No call to do any planning
• Can't test some theories without huge investment
• Inadequate testing
• No consensus on what testing is necessary
• Little review of process for improvement
• Increasing complexity of the environment
• Non-existent requirements
• Lack of planning
• Lack of active risk management
• We have to deal with some poorly developed code (legacy code?)
• Missing product management

Back-to-Work Action Plan:

By next week

• Revisit this and do a severe sanity check on it - Tony
• Look at SEP-PCH and CxOne charter templates and make CCSM Charter template - Brian
• Look SEP-RAA and CxOne to develop risk and asset process/template - Lawrence
• Draft up charters for CCSM and 5 components - Lawrence to lead, liaisons to do for components
• Discuss in defect tracking configuration agenda: Track defects including when injected, when detected, effort/cost to fix - Lawrence
• Send Earned Value references to Tony - Steve
• Get and read these books on testing: - Lawrence
  • Glenford Myers, The Art of Software Testing, Wiley, 1979
  • Cem Kaner, Jack Faulk, Hung Quoc Nguyen, Testing Computer Software, 2nd Ed., International Thompson Computer Press, 1993
  • Bill Hetzel, The Complete Guide to Software Testing, 2nd Ed., Wiley, 1988
• Send lessons-learned process and template to Tony - steve

By next month

• Make a presentation of all of this to the scientists - Tony
• "Finalize" charters for CCSM and 5 components (should be put under change control) - Jeff
• Update the risks list (look at assets also) (not necessarily under change control) - Lawrence
• Plan (for the software) based on charters, risks, and assets - Tony and Component Liaisons(?)
• Get Suzanne Robertson and James Robertson, Mastering the Requirements Process, Addison-Wesley, 1999. Pay particular attention to Volere Template and the "Fit Criteria" - Tony
• Put in place a change control process - Tony
• Start meetings of the atmosphere "change board" - Jeff
• Put planning checkpoint reviews on Jeff's radar screen (point him to relevant chapter in SPSG, point out that "no go" option is off the table) - Tony
• Have people who do planning read Chapter 7 (Lifecycles) of McConnell's Rapid Development
• Look at CxOne project plan and lite project plan templates, start thinking about your own project plan template - Tony
• Review the last 18 months/lessons learned - Tony
• Make sure lessons-learned is in the project plan template - Tony

By next quarter

• Develop a consistent approach to reviewing code - Brian
• Get and read the PMBOK (<a href="http://www.pmi.org">www.pmi.org</a>) - Tony
• Start meetings of the remaining "change board" - Jeff
• Complete a reading/discussion group on Suzanne Robertson and James Robertson, Mastering the Requirements Process, Addison-Wesley, 1999 - Tony
• Go investigate Earned Value - Tony
• Get copies of
  • Donald C. Gause and Gerald M. Weinberg, Exploring Requirements
  • Quality Before Design, Dorset House, 1989
• Look into peer reviews: - Lawrence
  • Daniel Freedman, Gerald Weinberg, Handbook of Walkthroughs, Inspections, and Technical Reviews
  • Evaluating Programs, Projects, and Products, Third Edition, Dorset House, 1990, David Wheeler, Bill Brykczynski, Reginald Meeson,
  • Software Inspection - An Industry Best Practice, IEEE Computer Society Press, 1996
  • CxOne peer reviews stuff

By next 6 months

• Extract defect tracking data and plot your own "Upstream, downstream" graph - Lawrence

By next year

• Start looking into the Cone of Uncertainty. Think about how it might be applied within the organization given all that's happened up to now - Tony

Background stuff

• Get a couple of copies of
  • Michael Doyle and David Straus, How to Make Meetings Work
  • The New Interaction Method, Reprinted Edition, Berkley Publishing Group, 1993
  • Tom DeMarco and Tim Lister, Peopleware, 10th Anniversary Edition, Dorset House, 1998
  • Robert Block, The Politics of Projects, Yourdon Press,  1984
  • Stephen Robbins, Essentials of Organizational Behavior, 6th Ed., Prentice Hall, 2000
  • Everett Rogers, Diffusion of Innovation, 4th Ed.(?), Free Press, 1994
  • Project Management Institute, Project Management Body of Knowledge (PM-BOK), available at www.pmi.org
• Think about Product Management

6.3 Appendix C: Terms

• CCSM: Community Climate System Model
• CSEG: CCSM Software Engineering Group
• CRB: Change Review Board
• CVS: a version control system