[an error occurred while processing this directive] [an error occurred while processing this directive]


CCSM Software Engineering Working Group Meeting
8 July 2004
Santa Fe, NM


The Software Engineering Working Group (SEWG) met during the 9th Annual CCSM Workshop on 8 July 2004 from 1:00 p.m. to 5:00 p.m. The following is a summary of that meeting.

Douglass Post, Los Alamos National Laboratory, opened the meeting with a presentation on the DARPA High Productivity Computer Systems (HPCS) Initiative. He described his role as application analyst for HPCS and discussed characteristics and typical lifecycles of successful projects. His presentation also included a discussion of the "coming crisis" in high performance computing, due to difficulties in validation of extremely complex models.

John Drake described recent and ongoing activities in the SciDAC project: Collaborative Design and Development of the Community Climate System Model for Terascale Computing.

After an overview of participants, science goals, and project goals, he discussed progress and plans for each of the CCSM component models, as well as general CCSM software engineering issues, such as vectorization, performance portability, parallel I/O, and single
executable CCSM.

Activities for each component model were divided into software engineering, in particular optimization and performance portability, model development, including both new algorithms and new physical processes, and science applications. Details follow.

Coupler: MCT (Model Coupling Toolkit) and MPH3 (Multi-Processor handshaking) were used to develop CPL6, the version of the coupler in CCSM3. Ongoing development includes vectorization, support for single executable CCSM (e.g., resolving name conflicts), and support for atmospheric and ocean (biogeo) chemistry.

Atmospheric Model: Vectorization, load balancing, and communication optimizations were developed, shown to improve performance, introduced into CAM3, and released as part of CCSM3. Gas-phase atmospheric chemistry with emissions, deposition, transport, and photochemical reactions for 89 species were developed and the impact of different levels of approximations evaluated. Subgrid orography was shown to produce realistic precipitation, snowcover, runoff, and river discharge, without requiring high-resolution dynamics.

Ongoing tasks include additional optimization on the vector systems, optimizations for high resolution, continued development of a version of the model with subgrid orography, and continued development of a chemical atmosphere model. Planned new model development activities include renewed development of the Semi-Lagrangian spectral dynamics. Software engineering plans include blocked/ESMF-compliant dynamics, utility layer standardization, and the introduction of parallel I/O options developed as part of ZIOLIB. Planned science applications include using subgrid orography with IPCC simulations and high-resolution downscaling of selected IPCC runs.

Land Model: Land model was completely rewritten to add vectorization. Decomposition algorithm was completely modified to significantly improve load balancing. River runoff history output was modified so that it appeared on the RTM grid rather than the model grid. Ongoing tasks include incorporation of new biogeochemistry, (CLM3-CN and CLM3-CASA'), an urban model, and dynamic land use.

Ocean Model: Vector modifications were introduced into the CCSM3 ocean model. Studies showed that high-resolution runs were able to resolve eddies visible in observations. Other studies examined (a) the impact of iron enrichment, (b) chlorophyll distribution during La Ninas and El Ninos, and (c) simulating global flux of DMS from the ocean to the atmosphere, all using POP. HYPOP development continues, with recent progress on the vertical coordinate system and a new time stepping algorithm. Work also continues on the LANL ecosystem model.

Sea Ice Model: The new incremental remapping scheme proved to be three times faster than MPDATA, resulting in a total model speedup of about 30 percent, and was added to CSIM, the ice model in CCSM3. Vectorization modifications were first introduced into CICE3.0, and then ported into CCSM3/CSIM. An Arctic Ocean Model Intercomparison Project (AOMIP) run, using global, 0.4 deg, coupled POP2.0 and CICE3.1_beta on the Cray X1, finished 1948-2002. Ongoing activities include sensitivity analysis and parameter tuning test of the CICE code using Automatic Differentiation (AD)-generated derivative code.

Cecelia Deluca described ongoing progress with the ESMF project, including the release of ESMF Version 2.0 on 23 June 2004. The release includes software for representing and manipulating modeling components, states, bundles of fields, fields, grids, and arrays, as well as utilities for time management, configuration, and logging. Major limitations of the current distribution are that it only supports logically rectangular grids; only supports sequentially executing components; and does not yet have framework-managed threading enabled. She also described interoperability experiments recently completed, which demonstrate how the framework can be used to create prototype applications out of never-before coupled model components. Prototype applications include CAM/NCEP SSI, CAM/MITgcm, and GFDL B-grid atmosphere/MITgcm.

Tom Henderson discussed plans for CCSM model unification. The objective of this effort is to bring together the stand-alone CAM and the full CCSM so that only a single CCSM model needs to be supported. This would greatly reduce the overhead necessary to maintain two separate and similar codes.

The meeting wrapped up with an open discussion of an evaluation plan for using ESMF in the CCSM. Some of the issues that came up were MPMD versus SPMD, how to handle name conflicts in SPMD, and what the appropriate code and strategy for evaluation would be.



Donald Anderson
David Bader
Anjuli Bamzai
Donald Batchelor
Byron Boville
Ilene Carpenter
George Carr, Jr.
Gerardo Cisneros
Matthew Cordery
Erik DeBenedictis
Jens Debernard
Cecelia DeLuca
John Drake
Brian Eaton
Michael Ham
Helen He
Thomas Henderson

  Robert Jacob
Philip Jones
Brian Kauffman
Tim Killeen
Erik Kluzek
Jay Larson
Nancy Norton
Jerry Olson
Douglass Post
William Putman
Juliana Rew
Mathew Rothstein
Brent Swartz
Michael Tobis
Mariana Vertenstein
Vincent Wayland
Patrick Worley


Community Climate System Model