next up previous contents index
Next: 11 Glossary Up: UsersGuide Previous: 9 Troubleshooting (Log Files,   Contents   Index


10 Post Processing

A completed run of CCSM3.0 will produce a large number of files, each with many variables. Post processing these files presents several unique challenges. Various techniques and tools have been developed to meet these challenges. Most involve software that does accompany the CCSM3 distribution. The user will have to download and install these tools separately. In the following sections, some of these tools will be described and their download location identified. Follow on sections will demonstrate various techniques using these tools.

10.1 Software Overview

10.1.1 The netCDF Operators (NCO)

The netCDF Operators are command line executables specifically designed for multi-file bulk processing. They are free, run on numerous operating systems, and are essential in reducing the number of CCSM files to a manageable level. While not officially supported, they are widely used and easy to learn. The operators and on line user's manual can be found at:

Table 7 lists the operators and gives a brief description. The user is directed to the web page listed above for specific details, limitations, and usage.

Table 7: netCDF Operators
Operator Name Function  
ncap arithmetic processor  
ncatted attribute editor  
ncbo binary operator (add, subtract, multiply, divide)  
ncea ensemble averager  
ncecat ensemble concatinator  
ncflint file interpolator  
ncks kitchen sick (variable extraction)  
ncra record averager  
ncrcat record concatinator  
ncrename renamer  
ncwa weighted averager  

Specific examples of these operators exist in sections 10.2.1 and 10.2.2.

10.1.2 The NCAR Command Language (NCL)

NCL is a free, interpreted computer language developed at NCAR. It is highly suited to meteorological and climatological data processing and visualization. It runs on UNIX, AIX, IRIX, LINUX, MACOS X, and Windows under cygwin. It handles netCDF, GRIB, and HDF data formats with ease and is officially supported. The following web site contains a large on line library of graphical and data processing scripts that can be downloaded and applied:

Questions about the software should be directed to or to Sylvia Murphy ( NCL exists on all NCAR supercomputers and CGD systems. It can be also be downloaded at:

Questions about the NCL binaries and their installing should be directed to Mary Haley ( of the Scientific Computing Division.

10.1.3 Component Model Processing Suite (CMPS)

CMPS is similar in functionality to the netCDF operators (see section 10.1). They are command line executables that are available on NCAR's supercomputers. Unlike the NCO, CMPS only works with CCSM model data. An on line user's guide is available at:

Table 8 lists the current operators and gives a brief description of their function. CMPS is officially supported. The user is advised to visit the web page above or contact Sylvia Murphy ( for more information and specific usage.

Table 8: Component Model Processing Suite
Operator Name Function  
add.scp addition  
diff.scp difference/anomalies  
e2d.scp extract all 2D variables  
esvc.scp extract selected variables and concatenate  
have.scp horizontal averaging  
iso.scp iso-level extraction  
iso_interp.scp interpolation of a variable to an iso surface  
mult.scp multiplication  
rmse.scp RMSE  
subext.scp sub-region extraction  
tsext.scp time series extraction  
ttest.scp t-test  
topbot.scp single level extraction  
trans.scp transects  
vert.scp vertical averaging  
volave.scp volume averaging  
vert.scp vertical averaging  
zave.scp zonal averaging  

10.1.4 Diagnostic Packages

Comprehensive diagnostic packages exist for each model component. Not all are publicly available. CCSM3 diagnostic packages are designed to do bulk file processing (e.g. concatenate files and average using the NCO), conduct the necessary data processing, and output NCL graphics in a web format. Table 9 contains a list of the currently available packages, their point of contact, and download location.

Table 9: Publicly available diagnostics
Component POC Download location
atm Mark Stevens (

10.1.5 Commercial Tools

People who run the CCSM use many commercial tools for post processing. These can include IDL, Fortran, MatLab, Yorick, and Vis5D. These are mentioned for information only. Use of these tools is not supported, and it is up to the user to download, install, and debug their own programs.

10.2 Post Processing Strategies

There is no way this guide can provide a detailed set of instructions for all of the post-processing tasks a typical CCSM3.O run requires. What follows is a short discussion of some of the more common tasks.

10.2.1 Selected Variable File Concatenation

CCSM output consists of many files each containing many variables. Often researchers are only interested in a subset of those variables. One strategy for file management is to extract the variables of interest and concatenate them into a smaller subset of files. The netCDF operators (see section 10.1) are well suited to this task. The following is a code snippet from a c-shell script that uses ncks to extract the selected variables T,U, and V and then uses ncrcat to combine the files together.

# extract variables using ncks
ls *.nc > file.txt                     # get the list of files
set fnlst = `cat file.txt`
foreach x ($fnlst)                     # loop over each file
  ncks -A -h -v T,U,V $x temp_$x       # extract variables to temp file
# concatinate variables with ncrcat
ncrcat -A -h temp_*.nc # concatinate all temp files together

10.2.2 Creating Climatologies

Climatologies can be created using the netCDF operators (see section 10.1) or NCL (see section 10.1.2). The code snippet below demonstrates the use of ncra to create annual averages for a set of years using monthly data for those years. ncra is then used again to average the annual averages to create an annual climatology.

@ yr_cnt      = $yr_strt                         # create year counter
@ yr_end      = $yr_strt + $num_yrs - 1          # put space btwn "-" and "1"
while ( $yr_cnt <= $yr_end )                     # loop over years
  set yr_prnt = `printf "%04d" {$yr_cnt}`        # format(change as required)

  ls ${DIR}${case}_${yr_prnt}*.nc > file.txt     # get list of files
  set files = `cat file.txt`
  ncra -O $files ${DIR}${case}_${yr_prnt} # create annual average
end while

ls ${DIR}${case}* > file.txt               # get list of ann average
set files = `cat file.txt`
ncra -O $files ${DIR}${case}        # create climatological avg

There is a web page devoted to creating climatologies using NCL

This page demonstrates just a few of the climatological related functions available. For a full list, see

10.2.3 Atmospheric Hybrid to Pressure Level Interpolation

The atmospheric model output is on hybrid coordinates (see the User's Guide to the NCAR Community Atmosphere Model 3.0 (CAM 3.0) for details). Often it is desirable to convert these to pressure coordinates. The following web page demonstrates this conversion using NCL (see section 10.1.2):

10.2.4 POP remapping

A special NCL (see section 10.1) library script (pop_remap.ncl) has been developed for POP remapping. This library script comes with the NCL distribution. In order to remap POP, special POP weight and grid files are necessary. SCRIP, which was developed at Los Alamos National Labs, is used to create these files. NCAR maintains a repository of grid and weight files for model configurations that have been scientifically validated (see section 1.5). If a map or weight file for one of these validated configurations is desired, contact Sylvia Murphy ( ) to have the files placed on an anonymous ftp server. The following web site describes how to use pop_remap.ncl and NCL to create other POP graphics:

next up previous contents index
Next: 11 Glossary Up: UsersGuide Previous: 9 Troubleshooting (Log Files,   Contents   Index