next up previous contents
Next: 12 The Sea-Ice Model Up: UsersGuide Previous: 10 The Atmosphere Setup   Contents


11 The Ocean Model Setup Script: ocn.setup.csh

The ocean setup script, $SCRIPTS/ocn.setup.csh, is called by when an active ocean model is selected in the $MODELS environment variable array. ocn.setup.csh prepares the ocean model component for execution. To do so, it defines the run environment, collects the ocean input data sets, then builds the dynamic ocean model by calling gmake.

If ocn.setup.csh is unable to complete any of these tasks, it will abort with a non-zero error status. The script checks the error status and will halt if an error is detected.

11.1 Document the ocean setup script

#! /bin/csh -fv
# Purpose:
#  (a) build an executable model (pop, a CSM active ocn component)
#  (b) gather or create necessary input files
# For help, see:
cat $0;$TOOLS/ccsm_checkenvs || exit -1            # cat this file, check envs

This first section documents the ocean setup script.

The first line of this section identifies this as a C-shell script. The "-f" option prevents the user's personalized $HOME/.cshrc file from being executed to avoid introducing aliases that could adversely affect the operation of this script.

The CVS lines document the revision control version of this script.

The echo lines document the purpose of this script. These output from the echo commands will appear in the component log files.

The cat command combines two functions on one line. The "cat $0" command prints a copy of the entire setup script into the output log file in order to document the exact options set by this script. Then $TOOLS/ccsm_checkenvs writes the environment variables that have been set by into the same output log file. If any of the required environment variables are not set, the setup script will exit with an error status of -1.

11.2 Set code location and resolution dependencies

echo -----------------------------------------------------------------
echo  setting up some internal Env variables and path names
echo -----------------------------------------------------------------

set SRCDIR  = $CSMCODE/ocn/pop
set my_path = $SCRIPTS/src.ocn

if ( $OCN_GRID ==  gx1v3) then
  setenv DT_COUNT  23
  setenv TIME_MIX_FREQ 17
  setenv INIT_TS_FILE ts_PHC2_jan_20010711

else if ( $OCN_GRID ==  gx3) then
  setenv DT_COUNT  12
  setenv TIME_MIX_FREQ 100
  setenv INIT_TS_FILE ts_PHC2_jan_20011012

  echo "$0 ERROR: Cannot deal with GRID = $GRID"
  exit -1

This section defines the ocean model source-code location and sets a few resolution dependent variables.

SRCDIR points to the directory for the POP model. Under SRCDIR are a number of subdirectories containing source code, documentation, input templates and other files needed by POP. SRCDIR is typically treated as an archive directory containing ``frozen'' code. Frozen source code refers to code that represents the exact released version of the model. Frozen code is never modified under any circumstances. To modify code, copy it from $SRCDIR into $my_path and make the necessary changes to the $my_path copy.

$my_path identifies a directory that contains user modified POP code. Any POP code that a user needs to modify should be placed in the $my_path directory. POP source code files in $my_path will have precedence over any copies of the same file that exist under SRCDIR. $my_path is designed to be only one level deep, with no subdirectories. If changes are so extensive that subdirectories are required, the ocn.setup.csh script will need to be modified accordingly.

Next, three resolution dependent variables are set. These variables will be input into the POP namelist via the pop_in file, which is modified in section b:

DT_COUNT is the number of timesteps per day.
TIME_MIX_FREQ is the number of timesteps between time-mixing
INIT_TS_FILE is the initial-condition temperature and salinity restart file used for continuation or startup runs.
See the POP users guide for full details.

11.3 Position initial files

# branch- or hybrid-run setup
if ($RUNTYPE == branch || $RUNTYPE == hybrid) then
  setenv INIT_TS_FILE ${REFCASE}.pop.r.$REFDATE-00000
  setenv TAVG_TS_FILE ${REFCASE}.pop.rh.$REFDATE-00000

# position the branch or hybrid files
  if !(-d rest ) mkdir -p rest
  cd rest
  $TOOLS/ccsm_getfile  $REFCASE/ocn/rest/$INIT_TS_FILE     || exit 99
  $TOOLS/ccsm_getfile  $REFCASE/ocn/rest/$INIT_TS_FILE.hdr || exit 99
  $TOOLS/ccsm_getfile  $REFCASE/ocn/rest/$TAVG_TS_FILE     || exit 99
  $TOOLS/ccsm_getfile  $REFCASE/ocn/rest/$TAVG_TS_FILE.hdr || exit 99
  cd ..

# generate the appropriate restart pointer files
cat >! $SCRIPTS/rpointer.ocn.restart << EOF

cat >! $SCRIPTS/rpointer.ocn.tavg << EOF


In this section, a number of the initial data and restart pointer files are identified and positioned for branch or hybrid runs. For hybrid or branch runs, the initial (INIT_TS_FILE) and time-averaged (TAVG_TS_FILES) input filenames are determined by resolving the name based on the $REFCASE and $REFDATE variables set in the script. These data files and their associated header files are placed in the ocean restart directory. Finally the restart pointer files, $SCRIPTS/rpointer.ocn.restart and $SCRIPTS/rpointer.ocn.tavg, are created.

11.4 Build the executable

echo -------------------------------------------------------------------------
echo  a. Build an executable in $OBJDIR
echo -------------------------------------------------------------------------

if ( -e ocn && $?DONTBUILD ) then
  echo "Note: using an existing binary"

  echo -------------------------------------------------------------------------
  echo  a.1 Calculate processor tiling based on $NTASK
  echo -------------------------------------------------------------------------
  if ($NTASK ==  4)   setenv NX  2
  if ($NTASK ==  8)   setenv NX  4
  if ($NTASK == 16)   setenv NX  4
  if ($NTASK == 32)   setenv NX  8
  if ($NTASK == 40)   setenv NX  5
  if ($NTASK == 48)   setenv NX  8
  if ($NTASK == 64)   setenv NX  8
  if !($?NX)          echo ERROR: ocn cant deal with NTASK=$NTASK
  if !($?NX)          exit -1
  @ ny = $NTASK / $NX; setenv NY $ny

The POP executable is built in this section. First the processor tiling is resolved, then the code necessary to build the model is copied in from a number of locations. Once the source code is in place, the POP executable is built with gmake.

This section attempts to automatically specify, for a variety of pre-determined processor counts, the values of NS and NY, the number of processors to assigned to the X and Y dimensions for a number of pre-determined processor counts. This is designed to be automatic. The values of NX and NY are passed as arguments to the gmake file, where they are used to set the values of the C-preprocessor parameters NPROC_X and NPROC_Y. Further documentation on NPROC_X and NPROC_Y is in the POP documentation:

  echo -----------------------------------------------------------------
  echo  a2. creating the internal directory structure
  echo -----------------------------------------------------------------

  set compile_dir = $OBJDIR/compile
  set source_dir  = $OBJDIR/source
  if !(-d $source_dir  ) mkdir -p $source_dir
  if !(-d $compile_dir ) mkdir -p $compile_dir

  echo ' '
  echo '-----------------------------------------------------------------'
  echo ' copy the necessary files into $source_dir                       '
  echo '-----------------------------------------------------------------'
  cd $source_dir
  cp -fp $SRCDIR/input_templates/${OCN_GRID}_model_size.F model_size.F
  cp -fp $SRCDIR/source/*.[FCc]             .
  cp -fp $SRCDIR/mpi/*.[FCc]                .
  cp -fp $CSMSHR/*.[F]*                   .
  cp -fp $CSMSHR/*.[Cc]                   .
  if (-d $my_path ) cp -fp $my_path/*.[FCc] .

The POP source code and compilation products are placed into two different directories, source_dir and compile_dir. Here, the two directories are created and all the necessary source code is copied into source_dir. Notice that the last copy is from $my_path, which overwrites any files that have the same name as the files in $my_path .

input_templates/gx1v3_model_size.F specifies the grid dimensions and number of tracers.

#  recompile if 2d decomp is changed
 diff $OBJDIR/ $OBJDIR/ocnres.old || touch `grep -l NPROC $OBJDIR/source/*`
 echo $OCN_GRID $NTASK $NX $NY >! $OBJDIR/ocnres.old

The POP processor tiling is compiled into the model executable. Any changes to the number of processors assigned to POP will change the tiling, which requires recompiling POP. This section compares the currently requested tiling with the tiling specified during the last build. If the two tilings are different, a recompilation of POP is forced.

  echo -----------------------------------------------------------------
  echo  a3. compile and copy the executable into $EXEDIR directory
  echo -----------------------------------------------------------------

  if ($BLDTYPE == 'true') then
    cd $compile_dir

    cc -o makdep $CSMBLD/makdep.c  || exit 2

    set EXEC = ocn_${NTASK}
    set THREAD = FALSE ; if ($NTHRD > 1) set THREAD = TRUE
    gmake -j 6 -f $CSMBLD/Makefile MACFILE=$CSMBLD/Macros.$OS \
	MODEL=pop NX=$NX NY=$NY THREAD=$THREAD                \
        VPATH=$source_dir EXEC=$OBJDIR/$EXEC                  || exit 2

# gmake -f $CSMBLD/Makefile MACFILE=$CSMBLD/Macros.$OS mostlyclean

  #--- cp & link into EXEDIR ---
    rm -f $EXEDIR/ocn
    cp $OBJDIR/$EXEC  $EXEDIR/ocn
    echo "BLDTYPE = $BLDTYPE"


A number of steps are required to build the POP executable using gmake. First, the dependency generator, makdep, is created. When gmake is called, makdep will build a list of source-code dependencies that identify the specific POP files that need to be recompiled if any of the source code is modified.

$EXEC specifies the name of the POP executable. The processor count NTASK is appended to the name to differentiate between executables built for different values of NTASK.

$THREAD allows for multiple MPI threads. However, since POP does not support threading, this should always be FALSE.

Finally, gmake is called using the generic CCSM gmake and macros files and the resulting executable is copied into the ocean-model execution directory.

11.5 Parse the date variables

echo ' '
echo '-------------------------------------------------------------------------'
echo " b. gather or create necessary input files in $EXEDIR                   "
echo '-------------------------------------------------------------------------'

set IYEAR0        = `echo $BASEDATE | cut -c1-4  | sed -e 's/^0*//'`
set IMONTH0       = `echo $BASEDATE | cut -c6-7  | sed -e 's/^0*//'`
set IDAY0         = `echo $BASEDATE | cut -c9-10 | sed -e 's/^0*//'`

if ($RUNTYPE == startup || $RUNTYPE == hybrid  )  @ IDAY0 = $IDAY0 + 1

The POP model receives the initial date information is a slightly different format than it is defined in the $SCRIPTS/ script. This short section does this text transformation.

IYEAR0, IMONTH0 and IDAY0 are the numeric representations for the coordinated base year, month and day set in Startup and hybrid runs with POP require that the rest of the coupled system run for 1 day before POP starts. This is accomplished by incrementing the IDAY0 setting.

11.6 Modify pop_in

# sed commands are of the form  s#OldString#NewString#

cat >! $EXEDIR/commands.sed << EOF

s#RUNID#$CASE#;                  s#LOG_FILENAME#$ocn_out#

s#IYEAR0#$IYEAR0#;               s#IHOUR0#0#
s#IMONTH0#$IMONTH0#;             s#IMINUTE0#0#
s#IDAY0#$IDAY0#;                 s#ISECOND0#0#


sed -f $EXEDIR/commands.sed $SRCDIR/input_templates/${OCN_GRID}_pop_in
                                           >! $EXEDIR/pop_in  || exit 3

# put pop_in where pop can find it (specific to mpirun, poe, etc)
ln -sf $EXEDIR/pop_in $EXEDIR/..
if (-d $EXEDIR/../all) ln -sf $EXEDIR/pop_in $EXEDIR/../all

Building the input namelist file for the ocean setup procedure differs from the other components. The text file ``pop_in'' contains a complete, resolution-dependent set of namelist input to POP, with several generically named values that are intended to be automatically modified by the ocean setup script. This section carries out this modification using the Unix stream editor, sed, to insert the desired namelist values into the pop_in file.

This set of sed commands modifies selected variables from the large generic pop_in namelist file in the directory $SRCDIR/input_templates. The sed commands are placed in the file commands.sed, then applied against the generic pop_in file. Additional changes to the pop_in file can be added here by following the examples in the script and appropriately modifying the pop_in file. The resulting pop_in file is linked into the directory in the next level up and to the /all directory for use on the SGI.

11.7 Define the ocean input datasets

echo '-------------------------------------------------------------------------'
echo   c. cp input datasets from $SRCDIR ...
echo '-------------------------------------------------------------------------'

if !( -d $EXEDIR/rest  )  mkdir -p $EXEDIR/rest   || exit 2
if !( -d $EXEDIR/hist  )  mkdir -p $EXEDIR/hist   || exit 2
if !( -d $EXEDIR/input )  mkdir -p $EXEDIR/input  || exit 2

cd $EXEDIR/input

foreach FILE (depth_accel history_contents \
              movie_contents region_ids  scalar_contents tavg_contents \
              transport_contents vert_grid )
  $TOOLS/ccsm_cpdata $SRCDIR/input_templates/${OCN_GRID}_$FILE $FILE || exit 99
foreach FILE (eos_coefficients.ieeer8 )
  $TOOLS/ccsm_cpdata $SRCDIR/input_templates/${OCN_GRID}_$FILE $FILE

This section collects the input datasets needed for the ocean model. The input datasets are all uniquely named by a description and a six digit number that documents the creation date (format: yymmdd) of the file. While the hard-wiring of filenames restricts the degree of automation, it ensures that the exact data that the user requests is input into the model.

First, the restart, history and input data directories are created if they don't already exist. Then text input files from the POP ``input_template'' directory are copied into the ocean input directory.

See the CCSM23.0 POP User's Guide, chapter 4, Model diagnostics and output, for a complete discussion of these files ( ).

11.8 Position the ocean input datasets

echo '-------------------------------------------------------------------------'
echo   c. acquire initial/boundary datasets disk or MSS
echo '-------------------------------------------------------------------------'

set OCNDATA = ocn/pop/$OCN_GRID  # point to disk copies

if ($RUNTYPE == startup ) then
$TOOLS/ccsm_getinput $OCNDATA/ic/${INIT_TS_FILE}.ieeer8         ts                  
$TOOLS/ccsm_getinput $OCNDATA/ic/${INIT_TS_FILE}.readme         ts.readme           
if ($OCN_GRID == gx1v3) then                                                        
 $TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20010402.ieeer8   horiz_grid         
 $TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20010402.readme   horiz_grid.readme  
 $TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20010709.ieeei4  region_mask        
 $TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20010709.readme  region_mask.readme 
 $TOOLS/ccsm_getinput $OCNDATA/grid/topography_20010702.ieeei4   topography         
 $TOOLS/ccsm_getinput $OCNDATA/grid/topography_20010702.readme   topography.readme  
 $TOOLS/ccsm_getinput $OCNDATA/forcing/shf_mm_all_85-88_20010308.ieeer8  shf        
 $TOOLS/ccsm_getinput $OCNDATA/forcing/shf_mm_all_85-88_20010308.readme  shf.readme 
 $TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_mm_all_85-88_20010320.readme shf.readme 
 $TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_mm_all_85-88_20010320.ieeer8 sfwf       
else if ($OCN_GRID == gx3) then                                                     
 $TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20001030.ieeer8 	 horiz_grid         
 $TOOLS/ccsm_getinput $OCNDATA/grid/horiz_grid_20001030.readme 	 horiz_grid.readme  
 $TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20001030.ieeei4  region_mask        
 $TOOLS/ccsm_getinput $OCNDATA/grid/region_mask_20001030.readme  region_mask.readme 
 $TOOLS/ccsm_getinput $OCNDATA/grid/topography_20001030.ieeei4   topography         
 $TOOLS/ccsm_getinput $OCNDATA/grid/topography_20001030.readme   topography.readme  
 $TOOLS/ccsm_getinput $OCNDATA/forcing/shf_20011030.ieeer8       shf                
 $TOOLS/ccsm_getinput $OCNDATA/forcing/shf_20011030.readme       shf.readme         
 $TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_20011030.ieeer8      sfwf               
 $TOOLS/ccsm_getinput $OCNDATA/forcing/sfwf_20011030.readme      sfwf.readme        



The variable OCNDATA defines the location of the collection of Initial and boundary datasets that are distributed with CCSM2.0. These variables are:

horiz_grid_20010402.ieeer8 Specifies the horizontal grid.
region_mask_20010709.ieeei4 Specifies the region mask.
topography_20010702.ieeei4 Specifies the topography (index of deepest
topography_20010702.readme ocean level at each horizontal grid point)
shf_mm_all_85-88_20010308.ieeer8 Contains monthly averaged fields of
shf_mm_all_85-88_20010308.readme sea-surface temperature needed by the
  surface heat flux forcing option ``restoring''.
sfwf_mm_all_85-88_20010320.ieeer8 Contains monthly averaged fields of
sfwf_mm_all_85-88_20010320.readme sea-surface salinity needed by the surface
  freshwater flux forcing option ``restoring''.

The POP input datasets are copied of from $OCNDATA to the $EXEDIR/input directory. Note that in the case of a ``startup'' run, initial temperature and salinity initial conditions are also copied to the input directory.

unset echo

echo ' '
echo '-----------------------------------------------------------------'
echo ' End of setup shell script                                       '
echo '-----------------------------------------------------------------'


Once the ocn.setup.csh script completes successfully, it will exit with a status value of 0.

next up previous contents
Next: 12 The Sea-Ice Model Up: UsersGuide Previous: 10 The Atmosphere Setup   Contents