CAM references PIO through both interface routines (which call PIO routines) and PIO routines proper. The interface routines control much of the setting up of PIO output and input, whereas the actual output and input is accomplished through direct PIO calls. The PIO routines are located in models/utils/pio. A description of the PIO package along with usage information may be found in the main PIO documentation. We will first describe the interface and then cover the direct calls.
The CAM interface to PIO consists of the pio_utils module, which is the file models/atm/cam/src/utils/pio_utils.F90.
Init_pio_subsystem is the first routine called. It is called from cam_init and initializes PIO in CAM. Init_pio_subsystem calls read_namelist_pio, which calls set_pio_parameters. The main parameters set includes the IO mode (netcdf vs pnetcdf), number of IO tasks, and IO stride.
Cam_pio_createfile and cam_pio_openfile create and open a PIO file, respectively. Cam_pio_createfile is called from cam_write_restart, h_define (called from wshist, which is called from write_restart_history), and atm_write_srfrest_mct.
Cam_pio_openfile is called from setup_initial (called from cam_initial), cam_read_restart, read_restart_history, atm_read_srfrest_mct, and wrapup (called from cam_run4).
Clean_iodesc_list, called from startup_initialconds and cam_read_restart, cleans the list of IO descriptors.
Get_phys_decomp and get_dyn_decomp create decompositions for physics and dynamics variables, respectively. Get_decomp is an interface to those routines.
Get_phys_decomp is directly called from infld_real_2d, infld_real_3d, infld_real_3dncol; read_restart_physics, write_restart_physics; and pbuf_read_restart, pbuf_write_restart.
Get_dyn_decomp is directly called from infld_real_2d and infld_real_3d.
Get_decomp is called from read_restart_history and dump_field (called from wshist, which is called from write_restart_history). We see that the only time the get_decomp interface is called is when dealing with history variables (where each variable contains a description telling whether it relates to dynamics or physics).
Get_phys_decomp and get_dyn_decomp have an optional column (or column_in) argument, which is used when defining history variables over a range of columns rather than the whole domain.
Whenever a decomposition is created, a description of that decomposition is stored in a list. When a new variable is catalogued, that list is searched to see if that needed decomposition already exists. The routine performing that search is find_iodesc (called from get_phys_decomp and get_dyn_decomp). This capability is supported only when the range of history output is the whole domain.
Get_phys_decomp calls get_phys_ldof (or get_column_ldof), and get_dyn_decomp calls get_dyn_ldof (or get_column_ldof). These routines do the bulk of the work in constructing the IO decompositions. Get_column_ldof is called when the history output is restricted to a subset of the domain.
Virtually all the calls to PIO routines reference the pio module, located in pio.F90. Many of the variables contained in the pio module are referenced as well. This section of the documentation focuses on these calls and surrounding environs and discusses data types only peripherally.
Write_restart_history (in control/cam_history.F90) controls the writing of a history file. Wshist writes the main variables that are on history tapes. Pio_put_var is used to write header information, and subroutine dump_field writes the main history variables, using primarily PIO_write_darray. Additional variables are then written using PIO_put_var. (Recall that PIO_write_darray isused to write distributed arrays.)
Read_restart_history (in control/cam_history.F90) controls the reading of a history file. Individual parameters are first read using pio_get_var. History variables are then read in using PIO_read_darray (which reads distributed arrays).
Cam_write_restart (in control/cam_restart.F90) controls the writing of a restart dump. The calling chain includes write_restart_hycoef (which writes the pressure coefficients), write_restart_dynamics, write_restart_physics, and write_restart_history. Write_restart_coef is in hycoef.F90 and uses PIO_put_var calls. Write_restart_dynamics is in restart_dynamics.F90 (in the dynamics subdirectory relevant to the active dynamical core) and issues primarily PIO_write_darray calls. Write_restart_physics is in physics/cam/restart_physics.F90. It calls a number of restart routines relating to chemistry and aerosols, such as pbuf_write_restart (in physics/cam/phys_buffer.F90), chem_write_restart (in chemistry/mozart), and write_prescribed_?????_restart (where ????? = ozone, ghg, aero, volcaero), located in chemistry/utils/prescribed_?????.F90. Pbuf_write_restart calls the relevant decomposition routine followed by PIO_write_darray. Chem_write_restart calls write_?????_restart (where ????? = tracer_cnst, tracer_srcs, linoz_data, spedata), located in chemistry/mozart/?????.F90. These routines typically use PIO_put_var. Write_restart_history is in cam_history.F90 and uses PIO_put_var.
Cam_read_restart (in control/cam_restart.F90) controls the reading of a restart dump. The calling chain corresponds to that for cam_write_restart, and includes read_restart_hycoef, read_restart_dynamics, read_restart_physics, and read_restart_history. Typical routines called are pio_get_var and PIO_read_darray.
Cam_init (in control/cam_comp.F90) calls cam_initial and phys_init. Cam_initial, which is located in the relevant dynamics subdirectory, calls initial_conds, located in control/startup_initialconds. Initial_conds calls read_inidat, which is located in the relevant dynamics subdirectory. Read_inidat calls infld, located in control/ncdio_atm.F90. Infld calls the relevant decomposition routine followed by PIO_read_darray.
Phys_init is located in physics/cam/phys_grid.F90 and calls a number of initialization routines. An important one is phys_inidat, which makes several calls to infld.