AMWG Metrics and Diagnostics

This page is intended to be a summary of the Atmosphere Model Working Group (AMWG) metrics and diagnostics used in developing the CESM Community Atmosphere Model version 6 (CAM6).

The inclusion of code for new or updated parameterizations or dynamical cores in scientifically supported versions of CESM is predicated on a positive evaluation following a number of metrics detailing agreement with observations. Although this does not determine an absolute pass or fail recommendation, it does inform the AMWG-wide decision as to whether to include major model changes in CAM. This is frequently a difficult decision since changes to the model often have to be balanced between improving the overall climate model (AMIP and coupled) simulations and including currently absent or more accurate representations of a particular physical process.

We request the AMWG input/feedback on the top 10 quantities, current metrics and other potential metrics. Please contact: AMWG liaison or AMWG co-chair.

Current metrics of performance for CAM and CESM simulations

Below is the current draft list of metrics that are commonly used to assess the fidelity of CAM standalone (AMIP) and CESM fully coupled simulations for annual means (ANN) and for winter and summer season averages (DJF; JJA).
The recommended AMIP simulation length is at least 10 years.
The recommended coupled simulation length is at least 50 years.
Code for calculating metrics (written in ncl) is/will be available on the AMWG diagnostics webpage.

Top 10 quantities

- Sea level pressure
- Shortwave cloud forcing
- Longwave cloud forcing
- Tropical land rainfall (30N - 30S)
- Tropical ocean rainfall (30N - 30S)
- Surface air temperature (at 2m) over land
- Equatorial Pacific zonal surface stress (5S - 5N)
- Zonal wind at 300 mb
- Relative humidity
- Temperature

Metrics and diagnostics

1. Global averages. Top of model residual energy balance (RESTOM) should be within 1-2 W/m2 for AMIP; close to zero for 1850 couple).

2. The average normalized bias and Root Mean Square Error (RMSE) from the top 10 list of Taylor diagram metrics should be competitive with CAM5 or CESM1-CAM5 simulations.

3. The individual RMSE, bias and correlation values of the top 10 list of Taylor diagram diagnostics should be competitive with CAM5 or CESM-CAM5 simulations. All variables are global unless otherwise stated .

4. Variance of nino3.4 Sea Surface Temperature anomalies (SSTA) should be greater than 0.5K^2, less than 1.5K^2 (Coupled only)

5. Phase of nino3.4 Sea Surface Temperature anomalies (SSTA) period should peak between 3 and 5 years). (Coupled only)

6. Simulation of the 'Climate Error Score'. This is based on error statistics of the northern hemisphere (30N - 90N) 200-mb height field and is meant to assess the model's ability to simulate the stationary wave field of the northern hemisphere (the Normalized Mean Square Errors (NMSE) has to be no greater than 50% higher than CAM5 or CESM1-CAM5 simulations).

Other potential metrics (ncl analysis scripts in development)

1. ITCZ metric (difference in maximum zonal precipitation between 0 and 10N minus the maximum zonal - mean precipitation between 0 and 10S) .

2. MJO metric (the ratio of eastward power to westward power for 3 - hourly averaged outgoing longwave radiation (OLR/FLNT) and averaged for wavenumbers 1 and 2 and frequency p eriods 30 to 90 days) .

3. Diurnal cycle over land metric (phase and amplitude of first diurnal harmonic from 3-hourly total precipitation output over the ARM - SGP site in Oklahoma) .