Metrics & Diagnostics

Last updated: June 2013

1. What do we do currently?
   
   
   
   The PCWG uses three types of evaluation methods:
   
   a. Compare a single number, seasonal cycle, or time series in the model and available observations/reanalysis using a CESM component model diagnostics package.
   
   b. Qualitatively compare a spatial map in the model and available observations/reanalysis using a CESM component model diagnostics package (mostly PCWG, AMWG polar set)
   
   c. Expert judgment informed by custom plots and evaluation.
   
   
   
   Top "metrics" used to evaluate development sea ice simulations within CESM:
   
   1) Annual mean and seasonal cycle of sea ice area and extent: model versus passive microwave satellite observations (method a)
   
   2)Annual spatial pattern of sea ice thickness: model versus available satellite/ground-­‐ based observations (methods b and c)
   
   3) Snow thickness/evolution (method c).
   
   4)Annual and seasonal mean atmospheric sea level pressure, surface wind stress, and surface air temperature patterns: model versus reanalysis) . Annual and seasonal mean radiative fluxes: model versus CERES EBAF (methods b and c, AMWG diags)
   
    
2. What could we do more of in the future?
   
   
   
   Proposed new target "metrics" -­‐ easy to implement/high-­‐priority:
   
   1) Single numbers (mean bias, rms) for annual mean and seasonal cycle of area and extent compared to observations. Add as a table to the sea ice diagnostics package.
   
   2) More quantitative comparisons of sea ice thickness/volume. Pattern correlations?
   
   
   
   Proposed new target "metrics" -­‐ medium ease to implement/medium-­‐priority:
   
   1) Sea ice velocity. Observations available from Pathfinder dataset.
   
   2) Fram Strait Ice Area Transport/Export -­‐ Example of an "integrative metric" that is relatively well observed. Observations available from ?.
   
   3) Ice age/area of first year ice. Compare to Maslanik observations.
   
   4) Ice albedo. Observations available from ?
   
   5) Combine several metrics into Taylor diagram.
   
   
   
   Proposed new target "metrics" -­‐ harder to implement/more thought required:
   
   1) Variability. ENSO-­‐like wave analysis? Modes of variability? Maps? Single numbers? Annual variance? Variance versus longitude (Antarctic)?
   
   2) Trend metric. Over what period?
   
   3) Regional diagnostics -­‐-­‐ means, timeseries, variability. How do we define the regions?
   
   4) Sea ice thickness PDF with available observations (e.g., ASPeCt, IceSat, CyroSat)
   
   5) Relating sea ice to atmospheric patterns (e.g., SAM/NAM, cyclones)