Metrics & Diagnostics
Last updated: June 2013
- What do we do currently?
The PCWG uses three types of evaluation methods:
a. Compare a single number, seasonal cycle, or time series in the model and available observations/reanalysis using a CESM component model diagnostics package.
b. Qualitatively compare a spatial map in the model and available observations/reanalysis using a CESM component model diagnostics package (mostly PCWG, AMWG polar set)
c. Expert judgment informed by custom plots and evaluation.
Top "metrics" used to evaluate development sea ice simulations within CESM:
1) Annual mean and seasonal cycle of sea ice area and extent: model versus passive microwave satellite observations (method a)
2)Annual spatial pattern of sea ice thickness: model versus available satellite/ground-‐ based observations (methods b and c)
3) Snow thickness/evolution (method c).
4)Annual and seasonal mean atmospheric sea level pressure, surface wind stress, and surface air temperature patterns: model versus reanalysis) . Annual and seasonal mean radiative fluxes: model versus CERES EBAF (methods b and c, AMWG diags)
- What could we do more of in the future?
Proposed new target "metrics" -‐ easy to implement/high-‐priority:
1) Single numbers (mean bias, rms) for annual mean and seasonal cycle of area and extent compared to observations. Add as a table to the sea ice diagnostics package.
2) More quantitative comparisons of sea ice thickness/volume. Pattern correlations?
Proposed new target "metrics" -‐ medium ease to implement/medium-‐priority:
1) Sea ice velocity. Observations available from Pathfinder dataset.
2) Fram Strait Ice Area Transport/Export -‐ Example of an "integrative metric" that is relatively well observed. Observations available from ?.
3) Ice age/area of first year ice. Compare to Maslanik observations.
4) Ice albedo. Observations available from ?
5) Combine several metrics into Taylor diagram.
Proposed new target "metrics" -‐ harder to implement/more thought required:
1) Variability. ENSO-‐like wave analysis? Modes of variability? Maps? Single numbers? Annual variance? Variance versus longitude (Antarctic)?
2) Trend metric. Over what period?
3) Regional diagnostics -‐-‐ means, timeseries, variability. How do we define the regions?
4) Sea ice thickness PDF with available observations (e.g., ASPeCt, IceSat, CyroSat)
5) Relating sea ice to atmospheric patterns (e.g., SAM/NAM, cyclones)