Metrics & Diagnostics

Last updated: June 2013

  1. What do we do currently?

    The PCWG uses three types of evaluation methods:

    a. Compare a single number, seasonal cycle, or time series in the model and available observations/reanalysis using a CESM component model diagnostics package.

    b. Qualitatively compare a spatial map in the model and available observations/reanalysis using a CESM component model diagnostics package (mostly PCWG, AMWG polar set)

    c. Expert judgment informed by custom plots and evaluation.

    Top "metrics" used to evaluate development sea ice simulations within CESM:

    1) Annual mean and seasonal cycle of sea ice area and extent: model versus passive microwave satellite observations (method a)

    2)Annual spatial pattern of sea ice thickness: model versus available satellite/ground-­‐ based observations (methods b and c)

    3) Snow thickness/evolution (method c).

    4)Annual and seasonal mean atmospheric sea level pressure, surface wind stress, and surface air temperature patterns: model versus reanalysis) . Annual and seasonal mean radiative fluxes: model versus CERES EBAF (methods b and c, AMWG diags)

  2. What could we do more of in the future?

    Proposed new target "metrics" -­‐ easy to implement/high-­‐priority:

    1) Single numbers (mean bias, rms) for annual mean and seasonal cycle of area and extent compared to observations. Add as a table to the sea ice diagnostics package.

    2) More quantitative comparisons of sea ice thickness/volume. Pattern correlations?

    Proposed new target "metrics" -­‐ medium ease to implement/medium-­‐priority:

    1) Sea ice velocity. Observations available from Pathfinder dataset.

    2) Fram Strait Ice Area Transport/Export -­‐ Example of an "integrative metric" that is relatively well observed. Observations available from ?.

    3) Ice age/area of first year ice. Compare to Maslanik observations.

    4) Ice albedo. Observations available from ?

    5) Combine several metrics into Taylor diagram.

    Proposed new target "metrics" -­‐ harder to implement/more thought required:

    1) Variability. ENSO-­‐like wave analysis? Modes of variability? Maps? Single numbers? Annual variance? Variance versus longitude (Antarctic)?

    2) Trend metric. Over what period?

    3) Regional diagnostics -­‐-­‐ means, timeseries, variability. How do we define the regions?

    4) Sea ice thickness PDF with available observations (e.g., ASPeCt, IceSat, CyroSat)

    5) Relating sea ice to atmospheric patterns (e.g., SAM/NAM, cyclones)