Journal cover Journal topic
Nonlinear Processes in Geophysics An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 1.699 IF 1.699
  • IF 5-year value: 1.559 IF 5-year
    1.559
  • CiteScore value: 1.61 CiteScore
    1.61
  • SNIP value: 0.884 SNIP 0.884
  • IPP value: 1.49 IPP 1.49
  • SJR value: 0.648 SJR 0.648
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 52 Scimago H
    index 52
  • h5-index value: 21 h5-index 21
Discussion papers
https://doi.org/10.5194/npg-2019-63
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/npg-2019-63
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Submitted as: research article 20 Jan 2020

Submitted as: research article | 20 Jan 2020

Review status
This preprint is currently under review for the journal NPG.

Beyond Univariate Calibration: Verifying Spatial Structure in Ensembles of Forecast Fields

Joshuah Jacobson1, William Kleiber1, Michael Scheuerer2,3, and Joseph Bellier2,3 Joshuah Jacobson et al.
  • 1Department of Applied Mathematics, University of Colorado Boulder
  • 2Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder
  • 3Physical Sciences Division, NOAA/Earth System Research Laboratory, Boulder, Colorado

Abstract. Most available verification metrics for ensemble forecasts focus on univariate quantities. That is, they assess whether the ensemble provides an adequate representation of the forecast uncertainty about the quantity of interest at a particular location and time. For spatially-indexed ensemble forecasts, however, it is also important that forecast fields reproduce the spatial structure of the observed field, and represent the uncertainty about spatial properties such as the size of the area for which heavy precipitation, high winds, critical fire weather conditions, etc. are expected. In this article we study the properties of a new diagnostic tool designed for spatially-indexed ensemble forecast fields. The metric is based on a level-crossing statistic that we term the fraction of threshold exceedance (FTE), and is calculated for the verification field, and separately for each ensemble member. The FTE yields a projection of a – possibly high-dimensional – multivariate quantity onto a univariate quantity that can be studied with standard tools like verification rank histograms. This projection is appealing since it reflects a spatial property that is intuitive and directly relevant in applications, though it is not obvious whether the FTE is sufficiently sensitive to misrepresentation of spatial structure in the ensemble. In a comprehensive simulation study we find that departures from uniformity of these so called FTE histograms can be indeed be related to forecast ensembles with biased spatial variability, and that these histograms detect shortcomings in the spatial structure of ensemble forecast fields that are not obvious by eye. For demonstration, FTE histograms are applied in the context of spatially downscaled ensemble precipitation forecast fields from NOAA's Global Ensemble Forecast System.

Joshuah Jacobson et al.

Interactive discussion

Status: open (until 16 Mar 2020)
Status: open (until 16 Mar 2020)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement

Joshuah Jacobson et al.

Data sets

Verifying Spatial Structure in Ensembles of Forecast Fields J. Jacobson, J. Bellier, M. Scheuerer, and W. Kleiber https://doi.org/10.5281/zenodo.3592495

Model code and software

FTE J. Jacobson https://doi.org/10.5281/zenodo.3592506

Joshuah Jacobson et al.

Viewed

Total article views: 174 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
129 44 1 174 0 0
  • HTML: 129
  • PDF: 44
  • XML: 1
  • Total: 174
  • BibTeX: 0
  • EndNote: 0
Views and downloads (calculated since 20 Jan 2020)
Cumulative views and downloads (calculated since 20 Jan 2020)

Viewed (geographical distribution)

Total article views: 150 (including HTML, PDF, and XML) Thereof 150 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 

Cited

Saved

No saved metrics found.

Discussed

No discussed metrics found.
Latest update: 23 Feb 2020
Publications Copernicus
Download
Short summary
Most verification metrics for ensemble forecasts assess the representation of uncertainty at a particular location and time. We study a new diagnostic tool based on fractions of threshold exceedance (FTE) which evaluates an additional important attribute: the ability of ensemble forecast fields to reproduce the spatial structure of observed fields. The utility of this diagnostic tool is demonstrated through simulations and an application to ensemble precipitation forecasts.
Most verification metrics for ensemble forecasts assess the representation of uncertainty at a...
Citation