Identifying the Necessary Ensemble Size for Estimating Extremes
Temperature anomalies in 2015-2019 relative to a base period of 1951-1980, showing large warming over most land areas. Image courtesy NASA
Ensembles of 20–25 members, smaller than traditional large ensembles, can accurately represent changes and uncertainty bounds in extremes of temperature and precipitation.
The Science
Estimating future extremes in temperature and precipitation is important for understanding a changing climate. This study shows how increasing ensemble sizes affect the precision of various estimates for changes in climate extremes in Earth system models and if such assessments can be conducted based on 5-member ensembles to determine the need for additional members.
This study used existing large ensembles and statistics to test the precision of estimates of changes in extremes based on different ensemble sizes. They found that a small ensemble (five members) allows them to anticipate the reduction of errors in estimating extreme changes as the ensemble size increases. The main result of this study shows that the errors of an estimate with only 20–25 members remain within the error bounds defined by a full ensemble of 40 or 50 members.
The Impact
Over the last few years, scientists and modelers have recognized the importance of exploring the noise in the climate system by running large-size ensembles with a climate model. Accordingly, modeling centers budget computational resources to run these expensive experiments. The size of these ensembles is generally around 40 or 50 members. Researchers show that for many questions related to the characterization of temperature and precipitation extremes, running ensembles larger than 25 members only marginally increases the data’s accuracy. This result is valuable for planning experiments and computer time allocations for large Earth system model setups.
Summary
This study considered the problem of estimating the ensemble sizes required to characterize mean changes and their variability for several extreme metrics. They took the perspective of a modeling center wanting to a priori estimate these sizes based on an existing small ensemble (5 members).
Figure 1: Error (as a fraction of the “true” 95% interval upper bound computed based on the full 40-member ensemble) in estimating the change in the temperature of the hottest night of the year by the end of the century under RCP8.5 (warming scenario of emissions, frequently referred to as “business and usual”) for ensembles with different member numbers, shown on the top of each panel. Errors are small for most grids with ensembles of 20–25 members.
Scientists used full ensemble sizes (40–50 ensemble members) as the ground truth, then estimated the standard error of a 5- member ensemble, and compared the expected and real error (Fig. 1). They used simple mean and standard error computations together with more sophisticated inferential statistics to characterize the signal and noise of six climate extremes metrics. The robustness of the results was tested using two large ensembles from different climate models at a variety of forcing levels, time horizons(two time periods: mid-century and near the end of the century), and spatial scales.
For a range of spatial scales, temporal scales, forcing levels, and two models, an ensemble size of 20 or 25 members appears to provide estimates (for the extreme metrics considered) that remain within small absolute and percentage errors. Additional members add only marginal precision to the estimate. This remains true when estimating variability in these extremes and how they might change under a high-emission scenario. While applications and metric definitions requiring larger statistical power and therefore larger ensemble sizes exist, these results suggest that an effective estimate of both mean changes and variability can be achieved with sizes below 30 members for a wide range of problems and scales. This smaller size would save computational resources for addressing additional sources of uncertainty in future climate projections.
Publication
Tebaldi, C., Dorheim, K., Wehner, M., & Leung, R. (2021). “Extreme Metrics and Large Ensembles”, Earth Syst. Dynam., 12, 1427–1501. https://doi.org/10.5194/esd-2021-53
Funding
- The U.S. Department of Energy Office of Science, Biological and Environmental Research Program supported portions of this research as part of the Regional and Global Model Analysis (RGMA), Atmospheric System Research (ASR), and Earth System Model Development (ESMD) program areas.
- This research used resources of the National Energy Research Scientific Computing Center (NERSC).
- This study was supported by the Energy Exascale Earth System Model (E3SM) project funded by the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research program. MW was supported by the CASCADE project, also funded by the DOE, Office of Science, Biological and Environmental Research program. Pacific Northwest National Laboratory is operated by Battelle for DOE under Contract DE-AC05-76RLO1830. Lawrence Berkeley National Laboratory is operated by DOE under Contract No. DE340AC02-05CH11231.
Contact
- Ruby Leung, Pacific Northwest National Laboratory
