E3SM Data Locations and Policies
Data Documentation
Did you know that E3SM makes all data from its major campaigns publicly available and that the list of available datasets is posted on E3SM Data Docs? This site, which offers an organized, informative, and user-friendly experience, is now linked from the the e3sm.org data page, which can now be found under the RESOURCES tab rather than its own top level tab on the E3SM home page.
For each model version and simulation campaign, a page dedicated to the campaign describes its objectives, resolutions and the individual experiments conducted. Links to the relevant published research papers are provided as well.
The primary access points for E3SM simulation data have been HPSS paths and ESGF nodes. Metadata on E3SM Data Docs always includes data size and the HPSS path where users can find the data. Depending on the model version and simulation campaign, additional metadata may include scripts to reproduce simulations, ESGF links and/or the HPSS URLs. ESGF links are available for download by the general public, as is data from the HPSS URLs. However, if you have an active account on the machine where data is stored on HPSS, downloading directly to that machine will be much faster.
Simulation materials for v1, v2, v2.1, v3.LR, SCREAMEv0 and SCREAMv1, as well as Artificial Intelligence (AI) Training Datasets are available. At present, over 4 PB of simulation data are archived. The team will soon be adding the v3 Large Ensemble (v3.LR.LE) datasets (~600 TB).
Data Availability Policy
Whereas E3SM Data Docs is a website to point users to data, the data itself is stored on HPSS, and sometimes on ESGF nodes. In the future, the project will focus on storing data on Department of Energy (DOE) HPSS systems and working with BER data management projects to identify the most efficient paths for making E3SM data available in the American Science Cloud.
Related Floating Points stories
- SCREAM and v2.1 data accessible
- E3SM v2.1 BGC Data Published and upcoming changes in native E3SM data availability
- Data availability: E3SM v3 LR DECK Plus and EAMv2,v3 AI Training Data
Contact
- Tony Bartoletti, Lawrence Livermore National Laboratory
- Ryan Forsyth, Lawrence Livermore National Laboratory
This article is a part of the E3SM “Floating Points” Newsletter, to read the full Newsletter check:
- E3SM Floating Points, May ’26: Looking Forward to the Summer All-Hands
