New zppy features: ILAMB Support, Extended E3SM Diags Capabilities and More
zppy (pronounced “zip-ee”), a post-processing toolchain for E3SM written in Python has had a number of improvements since we introduced it last year. It’s goal is to orchestrate the post-processing tasks for E3SM simulations and to speed them up by automating commonly-performed tasks. Recently released version 2.1.0 of zppy is included in E3SM Unified v1.7.0 environment.
Feature Highlights
zppy, which calls e3sm_diags (an E3SM diagnostic Python package: E3SM Diags) tasks, supports e3sm-diags’ new additional plots sets: cmip6 comparison (included in lat_lon set) (Fig 1.), annual_cycle_zonal_mean, streamflow_diags, zonal_mean_2d_stratosphere, Tropical Cyclone, aerosol_aeronet
Figure 1. One of the new features from E3SM Diags being included in zppy is a box whisker plot that compares Root Mean Square Error of selected fields between simulated data (black dots) and results from peer CMIP6 models, with E3SM v1 (blue triangles) and v2 (red triangles) results as baseline results.
Developed as part of the Coordinated Model Evaluation Capabilities CMEC, the International Land Model Benchmarks (ILAMB) System specializes in land model components. Collaborating with the RGMA funded SEATS project, the new ilamb task has been introduced (Fig.2). This added support facilitates ELM development and evaluation.
Figure 2. Example showing selected plots from ILAMB diagnostics that can be now performed through zppy task. Figure 2(a) shows temporarily integrated mean of Leaf Area Index (LAI) global distribution compared to the benchmark dataset: MODIS, and figure 2(b) shows the relationship between LAI and precipitation compare simulated and benchmark data: MODIS and GPCPv2.3.
zppy’s jobs can now be bundled together, which is particularly useful on machines which limit the number of concurrent job submissions (for example NERSC).
API Changes
- Improved output directory structure for diurnal climatology
- A directory atm/180x360_aave/clim_atm_monthly_diurnal_8xdaily_180x360aave will now be the much shorter atm/180x360_aave/clim_diurnal_8xdaily. This has two consequences for the API: (1) to generate diurnal climatology, users should now set frequency to diurnal_8xdaily rather than monthly_diurnal_8xdaily and (2) when running E3SM Diags with diurnal cycle diagnostics, climo_diurnal_frequency = "diurnal_8xdaily" should be specified.
- The e3sm_unified option has been removed. zppy will default to running the latest version of E3SM Unified unless specified otherwise with environment_commands.
- Tasks now have active = False by default, meaning users must explicitly set active = True for a task to run.
Details on New Features
- ts task: extra variables can be specified in the ts task using the extra_vars parameter.
- e3sm_diags task:
- Support for additional E3SM Diags sets — annual_cycle_zonal_mean, streamflow_diags, zonal_mean_2d_stratosphere<.
- Tropical Cyclone diagnostics are also included. This involves a new task called tc_analysis. Note that if multiple year sets are specified when running Tropical Cyclone analysis, the tc_analysis tasks will have to run sequentially rather than in parallel; the same is true for e3sm_diags when generating the Tropical Cyclone diagnostics.
- Model-vs-model functionality is enabled.
- global_time_series task: there is an option now to specify the input_subdir and to run without data from MPAS-Analysis (by setting atmosphere_only = True).
- The ilamb task has been introduced.
- Other features
- Jobs can now be bundled, which is particularly useful on machines which limit the number of concurrent job submissions.
- The last_year command-line option can be used to specify the year to post-process up to, even if later years are specified in the configuration file.
- environment_commands is now inferred based on what machine zppy is being run on. Users can still set this parameter in their configuration files.
- Campaign configurations exist for water_cycle, cryosphere, and high_res_v1.
- Settings for each parameter are printed to a file, so users will know what all their parameters were set as.
Resources
- Documentation: Zppy documentaion – zppy documentation
- GitHub repository: GitHub – E3SM-Project/zppy: E3SM post-processing toolchain
- Installation: Getting started – zppy documentation
- Tutorial: Tutorial – zppy documentation
- Detailed description in an E3SM news article: Introducing zppy
- See also: Running and Analyzing E3SM: Water Cycle Group’s Approach”, by Ryan Forsyth and Chris Golaz (PDF, YouTube video)
