zppy

zppy (pronounced “zip-ee”) is a post-processing toolchain for E3SM written in Python. The goal of zppy is to speed-up the post-processing of E3SM simulation output by automating commonly-performed tasks.

zppy is included in the E3SM Unified Environment and can be run with the simple command
zppy -c <configuration_file>. The configuration file specifies input and output directories, as well as the individual task configurations.

Supported tasks include generating climatology files and time series files (using ncclimo, NCO), E3SM Diagnostics plots, MPAS-Analysis plots, and global time series plots. Additionally, these tasks can include subtasks — for example, users can run both monthly and daily time series for both atmosphere and land.

To avoid unnecessary duplication, settings in zppy configuration files can be inherited and defined at the top level, task level, or subtask level. Users can then define parameters at the top level of the configuration file while providing more specific parameters (or overriding parameters) for individual tasks and subtasks. Sensible default values are provided for many parameters.

zppy launches batch jobs for each task, or if multiple year-sets are defined (e.g., 1-50, 50-100, etc.), then a single task may launch multiple jobs, one for each year-set. Each job comes with a batch-job file, log file, and status file. Users can check the current status by inspecting the status file (possible statuses include WAITING, RUNNING, OK, or ERROR). In case of errors, the batch-job and log files provide additional information. If the web server path is specified, zppy will move the plots onto the web server for viewing. zppy submits the batch jobs using SLURM and handles dependencies between tasks.