NCO Supports New Regridding and Timeseries Features
The netCDF Operators (NCO) toolkit allows researchers to manipulate and analyze data stored in netCDF-accessible formats, including DAP, HDF4, and HDF5. The latest version 4.9.1, “Skyglow”, contains powerful new features that accelerate post-processing of E3SM data. Foremost is the new weight-generation capability that can be used to regrid any finite-volume (FV) datasets including those from MPAS and the new “physics grids” targeted by EAM v2 and SCREAM. This new first-order conservative regridding algorithm scales well up to four OpenMP threads and so generates such weights on a laptop in under an hour—considered fast for SCREAM’s ~3 km grid with over 25 million gridcells.
A climatology of Antarctic sea-ice concentration (the fraction of sea ice in a model grid cell) from the MPAS-Seaice model averaged over June, July and August of the final ten years of an E3SM simulation at medium resolution (10 km around Antarctic) compared with NASA satellite observations. NCO was used to compute the climatology (the 10-year average) and then to remap the sea-ice data to a common grid with the observations so the bias (right panel) could be computed.
The latest version of NCO greatly simplifies the regridding of datasets that contain grid cells where surface area is shared by multiple model components, resulting in fractional areas per grid cell. For example, coastlines run through gridcells that are always shared by both the land (ELM) and ocean (MPAS-Ocean) models. In an even more complicated situation, sea ice forms and disappears through time, so the MPAS-Seaice model borrows and returns surface area from MPAS-Ocean as it does so. NCO treats the issues of fractional gridcells with a feature called Sub-GridScale (SGS) regridding. Now users can supply just the name of the variable that contains the sub-gridscale fraction, and choose whether the regridder should preserve the sub-gridcell (and global) integral or mean of the fields. It is often useful to conserve the integral of flux variables (like energy and mass fluxes), and the mean of state variables (like temperature and mixing ratios). The documentation for ncremap has all the details for regridding features.
Annual time series of (a) global mean net energy flux at the top-of-atmosphere and (b) global mean time series of 2-m air temperature for the 500 years of the E3SMv1 pre-industrial control (piControl) simulation. The global mean time series data was computed with the latest version of NCO.
Time (wall-clock minutes) to generate the global mean 500-year time series data shown above from native model output files (*cam.h0*.nc). The ‘old approach’ consisted of a custom shell script. The latest version of NCO can now accomplish the same operation substantially faster in a single command using the new --glb_avg option. The timing data was derived from two realizations each on a single Cori-KNL compute node. NCO parallelizes the work which explains part of the performance gain.
E3SM development has continued apace since the delivery of v1 datasets to the CMIP6 archive. Useful “quick-look” metrics for developers include the global mean timeseries of key indicators such as top-of-atmosphere fluxes and surface temperatures. Users can now construct such global-mean timeseries with the --glb_avg option to the ncclimo operator as explained in the “ncclimo netCDF Climatology Generator” section of the NCO documentation. This option area-weights the timeseries of all the specified variables and outputs them in CMIP6-like format, a parallel procedure that replaces the old serial methods. All of these features are available now in the NCO Conda package and will be included in the E3SM-Unified “super-package” after its next update.
NCO Links
- NCO
- Users Guide: HTML, PDF
- Code Repository: http://github.com/nco/nco
- Code Citation: DOI: 10.5281/zenodo.595745
Additional Resources
Contact: Charles (Charlie) Zender, University of California, Irvine
