The National Oceanic and Atmospheric Administration Earth Prediction Innovation Center (NOAA-EPIC)’s mission is to advance Earth system modeling by lowering barriers to innovation, expanding access to high-performance computing (HPC), and enabling consistent, reproducible model development across platforms. A core component of this effort is providing containerized software environments that allow researchers and operational users to focus on scientific development rather than system configuration.
EPIC combines compilers, libraries, and applications into containers, a software package that bundles an application together with all the libraries and dependencies it needs so it can run consistently across different computing environments without worrying about underlying system differences, enabling the support for large-scale numerical weather prediction and Earth System Models (ESMs). These environments enable simulations to run consistently across diverse supercomputing systems, fostering collaboration across NOAA, academia, and industry.
As EPIC’s container distribution processes matured, the team identified an opportunity to further modernize how modeling environments are built and shared with the Unified Forecast System (UFS) community. This refinement focused on strengthening portability, sustainability, and long-term distribution flexibility for UFS applications.
The EPIC team collaborated with experts at NOAA’s Geophysical Fluid Dynamics Laboratory (GFDL). Drawing on prior experience, the team implemented a two-stage container architecture led by NOAA-EPIC’s Edward Snyder, resulting in a lighter-weight runtime container, improving start-up time and preserving performance. This approach supports EPIC’s and NOAA’s larger goal of streamlining processes for evaluating and integrating contributions to the UFS that help improve weather prediction.
Additional enhancements allow development workflows to leverage host-installed compilers or a user-created sandbox derived from the Intel HPC Toolkit Docker image. Through extensive testing and debugging of the initial solution with academic partner George Mason University, the two-stage container is functioning successfully on a non-NOAA system.
Validated through UFS Weather Model (UFS-WM) regression testing, the new architecture ensures uninterrupted distribution of containerized UFS applications while supporting a growing community of more than 60,000 users.
For detailed, step-by-step instructions on using the externalized spack-stack container, please refer the publicly available container scripts guide.
