EPIC's Impacts
The National Oceanic and Atmospheric Administration Earth Prediction Innovation Center (NOAA-EPIC) implemented an innovative container architecture that preserves high-performance computing workflows while strengthening long-term distribution sustainability. Developed through collaboration with NOAA EPIC, NOAA Geophysical Fluid Dynamics Laboratory (GFDL) container experts, and academic partners, the solution enables continued delivery of Unified Forecast System (UFS) applications through streamlining processes. This work now supports a community of more than 60,000 users across NOAA, academia, and industry by providing reliable, scalable access to numerical weather and Earth system modeling applications. Read more >
Boosting UFS Performance with Simple Changes
NOAA EPIC’s Global Workflow – Atmospheric Rivers (GWAR) Team, including Wei Huang, and NOAA NWS’s EMC Global-Workflow team
While adding container capability to the Global Workflow (GW), the Earth Prediction Innovation Center (EPIC) team identified a significant performance issue: When running the workflow using the container for high-resolution forecasts, performance on the relatively new National Oceanic and Atmospheric Administration (NOAA) Research and Development (R&D) machines Ursa and Gaea C6, was poor. This is often the case when applications are brought to new platforms, especially if the associated capabilities are novel. Part of the break-in period for any new system and technology is identifying the new parameters and environment settings that are necessary to achieve peak performance.
A deep dive into the system configuration soon revealed that the host machine’s network distribution was the primary bottleneck. By refining how the job scheduler distributes tasks for large parallel jobs, we achieved dramatic speedups in high-resolution configurations. This modification not only improved container based forecast performance, but also delivered substantial gains when running the workflow in the traditional non-containerized manner.
These findings were shared with the NOAA Environmental Modeling Center (EMC), who verified the performance gains at C1152. EPIC submitted a pull request to incorporate these modifications into EMC’s code and the change was quickly merged into the primary GW repository. This is the kind of collaboration that enables NOAA, and the community, to efficiently adapt to new technologies and computing environments.
This small collaboration illustrates EPIC’s mission: To be the catalyst for community research and modeling system advances that continually inform and accelerate advances in our nation’s operational forecast modeling systems.
The EPIC GWAR team has successfully containerized NOAA’s Global-Workflow (GW). Running the GW using Singularity container runtime solves the High Performance Computing (HPC) portability problem. EPIC confirmed that the containerized GW could be executed with only a job scheduler (Slurm) and the container image present on the host machine (Ursa, Gaea, or AWS). Read more >
EPIC successfully led the porting of the Unified Forecast System (UFS) Weather Model to NOAA’s new Ursa high-performance computing system, significantly boosting speed and scalability for advanced weather prediction. Ursa’s AMD Genoa-based architecture, GPU capabilities, and optimized integration with the Spack Stack enable faster runtimes and expanded AI/ML applications in weather modeling. This achievement, driven by collaboration between EPIC, UFS, and Spack Stack teams, strengthens NOAA’s ability to deliver more accurate forecasts for high-impact weather events. Read more >
Dr. Jacob Radford (GSL/CIRA) developed Google Colab notebooks that make AI weather models like FourcastNet, GraphCast, and Aurora accessible using free cloud tools. Built on ECMWF’s ai-models, the notebooks guide users through setup, execution, and evaluation in ~70 minutes. EPIC applauds this open-source work for advancing education, equity, and innovation in AI weather forecasting. Read more >
NOAA’s EPIC Program, supported by Raytheon, advances Earth system modeling through open-science collaboration, cloud computing, and automation. It has improved forecasting accuracy, accelerated research-to-operations, and expanded community engagement. Key achievements include a 700% increase in model releases and a 1,700% reduction in peer-review times. EPIC’s impact has earned industry recognition, including the prestigious John Cassidy Award. Read more >
EPIC’s Code Management team developed an automated Jenkins pipeline for standalone unit testing in NOAA’s PSL Stochastic Physics repository. The pipeline enables automated unit tests for key physics schemes such as Stochastic Kinetic Energy Backscatter (SKEB), Stochastically Perturbed Physics Tendencies (SPPT), Specific Humidity Perturbations (SHUM), and Stochastic Parameter Perturbation (SPP). Read more >
Early and accurate warnings from NOAA’s National Hurricane Center (NHC), are integral to protecting any community threatened by hurricanes and tropical storms. This is the purpose of NOAA’s newest hurricane model, the Hurricane Analysis and Forecast System (HAFS), which significantly improves the accuracy of forecasts and further supports community action in preparation of such severe weather events. Read more >
The NOAA Earth Prediction Innovation Center has introduced two new test cases for the UFS Weather Model: an idealized dry baroclinic wave case and a July 2020 Convective Available Potential Energy (CAPE) case, both in atmosphere-only configurations. These tests are part of a new developmental framework that allows users to evaluate model changes and supports hierarchical system development within the UFS. The tests are easy to run on Tier-1 platforms and containers, with detailed instructions available in the updated UFS WM User’s Guide. Additional information >
EPIC collaborated with NOAA NOS and NCAR to establish the UFS Coastal App, integrating key ocean, wave, and weather models to support coastal forecasting. Leveraging Unified Workflow Tools and CI/CD pipelines, this project streamlined development, allowing for efficient testing and faster integration of model components. Read more >
The UFS replay dataset was created by the NOAA Physical Sciences Laboratory (PSL) and EPIC’s Senior Data Scientist, Mariah Pope, to create a useful reanalysis dataset that could be leveraged as a training dataset for machine learning models. Mariah Pope was instrumental in curating the Zarr replay datasets on Google Cloud Platform (GCP) through her help in staging the 1-degree FV3 and MOM6 data. Read more >
Kris Booker from EPIC collaborated with Ben Cash from George Mason University to develop a proof of concept using Apptainer (formerly Singularity) to run the UFS weather model on academic HPC platforms. This approach overcomes technical barriers, allowing containers to run without administrative privileges. While still in the refinement phase, this innovation will simplify UFS deployment processes, making weather modeling research and development more accessible to the UFS community. Read more >
Fast-Track Your SRW App Experiment
Christopher R., a new SRW App user, faced challenges in setting up his experiment. Our support team guided him through relevant documentation and provided configuration suggestions. The result? His experiment is now successfully running.
“This was outstanding! The model is now numerically integrating forward in time on Cheyenne… I appreciate your help!”
– Christopher R.
GitHub Training at UIFCW 2023
Lack of GitHub knowledge can hinder even the best scientists from contributing to the UFS. At UIFCW 2023, our training equipped attendees—from students like Delton W. to NOAA experts like Songyou H.—to contribute code on GitHub, paving the way for diverse contributions to the UFS.
