Collaboration Through a Common Infrastructure and Best Engineering Practices Expedites UFS Coastal App Development

Coastal communities need answers to critical questions about future water levels, flooding, and water quality on weather to climate scales. Making sound decisions requires easy access to reliable and timely coastal predictions. NOAA’s National Ocean Service (NOS) is uniquely positioned to lead the development of such a coastal and ocean prediction capability and to effectively deliver the range of applications and services needed to meet the threats facing impacted coastal communities along the U.S. coasts and the Great Lakes leveraging the NOAA Unified Forecast System (NOAA NOS Modeling Strategy, 2024).

Starting from December 2022, through the Bipartisan Infrastructure Act funds, the NOS, Earth Prediction Innovation Center (EPIC), and National Center for Atmospheric Research (NCAR) started upgrading the CoastalApp (now deprecated) infrastructure maintained by NOS to establish UFS Coastal Applications. This project leverages the Earth System Modeling Framework (ESMF) and the National Unified Operational Prediction Capability (NUOPC) layer to advance the coupling infrastructure for the UFS Coastal Applications. 

Leveraging the UFS Weather Model (WM) maintained and managed by EPIC and Environmental Modeling Center (EMC), a UFS Coastal Model has been established by forking UFS WM and adding model components such as ADCIRC ocean circulation model, Semi-implicit Cross-scale Hydroscience Integrated System Model (SCHISM), Finite Volume Community Ocean Model (FVCOM), and Regional Ocean Modeling System (ROMS), along with contributions to WaveWatchIII (WW3) and Consortium sea-ice model (CICE) infrastructures, to support coastal applications. UFS Coastal leverages the emerging standard for data ingest based on the Community Data Models for Earth Prediction Systems (CDEPS) to exercise the National Unified Operational Prediction Capability (NUOPC) caps in isolation and support hierarchical testing and model development approaches, and uses Community Mediator for Earth Prediction Systems (CMEPS) to provide more fine tuned interaction among the model components. Most of theUFS coastal ocean model components were also modified to be compatible with the CMEPS mediator.

The project motivated the future implementation of a newly developed generic NUOPC Driver (ESMX) that will benefit other modeling systems based on NUOPC infrastructure such as NCAR’s CESM, NAVY modeling systems, and the UFS applications. Additionally, the availability of a generic NUOPC Driver will facilitate and promote decentralized testing of component models (e.g., ocean, ice, wave, etc.) in isolated contexts using Continuous Integration (CI) toolchains, as a prerequisite to setting up the fully dynamic coupling. 

Leveraging the UFS Unified Workflow Tools (UW-tools) co-developed by Global Systems Laboratory, EPIC and other community members, a UFS Coastal App has been established in a short time frame of less than one year. The current development prototype of the App is the first UFS application to rely on UW-tools for coupled model configurations, and currently allows users to prepare, build, and run a SCHISM configuration coupled with data atmospheric forcing (DATM). The teams aim to implement similar support for WW3 configurations in future iterations. Collaboration between the UFS Coastal and the Unified WorkflowTeams helped identify future development priorities in both teams and promoted a co-development paradigm. 

Through this effort, EPIC partnered with NOS to provide technical recommendations on increasing development speed and using open-source technical tools to replace traditionally manual processes. EPIC had previously leveraged the use of Continuous Integration and Continuous Delivery (CI/CD) pipelines through Jenkins with other UFS applications. EPIC worked with the NOS and UFS Coastal team to share lessons learned and set up a basic pipeline.  This included supporting tools such as SonarQube (https://sonarqube.epic.oarcloud.noaa.gov) to automate the process of building and testing, as well as static code analysis to validate research applications before moving into operations. The first iteration of this UFS Coastal CI/CD pipeline enables the technical team to automate regression testing of the code base on selected platforms, and greatly improves the efficiency of development given the rapid evolution of UFS code-bases. 

The successful establishment of the UFS Coastal App leveraging UFS infrastructure in coupling, workflow, and testing framework in such a short time frame highlights the benefit of continuously enhancing a unified system architecture and infrastructure in a unified modeling framework for weather and earth system modeling development.

As more UFS applications will be coming online in the coming years, this partnership also allowed EPIC to better understand the state of additional applications which will help expedite the process of onboarding the future applications into the community. 

Some lessons learned that came out of this effort include:

1. Early communication and technical talks helped identify areas that will need focus before integration, 
2. Significant support from ESMF, NOS, EPIC, and UFS WM teams was crucial to lay out a minimum viable product (MVP) plan and get an application into a working state in a CI/CD pipeline, 
3. A final plan for follow on work and collaboration after the MVP is complete will allow for future efficiencies. 

The UFS Coastal project has benefited from a large community of contributors and collaborators. The partner institutions include:

• NOAA National Ocean Service, Office of Coast Survey (OCS)
• Earth Prediction Innovation Center, Weather Program Office, NOAA Oceanic and Atmospheric Research (NOAA/OAR/WPO/EPIC)
• Center for Operational Oceanographic Products and Services (CO-OPS) 
• National Center for Atmospheric Research (NCAR)
• University Corporation for Atmospheric Research (UCAR)
• NOAA  Environmental Modeling Center (NOAA-EMC)
• University of Notre Dame (UND)
• Virginia Institute of Marine Science, College of William & Mary, USA (VIMS)
• Helmholtz-Zentrum Hereon, Germany (Hereon)
• U.S. Army Engineer Research and Development Center (ERDC) 
• University of Massachusetts – Dartmouth (UMASS-Dartmouth)
• Rutgers, The State University of New Jersey (Rutgers)
• Oregon State University (OSU)
• Cooperative Institute for Research In Environmental Sciences (CIRES )
• Global Systems Laboratory (GSL)

Below is a non-exhaustive list of team members who have actively contributed code, but many others have been involved in the UFS Coastal development:

• NOAA NOS Team: 
  • Saeed Moghimi, OCS, NOS Earth System Model Coupling Lead
  • Jana Haddad, OCS, Technical project manager
  • Yunfang Sun, OCS, Scientist / Developer
  • Mansur Jisan, CO-OPS, Scientist / Developer
  • Panagiotis Velissariou, NHC- formerly at OCS, Scientist / Developer
• NCAR ESMF Team: 
  • Ufuk Turuncoglu,, Software Eng. / Lead UFS-Coastal Developer
  • Daniel Rosen, Software Eng. / Developer
  • Ann Tsay, ESMF Team Manager
• SCHISM: 
  • Y. Joseph Zhang, VIMS, Scientist  / Lead developer
  • Carsten Lemmen, Helmholtz-Zentrum Hereon, Scientist / Developer
• WW3: 
  • Saeideh Banihashemi, NOAA/EMC, Scientist /developer 
  • Ali Salimi, NOAA/EMC, Scientist /developer 
  • Ali Abdolali,ERDC, Scientist /developer 
  • Denise Worthen, NOAA/EMC, Scientist /developer 
  • Jessica Meixner, NOAA/EMC,Scientist /developer 
• ROMS: 
  • John Wilkin, Rutgers, Scientist /Lead developer
  • Hernan G. Arango, Rutgers, Scientist /developer 
• ADCIRC: 
  • Joannes Westerink ,UND, Scientist /Lead developer
  • Damrongsak Wirasaet ,UND, Scientist /developer
• FVCOM: 
  • Changsheng Chen, UMass-D, Scientist /Lead developer
  • Jianhua Qi, UMass-D, Scientist /developer
  • Siqi Li, UMass-D, Scientist /developer
• EPIC Contract team:
  • Keven Blackman, Raytheon/EPIC, Chief Engineer
  • Jong Kim, STC/EPIC, Product Owner
  • Kristopher Booker, Tomorrow.io/EPIC, Product Owner
  • Christopher Domanti, Raytheon/EPIC, Program Manager
  • Anna Kimball, Raytheon/EPIC, Scrum Master
  • Fredrick Gabelmann, Element 84/EPIC, Scrum Master
  • Brian Weir, Ratheon/EPIC, Software Engineer
• UFS UW Team:
  • Christina Holt, CIRES and NOAA GSL, Research Scientist
  • Paul Madden, CIRES and NOAA GSL, Senior Engineer
  • Emily Carpenter, CIRES and NOAA GSL, Software Engineer
  • Naureen Bharwani, CIRES and NOAA GSL, Software Engineer