Application Training

Contributing Code to the UFS

Outcomes and Value:

The EPIC Team put together an informative presentation on committing code to the Unified Forecast System (UFS) repositories. Twenty-two users signed up for the App Training. The event was designed to increase the community's knowledge on the UFS code contribution process. It included information on basic Linux and Git skills that are needed to add ideas into UFS code via pull request. The slide presentation shows how a user can start with very limited GitHub knowledge and leave with working examples of how to contribute code, which will be helpful to users in their development work. The presentation slides can be viewed here: Presentation Slides.

Short-Range Weather Application Version 2.2.0

Outcomes and Value:

We incorporated a significant amount of feedback from past events into the planning for this event to ensure that users felt empowered to recreate this work on their own. However, this Application Training ran into some technical hurdles with external dependencies running the SRW App on a Mac. The XCode program no longer allows the ability to push older versions of XCode tools onto a Mac, and the current version of XCode has some issues building the SRW App. HPC-Stack and miniconda built successfully with no issues, but when building the SRW App on a Mac outside of a container, newer versions of XCode tools were causing failures that would take some time to troubleshoot. The team rescheduled the event to early January because of these technical hurdles and worked through them, but a Mac update in early January caused additional issues building the SRW App. Going forward, we plan to mitigate further issues with low-tier systems by creating cloud virtual machines of Mac and PC systems where we run the SRW App regularly to make sure that it is working as expected. We could show users how to run HPC-Stack on a Mac, but we should redo this event after we get XCode fixed and build the low-tier virtual machines to regularly test Mac and PC systems.

Land Data Assimilation (DA) System

Outcomes and Value:

We incorporated a significant amount of feedback from this event to ensure that users felt empowered to recreate this on their own. There are working codes to run LandDA, infrastructure as code, and visualization notebooks to offer a complete experience. Students requested that the tutorials accompanying this video series be completed within a week so they can replicate it independently. Overall, I believe this format worked well, and we will incorporate any future feedback. There were 26 sign-ups for this event, with 16 participants attending.

Short-Range Weather Application Training: Running v2.1.0 Containers in AWS

Outcomes and Value:

The event hosted by our team marked our highest turnout with 20 participants and 38 registered members. Typically, we have observed that around half of the community members who sign up actually attend. The event focused on walking through two distinct methods for running SRW 2.1 on bare metal using containers and we covered the key requirements for building out these cloud instances. During the training we also addressed the inquiries raised by the attendees regarding replicating the work they saw beyond the event. As a team our multi-cloud packer scripts coupled with the excellent presentation by the AUS team enabled users to recreate the procedures they witnessed with ease. To access the presentation slides after the event, please visit: https://epic-sandbox-content.s3.amazonaws.com/EPIC+Seminar_Running_SRW_Singularity_Container_30Mar2023.pptx.pdf

Short-Range Weather Application Training: Porting over SRW/MRW tools to the Unified Workflow Toolkit

Outcomes and Value:

This Challenge was a way for the community to start working with UFS repositories from a modern take. The Unified Workflow repository is 100% python, and has unit testing, code standards, and code re-use built in. So, this was the first training that taught users how to checkout repositories, make contributions and then submit their peer-reviews for analysis. The unified workflow repository will be a great way for the broader community to learn about the UFS applications by starting with a repository that is more easily understood. This will drive users to let us know what other tools they would find beneficial, and be a  gateway to start understanding the more research intensive application repositories.

New ideas: It was brought up that industry uses more modern tools such as Jupyter notebooks, and more modern libraries. We should work with operations at NOAA to have a process to more easily scan libraries that the community would like to use so that we can help modern ideas continue to drive innovation, and shape legacy processes that need to modernize.

Outcomes and Value

The CodeSprint was aligned very well around teaching new community members how to run SRW locally on a MacBook.  You could show up to the event with a blank canvas, and leave with evertyhing that was needed to run a forecast. It is a great way to look at cost savings and be able to run the model locally instead of incurring large HPC costs. Ligia, who co-chairs the Release Coordination Cross-Cutting Team, was also in attendance.

Why I think this was a powerful event:

1. It is a challenge to setup this work to run SRW from nothing. When you log onto Tier-1 machines, HPC-Stack (Spack-stack), tools, packages are all available to you already, so setting those up yourself takes time and it very meticulous, and this event walked users through how to do this. 
2. Being able to run SRW locally for college students will save them a lot of money. Running SRW in the cloud can add up costs, but being able to run it on local hardware is an important step in allowing more members to have access.

Outcomes and Value

The code sprint participants emerged with both the ability to run the SRW App in the cloud and an understanding of how they can use this knowledge to assist the UFS community by building the infrastructure required to quickly test research. Three code sprint postdoc participants partook in a session on building the AWS cloud infrastructure for the SRW App. The event included a slide deck presentation on the code architecture followed by a hands-on process where each user built the SRW App in their own environment. Participants provided helpful feedback as UFS SRW App users and community members. They would like expanded documentation on how to utilize their own test case data and model configurations, and they would like more transparency about upcoming releases and what features—particularly physics implementations—will be included in releases. They would like to see documentation and roadmaps expand in these areas.

New ideas: Participants would like to see a code sprint like this with all CSPs and would like another event that walks through running and configuring the SRW application after the infrastructure is built out. They would like more information on how can they run local research in the cloud and stage appropriate data for their experiments. 

Outcomes and Value

The code sprint participants developed their knowledge of the SRW App, the infrastructure suite used to support it, and ways that they can assist the UFS community using this knowledge to quickly test research. Seven code sprint participants partook in a session on building the AWS cloud infrastructure for the SRW App and ran the SRW App after building the infrastructure to support the application.

New Ideas: The users communicated that they would like to see a follow-up event that walks through building out the infrastructure in AWS without the first portion of the event included. Rather than providing them with IAM, S3, and other baseline configurations, participants would set up these elements themselves.