Author: Zachary Shrader
EPIC Team: Code Management Systems Engineer & Code Manager
Wall Clock:
The following diagrams show the wall clock times of the UFS Weather Model regression tests run by the Code Management team against prospective code changes in active GitHub pull requests (PRs). The blue represents regression tests that utilize Intel compilers to build the physics suites for the tests; conversely the purple stacked columns represent GNU compilers. Every machine uses some version of Intel, while only Hercules and Hera have a GNU compiler option. The plots showcase the aggregate results of runs over the previous year by the Code Management team and provide the opportunity to compare machines and compiler versions. Derecho and Hercules appear to be the fastest machines, in terms of wall time per test measured in seconds, with no test running for longer than 1,400 and 1,300 seconds, respectively. Hera holds the longest run times at a high mark of 2,400 seconds combined for both Intel and GNU versions of some of the coupled tests. This could be due to Hera running somewhat older compiler versions: intel/2021.5.1 and gnu/9.2.0 for Hera and intel/2021.9.0 and gnu/12.2.0 for Derecho. Jet is also quite slow, reaching 2,000 seconds for some coupled tests. This is likely due to Jet being a smaller machine with fewer resources and less computational power.
Here we can see the number of regression tests per compiler on each machine. Most of the regression tests are Intel-based. The operational requirement tests, represented in green, are run almost exclusively with GNU compilers. While GNU compilers are generally slower, as compared to Intel, they are also less tolerant in the detection of bugs. This makes them a good choice for smoke testing PR code changes.