Capability Announcement: UFS-DA Diagnostics Toolkit v1.0
X (formerly Twitter) Instagram Vimeo Youtube Envelope

Capability Announcement: UFS-DA Diagnostics Toolkit v1.0

The Earth Prediction Innovation Center (EPIC) and the Unified Forecast System (UFS) community are proud to announce the public release of the UFS-Data Assimilation (DA) Diagnostics Toolkit v1.0, a new Python-based diagnostics framework for comprehensive analysis of UFS / Joint Effort for Data assimilation Integration (JEDI) DA experiments. The toolkit has been incorporated into the Consortium for Advanced Data Assimilation Research and Education (CADRE) 2026 Data Assimilation Training and Science Workshop, providing participants with a unified environment for evaluating analysis increments, observation-space behavior, error consistency, and scale-dependent characteristics of hybrid 3D-Var experiments. 

Developed through NOAA EPIC, in collaboration with CADRE, the UFS-DA Diagnostics Toolkit supports both training and research applications by providing a consistent set of diagnostics to understand how observations influence analyses and how DA configuration choices affect system performance. Key capabilities are listed below. 

Key Capabilities 

  • Model-Space Diagnostics: Evaluate analysis increments and background fields through global maps, zonal means, and vertical cross sections. These diagnostics help users understand where observations influence the atmospheric analysis and how analysis corrections are distributed horizontally and vertically throughout the model domain. 
  • Innovation-Space Diagnostics: Evaluate DA performance using observation-minus-background (O−B) and observation-minus-analysis (O−A) statistics, bias metrics, and observation-fit diagnostics. These tools provide direct insight into how effectively observations are utilized and help identify systematic behaviors across instruments and channels.
  • Error Consistency and Observation-Error Diagnostics: Estimate observation-error variance and background-error contributions using Desroziers innovation-space diagnostics (Desroziers et al, 2005) derived from O-B and O-A statistics. These capabilities support observation-error evaluation and tuning by providing quantitative measures of whether assumed error specifications are statistically consistent with DA performance.
  • Spectral Diagnostics: Characterize the scale-dependent behavior of background fields and analysis increments using power spectra, variance profiles, and spectral-ratio analyses. These diagnostics reveal how changes in background-error or observation-error configurations redistribute analysis variance across spatial scales and model levels, providing insight that is often difficult to obtain from traditional increment plots alone.
  • DA Consistency Monitoring: Perform automated Chi-square (𝝌²) and Jo/p consistency assessments using information extracted from JEDI outputs and log files. These diagnostics provide a high-level evaluation of the statistical balance among observations, background errors, and analysis increments, helping users identify potential issues in DA configuration and tuning.

 

Application for CADRE 

The UFS-DA Diagnostics Toolkit served as the primary evaluation framework during the EPIC sessions of the CADRE 2026 DA Training and Science Workshop. Participants used the toolkit to analyze Finite-Volume Cubed-Sphere (FV3)-JEDI hybrid Three-Dimensional Variational (3D-Var) DA algorithms through model-space diagnostics, innovation-space diagnostics, error-consistency metrics, and spectral analyses.

Figure 1. Surface temperature analysis increments at the lowest model level (Level 126, 998 hPa) from the CADRE 2026 FV3-JEDI hybrid 3D-Var experiments.
Figure 1. Surface temperature analysis increments at the lowest model level (Level 126, 998 hPa) from the CADRE 2026 FV3-JEDI hybrid 3D-Var experiments. Panel (a) shows the control experiment, panel (b) shows a sensitivity experiment with increased static covariance weighting in the hybrid background-error formulation, and panel (c) shows the corresponding increment difference. The increased static contribution yields larger, more spatially coherent temperature increments across several regions, illustrating the sensitivity of the analysis to the specification of background-error covariance.
Figure 2. Spectral diagnostics of temperature increments at Level 75 (505 hPa) comparing the control and hybrid-weight experiments.
Figure 2. Spectral diagnostics of temperature increments at Level 75 (505 hPa) comparing the control and hybrid-weight experiments. The (a) one-dimensional spectra, (b) variance-ratio profile, and (c) two-dimensional spectral-ratio analysis quantify how increasing the static covariance contribution modifies the distribution of analysis variance across spatial scales and vertical levels.

By providing a unified set of diagnostics across three workshop sessions, the toolkit enabled participants to connect changes in DA configurations to their impacts on analysis structure, observation fit, statistical consistency, and scale-dependent atmospheric variability.

A new user’s guide is available here to provide further information about the UFS-DA Diagnostics Toolkit. Also publicly available is the CADRE training repository, available here. Interested users can get support by submitting a question to support.epic@noaa.gov. 

Contributors: 

The inaugural UFS-DA Diagnostics Toolkit release is developed by the Earth Prediction Innovation Center (EPIC). The code is hosted on GitHub at ufs_da_diagnostics. Publicly available data is provided via the ufs-da-diagnostics AWS S3 bucket established as part of the NOAA Open Data Dissemination (NODD) Program.  

Acknowledgments: 

This release was funded by the NOAA Weather Program Office’s Earth Prediction Innovation Center.