At NSF NCAR, the local node of a global hackathon tackles km-scale challenges

The Digital Earths Hackathon, a first-of-its-kind “global pan-hackathon” organized by the World Climate Research Programme (WCRP), recently brought together over 650 participants across 10 sites worldwide. NSF NCAR hosted one of these hackathon nodes at its Foothills Lab in Boulder, Colorado, drawing about 50 enthusiastic and ambitious participants. 

The fully in-person, semi-synchronous, worldwide series of events took place from Monday, May 12 to Friday, May 16, 2025. The global hackathon’s primary aim was to address challenges and develop best practices for analyzing data from kilometer-scale or storm-resolving scale climate models, which produce vast amounts of complex data.

One of the NSF NCAR node’s organizers was John Clyne of the Computational and Information Systems Lab (CISL). "There's a big change slowly coming about in the climate-modeling or weather-modeling space: a move to kilometer-scale models, also called storm-resolving scale models," Clyne explained. The hackathon’s primary goal was to gain experience and establish efficient methods for analyzing the datasets, which are not only large, but also use "unstructured grids," adding another layer of complexity.

Participants worked in various breakout groups, delving into different topical areas using model outputs. They achieved significant scientific progress, generating numerous plots and Jupyter notebooks, and facilitating intercomparison between data from various models. Noted Clyne:

"Participants did a lot of science, they got a lot of experience, and they seemed to be teasing out things from these model outputs that they hadn't been able to do before." 

Some groups are aiming to publish papers based on their work, said Clyne, but for most, “the aim was really just to figure out what the best practices are. We were trying some new things, getting a handle on how to analyze these kilometer-scale model outputs in a way that's effective and efficient." 

The hackathon was conducted in-person at each site, with communication platforms enabling virtual collaboration among participants worldwide. Daily sync-ups, for everyone whose time zones permitted, allowed different nodes to share progress and exchange ideas. The event spanned a week, including three full days and two half-days, and featured invited speakers knowledgeable about the models used.

One of the key successes of the hackathon was the rapid productivity of the participants. Clyne expressed both surprise and satisfaction with the hackathon’s outcome, recalling:

"At the end of the first half-day, every single group had produced a number of plots that were meaningful. I was really shocked that things worked as well as they did."

The resulting science summary from each breakout group covers a wide range of topics. Some groups focused on cloud microphysics, examining the precipitation frequency predicted by global kilometer-scale models. Others worked on extreme weather events, analyzing extreme precipitation, using machine learning to predict extremes, and calculating the Hot-Dry-Windy Index. Another group compared models against GPM precipitation, validating model precipitation against satellite observations. General circulation and upscale effects, land-atmosphere interactions, and hydrology applications were also explored. Groups investigated mesoscale circulations like land-sea breezes, storm tracking, and the development of atmospheric convection.

“One of the key things is that we made available datasets for many different models; it wasn’t just one,” Clyne noted. For example, NSF NCAR contributed data from the Model for Prediction Across Scales (MPAS) and the Community Earth System Model (CESM), while the German Climate Computing Center (DKRZ) contributed data from the ICOsahedral Non-hydrostatic modeling framework (ICON) model.

Overall, the hackathon provided a uniquely broad reach for scientists worldwide to collaborate, share, and advance kilometer-scale model analysis—a field rapidly growing in importance. Said Clyne: “It was an opportunity for participants to connect with colleagues all across the world, and that’s not something that’s readily doable.”

The hackathon participants on site at NSF NCAR used the Casper cluster and CISL’s JupyterHub to conduct their analyses. NSF NCAR makes Classroom allocations available to instructors at U.S. two- or four-year academic institutions and their students in courses or training workshops. For researchers interested in analyzing Earth system science data from NSF NCAR-hosted data services, NSF NCAR makes Data Analysis allocations available to U.S. researchers. For more information, see NSF NCAR Allocations, or submit requests at Allocations Opportunities.