Scaling Method for Components of the Community Climate System Model (CCSM)

The left image is a global 1-degree Parallel Ocean Program (POP) grid that is subdivided into blocks. The right image is a space-filling-curve-based partitioning of the blocks onto eight processors. The partitioning based on space-filling curves allows better load balancing in POP and enables efficient execution of POP on 30,000 IBM Blue Gene processors. Increasing the scalability of the CCSM components will enable this key model to efficiently utilize the upcoming generation of supercomputers.

We examined the ability of the Blue Gene/L system to perform global eddy-resolving ocean modeling using the Parallel Ocean Program (POP). We used the POP 0.1-degree benchmark to examine the impact of several code modifications that improve scalability and simulation rate. We discovered that changes to the conjugate gradient solver and the use of space-filling curves to partition the computational domain across processors (see figure) significantly increase simulation rate. The code modifications increased the simulation rate from 4.0 to 7.9 simulated years per wall-clock day on 30,000 IBM Blue Gene processors.

Our discovery is significant because it demonstrates that it is possible to achieve simulation rates on the Blue Gene system sufficient to enable long-timeframe climate simulations. These results show that it may be possible to increase the scalability of the CCSM by two orders of magnitude. This increase in scalability would allow coupled climate simulations that are 100 times as computationally demanding to run on the upcoming NSF Petascale system. We are currently examining the scalability of each CCSM component at high resolution. A more detailed description of our efforts appears in Improving Scalability of CCSM Components.

This result arose from our work to advance NCAR's strategic priority of "Conducting research in computer science, applied mathematics, statistics, and numerical methods." It is supported through the NSF cooperative Grant NSF01, and through the Department of Energy CCPP program grant DE-FC03-97ER62402.