Scalable Numerical Methods for Partial Differential Equations

This chart compares iterations of a representative algorithm (GMRES) for various spatial resolutions (grid densities) for a typical problem in atmospheric physics. The three approaches are all optimal (flat lines), except that the time to solution for the currently used Block Jacobi method (blue line) increases somewhat with the number of computations being made, and the optimized Schwarz method (red line) requires significantly fewer iterations for the same computational cost. This test of one aspect of our method shows that a model can run to completion significantly faster using the same number of processors, and that this time savings even improves slightly as the resolution of the model increases.

To reach scientifically significant integration rates, implicit or semi-implicit time integration is required. This approach invariably leads to a large sparse matrix, representing an elliptic operator, needing to be inverted at each time step using iterative methods. Traditionally, because of the numerical techniques in use, especially for climate modeling, this problem was trivially invertible. Now, with efforts made toward using the latest available numerical methods, finding a solution to the linear system is a non-trivial challenge. Iterative methods converge quickly only if good preconditioning is available. One approach uses recently developed techniques like the optimized Schwarz method or algebraic multigrid.

Solvers are everywhere in almost every numerical model at NCAR. Improving the preconditioning technology yields immediate savings in direct computer time and helps scientists reach the solution to their problems faster. The larger university community using these models also benefits from these improvements.

In FY 2006, two next-generation models, HOMME and GASpAR, went into use as testbeds for the new preconditioners. Both of these models are in use by the larger community. The optimized Schwarz algorithm was implemented into HOMME with interesting results. It was discovered that, for an Eulerian semi-implicit dynamical core, a coarse solver was not required when optimized Schwarz (O2) was employed. This is a consequence of the advective CFL restriction that lowers the stiffness of the matrix as the spatial resolution is increased. This is quite a surprise, and it will enable almost perfect scaling on very large numbers of CPUs. This is th e kind of paradigm shift in algorithms required to create petascale models. In FY 2007 we will begin implementing optimized Schwarz in the GASpAR code for very large-scale turbulence simulations and into the ocean model POP.

This project supports NCAR's strategic priorities of "Conducting research in computer science, applied mathematics, statistics, and numerical methods," "Developing community models," and "Improving prediction of weather, climate, and other atmospheric phenomena." It is made possible through NSF Core funding.