Message from CISL Director Al Kellie
I am proud to present the FY2007 Annual Report of NCAR's Computational and Information Systems Laboratory (CISL). As you will find in this report, CISL has completed another productive year and is well prepared to continue its research and service mission to the atmospheric and related sciences. Our plans for the future are organized by these five broad initiatives:
- Replace NCAR's computing facilities to advance the leading edge of geosciences simulation
- Procure, secure, and deploy robust and highly capable cyberinfrastructure that can support such advances
- Strengthen and extend our research arm that enables geoscientists to accomplish more
- Develop and improve tools and research environments that extend the reach of our communities of researchers
- Ensure that our work remains meaningful to others in academia, government, industry, and the general public
Specifically, our top priorities for FY2008 include:
- Pursuing the expansion of the geosciences computing facility with our partners in Wyoming
- Increasing our support of science with Breakthrough Computational Campaigns and TeraGrid-enabled applications
- Stimulating rising talent in the computer sciences, applied mathematics, and statistics communities with research challenges in geosciences simulation
- Releasing and improving powerful new knowledge-based tools for production use by our constituents
- Increasing the relevance of our work to science communities and society
The following overview points to highlights of CISL's 2007 report via links from the images.
Ready to break ground for the NCAR Supercomputing Center:
CISL, with the support of the geosciences modeling community, various advisory bodies, and the NSF, has established a partnership with the University of Wyoming, the State of Wyoming, and Wyoming businesses to develop a modern, energy-efficient NCAR Supercomputing Center (NSC) in Cheyenne, Wyoming.
This vision for the NSC is well aligned with the NSF strategic plan and the NSF vision for cyberinfrastructure. For example, CISL has been prototyping future NSC operations through our TeraGrid integration work. The NSC project is driven by science and is being proposed in direct response to the exploding demand for both capability and capacity HPC resources needed by Earth system science researchers. Regional climate simulations of the future with resolutions approaching 10-20 kilometers will require scaling up to petascale resources.
Breakthrough Science initiative:
NCAR and NSF created a new level of supercomputing resource allocation early in FY2007. Before NCAR's newest and most capable supercomputer was released for production use, NSF program managers in OCE and EAR, and the CISL HPC Advisory Panel (CHAP) invited a small number of researchers with successful records for using large amounts of processor hours to run very large simulations. Because of its potential for discoveries through simulation, this initiative is named Breakthrough Science (BTS).
Six of the eight projects successfully used their large allocations, consuming almost 3 million processor hours on blueice. At end-FY2007, two of the six projects had submitted papers for publication, and three others have multiple papers in progress. Because of the BTS successes, the NCAR Executive Committee and CHAP both decided to continue this practice of allocating large amounts of computer time to a single project at a time.
This is significant because it shows that CISL can provide the necessary resources and that university researchers are ready and able to effectively use very large allocations for their geosciences research.
TeraGrid computing resources enter production phase:
NCAR's IBM Blue Gene/L (BG/L) supercomputer, named frost, became an operational TeraGrid resource on August 1, 2007, and it is expected to provide 4.5 million CPU hours annually to the TeraGrid research community. In addition to the computational resources, NCAR is also testing experimental systems and services on the TeraGrid. These include the wide-area versions of parallel file systems from IBM and Cluster File Systems, as well as a remote data visualization capability based on the VAPOR tool, an open source application developed by NCAR, the University of California at Davis, and Ohio State University under the sponsorship of the National Science Foundation.
The NCAR TeraGrid integration effort in FY2007 shifted from the equipment acquisition and deployment phase that dominated FY2006 to a new phase characterized by security-hardening TeraGrid components, CTSS software deployment, testing, and migration, and integration of accounting software. During this period, testing of the storage cluster capabilities expanded to experimentation with the capabilities of grid technologies to support wide-area parallel file systems and distributed scientific visualization workflows. The effort culminated on August 1, 2007 with the successful deployment of the frost resource on the TeraGrid.
Integrating math, statistics, and geosciences:
IMAGe, the math institute housed within CISL, strives to bring mathematical models and tools to bear on fundamental problems in the geosciences, and to be a center of activity for the mathematical and geophysical communities. Each year, IMAGe focuses on a particular area of the geosciences or applied mathematics that has an impact on NCAR's scientific mission and develops a series of workshops plus a summer school to that theme. In FY2007, the Theme-of-the-Year focused on statistics for numerical models. This theme was undertaken with the goal of matching cutting-edge statistical methods to the needs of geophysical model development and to make statistical scientists aware of the particular scientific issues and research in the geophysical modeling community.
In collaboration with the Statistical and Applied Mathematical Sciences Institute (SAMSI) and the Mathematical Sciences Research Institute (MSRI), four modeling groups at NCAR were engaged to present their models and highlight potential statistical connections at the first workshop. Several collaborations between NCAR scientists and statisticians at SAMSI and the broader statistical community were begun, and the results of these efforts were presented at the second workshop. The next workshop focused on random matrices. Finally, a summer school program on the Carbon Cycle was hosted at NCAR. These kinds of coordinated activities have the potential to significantly increase the multidisciplinary training of young scientists. This also brings new mathematical approaches to challenging geophysical problems.
Frameworks to standardize large-scale modeling efforts:
Again in FY2007, The Earth System Modeling Framework (ESMF) has made significant strides in helping researchers manage the growing complexity of developing Earth system models. Disparate model components representing physical domains and processesfor example, atmosphere, ocean, and sea iceare coupled into integrated systems to create realistic simulations. These models are computationally intensive and must run on a variety of parallel-processor supercomputing platforms. ESMF defines a set of standard software interfaces and a set of high-performance tools for common functions.
Now in its fifth year, ESMF has transitioned from NASA funding to multi-agency support and is the technical basis for the DoD Battlespace Environments Institute, the NASA Modeling Analysis and Prediction Program, and a host of smaller projects. The number of ESMF science components in the community is an important metric, since more standard components mean more options for researchers creating coupled systems. The adoption of ESMF grew steadily this year, with the number of available science components growing from 36 at end FY2006 to 58 at end FY2007.
Initiating the next generation of computational scientists:
A formalization of CISL's summer internship efforts, the Summer Internships in Parallel Computational Science (SIParCS) program is a prototype partnership between CISL and selected universities. SIParCS is sponsored and administered by CISL to provide opportunities for exceptional students with backgrounds in computational science, applied mathematics, computer science, or the computational geosciences.
This new program offers a significant opportunity to make a positive impact on the quality and diversity of the workforce needed to use and operate 21st century supercomputers. Ultimately, SIParCS aspires to help address shortages of trained scientists and engineers capable of using and maintaining high-end computer and data systemspeople desperately needed to achieve the goals of future computational geoscience research.
In return, SIParCS provides a framework for interns to gain practical experience with a wide variety of parallel computational science problems by working under the guidance of CISL mentors on HPC systems and applications relevant to NCAR's Earth System science mission.