The Earth System Grid

This snapshot of ESG capabilities shows powerful web-based portals to CCSM and IPCC data, virtual data services, model run tracking, large-scale data transfer, and access to the CCSM model itself. These components provide a global climate research and impacts community with readily accessible tools for locating, acquiring, and using climate change research data. ESG, a multi-agency collaboration, harnesses the power of Grid Computing to advance the process of scientific discovery.

The Earth System Grid (ESG) is a five-year project funded by the U.S. Department of Energy (DOE) to develop and deploy a Data Grid that makes our terascale climate simulation data readily available to a worldwide community. The project is a collaboration of NCAR (SCD, CGD, and HAO), Argonne National Labs (ANL), Oak Ridge National Labs (ORNL), Lawrence Livermore National Labs Program for Climate Model Diagnosis and Interpretation (LLNL/PCMDI), the University of Southern California Information Sciences Institute (USC/ISI), Lawrence Berkeley National Laboratory (LBNL), and Los Alamos National Laboratory (LANL).

ESG currently provides most of the joint NSF/DOE climate change simulations conducted over the last six years as well as the IPCC Fourth Assessment Report data holdings. Access is provided through a combination of web portal access as well as desktop applications that mediate large-scale transfers to the user. ESG currently has over 4,000 registered users worldwide and manages 160 TB of data in archives distributed around the nation. From this past year alone, more than 200 scientific journal articles have been published from analyses of data delivered by the ESG. ESG thus plays an important role in advancing NCAR's strategic plan, including "Developing and providing advanced services and tools," "Engaging a broader and more diverse community" in the atmospheric and geosciences, "Creating an Earth system knowledge environment," and advancing a global community's ability to advance climate research. It is also an excellent example of cross-agency collaboration to advance science.

Over the course of the last year, we have published a large volume of new datasets and IPCC assets and improved the function and performance of multi-file delivery as well as virtual data services. We made a significant migration to the Globus Toolkit V4 (particularly in monitoring and replica-location), improved the performance of OPeNDAP-g, delivered new versions of Data Mover Lite, and moved the entire software base to the Spring framework. We also worked with the CCSM project to develop new web-portal-based tools for instantiating and tracking model runs. We also completed a major refactoring of the system to migrate it to the TeraGrid in early FY 2007—a new Science Gateway for climate research.

We successfully completed our five-year project with a good outcome. Building on that, one of our primary activities for the year was developing a new proposal for DOE's SciDAC-2 solicitation, the second generation of DOE's highly successful Scientific Discovery Through Advanced Computing program. Developed as a collaboration across three agencies (DOE, NSF, and NOAA), five DOE labs, and the University of Southern California, we proposed to construct a global, distributed, federated data system. The proposed project, the Earth System Grid Center for Enabling Technologies (ESG-CET), aims to develop an integrated data management, access, analysis, and visualization system that spans the globe. Out of a multitude of proposals, DOE selected 13 Centers and Institutes to carry their program forward, and ESG-CET was among those. The project will address an integrated modeling, data, and analysis environment that will address core SciDAC, CCSM, and IPCC requirements into the year 2012. Our FY 2007 efforts will be focused on developing the long-range strategies and plans that will make this project a success for the research and impacts community.

Here is the ESG web page.

Primary support for this project is from DOE's Scientific Discovery Through Advanced Computing contracts DE-F02-01ER25452 and DE-F02-06ER25772, with additional support from NSF Core funds through the NCAR Cyberinfrastructure Strategic Initiative.