CISL Annual Report 2005 > Earth System Knowledge Environment accomplishments

Community software for geoscientific analysis and visualization

The demand from the scientific community is quickly increasing for more versatile, scalable, and wide-ranging tools for geoscientific data analysis and visualization. Members of SCD's Visualization and Enabling Technologies Section (VETS) have responded to this demand by continuing to strengthen and enhance existing software, and by broadening the scope of their development to include two other data analysis and visualization projects. This year, the focus of their development has been on four major software projects:

• NCL (NCAR Command Language), an internally developed scripting language that provides robust file handling, data analysis, and publication-quality two-dimensional visualizations

• PyNGL, a Python interface to NCL's visualization library

• PyNIO, a new Python interface to NCL's file input/output library that allows read and/or write access to a variety of data formats

• Vis5D+, a free OpenGL-based volumetric visualization program for scientific datasets in 3+ dimensions

This year, one of our major goals has been to gather better metrics on our users. We moved the downloadable NCL and PyNGL/PyNIO software to the Earth System Grid website. Based on information culled from the new download information, there are close to 800 scientists, researchers, teachers, and students worldwide that have a known interest in NCL, and new users come on board daily. NCL is used in almost 50 countries at sites ranging from foreign and domestic universities, national labs, supercomputing centers, government and military sites, commercial sites, and numerous research companies focusing on wide-ranging scientific disciplines: physics, forestry, space monitoring, urban impact, climate modeling, oceanography, disaster prevention, applied insurance, biogeography, genetic studies, electric power, marine investigation, hydrology, sea breezes, soil moisture, ocean biogeochemistry, sea circulation, atmospheric mercury, aircraft data analysis, harmful algal blooms, weather forecasting, tropical climate studies, remote sensing, polar research, renewable energy, monsoons, typhoons, air quality, in-flight icing, ocean diffusivity, wildfires, ecology, animal disease modeling, pollution, trans-Pacific transport, paleoclimatology, Middle East water cycles, and weather impact on society.

In FY2005, a major milestone was reached with the release of the new NCL website, a collaborative effort between SCD and CGD to consolidate the old NCL documentation website with the hundreds of NCL processing and visualization examples on CGD's CCSM processor website. This website uses XML technologies to better organize the hundreds of functions and resources available, and provides everything from detailed examples, download information, hardcopy documents, reference manuals, PowerPoint presentations, line-by-line tutorials, and email archives from a user support group.

This is the new home page for the upgraded NCAR Command Language (NCL) website.

A second major milestone was the release of the new "Nio" Python module. This module gives Python users a single, easy-to-use interface that allows them to handle the same multiple data formats that NCL supports, including netCDF, GRIB1, HDF4, HDF-EOS2, and CCM History tape. In addition, developers continued to fine-tune NCL and Nio's file management capabilities by adding several algorithms for significantly speeding up the reading of netCDF files, for controlling the interpolation methods for reading GRIB files, and for controlling the byte-ordering of binary files, allowing users to easily move binary files between computer systems.

Several versions of PyNGL were released in FY2005 since its initial release in FY2004, with the most recent version containing the Nio module. There have been many new PyNGL functions added in response to user requests, including functions for generating specialized visualizations like meteograms and skew-T plots.

Meteogram and skew-T plot, both generated by PyNGL.

To take steps toward including frameworks for tools that perform three-dimensional visualizations, the NCL/PyNGL team took on the maintenance of Vis5D+, which was no longer being supported by its original author. Based on high demand, Vis5D+ was ported to Mac OSX and provided to the user community.

It has been four years since the last NCL survey was conducted to about 400 users. A more general-purpose data analysis and visualization survey (with heavy focus on NCL, PyNGL, and Vis5D+) was released to over 1,000 users this year. The results will be essential in driving development efforts for said software for the next four to five years.