| I n what
has proved to be a popular service since its inception in January 2004,
SCD is supporting meteorlogical research by increasing the availability
of data from Unidata's Internet Data Distribution (IDD) system.
The IDD, developed by Unidata, is a system of software and a network
computing infrastructure that distributes meteorological data around
the world to subscribers via the Internet. IDD delivers about 50 gigabytes
of data per day to more than 130 universities.
Until January, anyone wanting that information had to set up a server
and run software to monitor the network and capture the data. If the
server went down, they would miss the data.
In Fall 2003, however, Doug Schuster, a software engineer in SCD's
Data Support Section (DSS), created permanent archives of a
selected subset of data from the IDD on NCAR's Mass Storage System.
He also set up two websites for accessing that material-one
for global observational
data, one for gridded
model data.
The websites went live in January '04. Each site holds three months'
worth of the most recent data in a rotating archive-old information
is aged off the website. The old data is available, back to December
'02, by request to Doug (schuster@ucar.edu).
Observational and model data
The observational data include meteorological readings from land
stations, ships, and buoys; surface synop and upper air reports; and
METAR data.
The gridded model data, produced by the National Centers for Environmental
Prediction and the European Centre for Medium-Range Forecasts, come
from predictive and analysis models that range from global simulations
to those covering the continental U.S. and North America.
"The universities have wanted IDD backup for years," says
DSS manager Steven Worley, "but it's not in Unidata's mission
to provide access to retrospective data archives. Unidata provides
software and computing technology to move data and allow people to
access that data, and they maintain a standing archive of the most
recent five days of IDD materials-but that's a very small chunk when
you think of weather patterns and research studies.
"SCD's new archives formalize a permanent backup of selected
IDD data circulating on the Net. Important subsets of model and
observational data are now completely archived on the MSS. Their
full history will be easily recoverable. This effort is an example
of fruitful collaboration between an NCAR division and UCAR's Office
of Programs."
The data are available free of charge via the web and anonymous
FTP. Unidata universities can use the software they already have to
display the data, although one doesn't have to be a Unidata software
user to access the files: the model data are in GRIB (GRIdded Binary)
and observations are in netCDF, both standard formats.
The service has gotten an enthusiastic response from the atmospheric
sciences community.
"It's a good resource," Steven says. "Anyone who's
lost the data or didn't collect the information can say, 'Let's go
to NCAR and get it.'"
A popular resource
Indeed, since the two websites went online in January, about 740
users have downloaded nearly 44 gigabytes of data. "The sites
have proven to be easy to use," Doug notes. "Of all the datasets
DSS provides for free, these rank in the top ten."
"It's a great set of information to help scientists and educators
study the weather," Steven adds. "They can acquire datasets
in near-real time and use them routinely in research labs and classrooms."
"For instance, a forecast model predicts two to seven days in
advance, and is immediately available. Later, people can compare observations
to the forecast generated at an earilier time to determine the effectiveness
of the model. Or, if they wanted to look at the conditions that led
to nine inches of snow in Boulder last week, they could see how the
models predicted the storm, and what data were available."
Doug concurs. "This is really something to be excited about—especially
the observation side. People like the ability to get real-time observational
data within two days-it allows them to be able to examine an event
near to the time it happens. That seems to be one of the major draws
for those who are utilizing the site."
SCD / Unidata collaboration
This data availability is just one part of an end-to-end collaboration
between SCD and Unidata. SCD houses and does administrative support
on a Unidata top-level IDD node, a server named motherlode. Unidata
runs a full suite of software, monitored by their staff, that receives
data from the IDD. These data are automatically supplied in real time
to other Unidata sites. SCD backs up the Unidata server to the MSS
and creates the retrospective archives described here.
For more information, see the DSS websites:
|
