Archiving on the MSS

This portion of the document is for any frost user with an NCAR account or a TeraGrid account. All information in these pages is important for users with TeraGrid accounts, whereas NCAR users may skip the parts marked TeraGrid. In preparing this document, we assume that you wish to archive data you have generated on frost, or data you need for running models or programs on frost. Archived data is thought of as long-term, meaning persisting for years; there are limits and charges associated with this type of storage. Please see web page TeraGrid Data Resources Page for information on all TeraGrid data resources, including the NCAR MSS.

Archival vs. Other Storage

Archived files are contrasted with ones you may place on a file server. It usually takes longer to read or write archived files, whereas you expect faster response from a server. Also, the lifetime of an archived file is usually expected to be longer than that of a file residing on a file server. Please be aware the files you place on the NCAR MSS are charged for storage and access against your account. TeraGrid users are subject to a 5 TB-year limit on their NCAR MSS files.

At this time, each file is limited to 12 GBs in size; larger files require users to compress them before archival, and to uncompress them after retrieval. Or, the user may split a file before archival, and merge the pieces after retrieval. In any case, merge and split operations, and their inverses, must be done by users outside of the MSS; there are no MSS commands provided for this purpose.

NCAR MSS Archival Commands

In this section we offer a few tips, followed by links to some of the more common NCAR MSS commands. See the NCAR Mass Storage Service User Reference for a complete description of the system and the commands, and for basic information you need for using the system, such as:

A description of the MSS pathnames;
How to create a directory (no mkdir equivalent; just write a file with that pathname name);
The default retention period for an MSS file is 32 days; please specify a longer period when you create a file, as per your archival needs;
The best way to archive files created by a model or program is to run the program, then run a small harvester job which archives the files -- this is a nice way of saying we prefer you not archive files from a running program or model. This way, you avoid problems if the job hangs or has other problems.

Note for TeraGrid users: when you logon to frost, you will land on node fr0103en. Typically you will use this node for editing files, submitting jobs etc. At this time, you must rsh to fr0102en in order to use the MSS commands.

Here are a few of the more common NCAR MSS commands:

msrcp - Copy files to or from the MSS.
msls - List MSS directory contents.
msrm - Move (rename) MSS files.
msretention - Set the retention period for an MSS file.

Sometimes you will need assistance from our MSS administrators for certain tasks, such as any that involve accessing or changing more than 10,000 MSS files in a single job, or for resolving password problems. To get help, NCAR users should contact CISL Customer Support via the link provided at the bottom of this page. TeraGrid users should request help by sending email to help@teragrid.org.

Next page | Table of contents - Frost user guide

If you have questions about this document, please contact CISL Customer Support. You can also reach us by telephone 24 hours a day, seven days a week at 303-497-1278. Additional contact methods: consult1@ucar.edu and during business hours in NCAR Mesa Lab Suite 39.

Address of this page: http://www.cisl.ucar.edu/docs/frost/mss.jsp