Table of Contents
This document describes the Distributed Computing Services (DCS) interface to the National Center for Atmospheric Research (NCAR) Mass Storage Service (MSS).
The MSS is a collection of hardware and software dedicated to providing archival storage services adequate for handling vast quantities of data. The major components are shown in this table:
Table 1.1. MSS Components
| Name | Description |
|---|---|
| DCS Client Systems | The computers where users run software (or scripts) that calls the DCS Client software in order to interact with the MSS. |
| DCS Server Systems | One or more computers that process the users' submitted DCS requests. The DCS Servers interact with the other MSS components on behalf of the users to accomplish their requests. |
| Mass Storage Control Processor (MSCP) | The computer that maintains the Master File Directory (MFD), which stores the metadata information describing all MSS files. The MSCP handles all resource scheduling for movement of MSS files into and out of the MSS, as well as the automatic migration of data between the various storage media. |
| Storage Manager (STMGR) | One or more computers that control the movement of MSS files between the DCS Client systems and other MSS components and the storage media of the disk cache and tape archives. |
| Disk Cache | A set of Redundant Array of Independent Disk (RAID) units that provide low latency access for most newly written files and frequently read files. |
| Tape Libraries | Several robotic tape library units that automatically handle the storage of tape media and mounts of that media into tape drives. MSS files stored in the tape libraries have an intermediate latency access time. |
| Offline Tape Archive | Open shelves that store tape cartridges, similar to shelves of books in a library. These tapes are mounted by hand when required. MSS files stored in the offline tape archive have a higher latency access time. |
The basic unit of exchange between the MSS and its clients is the MSS file, which is a named collection of one or more bytes (however, see Chapter 6, Usage Guidelines for cautions about storing small files). The maximum size of an MSS file increases from time to time, and the current maximum size is 12 billion (12,000,000,000) bytes. The MSS does not interpret the contents of MSS files, nor does it provide any data format conversion services. When an MSS file is transferred between the MSS and the user's system, the entire contents is moved in one operation. The MSS is not a file server; it is not possible to request the transfer of only a subset of an MSS file.
MSS files are stored for a limited amount of time, known as the retention period. If an MSS file has not been referenced for a period of days equal to its retention period, it is automatically purged from the MSS. After an MSS file has been purged, it is recoverable for a short period of time. See the section called “Retention Period” for more details.
The MSS supports directories that hold one or more MSS files and/or subdirectories. This structures the name space into a tree. Each authorized MSS user has a top-level directory. Directories are not persistent, they are created as needed and are automatically removed when empty. See the section called “Directory Behavior” for more details.