SCDzine
H I N T S

Using MSS commands on multiple files via the 'list' method

Handiest method ever. . .

tom parker
Tom Parker


Contents

Search

Article index

Back issues

Subscribe

Contact us

SCD

by Tom Parker


The Mass Storage System (MSS) commands include powerful file-selection techniques, namely, wild card characters and the msfind command. These techniques often allow you to conveniently perform operations on multiple MSS files.

However, sometimes you may wish to perform operations on multiple MSS files that can't be readily selected with wildcards or with msfind. This is where the "list" method may come in handy.

In the "list" method, you prepare a list of the MSS files you want to process and put this list in a file on your local computer. Then you can pass this list on to many of the available MSS commands.

The MSS commands that will work with this "list" method are:
msallinfo mschproj mscomment
msls mspasswd msrawinfo
msrecover msretention msrm
msstage mstouch

What makes the "list" method practical is use of the Unix command xargs. This command will read your file and pass the filenames along to the specified MSS command. xargs will actually break up the list of files into manageable chunks so that it does not exceed your shell's command line limit; hence the desired MSS command may actually be run several times, each with a different chunk of the filenames. The xargs command has several options, including the -t option that allows you to see the full command it's executing.


Check it out

Let's create a file called ms.filelist that contains the names of several MSS files that we are interested in:

    my/file/one 
    some/other/file     and/another     

      /PAT/yet/another/file             
     ocean/1998*                        
    last.file

Since there is no obvious pattern to the filenames, we can't make use of wildcards to select them. And there are enough files here that we probably wouldn't want to type them all in as arguments to an MSS command.

We can do an msls -l on these files using xargs, like this:

     xargs  msls -l < ms.filelist

or, if you prefer, with the equivalent command:

     cat ms.filelist | xargs  msls -l

If we want to see the actual command line that xargs is running, we can add the -t option to xargs:

     xargs -t  msls -l < ms.filelist


But wait, there's more

Here are some other points to be aware of:
  • The file containing the filenames can have:

    • One file per line
    • Many files per line
    • No files on a line

    Additionally:

    • Starting column is unimportant.
    • Filenames can be absolute file names (like /PAT/yet/another/file).
    • Filenames can be relative (like my/file/one). Relative filenames will be prefixed by the environment variable $NCAR_MSSPWD if it is set (normally by the mscd command); otherwise, the relative filename will be prefixed by your local host login name (e.g., /PAT).
    • Filenames can even make use of the normal MSS wildcard characters (like the line ocean/1998*).

  • Depending on the MSS command you wish to invoke, the filenames in your file could be MSS bitfile names and/or MSS directory names.

  • Since xargs reads its input from STDIN, you could conceivably have a script that outputs some MSS filenames that you could then feed in to one of the allowed MSS commands:

           some.script | xargs  msls -l

  • These MSS commands are not suitable for the "list" method:
    mscdsetup msfind msmv
    mspwd msrcp msread
    mswrite msstage mstouch

  • These MSS commands use their own kind of "list" method when processing multiple files (as described in their man pages).

    msexport   msimport

  • The list technique described here does not apply to the MASnet Internet Gateway Server (MIGS) or Internet Remote Job Entry (IRJE).


See also . . .

For a variety of information to help you run your jobs better, see SCD ConsultWeb, a website maintained by the SCD consultants.

rule
Contents || Search || Article index || Back issues || Subscribe || Contact us || SCD