Avoid submitting DCS metadata commands that examine or modify large numbers of files, especially with recursive operations on directories. A large metadata update is one that updates 10,000 or more files. A large msls operation is one that lists more than 100,000 files. Users having metadata needs at these scales are encouraged to contact the MSS group for consultation. Consider using the MSS reporting website for obtaining large file listings.
Users are advised that typical transaction rates during busy periods for metadata operations can be as low as these numbers: msls lists metadata entries at 10 per second, and metadata updates about 1-5 per second. This also applies to the exapansion rates for slow wildcard patterns.
When feasible, generate lists of MSS file names programmatically instead of using slow wildcard patterns. Capturing the output msls for a directory then using the grep on your local system may be faster than trying to use slow wildcard patterns in a directory containing many files.
If you are going to manipulate all files in a directory, use a recursive metadata
command (-R option) instead of a trailing
/* wildcard pattern.
Break up very large requests into pieces. Any single request that requires more than 24 hours to execute will automatically be terminated at the end of the 24 hour period.
Do not use mstouch on monthly basis to prevent an MSS file with the default retention period from being purged. If you need to keep some data longer than anticipated, consider using msretention to set a new retention period that reflects the new lifetime of that data, as opposed to using mstouch repeatedly.