Last update: 05/14/2008
CISL provides the tools described in this section to support your code development efforts. These tools can also facilitate good programming practices. Most of these tools are open source products available to the general UNIX programming community, and they may be familiar to you.
Multitudes of products are available, and you may not always find what you are looking for on NCAR systems. Version numbers may vary from one system to the next, and other inconsistencies may exist. If a problem like this is impeding your work, please contact Consulting Services at consult1@ucar.edu
One of the advantages of the UNIX operating system is its collection of tools. All of these tools are well adapted to specific tasks, and they flexibly perform those tasks. These tools also work very well together. UNIX tools tend to be more robust than tools that try to do too much.
Here are some of the UNIX tools used most often by NCAR supercomputer users.
vi, emacs - Editing files
vi and emacs are the two most widely available text editors. Excellent tutorials are available:
gmake - Generating executables
gmake is a tool for managing compilation and builds of software packages. gmake functions consistently across platforms. Here are some well-designed gmake tutorials:
tar - Concatenating files for transfer
tar is an archiver command (originally developed as a tape archiver). Because NCAR's Mass Storage System (MSS) is not good at handling large numbers of small files, the tar command provides a simple way to package groups of smaller files into a single large file for transport, storage, and retrieval. Details are available on the tar man page. Here are some usage examples:
To create an archive file:
tar -cvf archive_name.tar file_namesTo extract files from an archive:
tar -xvf archive_name.tar
compress, gzip - Reducing file size
Because text files often contain a large amount of redundant data and empty space, their size can be significantly reduced with tools such as compress and gzip. Details are available on the compress and gzip man pages. Here are some usage examples:
compress:
compress file_name
uncompress file_namegzip:
gzip file_name
gunzip file_name
CVS - Managing source code changes and versions
CVS stands for Concurrent Versions System. CVS is a tool that allows people who maintain text files, such as program source code files, to keep track of versions of those files, as well as the changes that were made between versions. CVS allows you to retrieve any version that you've stored during the document's development. CVS allows multiple people to work with source code simultaneously. CVS can also help you merge changes.
Here are some cvs tutorials:
Other tools - Perl, Python, Rexx
UNIX has a wide variety of other tools for manipulating program source code and other files. Some of these tools are installed on various systems at NCAR. Users can contact the CISL Support Team at the link at the bottom of this page to request permission to install tools like these on specific computers. User-contributed software is installed in the /contrib directory on individual computers for the benefit of the user community. You can read more about this in User-maintained software repositories (/contrib).
These other tools include Perl, Python, Rexx, and others.
Please note that these tools may or may not be supported on all systems.
As complex parallel codes are developed to run on larger numbers of processors -- both within a node and between nodes -- debugging becomes increasingly more complex. Simpler methods such as print statement debugging worked well with codes designed for fewer processors. However, parallel codes have grown to such large sizes that their communication requirements, multiprocessor requirements, and general complexity now require extensive coding to effectively place print statements in the code.
Print statement debugging is fast, easy, and effective for locating simple problems. It is appropriate when you know where the problem is located in the code. You can also use binary search techniques to help you place the print statements more effectively. However, each time you run your code with a new set of print statements, you use part of your computer time allocation. In addition, print statements can generate huge debug files that you must search through to find the error(s) in your code.
New debugging tools have been developed to control the execution of a code, display the line in the source code where the error occurred, display the contents of program variables, stack and heap, and the error status at the point the code is halted. These debuggers are designed to debug threaded symmetric multiprocessing (SMP) codes, including SMP codes that use message passing and run across a clustered network of nodes.
Examples of ways to use TotalView and Vampir appear below.
TotalView
TotalView is an interactive parallel code debugger with a graphical user interface. It is a full-screen debugger that displays the state of each message-passing process or SMP thread in a multipane processing window. It is supplied by Etnus, a third-party vendor.
TotalView is recommended by IBM for parallel debugging. We provide some simple examples for using it on bluevista, and the vendor provides comprehensive documentation:
TotalView requires you to spend time to learn how it works and how to understand its results, but it is a flexible, comprehensive parallel debugging tool. An important benefit is that you can explore your code without having to re-run it on the supercomputer. So using TotalView allows you to debug complex codes without diminishing your computer time allocation.
Overview of computing at NCAR - Table of contents
If you have questions about this document, please contact us via any of the methods (phone, email, ticket, or in person) described here: CISL Customer Support.
© Copyright 2003-2008. University Corporation for Atmospheric Research (UCAR). All Rights Reserved.
Address of this page: http://www.cisl.ucar.edu/docs/access/tools.html