lightning user document heading  
NCAR
Last update: 04/06/2005

Lightning user doc contents

Program development tools

Building and maintaining large codes

We highly recommend that you use makefiles and an appropriate source code control system (like CVS) to save time maintaining complex programs.

Building your programs using makefiles

The unix make utility lets you recompile only the source code with a more recent creation date than corresponding object files. If no compilation errors are encountered, then make will subsequently proceed to build a clean executable. Make is extremely useful for changing compiler options to do profiling, optimization, and/or debugging. GNU's automake is available for automatic makefile generation.

The following makefile example is short and simple. It demonstrates using the make command to build a mixed Fortran 77/C application. The executable generated by this simple makefile can be used to help diagnose how well an MPI program is performing. It reports on the performance of intra-node floating-point operations and inter-node communication. The code is set up to run on two nodes, but can be expanded to run on any number of nodes by changing the "MP_" variables in the env.csh or env.ksh file. To test more than 16 nodes, change the "NMACH=" variable in the makefile.

The example is contained in /usr/local/examples/makefiles/node_test.tar on lightning. Un-tar it in a convenient directory such as your home directory or in /ptmp/$LOGNAME. To make the executable "drv", type 'make'. To run the executable and understand the reports it generates, read the README file.

These are the contents of the makefile:

# The makefile should contain a set of suffix rules. All suffixes must
# be defined. In this case we will have .o for object files, .c for
# C files, and .f for Fortran files.
.SUFFIXES: .o .c .f

# Define the C and Fortran compilers to be used in this makefile:
CC=mpicc -c
FC=mpif90  -c

# Define flags to be used by the C and Fortran compilers:
CFLAGS =    
FFLAGS =

# Define include to be used by the C and Fortran compilers:
C_INCLUDES =    
F_INCLUDES =

# The linker executable in this case must be the MPI Fortran compiler
# to build a mixed C and Fortran MPI code:
LINK = mpif90

# Define values of parameters that appear in the source codes:
DEFINES =

# Define the list of object files for the linker. The linker will use
# those files to build the executable.
OBJECTS = newdrv.o  etime.o  flop.o

# The rule that makes the drv executable (note that libraries have
# been specified by the mpif90 linker):
drv: $(OBJECTS)
     $(LINK) -o drv $(OBJECTS)

# The rule that makes all object files from C sources:
.c.o:
    $(CC) $(CFLAGS) $(C_INCLUDES) $(DEFINES) $<

# The rule that makes all object files from Fortran sources:
.f.o:
    $(FC)  $(FFLAGS)  $(F_INCLUDES) $<

# The rule for deleting object files no longer needed after using
# make for drv:
clean:
    rm  *.o  drv

Important notes about makefile construction:

  • Programmers often write makefiles to the specification defined by GNU. If these makefiles don't function correctly, try using the gnumake tool provided in /usr/local/bin/gmake. This often needs to be in included in $PATH environment path before /usr/local/bin to work correctly.

  • In general, do not hard-code makefiles with MPI-related include paths, library paths, or library names. This can cause problems because the scripts are designed to provide the correct paths and libraries even if there are changes from one release to another or differences for signal vs. threads. When a makefile is hard-coded, any attempt Linux makes to keep such changes transparent to users is defeated.

Maintaining your programs using a source code control system

A source code control system (we recommend CVS) provides the capability to build and maintain large, complex source code libraries and datasets. Detailed information about CVS is provided in the CVS man page and on the CVS documentation page.

Note: You can centralize and maintain your CVS repository on your local system via SSH.

Debugging and performance analysis tools

The tools provided on lightning for code debugging and improvement include the pgdbx debugger (isolates code faults) and the pgprof profiler (identifies how your code uses computing resources).

Debugger pgdbx

The pgdbx debugger is from the PGI Server suite.

Profiler pgprof

The pgprof profiler is from the PGI Server suite.

Trapping floating-point exceptions

Compiler-loader options are available for trapping floating-point exceptions in your code.


Next page | Table of contents - Lightning user guide

If you have questions about this document, please contact CISL Customer Support. You can also reach us by telephone 24 hours a day, seven days a week at 303-497-1278. Additional contact methods: consult1@ucar.edu and during business hours in NCAR Mesa Lab Suite 39.

© Copyright 2004-2005. University Corporation for Atmospheric Research (UCAR). All Rights Reserved.

Address of this page: http://www.cisl.ucar.edu/docs/lightning/tools.jsp