SCD News: August 14, 2005

New IBM supercomputer coming to NCAR August 26

Storage disks and controllers for NCAR's next supercomputer, an IBM POWER5 system named bluevista, have already started to arrive in the Computer Room of the Mesa Laboratory. Standing behind the boxes of new equipment are Dick Valent, head of SCD Consulting Services; George Fuentes, head of the SCD Supercomputer Systems Group; Tom Bettge and Rich Loft, deputy directors of SCD; and Aaron Anderson, manager of SCD's Enterprise Services Section. Larger photo

A new supercomputer called bluevista will arrive at NCAR on August 26, 2005.

The new system is based on IBM's POWER5 processor and High-Performance Switch technology. It will have approximately the same sustained computing capacity as bluesky, NCAR's IBM POWER4 system.

The Scientific Computing Division (SCD) of NCAR's Computational and Information Systems Laboratory (CISL) has exercised an option in its Advanced Research Computing System (ARCS) contract to acquire the new supercomputer. The decision was made after consulting with NCAR scientists and management and with the SCD Advisory Panel.

SCD hopes to complete acceptance testing and early evaluation of bluevista by January 1, at which time it will be available for allocation and use by the university and NCAR communities and the Climate Simulation Laboratory (CSL). SCD will maintain and operate the system for three years thereafter.

A four-phase contract

The POWER5 system is the last in a sequence of supercomputer systems and system upgrades acquired under the ARCS contract, which has had four phases.

• The initial phase, in fall 2001, more than doubled the peak computational capacity of blackforest, an IBM POWER3 system, from 0.9 to 2.0 teraflops (Tflops).
  
• The second phase, in fall 2002, introduced bluesky, a POWER4 system with a peak computational capacity of 6.3 Tflops.
  
• The third phase, in fall 2003, increased bluesky's capacity to its current peak computational capacity of 8.3 Tflops.
  
  (In January 2006, blackforest, which had first been installed in August 1999, was decommissioned after serving NCAR for five-and-a-half years. Though it represented only 21% of the sustained computing capacity available to Community and CSL users, its removal has been felt by bluesky users, who have noticed longer turnaround times for their jobs.)

• Phase 4 of the ARCS contract with IBM will provide NCAR with an additional aggregate peak performance of 4.74 Tflops, 1,248 gigabytes of memory, and 55 terabytes of RAID disk.

Faster and more efficient model runs

At 1.9 gigahertz (GHz), the POWER5 processor runs at a faster clock speed than the POWER4 processor (1.3 GHz). In addition, the POWER5 processor also can sustain a higher memory bandwidth. Thus, NCAR applications on bluevista should show enhanced performance through the faster clock and a higher percent of peak computational capacity.

Figure 1 shows the relative performance of the Community Atmosphere Model (CAM) and the Parallel Ocean Program (POP) benchmarks on three systems: bluesky, lightning (NCAR's AMD Opteron, Linux-based system), and a test POWER5 system. SCD anticipates that most applications will show similar improvements on the POWER5 system, with some potentially approaching three times the speed of bluesky.

Processors and nodes

The bluevista system will consist of 78 IBM POWER5 p5-575 Symmetric Multi-Processor (SMP) nodes. Each node will have eight POWER5 simultaneous multithreading (SMT) processors and 16 gigabytes of memory.

Nodes will be allocated as follows:

• 72 nodes for computational workload
• 2 nodes for interactive login sessions
• 4 nodes for filesystem input/output, management of the 55-terabyte disk subsystem, and connectivity to NCAR's Mass Storage System

All nodes of the system will be interconnected with IBM's pSeries High-Performance Switch (HPS), previously known as the Federation switch. The HPS provides a single-link, unidirectional, point-to-point communication bandwidth of 2 gigabytes per second and latency of seven microseconds. Each node of the system will have two bidirectional link interfaces to the HPS.

A “hot” system

While bluevista will provide roughly the same sustained computational capacity as bluesky on NCAR models, it is a “denser, hotter” system. It will occupy only a third of the floor space as bluesky, but require two-thirds of bluesky's power and cooling.

SCD estimates bluevista will need 276 kilowatts of power to operate. (By comparison, the average U.S. home consumes 10.5 kilowatts.)

Bluevista will be located in the northwest corner of the SCD computational facility. Visitors standing in front of the glass observation area on level 1B of the Mesa Laboratory will see bluevista off to the right, toward the back of the Computer Room.

Future plans

SCD's ARCS contract with IBM will expire in December 2006. While the maintenance portion of that contract will last until late 2008, bluevista is the last equipment acquisition under the ARCS contract.

In cooperation with NCAR management, the SCD Advisory Panel, and the CSL allocations panel, SCD is developing a new Request for Proposal (RFP) for the competitive procurement of a new supercomputer system in late 2006 to replace bluesky. The new system will be called the Integrated Computing Environment for Scientific Simulation (ICESS).

SCD anticipates that ICESS will significantly enhance the computational capacity and capability at NCAR.

For more information

For more information on the IBM p5-575, see http://www.ibm.com/servers/eserver/pseries/hardware/highend/575.html 

NCAR is operated by the University Corporation for Atmospheric Research (UCAR) under the primary sponsorship of the National Science Foundation.