HPE SGI ICE XA - Cheyenne

Main content

Cheyenne - HPE SGI ICE XA cluster

In use: January 12, 2017 – December 31, 2023
Peak teraflops: 5,340 (5.34 petaflops)
Linpack teraflops: 4,788
Processors: 145,152
Processors per node: 36
Nodes per frame: 144
Frames: 28
Clock speed: 2.3 GHz
Memory (TB): 313 TB
Storage: 40 PB
Power consumption: 1.7 MW
Predecessor: Yellowstone (IBM iDataPlex)

The Cheyenne supercomputer was installed at the NCAR-Wyoming Supercomputing Center (NWSC) during the second half of 2016, run through a rigorous acceptance testing period, and began full-scale production in January 2017. Retired at the end of 2023 after seven years of service, Cheyenne was one of the longest-serving supercomputers in NCAR history.

A 5.34-petaflops system, Cheyenne tripled its predecessor’s power. At launch, it ranked as the 20th fastest supercomputer on the Top500.org list. The system was originally slated to be replaced after five years, but then the COVID-19 pandemic struck, severely disrupting supply chains. Cheyenne was called upon to serve an additional two years—maintained and supported by a dedicated staff of about 20 NWSC engineers—until the necessary parts were available to build its successor. 

Cheyenne was procured as an SGI ICE XA cluster—HPE purchased SGI prior shortly before the system was installed—with 4,032 nodes, each with two 18-core, 2.3-GHz Intel Xeon E5-2697v4 "Broadwell" processors, for a total of 145,152 cores. Cheyenne nodes had two memory sizes: 3,164 nodes had 64 GB and 864 had 128 GB. The nodes were connected by a Mellanox EDR InfiniBand interconnect in a 9-D enhanced hypercube topology. The system used the SuSE Linux operating system and the Altair PBS Pro scheduler. Accompanying the compute hardware, Cheyenne also included a 40-PB storage resource based on DDN hardware and IBM SpectrumScale file system.

Cheyenne continued to improve on the power efficiency of NCAR's large-scale systems. With a peak power consumption of 1.75 MW, Cheyenne consumed only 25% more power than its predecessor, Yellowstone, while delivering twice the sustained computing performance on the typical NCAR workload.

In its lifetime, Cheyenne delivered over 7 billion core-hours, served over 4,400 users, and supported nearly 1,300 NSF awards. It played a key role in education, supporting more than 80 university courses and training events. Nearly 1,000 projects were awarded for early-career graduate students and postdocs. Perhaps most tellingly, Cheyenne-powered research generated over 4,500 peer-review publications, dissertations and theses, and other works.

Researchers used Cheyenne to run increasingly detailed models simulating complex processes and how they might unfold in the future. Scientists also harnessed its computing power to run multiple simulations, or ensembles, to quantify a given event’s probability—providing needed intelligence for policy and resource planning.

Examples of transformative research with Cheyenne include:

“Cheyenne was a special machine that has certainly earned its retirement,” said NSF NCAR Director Everette Joseph. 

Previous Page
Geyser & Caldera
Next Page
CDC 3600