IBM Roadrunner
From Wikipedia, the free encyclopedia
Roadrunner is a supercomputer built by IBM at the Los Alamos National Laboratory in New Mexico, USA. Currently the world's fastest computer, the US$133-million Roadrunner is designed for a peak performance of 1.7 petaflops, achieving 1.026 on May 25, 2008,[1][2][3] and to be the world's first TOP500 Linpack sustained 1.0 petaflops system. It is a one-of-a-kind supercomputer, built from commodity parts, with many novel design features.
Contents |
[edit] Overview
IBM built the computer for the U.S. Department of Energy's (DOE) National Nuclear Security Administration.[4][5] It is a hybrid design with 12,960 IBM PowerXCell[6] 8i CPUs and 6,480 AMD Opteron dual-core processors[7] in specially designed server blades connected by Infiniband. The Roadrunner uses the Red Hat Enterprise Linux operating system and is managed with xCAT distributed computing software. It occupies approximately 6,000 square feet (557 m²)[8] and became operational in 2008.
The DOE plans to use the computer for simulating how nuclear materials age in order to predict whether the USA's aging arsenal of nuclear weapons is safe and reliable. Other uses for the Roadrunner include the sciences, financial, automotive and aerospace industries.
[edit] Hybrid design
 |
This article or section is missing citations or needs footnotes. Using inline citations helps guard against copyright violations and factual inaccuracies. |
Roadrunner differs from many contemporary supercomputers in that it is a hybrid system, using two different processor architectures for the heavy lifting. Usually supercomputers only use one, since it would be easier to design and program for. To tap the full potential of Roadrunner, all software will have to be written specially for this hybrid architecture which is uniquely complex. The hybrid design consists of dual-core Opteron server processors manufactured by AMD utilizing the standard x86 architecture. Attached to each Opteron core is a Cell processor manufactured by IBM using Power Architecture technology. As a supercomputer, the Roadrunner is considered an Opteron cluster with Cell accelerators as each node consists of a Cell attached to an Opteron core and the Opterons to each other.
The Opterons are very good general purpose processors, popular in supercomputer clusters and easy to program for. But they are not powerful enough to reach the desired 1 petaflops on their own. Building such a system would require at least 20 times as many processors and would be extremely expensive to house, build, power and cool. This is where the PowerXCell processors come in. The Cell processors are nearly 30 times more powerful than the Opterons on these operations, but they are weaker in all other respects, such as organizing the applications, running the operating systems and networking with other computer nodes, housekeeping work that is quite important in very large supercomputers. This leaves the design of a supercomputer of this scale solely based on Cell processors to be impractical.
[edit] Development
Roadrunner has been in development since 2002, and went online in 2006. Due to its novel design and complexity it is constructed in three phases and became fully operational in 2008.
[edit] Phase 1
The first phase of the Roadrunner was building a standard (albeit quite large) Opteron based cluster, while evaluating the feasibility to further construct and program the future hybrid version. This Phase 1 Roadrunner reached 71 teraflops and has been in full operation at Los Alamos National Laboratory doing advanced weapons simulations since 2006. Even if Roadrunner had not been greenlit for Phase 2, the Phase 1 form would still be a formidable supercomputer and would have ranked, at its time, in the top 10 list of the worlds fastest computers.
[edit] Phase 2
Phase 2 known as “AAIS” (Advanced Architecture Initial System) included building a small hybrid version of the finished system using an older version of the Cell processor. This phase was used to build prototype applications for the hybrid architecture. It went online in January 2007.
[edit] Phase 3
The goal of Phase 3 was to reach sustained performance in excess of 1 petaflops. Additional Opteron nodes and new PowerXCell processors were added to the design. These PowerXCell processors are five times as powerful as the Cell processors used in Phase 2. It was built to full scale at IBM’s Poughkeepsie, New York facility, where it broke the 1 petaflop barrier during its fourth attempt on May 25th, 2008. The complete system will be moving to its permanent location in New Mexico in the summer of 2008, where fine tuning of the applications will continue until final completion in 2009.
[edit] Technical specification
[edit] TriBlade
Logically, a TriBlade consists of two dual-core Opterons with 16 GB RAM and four PowerXCell 8i CPUs with 16 GB Cell RAM.[7]
Physically, a TriBlade consists of one LS21 Opteron blade, an expansion blade, and two QS22 Cell blades. The LS21 has two 1.8 GHz dual-core Opterons with 16 GB memory for the whole blade, providing 4GB for each CPU. Each QS22 has two PowerXCell 8i CPUs, running at 3.2 GHz and 8GB memory, which makes 4 GB for each CPU (like on the LS21). The expansion blade connects the two QS22 via four PCIe x8 links to the LS21, two links for each QS22. It also provides outside connectivity via an Infiniband 4x DDR adapter. This makes a total width of four slots for a single TriBlade. Three TriBlades fit into one BladeCenter H chassis.
[edit] Connected Unit (CU)
A Connected Unit is 60 BladeCenter H full of TriBlades, that is 180 TriBlades. All TriBlades are connected to a 288-port Voltaire ISR2012 Infiniband switch. Each CU also has access to the Panasas file system through twelve System x3755 servers.[7]
CU system information:[7]
- 360 dual-core Opterons with 2.88 TiB RAM
- 720 PowerXCell 8i cores with 2.88 TiB RAM
- 12 System x3755 with dual 10-GBit Ethernet each
- 288-port Voltaire ISR2012 switch with 192 Infiniband 4x DDR links (180 TriBlades and twelve I/O nodes)
[edit] Roadrunner cluster
The final cluster is made up of 18 connected units, which are connected via eight additional (second-stage) ISR2012 switches. Each CU is connected through twelve uplinks for each second-stage switch, that makes a total of 96 uplink connections.[7]
Overall system information:[7]
- 6,480 Opteron cores with 51.8 TiB RAM (in 3,240 LS21 blades)
- 12,960 Cell cores with 51.8 TiB RAM (in 6,480 QS22 blades)
- 216 System x3755 I/O nodes
- 26 288-port ISR2012 Infiniband 4x DDR switches
- 296 racks
- 3.9 MW power
[edit] References
- ^ Sharon Gaudin (2008-06-09). IBM's Roadrunner smashes 4-minute mile of supercomputing. Computerworld. Retrieved on 2008-06-10.
- ^ Military supercomputer sets record - CNET News.com.
- ^ Supercomputer sets petaflop pace. BBC (2008-06-09). Retrieved on 2008-06-09.
- ^ IBM to Build World's First Cell Broadband Engine Based Supercomputer. IBM (2006-09-06). Retrieved on 2008-05-31.
- ^ IBM Selected to Build New DOE Supercomputer. NNSA (2006-09-06). Retrieved on 2008-05-31.
- ^ International Supercomputing Conference to Host First Panel Discussion on Breaking the Petaflop/s Barrier
- ^ a b c d e f RR Seminar - System Overview. Los Alamos National Laboratory (2008-03-13). Retrieved on 2008-05-31.
- ^ Los Alamos computer breaks petaflop barrier. IBM (2008-06-09). Retrieved on 2008-06-12.
[edit] External links
- Los Alamos National Laboratory Roadrunner Home Page. Los Alamos National Laboratory (2007-03-30). Retrieved on 2008-05-31.
- In Pictures: A look inside what may be the world's fastest supercomputer. Computerworld (2008-05-13). Retrieved on 2008-05-31.
