ORNL Summit Supercomputer With 200 Petaflops To Become Operational in Early 2018 – Powered by NVIDIA Volta GPUs and IBM Power9 CPUs

Hassan Mujtaba

The world of supercomputing is being infused with magnitude of performance which is being delivered with the latest hardware capabilities. A few years ago, Oak Ridge National Laboratory was proud of their Titan HPC super computer but that ha since been replaced in terms of performance by two supercomputers from China, Tianhe-2 and the recently announced Sunway TaihuLight. Now US is planning to make a comeback to take the crown of the fastest supercomputer with their upcoming Sierra and Summit supercomputer machines.

The Summit Supercomputer will be the fastest in the world when it launches in early 2018.

Oak Ridge's Summit Super computer Announced With 200 Petaflops Peak Performance - Powered by Volta GPUs and Power9 CPUs

The announcement of the Summit supercomputer comes just a few days after China unveiled their Sunway TaihuLight supercomputer which is around 3 times faster than the Tianhe-2 supercomputer and also based on China's own hardware technology as seen during the reveal. The machine is currently the fastest supercomputer in the world and is placed at China's National Supercomputing Center in Wuxi. The system is able of delivering a peak performance of 125.4 PFLOPs which is insane.

The top 10 supercomputers in the world. (Image Credits: Top500.org)

The system achieves these numbers with China's custom made ShenWei CPUs that were made by the Jiangnan Computing Research Lab in Wuxi. These chips utilize an RISC (reduced instruction set computer) architecture. The supercomputer deploys several of these cores in nodes and each processor is capable of 3 TF (Teraflops) compute output. Each chip has 260 cores and the entire machine features 10.65 million cores. You can say that the reason China developed their own CPUs for supercomputing was the ban on sale of Intel's high-end micro processors to China. China's Tianhe-2 supercomputer is based on Intel Xeon-Phi processors which also power several US based machines.

The  Sunway TaihuLight is the fastest supercomputer in the world. (Image Credits: Pugam)

But now that the crown of supercomputing lands in Chinese hands, US is expecting to make a comeback early next year with their own machines. The US Department of Energy (DOE) has already announced two brand new supercomputers for 2018, Summit from Oak Ridge National Laboratory and Sierra from Lawrence Livermore.

The DOE, in a statement responding to the news Monday about China's supercomputer, said U.S. supercomputing capabilities "have grown exponentially by a factor of 300,000" since 1993. "High-performance computing remains an integral priority for the DOE," the statement said. via Computerworld

The DOE said the government is "pursuing the goals of the National Strategic Computing Initiative: Accelerating the delivery of exascale computing; increasing the coherence between the technology base used for modeling and simulation and that for data analytic computing; charting a path forward to a post-Moore's Law era; and building the overall capacity and capability of an enduring national HPC ecosystem." via Computerworld

Top 500 Supercomputer Features (Top500.org):

  • There are 95 systems with performance greater than a petaflop/s on the list, up from 81 six months ago.
  • Intel continues to provide the processors for the largest share – 455 systems or 91 percent – of the TOP500 systems. The share of IBM Power processors is now at 23 systems, down from 26 systems six month ago. The AMD Opteron family is used in 13 systems (2.6 percent), down from 4.2 percent on the previous list.
  • Hewlett Packard Enterprise has the lead in the total number of systems with 127 systems (25.4 percent) followed by Lenovo with 84 systems. Cray now has 60 systems, down from 69 systems six months ago. HPE had 155 systems six months ago. IBM is now fifth in the systems category with 38 systems.
  • A total of 93 systems on the list are using accelerator/coprocessor technology, down from 104 in November 2015. Sixty-seven of these use NVIDIA chips, 26 systems with Intel Xeon Phi technology, three use ATI Radeon, and two use PEZY technology. Three systems use a combination of NVIDIA and Intel Xeon Phi accelerators/coprocessors.The average number of accelerator cores for these systems is 76,000 cores per system.

NVIDIA and IBM Hardware Being Used To Built The Fastest Supercomputer in The World

Rated at a peak performance of 150-300 PFLOPS, the Summit supercomputer will be based on more than 3400 compute nodes with each node consistin of several next generation IBM POWER9 CPUs and NVIDIA Volta architecture based Tesla GPUs. Each node will deliver around 40 TFLOPs of compute performance and is said to be enough to outperform an entire rack of top of the line Haswell based x86 CPU servers.


With Summit come a few other enabling technologies such as NVLINK which is already shipping with Pascal GPU from NVIDIA. The Pascal GP100 based Tesla P100 is the first to introduce NVLINK which is the next generation Unified Virtual Memory link with Gen 2.0 Cache coherency features and 5 – 12 times the bandwidth of a regular PCIe connection. This will solve many of the bandwidth issues that high performance GPUs currently face. The fast speed interconnect will allow several nodes to balance utilization when working in parallel with cross sharing of data and information through a speedy interface.

A comparison between the Titan and Summit Supercomputer.

On summit, each compute node will feature 512 GB of memory through dense server DDR4 memory and high bandwidth HBM stacked designs which will work in complete coherency. Extending the impressive amount of memory is an additional 800 GB of NVRAM per node, which can be configured either as a burst buffer or as extended memory.

Summit is the next leap in leadership-class computing systems for open science. With Summit we will be able to address, with greater complexity and higher fidelity, questions concerning who we are, our place on earth, and in our universe.

Summit will deliver more than five times the computational performance of Titan’s 18,688 nodes, using only approximately 3,400 nodes when it arrives in 2017. Like Titan, Summit will have a hybrid architecture, and each node will contain multiple IBM POWER9 CPUs and NVIDIA Volta GPUs all connected together with NVIDIA’s high-speed NVLink. Each node will have over half a terabyte of coherent memory (high bandwidth memory + DDR4) addressable by all CPUs and GPUs plus 800GB of non-volatile RAM that can be used as a burst buffer or as extended memory. To provide a high rate of I/O throughput, the nodes will be connected in a non-blocking fat-tree using a dual-rail Mellanox EDR InfiniBand interconnect.

Upon completion, Summit will allow researchers in all fields of science unprecedented access to solving some of the world’s most pressing challenges. via OLCF

The system is interconnected with dual-rail Mellanox EDR InfiniBand, using the full, non-blocking fat-tree design. Summit will also include the GPFS parallel file system with 1 TB/s of I/O bandwidth and 120 PB of disk capacity. The summit computer will be able to achieve 200 PFLOPs of compute performance at just 10MW peak power consumption, the Titan supercomputer achieved a peak 27 TFLOPs performance at 9 MW and with 18,688 compute nodes.

Intel is also working in-cooperation with Cray to deliver two supercomputers for Argonne National Laboratory, named as Aurora and Theta. Both supercomputers advance the hyper computing world with unparalleled amounts of performance, pushing past the PetaFlops barrier with several thousand nodes. Built specifically for scientific purposes, these supercomputers will consume power in the Mega Watts range which gives us a perspective of the size of these monsters. More on those here.

Share this story