NVIDIA detailed its next-gen ConnectX-8 NIC for Blackwell systems, which it states is so advanced that it should be referred to as a SuperNIC.
NVIDIA's ConnectX-8 SuperNIC Is Designed For The Latest Blackwell Systems & Offers Up To 800G Speeds
According to NVIDIA, AI training and AI Inference are two different workloads and require a fungible end-to-end network policy. Inference is a disaggregated, partitioned workload that is latency sensitive, & has large interface requirements with the outside world, whereas Training is a synchronized, long-lasting workload where tail latency impacts efficiency and has minimal interface with the outside world.
The ConnectX-8 NIC, as mentioned above, is called a SuperNIC and is supported on both Spectrum-X Ethernet & Quantum-X Infiniband.
The following are some of the highlights of this NIC:
- RDMA technology deployed over millions of GPUs
- 800G RDMA hardware pipeline, designed for AI bandwidth, latency, and scale
- Integrated load balancing, congestion control, and reliability
- Deep data-path programmability for AI workload and data center versatility
- Tight integration with system architecture
- Enterprise-class security
In terms of specs, the ConnectX-8 800G SuperNIC is compliant with Verbs, NCCL, NIXL, DOCA APIs, and features an 800 Gb/s Infiniband XDR or 2x400G Ethernet Integrated Spectrum-X Ethernet interface with up to 8 ports. The Host I/O interface used is PCIe Gen6 with 48 lanes, offered through an integrated PCIe switch.
NVIDIA states that ConnectX-8 RDMA offers limitless scaling at 800G at different message sizes (64 KB to 1 MB). The SuperNIC is more than just an NIC, but it acts as an ASIC that connects the GPU to the other clusters.
The NIC is first being deployed within the latest NVIDIA Blackwell GB300 NVL72 systems. These are based on the Blackwell Ultra GPU, which was detailed yesterday. The CX8 PCIe switch enables higher savings of NVLINK C2C bandwidth, QoS between networking.
Each CX8 PCIe switch, two on the NVL72 board, has Gen5 x16 lanes for the Grace CPU and Gen6 x16 lanes for Blackwell Ultra GPU; the SSD also gets a PCIe Gen5 x4 lane.
NVIDIA also shows how ConnectX-8 scales across 64 GPUs:
Since AI is all about scaling up these days, NVIDIA has devised the ConnectX-8 Integrated Spectrum-X Ethernet Switch, which is an extension of the Spectrum-X Ethernet switch infrastructure and offers load balancing and congestion control with switch functions. ConnectX-8 RDMA allows scaling of traditional OS services for AI networking, while the ConnectX-8 Packet Processor enables securing and routing of AI factories.
To keep the network running efficiently, ConnectX-8 includes a Data Path Accelerator, which is a 16T RISC-V event processor. With all of these technologies and hardware-level integrations, Spectrum-X Ethernet delivers a 60% shorter training step time and near-zero tail latency under load when compared to an OTS RDMA NIC / Switch.
Looking at some Spectrum-X performance figures shared by NVIDIA, we get:
- 1.6x Higher Effective Bandwidth (Load Balancing)
- 1.3x Higher Collective Bandwidth (Tail Latency)
- 2.2x Higher all-reduce Bandwidth (Noise Isolation)
- 1.3x Higher all-to-all Bandwidth (Resilient Performance)
- 1000x Faster Telemetry Collection (High-Frequency Telemetry)
Spectrum-X and ConnectX-8 SuperNIC bring 800G to the table along with PCIe Gen6 support. The company is already dishing out solutions with these technologies in the Blackwell systems, so expect additional details in the coming months.
Follow Wccftech on Google to get more of our news coverage in your feeds.
