NVIDIA Rubin GPUs Bring 10x Increase in Agentic AI Performance Versus Blackwell as Its Architecture Gets Fully Unpacked, Featuring 336 billion Transistors

NVIDIA has fully disclosed its Rubin AI GPU architecture, bringing a monumental increase over Blackwell for the Agentic era.

NVIDIA Reimagines Data Center With Its Rubin GPUs, Bringing The Fastest AI Performance In The World

The Rubin GPU is at the heart of NVIDIA's next-generation data center platforms; it is a chip that took several years to make and is now finally entering general availability. With the first Rubin AI platforms already landing across the globe at major tech firms, it was high time that NVIDIA disclosed the full details of the architecture that went into making this supercharged AI chip that will shape the Agentic landscape.

According to NVIDIA, the Rubin chip delivers 10x higher agentic throughput per unit of energy versus Blackwell, and it does so with three major architectural changes:

• Enhanced Tensor Cores with Expanded Precision Capabilities
• New HBM4 Memory Subsystem
• 3rd Generation Transformer Engine

But first, let's start with the architectural breakdown by looking inside the chip.

Rubin Architecture Dissected - A Multi-Billion Transistor Monster

NVIDIA's Rubin chip is made on TSMC's 3nm (N3P) process node and features two reticle-limited dies to achieve high-density and efficiency characteristics that define the chip. These two dies are connected in a single unified package through NVIDIA's High-Bandwidth Interface (NV-HBI). The chip packs a total of 336 billion transistors, a 62% increase over the Blackwell GB300 chip.

Under the hood of each Rubin GPU are 8 GPCs or Graphics Processing Clusters with a total of 224 SMs or Streaming Multiprocessors and 896 Tensor Cores. That should round up to 28 SMs & 112 Tensor Cores per GPC, but based on the block diagram, there are two types of GPCs, one with 30 SMs, and the other with 26 SMs.

There are four GPCs per die, which are connected to two blocks of large decentralized L2 cache, a Gigathread Engine, four HBM channels (4096-bit per die), and an NVLink interface. The chip also houses a PCIe Gen6 controller to interface with the host CPU, and the NVLink v6 IO provides 3600 GB/s linked with the NVLink switch. The MIG controller partitions the GPU for multiple workloads & there's also NV-DEC for accelerated decoding.

The chip also features Confidential Computing through TEE-I/O, which secures data at rest, in transit, and in use across AI workflows.

The Fastest HBM4 on The Market For Faster Data Movement

On the memory front, NVIDIA Rubin makes use of HBM4, which is the latest HBM standard. The need for faster data movement has become critical in today's AI landscape, and Rubin goes all out by bringing 288 GB of capacity across eight 12-Hi stacks (that's 36 GB per stack &3 GB per DRAM). These stacks deliver a peak bandwidth of 22 TB/s, making them the fastest HBM4 solution on the market.

NVIDIA also utilizes its new and enhanced Tensor Memory Accelerator (TMA) for moving highly complex data efficiently through the chip, while NVLink 6 provides 3600 GB/s of scale-up speeds for all-to-all GPU-GPU communication. The NVLink-C2C interface provides 1800 GB/s of CPU-to-GPU communication, while the PCIe Gen6 x16 lanes provide 256 GB/s bandwidth for host CPU connectivity.

HBM4 Boosts Token Generation

The HBM4 memory for Rubin has been designed for maximum power and compute efficiency, leading to faster decoding and token generation capabilities. Most modern workloads spend time doing long contexts, large KV caches, and interactive token generation, which make bandwidth even more critical. With a 2.8x increase over Blackwell's memory solution (22 TB/s vs 8 TB/s), the HBM4 solution provides:

• Capacity: Supports model residency, larger context windows, larger KV caches, and higher concurrency without unnecessary KV-cache offload.
• Bandwidth: Supports the token-by-token generation phase, during which model weights and KV state must be moved rapidly enough to keep compute engines productive.
• Memory subsystem: TMA and memory-locality strategies help software use the memory subsystem efficiently, supporting high achieved bandwidth for complex data layouts.

HBM4 amplifies Rubin, which is essential for supporting multi-trillion-parameter models, extending context length without the hassle of KV cache offloading, and supporting much higher concurrency rates as well as long-horizon inference workloads.

Accelerating MoE & Tokens Through Enhanced Tensor Memory Accelerator

Rubin also comes with an enhanced Tensor Memory Accelerator, or TMA, which efficiently locates and moves expert weights as the number of experts grows within MoE models. TMA reduces the data movement overhead through improved descriptor handling that allows software to route and work more efficiently. The Inline Descriptor also sees updated support for TMA and allows the kernel to keep one unified descriptor for tensors that share the same layout & override fields such as memory pointer and stride directly in the TMA instruction at runtime.

The result is more efficient MoE model scaling even with increased expert counts. This enables the chip to allocate more time towards inference computation for higher throughput.

Faster Tensor Cores For Faster Throughput

The enhanced Tensor Cores have been pumped up to offer double the throughput per clock cycle. This is achieved by doubling the amount of data a tensor core can process along the K Dimension. This helps throughput-bound kernels and memory/latency bound kernels.

NVIDIA explains that the benefit of a larger K dimension is fewer K loop iterations. For example, a GEMM that required four K iterations on Blackwell can be completed on just two iterations on Rubin chips.

There are also improvements to attention made by combining activation sparsity with adaptive compression, and improved softmax throughput. The attention pipeline begins with a dense QK^T computation to generate the intermediate attention scores.

Rubin can then load that intermediate data from Tensor Memory into a structured 2:4 sparse compressed form, generating both nonzero values and the metadata needed to use them efficiently while reducing the scores’ write cost and storage requirements. This allows the later attention stages to operate on less data while preserving the dense output format expected by the rest of the model.

There's also an increase in Rubin's exponential throughput capabilities over baseline Blackwell. Rubin offers 2x the FP32 throughput (per clock per SM) versus GB200 and the same as GB300, while the BF16/FP16 throughput is quadrupled over baseline Blackwell (GB200) and double vs Blackwell Ultra (GB300).

Rubin also enables more fine-grained coordination between dependent kernels. This allows consumer work to begin earlier as required input data becomes available, rather than waiting for a larger set of producer work to complete.

The result is a more tightly packed GPU timeline, reduced idle gaps, and improved overlap between dependent kernels. This is particularly valuable for agentic inference, where activations move sequentially through the model and kernel-to-kernel latency directly affects tokens per second per user.

Incremental Gains To Power Efficiency With DSX MaxLPS

Another factor that is crucial for today's AI data centers is power efficiency. The Vera Rubin NVL72 are state of the art systems and features some new tricks to expand on this, such as power steering and Intelligent Power Smoothing with energy storage into a single execution domain designed to maximize useful token output within a fixed power envelope.

Transient peaks can hamper the overall throughput by stranding the available capacity at an AI factory. To address this, Vera Rubin systems are equipped with SoC (state-of-charge) intelligent power-smoothing power supplies that absorb these transient swings to reduce average power consumption by approximately 10% versus the prior generation. They also reduce 50ms peak power by 20%, delivering a streamlined power profile that can sustain max power requirements.

For AI factories, NVIDIA DSX OS, which includes DSX MaxLPS, extends across GPUs, racks, liquid cooling, and workloads, offering an operational layer for scheduling, lifecycle management, and health automation that enables the provision of 40% more GPUs within the same power budget, increasing the throughput of each deployment.

All of this combined is what makes Rubin a state-of-the-art chip for AI data centers, and a chip that will embark on a new journey for future AI capabilities, extending to Agentic AI workflows & to what comes beyond.

Follow Wccftech on Google to get more of our news coverage in your feeds.