OpenAI has published a new patent in which it discloses an AI chip housing several compute chiplets, surrounded by a large number of HBM memory stacks.
One Compute Chiplet, Several HBM Memory Stacks: This Could Be OpenAI's Future AI Chip Plans
In a new patent titled "Non-Adjacent Connection of High-Bandwidth Memory Chiplets, I/O Chiplets, And Compute Chiplets Through Embedded Logic Bridges", OpenAI shares plans for an AI chip solution that is going to house several HBM chiplets and compute chiplets, all connected using Embedded Logic Bridges.
The research proposes the idea of leveraging these embedded logic bridges for high-speed interconnects between longer distances, enabling more chiplets to connect together to support high-performance computing and AI workloads which require access to large amounts of memory for efficient operation.
Current packaging solutions have certain limitations regarding HBM integration as HBM memory communicates with other chiplets on the package using metal wires on the base layer. The current JEDEC standard requires HBM memory to be placed adjacent to the compute chiplet, and there are physical limitation with this approach since the metal wires are required to be less than 6mm in length from the PHY controller on the main chiplet itself.
To overcome this limitation, the OpenAI patent proposes the use of embedded logic bridges, which can extend the distance of 6mm to 16mm. These bridges have two advantages: they not just enable longer communication distances, but they can also provide functionality of a controller for the HBM stack, or provide the functionality of a high-speed PHY for communicating between the chiplets within a package. This D2D (Die-to-Die) interface complies with the UCIe (Universal Chiplet Interconnect Express) standard.
In an example, OpenAI showcases a compute chiplet with 20 HBM memory stacks using the Embedded Logic Bridges. With traditional approaches, you are limited to four, six, or eight stacks. This scales up the memory capacity significantly, leading to chips that are potent for larger AI models.
But this research also goes hand in hand with a technology that someone is already working on. That's Intel's EMIB or Embedded Multi-Interconnect Bridge solution. EMIB is an advanced packaging solution that acts like a bridge.
It is designed to tackle the 2.5D packaging tech, leveraging tiny bridges to expand the capabilities and designs of high-performance chips. EMIB and its successor, EMIB-T, have various advantages, they are simple, they are small, they expand beyond the reticle limits of current interposers, and the are cost effective solutions.
Can we see a future where Intel's EMIB technology gets used by OpenAI for the creation of its custom AI chips with many chiplets and large amounts of HBM memory? Well, this patent seems be pointing towards that.
News Source: SETI Park
About the author: A Software Engineer by training and a PC enthusiast by passion, Hassan Mujtaba serves as Wccftech's Senior Editor for hardware section. With years of experience in the industry, he specializes in deep-dive technical analysis of next-generation CPU and GPU architectures, motherboards, and cooling solutions. His work involves not only breaking news on upcoming technologies but also extensive hands-on reviews and benchmarking.
Follow Wccftech on Google to get more of our news coverage in your feeds.
