As memory continues to dominate the AI hardware conversation, a research note from Korea Investment & Securities (KIS) outlines that prices might not drop. Since memory is directly tied to the productivity of an AI GPU, hyperscalers have placed long-term orders for capacity. KIS believes that even if this time period elapses, the impact of the shortage on memory prices might persist due to the chips' ability to drive key metrics of AI GPU performance.
Memory Chips' Ability To Drive Up System Level Performance Could Keep Prices High, Says Research Firm
The research note explains that since expanding memory capacity increases a GPU's utilization, it ends up making a GPU more cost efficient since more tokens can be processed. Therefore, AI companies have ordered more memory chips in order to beef up efficiency and reduce the cost per token processed. Since more memory enables more tokens that lie in close proximity to a GPU, the GPU's ability to process more tokens is increased since they do not have to be 'ferried in' from storage devices.
According to KIS, not only has the rise in efficiency led to the hyperscalers placing orders for long term capacity, but this trend might continue in the future as well.
As a result, DRAM memory prices, which are currently three times higher annually might stay higher even though the market expects either them or the demand to drop, explains the research firm. KIS adds that the market's viewpoint might be flawed since even though the prices are higher, the payoff from the higher utilization of the GPUs is higher as well since more memory enables greater system level performance rather than per chip performance.
The firm adds that high demand for HBM and DRAM memory modules and the resulting tight capacity is also leading to higher demand for NAND chips. This demand goes against the belief that higher demand for NAND chips might ease the pressure in the DRAM market since firms are working towards integrating NAND into AI systems. Another factor supporting sustained high NAND demand is the lower price compared to DRAM which allows for more elasticity in case of even extremely high demand.
Tight HBM capacity and AI hyperscalers' demand is also driving innovation in the sector with firms such as SK hynix seeking to increase the number of memory dies in a single chip through techniques such as hybrid bonding to remove copper bumps from the package.
About the author: Ramish is a seasoned technology writer and editor with more than a decade of experience. He specializes in semiconductor fabrication and market analysis. With a background in finance and supply chain management - via his bachelors in Finance and a micromasters in supply chain management from MIT - Ramish combines financial rigor with deep industry insight to deliver accurate and authoritative coverage.
Follow Wccftech on Google to get more of our news coverage in your feeds.
