Well, first it was GPUs, then it was memory, and now it is CPUs, which are being affected by severe shortages due to the Agentic AI boom.
Amazon & Cloud Providers Have Ran Out of CPUs As Agnetic AI Demand Enters Frenzy
As Agentic AI takes the tech markets by storm, it is also becoming a nuisance for cloud providers and chipmakers who are failing to meet the growing demand at the very end. The market is yet to figure its way out of one shortage, and we are already looking at shortages related to a new tech component, & that's CPUs.
Reported by Dylan Patel of Semianalysis, the latest is that GPUs are no longer the constraint for cloud providers, as that role is shifted to CPUs now. Previously, GPUs doing AI were doing simple inferencing tasks, and these evolved as new models landed.
But with the surge of models, and the needs growing at an exponential rate, Agentic AI is now being used to call databases and also for interfacing with some tasks that rely heavily on CPUs, such as Physics and Simulation. So these calls being made to databases and more CPU-heavy tasks have led to a sharp increase in CPU use at cloud data centers.
While GPUs maintained the dominant role in cloud servers serving the AI populace, for example, 8 GPUs running on a single CPU per rack, this difference is now being reduced & both GPUs/CPUs share a similar ratio in servers.
This huge demand has already caused instability in databases such as GitHub. Several users are reporting downtimes, and most of the time, it fails to commit.
Yeah, so we've been like checking GitHub's like stats on like how often is it down, how often does it fail to commit, like, you know, whatever, right? It's terrible.And that's because Microsoft sold all their CPUs that they had spare to other people, right? Either internal use for their lab, but, you know, not really, more so like external labs that sign deals with Entropic and OpenAI.
And so they just have like no CPUs left, right? And we've seen, you know, the same at many other firms, right? So whereas before you'd have like, you know, many GPU servers per CPU server. And so you'd be like, you know, 100 megawatts of GPUs would be served by even like one megawatt or less of CPUs.Nowadays, the ratio is like getting much, much closer, both for RL training and for inference, agentic inference. So then you've just seen everyone run out of CPUs. Amazon's volumes on CPUs.
Dylan Patel (Semianalysis)
The reason behind this is reportedly that the CPU demand is so high that cloud providers have exhausted their entire CPU supply. Dylan states that Amazon & Microsoft have sold out their entire CPU supply to AI firms such as OpenAI and Entropic. The thing is so bad that despite Amazon tripling its CPU servers year over year, they have run out of supply and are now failing to meet future demands.
Furthermore, Arm CPUs have been hit worse due to OpenAI's switch from x86 to Arm. This was done as Amazon had more CPU resources open before the shortages, and with the x86 supply being lower in the past, firms ported their codebase from x86 to Arm. That move has now hit AI firms hard.
So, where does that lead the tech market to? The answer is a huge and imminent CPU shortage. The shortages of cloud CPUs will result in other vendors dialing up their production to meet demand.
This won't just be limited to ARM chips, but x86 too. AMD and Intel supply a huge portion of their x86 chips to cloud providers, and NVIDIA is also ramping up its Vera CPU racks, consisting of multiple chips and lots of DRAM. Yes, so the DRAM supply will still be heavily focused on AI, and now, CPUs will also be heavily focused on AI. This means that production lines dedicated to consumer and business segments can be adversely affected as CPUs are prioritized for the AI segment. This means supplies will be short, prices will be high, and whoever pays the most will get the chips first.
About the author: A Software Engineer by training and a PC enthusiast by passion, Hassan Mujtaba serves as Wccftech's Senior Editor for hardware section. With years of experience in the industry, he specializes in deep-dive technical analysis of next-generation CPU and GPU architectures, motherboards, and cooling solutions. His work involves not only breaking news on upcoming technologies but also extensive hands-on reviews and benchmarking.
Follow Wccftech on Google to get more of our news coverage in your feeds.
