Amazon's CEO, Andy Jassy, issued a letter to shareholders about the company's custom chip business, and his words showed strong optimism about the in-house infrastructure stack.
Amazon's Andy Jassy Hints Towards Selling Their Custom Silicon To External Customers, Given Their Internal Success
There's no doubt that hyperscalers are experiencing what we call a 'compute crunch' today, with current infrastructure unable to meet the demand driven by AI and related services. We have seen giants like Amazon, Google, and Meta resort to custom silicon efforts as a way to address this gap in compute capabilities, but out of the bunch, Amazon's Trainium chips and Graviton server CPUs have proven themselves to be a mature offering, which is why CEO Andy Jassy has shown immense confidence in them. While speaking with shareholders, Jassy says their custom chip business is scaling to achieve $50 billion in ARR, surprisingly higher than what AMD and Intel have achieved.
Having our own hotly demanded AI chip opens up many possibilities, but perhaps none larger than the ability to lower costs for customers and secure better economics for AWS. At scale, we expect Trainium will save us tens of billions of capex dollars per year, and provide several hundred basis points of operating margin advantage versus relying on others’ chips for inference.
- Amazon's CEO
The $50 billion ARR figure is an estimate of what Amazon's custom chip business would be if it were to spin off into a separate entity and become a compute provider like NVIDIA. The growth of Trainium and Graviton primarily comes from AWS's cloud services, and Amazon still hasn't offered up its custom silicon to those who need compute. Jassy says that the economics brought in by AWS and the ASIC option are superior to NVIDIA's, and while he iterates his commitment to NVIDIA, he adds a note that customers want an option that dominates "price-performance", and he believes Trainium serves this a lot better.
Amazon's CEO also throws shade at Intel and its CPU market share, saying that after the debut of the ARM-based Graviton chips, AWS's infrastructure is now dominated by them. Jassy says a similar situation is playing out between training and inference with Trainium. There's an interesting thesis with his statement, one that the industry probably doesn't realize. The idea of custom infrastructure isn't to replace mainstream compute options, such as those from NVIDIA, but rather to address the compute gap so large that hyperscalers like Amazon cannot rely solely on GPU manufacturers.
Interestingly, Amazon also hinted at selling its racks to third-party customers, suggesting it could very well compete directly with NVIDIA and others. Backed by the 'hundreds of billions' in CapEx the firm plans to invest into the business, it would be interesting to see how the Trainium + Graviton stack evolves moving ahead.
About the author: Muhammad Zuhair is a hardware and technology reporter for Wccftech, specializing in the semiconductor industry and the complex interplay between technology, manufacturing, and geopolitics. His coverage focuses on the corporate strategies and technological roadmaps of industry giants like TSMC, NVIDIA, Samsung, and Intel. Zuhair's expertise lies in deconstructing complex topics such as fabrication nodes (e.g., 2nm process), the economic impact of policies like the CHIPS Act, and the strategic development of AI infrastructure from NVIDIA, AMD and Intel.
Follow Wccftech on Google to get more of our news coverage in your feeds.
