Elon Musk has revealed that the next-gen Grok 3 LLM model has been trained on a whopping 100,000 NVIDIA H100 AI GPUs.
Elon Musk's AI Startup Plans To Achieve Massive Breakthrough With Grok LLM Models, Steps Up The Game By Utilizing 100K NVIDIA H100 AI GPUs
Well, it looks like every other AI-focused firm is battling each other to push out advanced versions of their LLM models to cater to the growing need for AI chatbots. A new firm to have joined the race is Elon Musk's AI startup, the xAI, which previously announced the development of the "Grok" set of LLM models, designed specifically for its integration as an AI assistant for premium users on X.
The firm has released newer versions of the original model, and xAI is currently preparing for Grok 2; however, Elon, in his latest tweet, has started to promote the next-gen Grok 3 model, claiming that it's going to be much larger than its predecessors.
Grok 3 end of year after training on 100k H100s should be really something special
— Elon Musk (@elonmusk) July 1, 2024
The disclosure of the hardware used to train Grok 3 is something surprising and exciting at the same time because utilizing 100,000 units of NVIDIA's high-end H100 AI GPUs for model training will produce a result that the markets haven't seen yet.
To back this statement, it is rumored that OpenAI's GPT-4 LLM model was trained on 40,000 of NVIDIA's A100 AI GPUs, which are relatively outdated compared to the H100s and lesser in quantity as well; hence, you can imagine the capabilities Grok 3 AI model will hold within itself.
In a previous tweet, Elon Musk claimed that training LLMs through internet data is hectic and requires a "lot of work," especially when it comes to computing resources. He said that the Grok 2 AI model, which is slated for August, is an improvement in this regard and that the next-generation Grok 3 LLM model will build upon the previous models, ultimately being a massive product by xAI.
Apart from that, Elon Musk previously announced that he plans to acquire NVIDIA's flagship Blackwell B200 AI accelerators for xAI as well, apparently valued at $9 billion, which is a gigantic amount considering the dynamics of the markets. With Grok 3 utilizing 100,000 units of NVIDIA's H100s, the model is supposed to cost around $3 billion in just training, justifying the fact that Elon is maxing out on everything with xAI.
It will be interesting to see how xAI and its ventures contribute to the AI hype, and by the looks of it, Elon's AI startup might be the next big thing.
About the author: Muhammad Zuhair is a hardware and technology reporter for Wccftech, specializing in the semiconductor industry and the complex interplay between technology, manufacturing, and geopolitics. His coverage focuses on the corporate strategies and technological roadmaps of industry giants like TSMC, NVIDIA, Samsung, and Intel. Zuhair's expertise lies in deconstructing complex topics such as fabrication nodes (e.g., 2nm process), the economic impact of policies like the CHIPS Act, and the strategic development of AI infrastructure from NVIDIA, AMD and Intel.
Follow Wccftech on Google to get more of our news coverage in your feeds.
