Here Is Why OpenAI Is Much More Likely to Release GPT-4.5 This Year Instead of GPT-5

This is not investment advice. The author has no position in any of the stocks mentioned. Wccftech.com has a disclosure and ethics policy.

Although GPT-4 is currently at the apex of the increasingly convoluted world of generative artificial intelligence, its competitors, including Anthropic's Claude and Meta's open-source Llama, keep getting better, necessitating another iteration of OpenAI's flagship Large Language Model (LLM). While many expect Sam Altman's non-profit to release GPT-5 in 2024, some analysts are now asserting that those expectations remain far-fetched, especially given the scale of resources required.

GPT-5 Would Need Around ~100x the Computational Resources of GPT-4, or 3 Months of ~1 Million H100s

GPT-5 doesn't seem likely to be released this year.
Ever since GPT-1, the difference between GPT-n and GPT-n+0.5 is ~10x in compute.
That would mean GPT-5 would have around ~100x the compute GPT-4, or 3 months of ~1 million H100s.
I doubt OpenAI has a 1 million GPU server ready. https://t.co/asJWNLkO23

— Dan Hendrycks (@DanHendrycks) April 22, 2024

According to Dan Hendrycks, the director of the Center for AI Safety, each incremental iteration of OpenAI's GPT LLM has required a 10x increase in computational resources. Consequently, if OpenAI were to skip GPT-4.5 and directly jump to GPT-5, it would translate into around a 100x increase in computational requirements relative to GPT-4, equivalent to around 1 million H100 chips running for three straight months.

Sam said they will release an amazing model this year, but he doesn't know what it will be called. Dario said the ones training right now are $1 billion runs. Which would kind of line up with GPT-4.5 being 10x more compute than GPT-4.https://t.co/Nyvv3j2uri

— MachDiamonds (@andromeda74356) April 22, 2024

This thesis is supported by the comments from Anthropic's CEO, Dario Amodei, who recently noted that it currently costs around $1 billion to train a state-of-the-art LLM, with this cost expected to balloon to between $5 billion and $10 billion by 2025/26. Crucially, $1 billion in training costs aligns with the 10x jump in computational resources that can be reasonably inferred for GPT-4.5.

We noted recently that NVIDIA's H100 units deployed this year are expected to consume around 13,000 GWh of electricity annually, equivalent to the annual electricity consumption of countries such as Lithuania and Guatemala. By 2027, the global power consumption of data centers is expected to rocket to between 85 and 134 TWh (terawatt-hours)!

Llama3-70B has settled at #5. With 405B still to come next...

I remember when GPT-4 released in March 2023, it looked like it was nearly-impossible to get to the same performance.

Since then, I've seen @Ahmad_Al_Dahle and the rest of the GenAI org in a chaotic rise to focus,… https://t.co/xUMHhW8sIX

— Soumith Chintala (@soumithchintala) April 22, 2024

Of course, GPT-4's competition is rapidly catching up. Look no further than Meta's Llama 3 LLM (70 billion parameters), which now ranks fifth on the Arena leadership board. Critically, Llama 3 is now outperforming all other open-source LLMs, and that's in the absence of the upcoming 405-billion parameter model.

I suspect that GPT-5 will be delayed more than the lack of GPU resources. It's the realization after GPT-4's release of the need to change the original curriculum. Ingesting poorly curated human conversations and a naive training curriculum were low-hanging fruit. Most others… https://t.co/7QO7Zykk2e

— Carlos E. Perez (@IntuitMachine) April 22, 2024

What's more, some experts now believe that for GPT-5, OpenAI will have to change the "original curriculum," which currently involves leveraging "poorly curated human conversations" and an overall "naive" training process. This appends with our original thesis that OpenAI is likely to release an iterative GPT-4.5 model this year instead of upending the stakes altogether with GPT-5.

Follow Wccftech on Google to get more of our news coverage in your feeds.