It has only been a couple of months since Google launched Gemini 1.0. Now, the company has announced Gemini 1.5, the next-generation model that has "dramatically enhanced performance." Google claims that one of the best things about the new update is that it will offer a much larger context window, so if you are like me and like to use the chatbot, then this is certainly something you will be interested in.
Gemini 1.5's prowess shows that Google is not messing around in the race to become one of the best companies with artificial intelligence
Here is what Google has to say about Gemini 1.5:
An AI model’s “context window” is made up of tokens, which are the building blocks used for processing information. Tokens can be entire parts or subsections of words, images, videos, audio or code. The bigger a model’s context window, the more information it can take in and process in a given prompt — making its output more consistent, relevant and useful.
It is also worth noting that Gemini 1.5 Pro is still Google's mid-tier offering, and its standard context window is 128,000 tokens. For the sake of comparison, the older version has 32,000 tokens. The new update is capable of translating over 700,000 words, as well as codebases with over 30,000 lines of code, 11 hours of audio, and 1 hour of video. GPT-4 turbo also offers 128,00 tokens. Comparatively, we have Claude 2.1, which offers 200,000 tokens. If you are wondering how the new update will work, here is an explanation Google provided:
“1.5 Pro can seamlessly analyze, classify and summarize large amounts of content within a given prompt. For example, when given the 402-page transcripts from Apollo 11’s mission to the moon, it can reason about conversations, events and details found across the document.”
In addition to that, here is more that Gemini 1.5 Pro is capable of:
“1.5 Pro can perform highly-sophisticated understanding and reasoning tasks for different modalities, including video. For instance, when given a 44-minute silent Buster Keaton movie, the model can accurately analyze various plot points and events, and even reason about small details in the movie that could easily be missed.”
Moving further, Gemini 15 Pro:
can perform more relevant problem-solving tasks across longer blocks of code. When given a prompt with more than 100,000 lines of code, it can better reason across examples, suggest helpful modifications and give explanations about how different parts of the code works.”
Another thing to note here is that Google is running up to 1 million tokens in production, which will be available to early testers. At the same time, the company has successfully tested over 10 million tokens. This shows that Gemini 1.5 and future updates are not going to slow down.
Google claims that all these Gemini 1.5 advancements are made possibly using the new Mixture-of-Experts MoE architecture where models are “divided into smaller ‘expert’ neural networks.” And thanks to this, the new update is more efficient to train as well as to serve.
Depending on the type of input given, MoE models learn to selectively activate only the most relevant expert pathways in its neural network. This specialization massively enhances the model’s efficiency.
Regarding performance, Google claims that the Gemini 1.5 Pro can outperform the older version on 87% of the benchmarks, spanning across various fields, including text, code, image, audio, and video evaluations. Google further claims that, at the moment, the performance is comparable to the 1.0 Ultra:
Gemini 1.5 Pro also shows impressive “in-context learning” skills, meaning that it can learn a new skill from information given in a long prompt, without needing additional fine-tuning.
Sadly, you are going to have to wait to get started with the Gemini 1.5 Pro, as Google states that the larger context window is currently rolling out in a limited capacity to the developers and enterprise customers. In its current stage, it is still experimental, so you might have to wait a bit longer before it is available to everyone.
Coming soon, we plan to introduce pricing tiers that start at the standard 128,000 context window and scale up to 1 million tokens, as we improve the model.
News Source: Google Blog
Follow Wccftech on Google to get more of our news coverage in your feeds.
