Google Deepmind has released a new generation of their AI model, Gemini, with a new advanced version 1.5.

The updated model claims to offer a ‘breakthrough’ in long-context understanding. Essentially, the model can process more information than before, running up to 1 million tokens consistently, which Google says is the longest context window of any large-scale foundation model out so far.

Gemini 1.5 is supposed to offer “dramatic improvements” in various dimensions, Sundar Pichai, the CEO of Google and Alphabet said.

The first model to be released for early testing, Gemini 1.5 Pro, is a mid-size multimodal model, which Google claims works at a similar level as their largest current model, 1.0 Ultra, with less computing power.

While the standard Gemini 1.5 Pro will come with a 128,000 token context window, a limited group of developers and enterprise customers will have access to a context window of up to 1 million tokens.

While the 1m token model is rolled out, Google says they will continue to work on optimising its features and latency, as well as attempting to continually reduce its computing requirements.

It is currently unclear what the time scale of the rollout will look like, but Google’s demonstration of Gemini 1.5 Pro’s power was promising.

Recommended reading

• Hackers: AI Unlikely to Replace Human Cybersecurity Skills
• How Much Have Blockchain Hackers Stolen This Year?
• ChatGPT, How Do Cyber-criminals Really Feel About You?

It was able to summarise and pinpoint detail in the entire Apollo 11 transcript – totalling 402 pages – and even locate humour. Further, it was able to find and discern scenes in the a 44-minute silent film.

Google has credited these advancements to the model’s use of mixture-of-experts architecture – this method divides the neural network AI uses to process information into separate pieces, and only activates the parts relevant to a task rather than using the entire neural network for every task. Google Deepmind is not the only large language model to employ this design, however.

The model is currently only available as a limited preview to developers and enterprise customers via Google’s AI Studio and Vertex AI.