Google DeepMind Unveils Gemini Robotics AI Models That Can Control Robots in the Real World

Google DeepMind unveiled two new artificial intelligence (AI) models on Thursday, which can control robots to make them perform a wide range of tasks in real-world environments. Dubbed Gemini Robotics and Gemini Robotics-ER (embodied reasoning), these are advanced vision language models capable of displaying spatial intelligence and performing actions. The Mountain View-based tech giant also revealed that it is partnering with Apptronik to build Gemini 2.0-powered humanoid robots. The company is also testing these models to evaluate them further, and understand how to make them better.

Google DeepMind Unveils Gemini Robotics AI Models

In a blog post, DeepMind detailed the new AI models for robots. Carolina Parada, the Senior Director and Head of Robotics at Google DeepMind, said that for AI to be helpful to people in the physical world, they would have to demonstrate “embodied” reasoning — the ability to interact and understand the physical world and perform actions to complete tasks.

Gemini Robotics, the first of the two AI models, is an advanced vision-language-action (VLA) model which was built using the Gemini 2.0 model. It has a new output modality of “physical actions” which allows the model to directly control robots.

DeepMind highlighted that to be useful in the physical world, AI models for robotics require three key capabilities — generality, interactivity, and dexterity. Generality refers to a model's ability to adapt to different situations. Gemini Robotics is “adept at dealing with new objects, diverse instructions, and new environments,” claimed the company. Based on internal testing, the researchers found the AI model more than doubles the performance on a comprehensive generalisation benchmark.

The AI model's interactivity is built on the foundation of Gemini 2.0, and it can understand and respond to commands phrased in everyday, conversational language and different languages. Google claimed that the model also continuously monitors its surroundings, detects changes to the environment or instructions, and adjusts its actions based on the input.

Finally, DeepMind claimed that Gemini Robotics can perform extremely complex, multi-step tasks that require precise manipulation of the physical environment. The researchers said the AI model can control robots to fold a piece of paper or pack a snack into a bag.

The second AI model, Gemini Robotics-ER, is also a vision language model but it focuses on spatial reasoning. Drawing from Gemini 2.0's coding and 3D detection, the AI model is said to display the ability to understand the right moves to manipulate an object in the real world. Highlighting an example, Parada said when the model was shown a coffee mug, it was able to generate a command for a two-finger grasp to pick it up by the handle along a safe trajectory.

The AI model performs a large number of steps necessary to control a robot in the physical world, including perception, state estimation, spatial understanding, planning, and code generation. Notably, neither of the two AI models is currently available in the public domain. DeepMind will likely first integrate the AI model into a humanoid robot and evaluate its capabilities, before releasing the technology.