Google DeepMind's SIMA Is an AI Agent That Can Play 3D Video Games

Google DeepMind collaborated with eight game studios to train and test SIMA.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 14 March 2024 14:14 IST

Google DeepMind's SIMA Is an AI Agent That Can Play 3D Video Games

Photo Credit: Google DeepMind

Google DeepMind’s SIMA follows natural-language instructions to carry out tasks

Highlights

SIMA was evaluated across 600 basic skills such as navigation, and more
SIMA plays video games to learn how to interact with the game environment
Google recently introduced Genie, an AI 2D platformer game generator

Google DeepMind introduced a unique artificial intelligence (AI) model on Wednesday that can play 3D video games just like a human would. The AI model is called Scalable Instructable Multiworld Agent or SIMA, and it is learning how to interact with various gaming environments and carry out different tasks. The model is currently in research and is still being trained to build expertise in performing more complex movements. Google says that once perfected, the AI model can have a wide range of use cases in both the online and real world.

In a blog post, DeepMind's SIMA team explained that the AI model is not being developed to be a super-intelligent gamer that can beat any game. Instead, the aim is to teach it how to move around in a 3D game in open-world situations and understand how a human would react and interact with it using natural-language instructions. Google highlights that it is a significantly difficult task and one that should be a focus.

“This is an important goal for AI in general, because while Large Language Models have given rise to powerful systems that can capture knowledge about the world and generate plans, they currently lack the ability to take actions on our behalf,” the SIMA team stated.

OpenAI’s AI Video Generator Sora to Be Available Later This Year: Report

Tasks performed by AI model SIMA
Photo Credit: Google DeepMind

To create a learning environment for the AI model, Google DeepMind partnered with eight game studios and trained SIMA on nine different video games. Some of them include No Man's Sky by Hello Games, Teardown by Tuxedo Labs, Goat Simulator 3 and Valheim by Coffee Stain Studios, and more. The AI model was exposed to new interactive worlds in each game and had to learn navigating through the world, interaction with objects, menu use, and more.

Devin, AI Software Engineer That Can Write Codes and Build Apps, Unveiled

The company also created four research environments including a new environment built with Unity called the Construction Lab. In this particular environment, the AI model was tasked to build sculptures from building blocks to test its object manipulation and intuitive understanding of the physical world.

Google highlighted that the current version of SIMA was evaluated across 600 basic skills including navigation such as turning left and driving a car, object interaction such as climbing a ladder and crafting a helmet, and more. The tasks were largely simple and could be completed within 10 seconds.

You Can Now Adjust the Brightness of Your PS5's Power Indicator

According to the company, training an AI model on a diverse set of 3D video games where it follows human instructions and acts accordingly can have a major impact in future. The company is now focusing on training SIMA with more complex instructions “that require high-level strategic planning and multiple sub-tasks to complete, such as find resources and build a camp.” Google hints that these tasks can translate to the real world and the company is working on finding ways it can be helpful to humans.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Google DeepMind, Artificial intelligence, AI, Google

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More