Search

Microsoft Muse AI Model Unveiled; Can Generate Game Visuals and Controller Actions

Microsoft has published a paper on the World and Human Action Model (WHAM), also known as Muse, in the Nature journal.

Advertisement
Highlights
  • The Muse AI model can understand 3D game worlds and game physics
  • Microsoft said it can analyse how players interact with a game
  • Muse was trained on human gameplay data from the Xbox game Bleeding Edge
Microsoft Muse AI Model Unveiled; Can Generate Game Visuals and Controller Actions

The Muse AI model was developed by Microsoft researchers and Xbox Games Studios’ Ninja Theory

Photo Credit: Microsoft

Microsoft researchers introduced a new artificial intelligence (AI) model on Wednesday that can generate 3D gameplay environments. Dubbed the World and Human Action Model (WHAM) or Muse, the new AI model was developed by the tech giant's Research Game Intelligence and Teachable AI Experiences (Tai X) teams in collaboration with Xbox Games Studios' Ninja Theory. The company said that the large language model (LLM) can help game designers in the ideation process, as well as help generate game visuals and controller actions to help creatives in game development.

Microsoft Unveils Muse AI Model

In a blog post, the Redmond-based tech giant detailed the Muse AI model. This is a research product currently, although the company said that it is open-sourcing the weights and sample data of the model for the WHAM Demonstrator (a concept prototype of a visual interface to interact with the AI model). Developers can try out the model on Azure AI Foundry. A paper detailing the technical aspects of the model is published in the Nature journal.

To train a model on such a complex area is a difficult proposition. Microsoft researchers collected a large amount of human gameplay data from the 2020 game Bleeding Edge, a game published by Ninja Theory. The LLM was trained on a billion image action pairs, which is equivalent to seven years of human gameplay. The data is said to be collected ethically and is used only for research purposes.

The researchers said that scaling up the model training was a major challenge. Initially, Muse was trained on a cluster of Nvidia V100 GPUs, but then it was scaled to multiple Nvidia H100 GPUs.

Coming to the functionality, the Muse AI model accepts text prompts as well as visual inputs. Additionally, once a game environment is generated, it can be further enhanced using controller actions. The AI responds to the movements made by the user to render new environments aligned with the initial prompt, and consistent with the rest of the gameplay.

Due to being a unique AI model, typical benchmark tests cannot properly evaluate its capabilities. The researchers highlighted that they have internally tested the LLM on metrics such as consistency, diversity, and persistence. Since it is a research-focused model, the outputs have been limited to just 300x180p resolution.

Affiliate links may be automatically generated - see our ethics statement for details.

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

 
Show Full Article
Please wait...
Advertisement

Related Stories

Popular Mobile Brands
  1. Xiaomi 15s Pro Design, Camera Details Teased Ahead of Launch Today
  2. Sam Altman Reportedly Drops Clues About 'Secret' AI Device With Jony Ive
  3. Vi Rolls Out 'Nonstop Hero' Plan With Truly Unlimited Data and Calls
  4. LinkedIn Is Now Letting Users Search for Ideal Jobs Using GenAI
  5. Tecno Pova Curve 5G India Launch Date Announced
  6. Mistral's Coding Agent Devstral Outperforms OpenAI's GPT-4.1 Mini
  1. Motorola Razr 60 Set to Launch in India Next Week; Specifications Revealed
  2. Signal’s New Update Prohibits Microsoft’s AI-Powered Recall Feature From Taking Screenshots
  3. iPhone 7 Plus and iPhone 8 Added to Apple's Vintage and Obsolete Products List
  4. Tecno Pova Curve 5G India Launch Date Confirmed; to Be Available on Flipkart
  5. Vodafone Idea (Vi) Rolls Out ‘Nonstop Hero’ Plan With Truly Unlimited Data, Calls in Kolkata and Other Circles
  6. OpenAI’s Sam Altman Reportedly Hints at New AI Device Being Developed With Jony Ive
  7. Vodafone Idea Updates Family Postpaid Plans to Let Users Add Up to 8 Additional Members
  8. Amazon Begins Testing AI-Powered Audio Product Summaries Feature on Its Platform
  9. Mistral Releases Devstral, an Open-Source Agentic Coding AI Model That Outperforms GPT-4.1 Mini
  10. Sony to Fully Shut Down PlayStation Stars Loyalty Program Next Year
Gadgets 360 is available in
Download Our Apps
App Store App Store
Available in Hindi
App Store
© Copyright Red Pixels Ventures Limited 2025. All rights reserved.
Trending Products »
Latest Tech News »