DeepMind AI Can Play Quake III Arena Capture the Flag Better Than a Human

DeepMind, part of the Alphabet group after being acquired by Google in 2014, is focused on artificial intelligence. On Tuesday, it announced that it had taught AI programs to learn how to team up efficiently with other AI programs as well as humans in a video game, at human-level. Researchers at the company have taught an AI agent to play a customised version of Quake III Arena like a human. Interestingly, the trained AI is now better than most human players in the game, even when it is playing with a human. The trained AI agents were found to have a higher win-rate (Elo rating)and be more collaborative than humans. With such AI-based research, the video gaming space is expanding the scope for NPC or non-player characters to do better and work more effectively.

As per the results of research and experiments shared by DeepMind, several AI systems were trained to play 'Capture the Flag' on Quake III Arena, which is a multiplayer first-person shooter game. An AI agent based on the For The Win (FTW) architecture was made to play roughly 450,000 rounds of the game to eventually gain its dominance over most human players and also to understand how to effectively work with other machines and humans. The teamwork training, according to DeepMind, is being called as multi-agent learning.

Mastering strategies, tactical understanding, and team play involved in multiplayer video games is an important part of AI research. DeepMind, in the blog post, says, "We train agents that learn and act as individuals, but which must be able to play on teams with and against any other agents, artificial or human." It added, "From a multi-agent perspective, CTF requires players to both successfully cooperate with their teammates as well as compete with the opposing team, while remaining robust to any playing style they might encounter."

The reason for choosing Quake III Arena for the experiment, DeepMind says, was that the game has laid the foundations for several modern first-person video games, and has also "attracted a long-standing competitive e-sports scene."

Such experiments, DeepMind says, are based on three general ideas for reinforcement learning. Firstly, instead of training a single agent, the researchers trained a population of agents. Each agent then learns its own internal reward signal that enables them to generate their own internal goals, such as capturing a flag. Finally, these agents operate at two timescales - fast and slow - and that enhances their ability to use memory and generate consistent action sequences.

In a tournament with and against 40 human players, the machine-only teams had a high chance of winning against human-only teams. Interestingly, it also had a very high chance of winning against human-machine teams. A survey of human participants found that the FTW agent was more collaborative than human teammates. What is even more interestingly, is that the rules of the game were not given to machines before hand, yet over time, FTW learned most of the basic strategies "to a very high standard."

While machines have beaten humans at games before, each new game offers newer challenges for AI to solve. To recall, last year DeepMind drew headlines by creating an AI system AlphaGo Zero that defeated the world champion of the game Go. Recently, OpenAI has announced that it will take on a Pro Dota 2 Team at The International, one of the world's most popular video game tournaments. Also, last year, DeepMind had released a set of tools along with Blizzard Entertainment to accelerate AI research with help from the real-time strategy game Starcraft.