Artificial intelligence that’s beastly at World of Warcraft might not lie too far into the distant future, if OpenAI has its way. The San Francisco research nonprofit today released Neural MMO, a “massively multiagent” virtual training ground that plops agents in the middle of an RPG-like world — one complete with a resource collection mechanic and player versus player combat.
“The game genre of Massively Multiplayer Online Games (MMOs) simulates a large ecosystem of a variable number of players competing in persistent and extensive environments,” OpenAI wrote in a blog post. “The inclusion of many agents and species leads to better exploration, divergent niche formation, and greater overall competence.”
AI agents spawn randomly in Neural MMO environments, which contain automatically generated tile maps of a prespecified size. Some tiles are traversable, like “forest” (which bears food) and “grass,” while others aren’t (such as water and stone). Agents observe the square crops of tiles centered on their respective positions and make one movement and one attack per timestamp (or tick), tackling tasks like foraging for limited “food” and “water” resources (by stepping on forest tiles or next to water tiles) and engaging in combat (“melee,” “range,” and “mage”) with other agents.
OpenAI used Neural MMO to train an AI system by rewarding agents for their lifetime — i.e., how long they managed to stay alive — and found that the longer the agents interacted with each other, the better they became at certain tasks, and that increasing the maximum number of concurrent agents “magnified” their exploration. Intriguingly, they also found that increasing the agents’ population size prompted them to spread out within different parts of the map and that agents trained in larger settings “consistently” outperformed those trained in smaller settings.
“As entities cannot out-compete other agents of their own population (i.e. agents with whom they share weights), they tend to seek areas of the map that contain enough resources to sustain their population,” OpenAI wrote. “In the natural world, competition among animals can incentivize them to spread out to avoid conflict. We observe that map coverage increases as the number of concurrent agents increases. Agents learn to explore only because the presence of other agents provides a natural incentive for doing so.”
Neural MMO, available on GitHub, is designed to support a large number of agents (up to 128 in each of 100 concurrent servers). It packs in baselines (trained on over 100 worlds) against which the agents’ performance can be compared, and the computational overhead is relatively low — training only requires a single desktop CPU.
It’s far from the first of its kind, it’s worth noting. In December, OpenAI released CoinRun, a classic platformer designed to measure agents’ ability to transfer their experiences to unfamiliar scenarios. And in August, researchers at the University of Adger in Norway open-sourced an environment for AI training in real-time strategy games.
Beyond simulated learning environments, data scientists have set loose AI on Starcraft II, Montezuma’s Revenge, Dota 2, Quake III, and other games, all in pursuit of systems that might one day diagnose illnesses, predict complicated protein structures, and segment CT scans. “The reason we test ourselves and all these games is … that [they’re] a very convenient proving ground for us to develop our algorithms,” DeepMind cofounder Demis Hassabis told VentureBeat in a recent interview. “Ultimately, [we’re developing algorithms that can be] translate[ed] into the real world to work on really challenging problems … and help experts in those areas.”