Not a member yet? Register for full benefits!

A Turing Test of Actions, Not Words

The traditional Turing Test, like the Loebner Prize based on it, involves an artificial intelligence chatbot, and a human user in front of a computer. A panel of judges are presented in random order the chatbot, interacting over a local area connection, and the human interacting over a local area connection. The judges' task is to tell which is the human and which is the AI. As far as the judges know, they may both be human, or both be AI; they have to determine what each is.

So far, the Loebner Prize has never been won.

However, on the 26th of September 2012, on the centenary of inventor of the test, mathematician Alan Turning's birth, a different form of Turing Test was carried out and succeeded.

UT^2 game bot faces off against an opponent

In this case, it was a game-bot, an AI embodied in a virtual avatar form. It convinced a panel of judges that it was more human-like than half the humans it competed against.

The setting was the 2012 IEEE conference on Computational Intelligence and Games, whilst the environment used was the virtual world of "Unreal Tournament 2004," a first-person shooter gameworld. Like most FPS games, it relies on quick thinking, good reflexes and the ability to make tactical choices and keep a map in mind of where other players are going to be based on their prior observed behaviour and what you have learnt about the layout.

"The idea is to evaluate how we can make game bots, which are nonplayer characters (NPCs) controlled by AI algorithms, appear as human as possible," said Risto Miikkulainen, professor of computer science in the College of Natural Sciences at The University of Texas at Austin. Miikkulainen created the winning bot, called the UT^2 game bot, with doctoral students Jacob Schrum and Igor Karpov.

The bots face off in a tournament against one another and about an equal number of humans, with each player trying to score points by eliminating its opponents. Each player also has a "judging gun" in addition to its usual complement of weapons. That gun is used to tag opponents as human or bot.

UT^2 killing a human opponent who supposedly had the upper hand

"When this 'Turing test for game bots' competition was started, the goal was 50 percent humanness," said Miikkulainen. "It took us five years to get there, but that level was finally reached last week, and it's not a fluke."

The complex gameplay and 3D environments of "Unreal Tournament 2004" require that bots mimic humans in a number of ways, including moving around in 3-D space, engaging in chaotic combat against multiple opponents and reasoning about the best strategy at any given point in the game. Even displays of distinctively human irrational behaviour can, in some cases, be emulated.

"People tend to tenaciously pursue specific opponents without regard for optimality," said Schrum. "When humans have a grudge, they'll chase after an enemy even when it's not in their interests. We can mimic that behaviour."

In order to most convincingly mimic as much of the range of human behaviour as possible, the team takes a two-pronged approach. Some behaviour is modelled directly on previously observed human behaviour, while the central battle behaviours are developed through a process called neuroevolution, which runs artificially intelligent neural networks through a survival-of-the-fittest gauntlet that is modelled on the biological process of evolution.

Networks that thrive in a given environment are kept, and the less fit are thrown away. The holes in the population are filled by copies of the fit ones and by their "offspring," which are created by randomly modifying (mutating) the survivors. The simulation is run for as many generations as are necessary for networks to emerge that have evolved the desired behaviour.

"In the case of the BotPrize," said Schrum, "a great deal of the challenge is in defining what 'human-like' is, and then setting constraints upon the neural networks so that they evolve toward that behaviour.

"If we just set the goal as eliminating one's enemies, a bot will evolve toward having perfect aim, which is not very human-like. So we impose constraints on the bot's aim, such that rapid movements and long distances decrease accuracy. By evolving for good performance under such behavioural constraints, the bot's skill is optimized within human limitations, resulting in behaviour that is good but still human-like."

Miikkulainen said that methods developed for the BotPrize competition should eventually be useful not just in developing games that are more entertaining, but also in creating virtual training environments that are more realistic, and even in building robots that interact with humans in more pleasant and effective ways.

In other words, they are a good step towards creating intelligent NPCs for all virtual enironments, including non-gaming.


Artificially intelligent game bots pass the Turing test on Turing's centenary

Unreal Tournament 2004

2012 IEEE conference on Computational Intelligence and Games

Loebner Prize


Staff Comments


Untitled Document .