Facebook's Attempt To Build 'Starcraft II' AI Bot Fails Miserably

Monday, 09 October 2017 - 10:41AM
Science News
Monday, 09 October 2017 - 10:41AM
Facebook's Attempt To Build 'Starcraft II' AI Bot Fails Miserably
< >
Image credit: modified from original by Blizzard Entertainment
Here's a story that should put a smile on the face of anyone who has a soft spot for inspirational underdog sports movies.

Facebook's AI programmers have created a bot named CherryPi, which the company entered in a high profile Starcraft II tournament in an effort to prove their worth against other similar coders. The AAIDE Starcraft AI Competition is an annual battle of wits for programmers around the world, who all seek to create the most successful, advanced bot possible in order to defeat each others and win bragging rights for the coming year.

Essentially, it's like certain other AI robot fighting tournaments, but with a far, far more complicated game.

On paper, Facebook seemed like a pretty big deal. The website's AI division boasts 80 employees, is overseen by a professor at New York University, and has even published several papers on the subject of AI programming for Starcraft. This looked for all the world like a formidable team of programmers with all the knowledge, skill, and resources to make an unbeatable Starcraft bot that could wipe the floor with its opponents.

So how well did CherryPi fare against its rivals? Facebook's AI ultimately came sixth in the overall tournament. There were several bots at the event that outperformed Facebook, and they were all created by amateurs. 

What's particularly fun is that the winner of the entire competition, Australian software developer Chris Coxe, is a single-man team who ultimately isn't all that impressed with his own work, stating:

Opening quote
"It was supposed to be a proof of concept. The source code isn't all that great."
Closing quote

Great at coding, and humble too! Here's the kind of underdog tale we can get behind, wherein a part-time hobbyist has managed to defeat the entire mighty force of Facebook's entire AI team.

While Facebook representatives reporting on the event at the recent AAIDE conference have tried to play this off as a research mission to help them establish a "baseline" for AI competence, it must still be galling to have been beaten by far smaller teams, especially considering the success that other AI initiatives are having around the world.

Some companies are successfully teaching AI to drive using video games, while Elon Musk's OpenAI has managed to beat the world's best human players of Dota 2 in a match setting.

Facebook may not have been (supposedly) trying all that hard to win the AAIDE Starcraft AI Competition, but some good publicity for the company's AI programs wouldn't be unwelcome, especially considering the bad press the AI division has been suffering from as a result of one of their bots going rogue and developing its own incomprehensible language.

For the moment, at least, it seems that a single hobbyist coder can still build a better AI than the best talent Facebook can put together. There's still a lot of work to go into teaching robots everything there is about gaming (including how to program games themselves).

This just goes to show that no matter how big and important a major AI laboratory may seem, it doesn't necessarily mean that the underdog can't still prevail.
Artificial Intelligence
Science News