Man Trains for Weeks to Beat Google's AI at Image Recognition - Just Barely

Thursday, 19 February 2015 - 3:00PM
Technology
Thursday, 19 February 2015 - 3:00PM
Man Trains for Weeks to Beat Google's AI at Image Recognition - Just Barely

It looks like alarmists about artificial superintelligence might not be that far off. Stanford graduate student Andrej Karpathy went up against Google's most advanced image recognition AI and lost, until he spent weeks of his life training for the test and finally beat the machine- by less than two percent.

 

It all started in 2011, when Stanford researchers built the most sophisticated image recognition software up to that point. Karpathy took the same test as the AI, in which they both classified 50,000 images into one of ten relatively simple categories such as "dog," "horse," and "truck." The machine classified the images correctly approximately 80% of the time, while Karpathy had an accuracy level of 94%.

 

"[I]t will be hard [for AI] to go above 80 percent," Karpathy wrote in a blog post at the time, "but I suspect improvements might be possible up to range of about 85-90 percent."

 

That prediction turned out to be completely erroneous, as Google's new ImageNet performed much better on a similar test, at an accuracy rate of 93.4. Karpathy and his colleagues, in comparison, scored an average of a paltry 85%.

 

To be fair, the test was much more complex in certain ways; for example, it expected the test-takers to know 200 different dog breeds, and it's more reasonable to expect this type of information to be programmed into machines than to expect humans to know it off the top of their heads. So Karpathy studied for two weeks and 50 hours in order to beat the machine, and he did. He scored a 94.9% accuracy level, a difference of 1.7%.

 

"It was a bit draining, but I felt that it was very important to get the human accuracy," he said to Wired

 

So this doesn't exactly point to the singularity just yet, but it does indicate an amazing, maybe even alarming rate of advancement for artificial intelligence. Only four years ago, humans were beating AI at image recognition by a wide margin, and now they have to train for weeks on end in order to score relatively on par. Image recognition is only one small facet of artificial intelligence, but this is still a remarkable step forward for the technology.

Science
Artificial Intelligence
Technology

Load Comments