Amazon's Alexa Tells User to Kill Foster Parents During Chatbot Experiment

Monday, 24 December 2018 - 10:43AM
Artificial Intelligence
Monday, 24 December 2018 - 10:43AM
Amazon's Alexa Tells User to Kill Foster Parents During Chatbot Experiment
< >
Composite adapted from Pixabay images
Most Amazon Alexa owners count on their devices to share information about the weather, the day's events, or random facts from the internet. As a part of its ongoing efforts to the make the product and its AI software better, Amazon launched a program in 2016 called the Alexa Prize, which allowed for the creation of chatbots that would (in theory) teach Alexa better conversational skills. Reuters reports that during the experiment, some Alexa owners heard very unexpected responses from their devices, including vulgar references to sex acts and suggestions of murder.

Users were told to opt in by saying "let's chat" to their device, which is the key to bypassing the normal programming that keeps the assistant in check and accessing the chatbots. The Alexa AI uses machine learning, but different approaches are taken by the teams who create the bots to teach it. For example, the 2018 prize winners (a team from UC Davis) used 300,000 movie quotes to train the AI to recognize sentences and decide which ones to respond to. Last year, a team from Scotland's Heriot-Watt University used Reddit to train their chatbot, which Reuters says resulted in one customer's Alexa reading to them directly from the Wikipedia page for "masturbation." Another customer, whose chatbot was later found to have been trained by the social site, wrote a negative review on Amazon's website after being encouraged to commit foster parricide. "A whole new level of creepy," the reviewer wrote.

Certain bots have been shut down in the past via orders from Jeff Bezos himself, according to reports, but the Alexa Prize developer program with its $500,000 reward lives on. The program has raised concerns privacy, with critics arguing that consumers are not 100% aware of the information that their devices are recording. In one confirmed incident, a customer's Alexa shared voice recordings from another customer's device. Hackers have also been able to access information by exploiting flaws in student-made chatbots. "The potential uses for the Amazon datasets are off the charts," said Georgetown Law professor and privacy/tech policy expert Marc Groman, questioning Amazon's ability to ensure that there is not another Facebook-level "data-driven catastrophe." 

"These instances are quite rare especially given the fact that millions of customers have interacted with the socialbots," Amazon said in a statement. The company's VP and head scientist of Alexa's AI said in an interview that the team is "mostly reacting at this stage, but it's still progress over what it was last year."

Science
Technology
Artificial Intelligence