Researchers build AI-driven sarcasm detector | Artificial intelligence (AI)
Never mind that it can pass the bar exam, ace medical tests and read bedtime stories with emotion, artificial intelligence will never match the marvel of the human mind without first mastering the art of sarcasm.
But that art, it seems, may be next on the list of the technology’s dizzying capabilities. Researchers in the Netherlands have built an AI-driven sarcasm detector that can spot when the lowest form of wit, and the highest form of intelligence, is being deployed.
“We are able to recognise sarcasm in a reliable way, and we’re eager to grow that,” said Matt Coler at the University of Groningen’s speech technology lab. “We want to see how far we can push it.”
There is more to the project than teaching algorithms that sometimes even the most effusive comments cannot be taken literally and must, instead, be interpreted as the diametric opposite. Sarcasm permeates our discourse more than we might appreciate, Coler said, so understanding it is crucial if humans and machines are to communicate seamlessly.
“When you start studying sarcasm, you become hyper-aware of the extent to which we use it as part of our normal mode of communication,” Coler said. “But we have to speak to our devices in a very literal way, as if we’re talking to a robot, because we are. It doesn’t have to be this way.”
Humans are generally adept at spotting sarcasm, though the limited cues found in text alone make it tougher than in a face-to-face interaction when delivery, tone and facial expressions all reveal the speaker’s intent. In developing their AI, the researchers found multiple cues mattered too for the algorithm to distinguish the sarcastic from the sincere.
In work presented at a joint meeting of the Acoustical Society of America and the Canadian Acoustical Association in Ottawa on Thursday, Xiyuan Gao, a PhD student at the lab, described how the group trained a neural network on text, audio and emotional content of video clips from US sitcoms including Friends and The Big Bang Theory. The database, known as Mustard, was compiled by researchers in the US and Singapore, who annotated sentences from the TV shows with sarcasm labels to build their own detector.
One scene the AI trained on was Leonard’s futile effort to escape from a locked room in The Big Bang Theory, prompting Sheldon to observe: “It’s just a privilege to watch your mind at work.” Another from Friends has Ross invite Rachel to come over and join Joey and Chandler in putting together some furniture, prompting Chandler to comment: “Yes, and we’re very excited about it.”
After training on the text and audio, along with scores that reflected the emotional content of words spoken by the actors, the AI could detect sarcasm in unlabelled exchanges from the sitcoms nearly 75% of the time. Further work at the lab has used synthetic data to bump up the accuracy further, but that research is awaiting publication.
Shekhar Nayak, another researcher on the project, said as well as making conversations with AI assistants more fluid, the same approach could an be used to detect negative tone in language and detect abuse and hate speech.
Gao said additional improvements could come from adding visual cues into the AI’s training data, such as eyebrow movements and smirks. Which raises the question of how accurate is accurate enough? “Are we going to have a machine that is 100% accurate?” said Gao. “That’s not something even humans can achieve.”
Making programs more familiar with how humans really speak should help people converse with devices more naturally, Coler adds, but he wonders what will happen if machines embrace their newfound skills and start throwing sarcasm back at us. “If I ask: ‘Do you have time for a question?’ And it says: ‘Yeah, sure,’ I might think: well does it or doesn’t it?”
