Conversational AI is a catch-all term for natural language models for artificial intelligence that can interpret human words, speak to people, or carry out tasks or computation with natural language.
But talk to any of the best-known AI assistants today — Alexa, Siri, Google Assistant — and they’re not exactly conversational. They can tell you jokes, answer factual questions, and even respond to multiple queries without the need to keep repeating a wake word, but conversation or chit-chat is still very much a human endeavor.
To share progress towards deep learning designed to carry a conversation, Google today introduced Meena, a neural network with 2.6 billion parameters. Meena can handle multiturn dialogue, and Google claims it’s better than other AI agents built for conversation and available online today. It even told an off-the-cuff joke.
Google today also released Sensibleness and Specificity Average (SSA), a metric created by Google researchers to measure the ability of a conversational agent to maintain responses in conversation that make sense and are specific. Humans rank around 86% in SSA, and in initial tests, Meena scores a high of 79%. Mitsuku, an AI agent created by Pandora Bots that’s won the Loebner Prize for the past four years, got a 56%, while Microsoft’s XiaoIce, which speaks Mandarin Chinese, got a score of 31%.
The work is detailed in “Towards a Human-like Open Domain Chatbot,” a paper published Monday on preprint repository arXiv.
Meena is trained on 40 billion words and utilizes a seq2seq model and a variation of the popular Transformer architecture. Google first released Transformer in 2017, but since then the language has grown to rank among the highest performing language models around.
SSA evaluates dialogue based on static performance with a fixed set of prompts or interactive performance, which allows for free-flowing conversation. Each evaluated conversation was required to last at least 14 turns and no more than 28 turns. Results are then arrived at based on the percentage of turns considered specific or sensible. SSA penalizes responses with generic responses.
The SSA standard Google proposes is different than the metric other AI assistants have set for assessing a truly conversational AI.
Now in its third year, the Alexa Prize is a challenge for teams of student developers to create AI that can hold a conversation for up to 20 minutes. The finalist last year got up to about 10 minutes. The latest round of finalists will be announced in May. You can speak with last year’s finalists by simply saying “Alexa, let’s chat.”
Amazon is already beginning to grow its multiturn dialogue offerings. Conversations is a feature that packages voice app recommendations in conversational multiturn dialogue. At the time of launch last summer, Amazon VP of devices David Limp called it “the holy grail of voice science.”
Microsoft acquired the company Semantic Machines in 2018 and last year began to showcase more multiturn dialogue for users of the Microsoft Bot Framework.
AI assistants that can maintain a conversation may be able to secure closer bonds with humans and do things like provide emotional support to people, or cure the loneliness epidemic, as former Alexa Prize head and current Google Research director Ashwin Ram put it in 2017.
.jpg "Coat Giveaway Becomes a Powerhouse Nonprofit Making Life Better for Thousands of Economically Disadvantaged Students")