Google's new chatbot Meena can chat about anything a user wants

Meena has 1.7x greater model capacity and was trained on 8.5x more data as compared to OpenAI GPT-2, an existing state-of-the-art generative model.

Google just launched a multi-turn open-domain chatbot named Meena that learns to respond sensibly to a given conversational context. The human-like chatbot, as Google claims, is more sensible and better compared with that of other well-known open-domain chatbots including Mitsuku, Cleverbot, XiaoIce, and DialoGPT.

The end-to-end trained neural conversational model has 2.6 billion parameters and is trained on 341 GB of text from social media conversations. According to Google, Meena has 1.7x greater model capacity and was trained on 8.5x more data as compared to OpenAI GPT-2, an existing text-generating language model.

"Current open-domain chatbots have a critical flaw, they often don't make sense. They sometimes say things that are inconsistent with what has been said so far, or lack common sense and basic knowledge about the world. Moreover, chatbots often give responses that are not specific to the current context," Daniel Adiwardana, Senior Research Engineer, and Thang Luong, Senior Research Scientist, Google Research, Brain Team wrote in a blog post.

The tech giant also introduced a new human evaluation metric for open-domain chatbots. Dubbed Sensibleness and Specificity Average (SSA), the new human evaluation metric captures key elements of a good conversation. Meena scored 79 percent of SSA, 23 percent higher than the next highest scoring chatbot, Mitsuku (56 percent). Humans, on the other hand, achieved 86 percent scores, Cleverbot got 56 percent, followed by XiaoIce that achieved 31 percent, and lastly DialoGPT that scored 48 percent of SSA.

While we have focused solely on sensibleness and specificity in this work, other attributes such as personality and factuality are also worth considering in subsequent works. Also, tackling safety and bias in the models is a key focus area for us, and given the challenges related to this, we are not currently releasing an external research demo.

Google Research