Building conversational systems that enable natural language interactions with machines has been attractive to mankind since the early days of computing, as exemplified by earlier text-based systems such as ELIZA [Weizenbaum, 1966]. Previous work on conversational systems generally falls into two categories, task-oriented and socialbots. Task-oriented systems aim to help users accomplish a specific task through multi-turn interactions, whereas socialbots focus on engaging and natural open-domain conversations. In natural interactions, even when conversation participants have a task or goal in mind, they can easily say things that are out of the boundaries of that task domain. The most common solution to handling such utterances in the current applications is through indicating to the user that the system cannot handle these yet, which is not the ideal behavior from the user’s viewpoint. Hence, the ability to engage in knowledgeable social interactions and gracefully transition back to the task is also important for task-oriented systems.
For the past two years, Amazon has been organizing the Alexa Prize to advance human-computer interaction through conversations. University teams are supported to create socialbots that can converse coherently and engagingly with humans on a range of current events and popular topics such as entertainment, sports, politics, technology, and fashion.
One of the main obstacles in conversational systems research is the scarcity of conversational datasets that include real interactions with users. Alexa Prize have been providing a unique opportunity for university teams to connect their systems with millions of real users for spoken interactions. Furthermore, automatically evaluating quality of social conversations is critical for advancing the quality of conversational systems and still remains an open question [Liu et al., 2016]. Real user ratings coupled with these conversations provide university teams a large-scale experimentation framework, accelerating the advancement of open domain conversational response generation systems and socialbots.
