HOKIEBOT: Towards personalized open-domain chatbot with long-term dialogue management and customizable automatic evaluation

With the increasing prevalence of smart speakers and virtual chatbots, dialogue systems have become a prominent research area in natural language processing. The primary objective of designing a dialogue agent is to converse with humans on a wide range of requests, ranging from general chat to information seeking, event discussion, and more. While existing dialog systems still face numerous challenges, we mainly focus on addressing the following three essential ones: (1) managing diverse conversational purposes, (2) facilitating effective memory management for long-term conversations, and (3) automating the evaluation of machine-generated dialog responses. To overcome these challenges, we introduce HOKIEBOT, an open-domain chatbot developed for the Alexa Prize SocialBot Grand Challenge 5. HOKIEBOT employs a diverse set of dialog responders, including retrieval-based, neural network-based, and large language model-based (such as BlenderBot and Alpaca) to cater to a broad spectrum of user requests. To enhance engagement and foster long-term conversations, we introduce a novel topic-aware responder that keeps track of user preferences toward various topics from previous interactions, stores them in memory, and dynamically utilizes them to generate consistent responses. Additionally, we investigate various ranking strategies to evaluate and select the most suitable responses from a diverse array of candidates. The integration of these components enables HOKIEBOT to produce user-preferred responses and maintain consistency in long-term interactions, thereby offering an improved conversational experience across a wide range of conversational topics.