Building multi-turn RAG for customer support with LLM labeling

Customer service in e-commerce often relies on human agents to handle inquiries related to orders, returns, and product information. While this approach is effective, it can be expensive and difficult to scale during periods of high demand. Recent advances in intelligent chatbots, particularly those based on Retrieval Augmented Generation (RAG) models, have significantly improved customer service efficiency by combining large language models with external knowledge sources. In the context of e-commerce, these systems can access up-to-date information from order databases, product catalogs, and support documents to manage complex, multi-turn interactions with customers. However, developing a multi-turn RAG chatbot for real-world customer service introduces additional challenges such as adaptive retrieval and query reformulation across dialogue turns. These components typically require large volumes of annotated data, which are often unavailable. To address this limitation, we propose methods that leverage large language models to automatically generate labels from real customer-agent dialogues. Specifically, we introduce two LLM-assisted labeling strategies for adaptive retrieval: an intent-guided strategy and an explanationbased strategy. For query reformulation, we explore two approaches: natural language reformulation and keyword-based reformulation. Our experiments show that the explanation-based strategy achieves the best results for adaptive retrieval, while keyword-based reformulation improves the quality of retrieved documents. These findings provide practical insights for developing scalable and intelligent customer support solutions in the e-commerce industry.