CL-QR: Cross-lingual enhanced query reformulation for multi-lingual conversational AI agents

The growing popularity of conversational AI agents such as Alexa, Google Assistant, and Siri relies on accurate spoken-language comprehension. The query reformulation (QR) method, which reformulates defective user queries, has been broadly adopted to mitigate the challenges posed by understanding the user’s intent from an imperfect spoken recognition result. However, due to the scarcity of non- English QR labels, providing high-quality QR for non-English users still remains a challenge. This work proposes a novel cross-lingual QR framework, CL-QR, to leverage the abundant reformulation resources in English to improve non-English QR performance. The proposed work also proposes a Module-wise Mutually supervised Feedback learning (MMF) algorithm to enable the continual self-improving of the CL-QR, which alleviates the lack of cross-lingual QR training data and enhances the delivery of high-quality reformulations learned in English for multilingual queries. Both offline evaluation and online A/B testing demonstrate the effectiveness of the proposed method.