Paraphrase label alignment for voice application retrieval in spoken language understanding

Spoken language understanding (SLU) smart assistants such as Amazon Alexa host hundreds of thousands of voice applications (skills) to delight end-users and fulfill their utterance requests. Sometimes utterances fail to be claimed by smart assistants due to system problems such as model incapability or routing errors. The failure may lead to customer frustration, dialog termination and eventually cause customer churn. To avoid this, we design a skill retrieval system as a downstream service to suggest fallback skills to unclaimed utterances. If the suggested skill satisfies customer intent, the conversation will be recovered with the assistant. For the sake of smooth customer experience, we only present the most relevant skill to customers, resulting in partial observation problem which constrains retrieval model training. To solve this problem, we propose a two-step approach to automatically align claimed utterance labels to unclaimed utterances. First, it measures lexical and semantic similarity to detect claimed utterances as paraphrases for each unclaimed utterance. Second, it trains a neural model on two objectives: skill label prediction and alignment to unclaimed utterances and paraphrases. In the end, the skill predicted with the highest relevancy score is retrieved and suggested to customers. Extensive experiments on two real-world datasets demonstrate that our proposed model significantly outperforms a number of strong alternatives in both paraphrase detection and skill retrieval.