Search results

This repository contains two abstractive question answering datasets that are reduced from task-oriented parsing (TOP) datasets. They are used in the semantic parsing as abstractive question answering work. We also provide the related data processing scripts.

We present a very simple method for extending pretrained machine translation metrics to incorporate document-level context. We apply our method to four popular metrics: BERTScore, Prism, COMET, and the reference-free metric COMET-QE. We evaluate our document-level metrics on the MQM annotations from the WMT 2021 metrics shared task and find that the document-level metrics outperform their sentence-level

In this work, we propose an effective and intuitive termination criterion for BO that automatically stops the procedure if it is sufficiently close to the global optimum. Our key insight is that the discrepancy between the true objective (predictive performance on test data) and the computable target (validation performance) suggests stopping once the suboptimality in optimizing the target is dominated

Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectures based on large language models that have been pretrained on vast amounts of text. To better leverage that pretraining, recent work has explored a reformulation of semantic parsing whereby the output sequences are themselves

The Alexa Voice Service (AVS) enables developers to integrate Alexa directly into their products, bringing the convenience of voice control to any connected device. AVS provides developers with access to a suite of resources to build Alexa-enabled products, including APIs, hardware development kits, software development kits, and documentation.

Leveraging the characteristics of convolutional layers, neural networks are extremely effective for pattern recognition tasks. However in some cases, their decisions are based on unintended information leading to high performance on standard benchmarks but also to a lack of generalization to challenging testing conditions and unintuitive failures. Recent work has termed this ”shortcut learning” and addressed

Pre-trained language models (PLM) have advanced the state-of-the-art across NLP applications, but lack domain-specific knowledge that does not naturally occur in pre-training data. Previous studies augmented PLMs with symbolic knowledge for different downstream NLP tasks. However, knowledge bases (KBs) utilized in these studies are usually large-scale and static, in contrast to small, domain-specific, and

The FoodOrdering dataset is a task-oriented parsing dataset in the food-ordering domain with utterances and annotations derived from the menus of five venues characteristic of that business vertical: burgers, burritos, coffees, pizzas, and subs.

A 20 billion parameter multilingual seq2seq model called Alexa Teacher Model (AlexaTM 20B), which achieves state-of-the-art (SOTA) performance on 1-shot summarization tasks, outperforming a much larger 540B PaLM decoder model. AlexaTM 20B also achieves SOTA in 1-shot machine translation, especially for low-resource languages, across almost all language pairs supported by the model (Arabic, English, French

Automatic speech recognition (ASR) is increasingly being used in specialized domains such as medical ASR and news transcription. Owing to the lack of high quality annotated speech data in such domains, off-the-shelf models are commonly employed by fine-tuning on domain-specific data. This poses a significant challenge in transcribing long-tail expressions and out-of-vocabulary (OOV) named entities. On the

Current question answering (QA) systems primarily consider the single-answer scenario, where each question is assumed to be paired with one correct answer. However, in many real-world QA applications, multiple answer scenarios arise where consolidating answers into a comprehensive and non-redundant set of answers is a more efficient user interface. In this paper, we formulate the problem of answer consolidation

Project Description This is a test framework for the bias bounties project. Getting Started as a Bounty Hunter If you are interacting with this codebase as a "bounty hunter", you'll need to have a way to run Jupyter notebooks. The easiest way to do this is to download Anaconda, which will also manage all of your python packages for you. See here for installation instructions: https://docs.anaconda.com/anaconda

This module implements a set of functions to perform MAXENT from a causal perspective. The code here can be used to reproduce the results in the publication Obtaining Causal Information by Merging Datasets with MAXENT. The parts of the plots using KCI are, unfortunately, not available. To reproduce the results in the article create a python 3.6+ environment, pip install all the requirements.txt file and

As generic machine translation (MT) quality has improved, the need for targeted benchmarks that explore fine-grained aspects of quality has increased (Freitag et al., 2021; Isabelle et al., 2017). In particular, gender accuracy in translation (Choubey et al., 2021; Saunders and Byrne, 2020) can have implications in terms of output fluency, translation accuracy, and ethics. In this paper, we introduce MTGenEval

Robots operating in human spaces must be able to engage in natural language interaction, both understanding and executing instructions, and using conversation to resolve ambiguity and correct mistakes. To study this, we introduce TEACh, a dataset of over 3,000 human–human, interactive dialogues to complete household tasks in simulation. A Commander with access to oracle information about a task communicates

Skill Components are “ready to use” experiences that you can easily add to your skills, and configure them according to your needs. Each component is a collection of skill primitives, such as VUI dialogs & intents, Alexa Presentation Language (APL) documents, skill code, skill connection tasks, and skill events. They bring together design best practices and pre-built voice experiences, which solve for a

Tartan is a multi-domain task-oriented bot that assists users with two different tasks: 1. cooking with recipes from Whole Foods and 2. doing projects from WikiHow. The bot’s system is divided into two stages. In the first stage, the bot assists the user in identifying a task that they want to do. In the second stage, the bot guides the users through sequences of step-by-step instructions. We developed

We present TacoBot, a task-oriented dialogue system built for the inaugural Alexa Prize TaskBot Challenge, which assists users in completing multi-step cooking and home improvement tasks. TacoBot is designed with a user-centered principle and aspires to deliver a collaborative and accessible dialogue experience. Towards that end, it is equipped with accurate language understanding, flexible dialogue management

This paper describes the vision, scientific contributions, and technical details of the Task Wizard (TWIZ) team’s participation in the Alexa TaskBot Challenge 2021. Our bot design envisions the support of an engaging experience, where users are guided through multimodal conversations, towards the successful completion of the selected task. This is achieved through four key principles: a) robust dialog interaction

Conversational Artificial Intelligence (AI) has been a long-standing area of exploration in the research community and has now penetrated both academia and industries with products such as Siri and Alexa. In this work, we present COoking-aNd-DIy-TAsk-based (Condita) ChatBot, a task-oriented dialogue system, for the 2021 Alexa Prize TaskBot Challenge. Condita provides an engaging multi-modal agent that assists