Search results

We propose and implement an interpretable machine learning classification model for Explainable AI (XAI) based on expressive Boolean formulas. Potential applications include credit scoring and diagnosis of medical conditions. The Boolean formula defines a rule with tunable complexity (or interpretability), according to which input data are classified. Such a formula can include any operator that can be

Applied Scientist

Proper estimation of predictive uncertainty is fundamental in applications that involve critical decisions. Uncertainty can be used to assess the reliability of model predictions, trigger human intervention, or decide whether a model can be safely deployed in the wild. We introduce Fortuna, an open-source library for uncertainty quantification. Fortuna provides calibration methods, such as conformal prediction

Xtr-WikiQA is an Answer Sentence Selection (AS2) dataset in 9 non-English languages, proposed in our paper accepted at ACL 2023 (Findings): Cross-Lingual Knowledge Distillation for answer sentence selection in low-resource languages. This dataset is based on an English AS2 dataset, WikiQA (Original, Hugging Face). For translations, we used Amazon Translate.

TyDi-AS2 and Xtr-TyDi-AS2 are multilingual Answer Sentence Selection (AS2) datasets comprising 8 diverse languages, proposed in our paper accepted at ACL 2023 (Findings): Cross-Lingual Knowledge Distillation for answer sentence selection in low-resource languages. Both the datasets were created from TyDi-QA, a multilingual question-answering dataset. TyDi-AS2 was created by converting the QA instances in

Missing information is a common issue of dialogue summarization where some information in the reference summaries is not covered in the generated summaries. To address this issue, we propose to utilize natural language inference (NLI) models to improve coverage while avoiding introducing factual inconsistencies. Specifically, we use NLI to compute fine-grained training signals to encourage the model to

RefChecker provides a standardized assessment framework to identify subtle hallucinations present in the outputs of large language models (LLMs). Highlighted Features Finer granularity — RefChecker breaks down the claims in the LLM’s response into knowledge triplets as opposed to paragraphs, sentences, or sub-sentences. Detecting at the knowledge-triplet level will test the truthfulness of facts. Importantly

Recent work in scientific machine learning (SciML) has focused on incorporating partial differential equation (PDE) information into the learning process. Much of this work has focused on relatively “easy” PDE operators (e.g., elliptic and parabolic), with less emphasis on relatively “hard” PDE operators (e.g., hyperbolic). Within numerical PDEs, the latter problem class requires control of a type of volume

Regulations introduced General Data Protection Regulation (GDPR) in the EU or California Consumer Privacy Act (CCPA) in the US have included provisions on the right to be forgotten that mandates industry applications to remove data related to an individual from their systems. In several real world industry applications that use Machine Learning to build models on user data, such mandates require significant

Minimax-fair machine learning minimizes the error for the worst-off group. However, empirical evidence suggests that when sophisticated models are trained with standard empirical risk minimization (ERM), they often have the same performance on the worst-off group as a minimax-trained model. Our work makes this counterintuitive observation concrete. We prove that if the hypothesis class is sufficiently expressive

This is the repo for ReCode (arXiv), providing a comprehensive evaluation for the practical robustness of code generation models like CodeGEN, Incoder, GPT-J. In specific, this benchmark provides over 30 different general perturbations on docstrings, function names, and codes. The perturbations are carefully selected and implemented such that the perturbed datasets are naturally and semantically close to

KILM (short for Knowledgeable Injection into Language Model) is an approach for injecting knowledge into pre-trained language models. This repository provides the necessary code for KILM. KILM propose a new (second-step) pre-training method to inject information about entities (such as entity descriptions) into pre-trained language models. This repository contains code for Creating the dataset needed for

This is a package to apply clustering algorithms to utterances, embedded with a fine-tuned version out of the Supervised Intent Clustering package. Modern virtual assistants are trained to classify customer requests into a taxonomy of predesigned intents. Requests that fall outside of this taxonomy, however, are often unhandled and need to be clustered to define new experiences. Recently, state-of-the-art

Language models have been shown to perform better with an increase in scale on a wide variety of tasks via the in-context learning paradigm. In this paper, we investigate the hypothesis that the ability of a large language model to in-context learn-perform a task is not uniformly spread across all of its underlying components. Using a 66 billion parameter language model (OPT-66B) across a diverse set of

In this work, we introduce Fleet2D, a fast, lightweight simulator designed for long-term human-robot interactions in the home. By abstracting away low-level physics, aggressively caching compute-intensive operations, and operating in a simplified two-dimensional world, we are able to perform realistic simulations of robot behavior at more than 10,000 times real time. We present the design, development,

Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task which involves four elements from user-generated texts: aspect term, aspect category, opinion term, and sentiment polarity. Most computational approaches focus on some of the ABSA sub-tasks such as tuple (aspect term, sentiment polarity) or triplet (aspect term, opinion term, sentiment polarity) extraction using either pipeline

OC-MOT is a framework designed to perform multiple object tracking on object-centric representations without object ID labels. It consists of an index-merge module that adapts the object-centric slots into detection outputs and an unsupervised memory module that builds complete object prototypes to handle occlusions. Benefited from object-centric learning, we only requires sparse detection labels for object

This repository contains code to reproduce the results from Geographical erasure in language generation by Schwöbel et al., from EMNLP 2023 Findings. The goal of this work is to study and operationalise a form of geographical erasure, wherein language models underpredict certain countries.

This repository contains the annotations and script to generate the data for the STREET dataset. Requirements: Python 3 Execute the following python script: python run_street_data_generation.py The script will download some of the original datasets in which STREET is built and merge them with our reasoning and explanation annotations. The final data will be contained within the ./data/$STREET_TASK$/reasoning_annotated

Automatic melody-to-lyric generation is a task in which song lyrics are generated to go with a given melody. It is of significant practical interest and more challenging than unconstrained lyric generation as the music imposes additional constraints onto the lyrics. The training data is limited as most songs are copyrighted, resulting in models that underfit the complicated cross-modal relationship between