Search results

NeurIPS 2023

The Neural Information Processing Systems (NeurIPS) annual meeting fosters the exchange of research on neural information processing systems in their biological, technological, mathematical, and theoretical aspects. The core focus is peer-reviewed novel research which is presented and discussed in the general session, along with invited talks by leaders in their fields.

Machine learning

December 10 - 16, 2023

New Orleans, Louisiana

EMNLP 2023

Empirical Methods in Natural Language Processing is a leading conference in the area of natural language processing and artificial intelligence.

Conversational AI / Natural-language processing

December 6 - 10, 2023

Singapore

NeurIPS 2023 Workshop on Efficient Natural Language and Speech Processing (ENLSP)

Zizhao Zhang

Applied Scientist

Baxter Black

Data Scientist

Video streaming

ProcSim: Proxy-based confidence for robust similarity learning

Oriol Barbany Mayor, Xiaofan Lin, Muhammet Bastan, Arnab Dhua

WACV 2024

2023

Deep Metric Learning (DML) methods aim at learning an embedding space in which distances are closely related to the inherent semantic similarity of the inputs. Previous studies have shown that popular benchmark datasets often contain numerous wrong labels, and DML methods are susceptible to them. Intending to study the effect of realistic noise, we create an ontology of the classes in a dataset and use

Computer vision

FundRecLLM: Fund recommendation based on financial news and research analyst report

Guang Yang, Peiyang He, Xuefeng Liu

FSDM 2023

2023

Adopting AI in financial advisory is a challenging task as there exists multiple sources of information to digest and interpret. Such information consumption process are very lengthy for financial advisors, reducing the efficiency and timeliness for their advice and recommendation given to their clients. In this work, we introduce a multi-step framework that consumes and combines news and industry-focused

Conversational AI / Natural-language processing

Semi-supervised semantic segmentation under label noise via diverse learning groups

Peixia Li, Pulak Purkait, Ajanthan Thalaiyasingam, Majid Abdolshah, Ravi Garg, Hisham Husain, Chenchen Xu, Stephen Gould, Wanli Ouyang, Anton van den Hengel

ICCV 2023

2023

Semi-supervised semantic segmentation methods use a small amount of clean pixel-level annotations to guide the interpretation of a larger quantity of unlabelled image data. The challenges of providing pixel-accurate annotations at scale mean that the labels are typically noisy, and this contaminates the final results. In this work, we propose an approach that is robust to label noise in the annotated data

Computer vision

MIVC: Multiple instance visual component for visual-language models

Wenyi Wu, Qi Li, Wenliang Zhong, Junzhou Huang

WACV 2024

2023

Vision-language models have been widely explored across a wide range of tasks and achieve satisfactory performance. However, it’s under-explored how to consolidate entity understanding through a varying number of images and to align it with the pre-trained language models for generative tasks. In this paper, we propose MIVC, a general multiple instance visual component to bridge the gap between various

Computer vision

Topic knowledge based controlled generation for long documents using retrieval-based language models

Xuefei Zhang, Peiyang He, Tomal Deb, Guang Yang, Xuefeng Liu, Ziqing Hu, Tianyi Mao

FSDM 2023

2023

Current LLM summarization systems Produce broad overviews which are disconnected from people specific interests and expectations. Basically, people preferences (topics) can be expressed by a collection of semantic keywords. Previous work exploit these keywords as extra input to generate summary. That requires additional human annotations. To tackle these constraints, we propose a novel framework, Topic

Conversational AI / Natural-language processing

Augment the pairs-semantics-preserving image-caption pair augmentation for grounding-based vision and language models

Jingru Yi, Burak Uzkent, Oana Ignat, Zili Li, Amanmeet Garg, Xiang Yu, Linda Liu

WACV 2024

2023

Grounding-based vision and language models have been successfully applied to low-level vision tasks, aiming to pre- cisely locate objects referred in captions. The effectiveness of grounding representation learning heavily relies on the scale of the training dataset. Despite being a useful data en- richment strategy, data augmentation has received minimal attention in existing vision and language tasks

Computer vision

Integrating summarization and retrieval for enhanced personalization via large language models

Chris Richardson, Yao Zhang, Kellen Gillespie, Sudipta Kar, Arshdeep Singh, Zeynab Raeesy, Omar Zia Khan, Abhinav Sethy

CIKM 2023 Workshop Personalized Generative AI

2023

Personalization, the ability to tailor a system to individual users, is an essential factor in user experience with natural language process- ing (NLP) systems. With the emergence of Large Language Models (LLMs), a key question is how to leverage these models to better personalize user experiences. To personalize a language model’s output, a straightforward approach is to incorporate past user data into

Conversational AI / Natural-language processing

What is lost in knowledge distillation?

Peyman Passban, Tanya Roosta, Manas Mohanty

2023

Deep neural networks (DNNs) have improved NLP tasks significantly, but training and maintaining such networks could be costly. Model compression techniques, such as, knowledge distillation (KD), have been proposed to address the issue; however, the compression process could be lossy. Motivated by this, our work investigates how a distilled student model differs from its teacher, if the distillation process

Machine learning

Generating and exploiting automated reasoning proof certificates

Haniel Barbosa, Clark Barrett, Byron Cook, Cesare Tinelli, Andrew Reynolds

Communications of the ACM

2023

Moving toward a full suite of proof-producing automated reasoning tools with SMT solvers that can produce full, independently checkable proofs for real-world problems.

Automated reasoning

Himanshu Gupta

Applied Scientist

Smita Bailur

Applied Scientist

Pengju Chen

Data Scientist

Sachin Garg

Data Scientist

Fabio Valongo Ferreira

Applied Scientist

Maira Ladeira Tanke

Data Scientist

Roberto Perales

Research Scientist

Bugra Cetingok

Data Scientist

Virginia Chen

Applied Scientist

Travis Field

Research Scientist

Asma Benhenda

Economist

Radek Danecek

Applied Scientist

CALICO: Conversational agent localization via synthetic data generation

Andy Rosenbaum, Pegah Kharazmi, Ershad Banijamali, Lu Zeng, Christopher DiPersio, Vivi Wei, Gokmen Oz, Clement Chung, Karolina Owczarzak, Fabian Triefenbach, Wael Hamza

NeurIPS 2023 Workshop on SyntheticData4ML

2023

We present CALICO, a method to fine-tune Large Language Models (LLMs) to localize conversational agent training data from one language to another. For slots (named entities), CALICO supports three operations: verbatim copy, literal translation, and localization, i.e. generating slot values more appropriate in the target language, such as city and airport names located in countries where the language is

Conversational AI / Natural-language processing

Sneha Kumar

Data Scientist

Maya Cakmak builds teachable robots to help humans

Staff writer

November 21, 2023

Maya Cakmak, an associate professor at the University of Washington, is seen speaking at an Amazon event, she is holding a mic while giving a talk

Through the UW + Amazon Science Hub, the UW associate professor and Science Hub advisory board member is helping to realize a future where robots and people collaborate on tasks.

Robotics

Chen Xue

Applied Scientist

Alon Halevy

Senior Principal Scientist

Kommy Weldemariam

Director, Research Science

Gabriel Bénédict

Applied Scientist

Austin Roberts

Research Scientist

Nimra Nauraiz

Data Scientist

Ashwani Anand

Applied Scientist

Hasta Vanchinathan

Senior Manager, Applied Science

Chris Pecora

Data Scientist

Responsible AI in the wild: Lessons learned at AWS

Michael Kearns, Aaron Roth

November 16, 2023

Real-world deployment requires notions of fairness that are task relevant and responsive to the available data, recognition of unforeseen variation in the “last mile” of AI delivery, and collaboration with AI activists.

Machine learning

Federated multi-objective learning

Haibo Yang, Zhuqing Liu, Jia Liu, Chaosheng Dong, Michinari Momma

Shreyas S, Prakash Mandayam Comar, Sivaramakrishnan (Siva) Kaveri

2023

In recent years, multi-objective optimization (MOO) emerges as a foundational problem underpinning many multi-agent multi-task learning applications. However, existing algorithms in MOO literature remain limited to centralized learning settings, which do not satisfy the distributed nature and data privacy needs of such multi-agent multi-task learning applications. This motivates us to propose a new federated

Machine learning

NNCR: Revising classifications using embedding based k-nearest-neighbor search

Yu Hsuan Kuo, Saaransh Gulati, Xiaoyu Chu, Pulkit Garg

IEEE BigData 2023

2023

The global e-commerce store needs to ensure compliance with various regulations at local, national, and international levels. One business use case is to identify face masks to avoid price gouging during times of high demand. In order to keep billions of items safe and legally compliant, it is important to ensure accurate classifications. Classification revisers aim to enhance classification accuracy by

Machine learning

DynaPipe: Optimizing multi-task training through dynamic pipelines

Chenyu Jiang, Zhen Jia, Shuai Zheng

EuroSys 2023

2023

Multi-task model training has been adopted to enable a single deep neural network model (often a large language model) to handle multiple tasks (e.g., question answering and text summarization). Multi-task training commonly receives input sequences of highly different lengths due to the diverse contexts of different tasks. Padding (to the same sequence length) or packing (short examples into long sequences

Cloud and systems

ForeSeer: Product aspect forecasting using temporal graph embedding

Zixuan Liu, Gaurush Hiranandani, Kun Qian, Eddie Huang, Yi Xu, Belinda Zeng, Karthik Subbian, sheng wang

CIKM 2023

2023

Developing text mining approaches to mine aspects from customer reviews has been well-studied due to its importance in understand-ing customer needs and product attributes. In contrast, it remains unclear how to predict the future emerging aspects of a new product that currently has little review information. This task, which we named product aspect forecasting, is critical for recommending new products

Machine learning

Adversarial density ratio estimation for change point detection

CIKM 2023

2023

Change Point Detection (CPD) models are used to identify abrupt changes in the distribution of a data stream and have a widespread practical use. CPD methods generally compare the distribution of data sequences before and after a given time step to infer if there is a shift in distribution at the said time step. Numerous divergence measures, which measure distance between data distributions of sequence

Machine learning

Amazon Books editors announce best 2023 science books

Staff writer

November 16, 2023

The 20 best science books of 2023 are seen in a gird

Outlive: The Science and Art of Longevity by Peter Attia named as the best science book of 2023.

GeMQuAD: Generating multilingual question answering datasets from large language models using few shot learning

Amani Namboori, Shivam Mangale, Andy Rosenbaum, Saleh Soltan

NeurIPS 2023 Workshop on SyntheticData4ML

2023

The emergence of Large Language Models (LLMs) with capabilities like In-Context Learning (ICL) has ushered in new possibilities for data generation across various domains while minimizing the need for extensive data collection and modeling techniques. Researchers have explored ways to use this generated synthetic data to optimize smaller student models for reduced deployment costs and lower latency in downstream

Conversational AI / Natural-language processing

Tree-regularized tabular embeddings

Xuan Li, Yun Wang, Bo Li

NeurIPS 2023 Workshop on Table Representation Learning

2023

Tabular neural network (NN) has attracted remarkable attentions and its recent advances have gradually narrowed the performance gap with respect to tree-based models on many public datasets. While the mainstreams focus on calibrating NN to fit tabular data, we emphasize the importance of homogeneous embeddings and alternately concentrate on regularizing tabular inputs through supervised pretraining. Specifically

Machine learning

Testing the limits of unified seq2seq LLM pretraining on diverse table data tasks

Soumajyoti Sarkar, Leonard Lausen

NeurIPS 2023 Workshop on Table Representation Learning

2023

Tables stored in databases and tables which are present in web pages and articles account for a large part of semi-structured data that is available on the internet. It motivates the need to develop a modeling approach with large language models (LLMs) which can be used to solve diverse table tasks such as semantic parsing, question answering as well as classification problems. Traditionally, there existed

Machine learning

Weiwei Cheng

Applied Scientist

Gaurav Gupta

Applied Scientist

Julian Bitterwolf

Applied Scientist

Automated few-shot classification with instruction-finetuned language models

Rami Aly, Xingjian Shi, Kaixiang Lin, Aston Zhang, Andrew Wilson

Vinayak Puranik, Anirban Majumder, Vineet Chaoji

2023

A particularly successful class of approaches for few-shot learning combines language models with prompts — handcrafted task descriptions that complement data samples. However, designing prompts by hand for each task commonly requires domain knowledge and substantial guesswork. We observe, in the context of classification tasks, that instruction-finetuned language models are remarkably robust towards some

Conversational AI / Natural-language processing

Protege: Prompt-based diverse question generation from web articles

2023

Rich and diverse knowledge-bases (KB) are foundational building blocks for online knowledge-sharing communities such as StackOverflow and Quora and applications such as conversational assistants (aka chatbots). A popular format for knowledge bases is question-answer pairs (or FAQs), where questions are designed to accurately match a multitude of queries. In this paper, we address the problem of automatic

Conversational AI / Natural-language processing

MultiCoNER v2: A large multilingual dataset for fine-grained and noisy named entity recognition

Besnik Fetahu, Zhiyu Chen, Sudipta Kar, Oleg Rokhlenko, Shervin Malmasi

Sai Muralidhar Jayanthi, Devang Kulshreshtha, Saket Dingliwal, Srikanth Ronanki, Sravan Bodapati

2023

We present MultiCoNER V2, a dataset for fine-grained Named Entity Recognition covering 33 entity classes across 12 languages, in both monolingual and multilingual settings. This dataset aims to tackle the following practical challenges in NER: (i) effective handling of fine-grained classes that include complex entities like movie titles, and (ii) performance degradation due to noise generated from typing

Conversational AI / Natural-language processing

Retrieve and copy: Scaling ASR personalization to large catalogs

2023

Personalization of automatic speech recognition (ASR) models is a widely studied topic because of its many practical applications. Most recently, attention-based contextual biasing techniques are used to improve the recognition of rare words and/or domain-specific entities. However, due to performance constraints, the biasing is often limited to a few thousand entities, restricting real-world usability.

Conversational AI / Natural-language processing

InsightNet: Structured insight mining from customer feedback

Sandeep Mukku, Manan Soni, Chetan Aggarwal, Jitenkumar Rana, Promod Yenigalla, Rashmi Patange, Shyam Mohan

2023

We propose InsightNet, a novel approach for the automated extraction of structured insights from customer reviews. Our end-to-end machine learning framework is designed to overcome the limitations of current solutions, including the absence of structure for identified topics, non-standard aspect names, and lack of abundant training data. The proposed solution builds a semi-supervised multi-level taxonomy

Conversational AI / Natural-language processing

Shyam Mohan

Generative speech recognition error correction with large language models and task-activating prompting

Chao-Han Huck Yang, Yi Gu, Yi Chieh Liu, Shalini Ghosh, Ivan Bulyko, Andreas Stolcke

ASRU 2023

2023

We explore the ability of large language models (LLMs) to act as speech recognition post-processors that perform rescoring and error correction. Our first focus is on instruction prompting to let LLMs perform these task without fine-tuning, for which we evaluate different prompting schemes, both zeroand few-shot in-context learning, and a novel “task activation” prompting method that combines causal instructions

Conversational AI / Natural-language processing

TouchUp-G: Improving feature representation through graph-centric finetuning

Jing Zhu, Xiang Song, Vasileios Ioannidis, Danai Koutra, Christos Faloutsos

KDD 2023 Workshop on Knowledge Augmented Methods for NLP

2023

How can we improve node features obtained from Pretrained Models (PMs) for downstream graph tasks? Graph Neural Networks (GNNs) have demonstrated promising results in various graph learning tasks, including node classification and link prediction. Despite their remarkable success in high-impact applications, we have observed that for feature-rich graphs, it is a common practice to directly employ a PM for

Machine learning

The need for a durability index framework for electrical and electronic equipment to support a circular economy

Hamidreza Habibollahi Najaf Abadi, Maxim Nikiforov, Aaron Krive, Jeffrey W. Herrmann, Mohammad Modarres

ESREL 2023

2023

Enabling a circular economy aims to reduce the amount of global waste generated from electrical and electronic equipment, mitigate the associated risk to the ecosystem and human health, and address concerns over limited material resources. Durability is a critical concern because keeping products in use for a longer time should reduce resource consumption and waste. Assessing the durability of products

Economics

Maxim Nikiforov

Sr. HW Reliability Engineer

Aaron Krive

HW Reliability Manager

Adversarial robustness for large-language NER models using disentanglement and word attributions

Xiaomeng Jin, Bhanu Vinzamuri, Sriram Venkatapathy, Heng Ji, Pradeep Natarajan

Anselmo Talotta, Valentin Radu, Lorenzo Sorgi

2023

Large language models (LLMs) have been widely used for several applications such as question answering, text classification and clustering. While the preliminary results across the aforementioned tasks looks promising, recent work (Qin et al., 2023; Wang et al., 2023a) has dived deep into LLMs' performing poorly for complex Named Entity Recognition (NER) tasks in comparison to fine-tuned pre-trained language

Conversational AI / Natural-language processing

Clinical note section classification on doctor patient conversations in low resourced settings

Zhuohao Chen, Jangwon Kim, Yang Liu, Shrikanth Narayanan

NLPMC 2023 Workshop on NLP for Medical Conversations

2023

In clinical visits, clinical note writing is a timeconsuming and cost-prohibitive manual task for clinicians. Although virtual medical scribes have been proposed to generate clinical notes (semi-)automatically, the data sparsity issue is still a challenging problem in practice. Identifying the topic of clinical utterances in doctorpatient conversations is one of the key strategies for automation. In this

Conversational AI / Natural-language processing

Floorplan generation from noisy point cloud

ACM SIGSPATIAL 2023 Workshop on Spatial Big Data and AI for Industrial Applications

2023

Floorplans are useful for navigating indoor spaces, for resource allocation and for indoor space management among many others. But in the absence of readily available digital floorplans, these are hard to generate. In this work, we enhance the generation of floor- plans from point clouds to be more robust to noisy measurements of sensor data. In our approach, we train an object detector to expose room shapes

Computer vision

Search optimization with query likelihood boosting and two-level approximate search for edge devices

Jianwei Zhang, Helian Feng, Cynthia He, Grant Strimel, Farhad Ghassemi, Ali Kebarighotbi

CIKM 2023

2023

We present a novel search optimization solution for approximate nearest neighbor (ANN) search on resource-constrained edge devices. Traditional ANN approaches fall short in meeting the specific demands of real-world scenarios, e.g., skewed query likelihood distribution and search on large-scale indices with a low latency and small footprint. To address these limitations, we introduce two key components:

Search and information retrieval

Too much of product information: Don’t worry, let’s look for evidence!

Aryan Jain, Jitenkumar Rana, Chetan Aggarwal

2023

Product question answering (PQA) aims to provide instant responses to customer questions posted on shopping message boards, social media, brand websites and retail stores. In this paper, we propose a distantly supervised solution to answer customer questions by using product information. Auto-answering questions using product information poses two main challenges: (i) labelled data is not readily available

Conversational AI / Natural-language processing

HYTREL: HYpergraph-enhanced Tabular data Representation Learning

Pei Chen, Soumajyoti Sarkar, Leonard Lausen, Balasubramaniam Srinivasan, Sheng Zha, Ruihong Huang, George Karypis

Abishek Sankararaman, Balakrishnan (Murali) Narayanaswamy

2023

Language models pretrained on large collections of tabular data have demonstrated their effectiveness in several downstream tasks. However, many of these models do not take into account the row/column permutation invariances, hierarchical structure, etc. that exist in tabular data. To alleviate these limitations, we propose HYTREL, a tabular language model, that captures the permutation invariances and

Machine learning

Online robust non-stationary estimation

2023

The real-time estimation of time-varying parameters from high-dimensional, heavy tailed and corrupted data-streams is a common sub-routine in systems ranging from those for network monitoring and anomaly detection to those for traffic scheduling in data-centers. For estimation tasks that can be cast as minimizing a strongly convex loss function, we prove that an appropriately tuned version of the clipped

Machine learning

MReFinED: An efficient end-to-end multilingual entity linking system

Peerat Limkonchotiwat, Weiwei Cheng, Christos Christodoulopoulos, Amir Saffari, Jens Lehmann

NeurIPS 2023 Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response (AI + HADR)

2023

End-to-end multilingual entity linking (MEL) is concerned with identifying multilingual entity mentions and their corresponding entity IDs in a knowledge base. Prior efforts assume that entity mentions are given and skip the entity mention detection step due to a lack of high-quality multilingual training corpora. To overcome this limitation, we propose mReFinED, the first end-to-end MEL model. Additionally

Conversational AI / Natural-language processing

Enabling decision-support systems through automated cell tower detection

Natasha Krell, Will Gleave, Daniel Nakada, Justin Downes, Amanda Willet, Matthew Baran

2023

Cell phone coverage and high-speed service gaps persist in rural areas in sub-Saharan Africa, impacting public access to mobile-based financial, educational, and humanitarian services. Improving maps of telecommunications infrastructure can help inform strategies to eliminate gaps in mobile coverage. Deep neural networks, paired with remote sensing images, can be used for object detection of cell towers

Cloud and systems

Latent skill discovery for chain-of-thought reasoning

Zifan Xu, Haozhu Wang, Dmitriy Bespalov, Yanjun (Jane) Qi

NeurIPS 2023 Workshop on Robustness of Zero/Few-shot Learning in Foundation Models (R0-FoMo)

2023

Recent advances in Large Language Models (LLMs) have led to an emergent ability of chain-of-thought (CoT) prompting, a prompt reasoning strategy that adds intermediate rationale steps between questions and answers to construct prompts. Conditioned on these prompts, LLMs can effectively learn in context to generate rationales that lead to more accurate answers than when answering the same question directly

Machine learning

SpotTarget: Rethinking the effect of target edges for link prediction in graph neural networks

Jing Zhu, Yuhang Zhou, Vasileios Ioannidis, Shengyi Qian, Wei Ai, Xiang Song, Danai Koutra

KDD 2023 Workshop on Mining and Learning with Graphs

2023

Graph Neural Networks (GNNs) have demonstrated promising outcomes across various tasks, including node classification and link prediction. Despite their remarkable success in various high-impact applications, we have identified three common pitfalls in message passing for link prediction, especially within industrial settings. Particularly, in prevalent GNN frameworks (e.g., DGL and PyTorchGeometric), the

Search and information retrieval

T3GDT: Three-tier tokens to Guide Decision Transformer for offline meta reinforcement learning

Zhe Wang, Haozhu Wang, Yanjun (Jane) Qi

NeurIPS 2023 Workshop on Robot Learning

2023

Offline meta-reinforcement learning (OMRL) aims to generalize an agent’s knowledge from training tasks with offline data to a new unknown RL task with few demonstration trajectories. This paper proposes T3GDT: Three-tier tokens to Guide Decision Transformer for OMRL. First, our approach learns a global token from its demonstrations to summarize a RL task’s transition dynamic and reward pattern. This global

Machine learning

Your representations are in the network: composable and parallel adaptation for large scale models

Yonatan Dukler, Alessandro Achille, Hao Yang, Ben Bowman, Varsha Vivek, Luca Zancato, Avinash Ravichandran, Charless Fowlkes, Ashwin Swaminathan, Stefano Soatto

Sumegh Roychowdhury, Karan Gupta, Siva Rajesh Kasa, Prasanna Srinivasa Murthy, Alok Chandra

2023

We present a framework for transfer learning that efficiently adapts a large basemodel by learning lightweight cross-attention modules attached to its intermediate activations. We name our approach InCA (Introspective-Cross-Attention) and show that it can efficiently survey a network’s representations and identify strong performing adapter models for a downstream task. During training, InCA enables training

Computer vision

Tackling concept shift in text classification using entailment-style modeling

NeurIPS 2023 Workshop on Distribution Shifts (DistShifts)

2023

Pre-trained language models (PLMs) have seen tremendous success in text classification (TC) problems in the context of Natural Language Processing (NLP). In many real-world text classification tasks, the class definitions being learned do not remain constant but rather change with time - this is known as concept shift. Most techniques for handling concept shift rely on retraining the old classifiers with

Conversational AI / Natural-language processing

PreDiff: Precipitation nowcasting with latent diffusion models

Zhihan Gao, Xingjian Shi, Boran Han, Hao Wang, Xiaoyong Jin, Danielle Maddix Robinson, Yi Zhu, Mu Li, Yuyang (Bernie) Wang

Prasenjit Dey, Srujana Merugu, Sivaramakrishnan (Siva) Kaveri

2023

Earth system forecasting has traditionally relied on complex physical models that are computationally expensive and require significant domain expertise. In the past decade, the unprecedented increase in spatiotemporal Earth observation data has enabled data-driven forecasting models using deep learning techniques. These models have shown promise for diverse Earth system forecasting tasks. However, they

Machine learning

Conformal prediction sets for ordinal classification

2023

Ordinal classification (OC), i.e., labeling instances along classes with a natural ordering, is common in multiple applications such as disease severity labeling and size or budget based recommendations. Often in practical scenarios, it is desirable to obtain a small set of likely classes with a guaranteed high chance of including the true class. Recent works on conformal prediction (CP) address this problem

Machine learning

An inductive bias for tabular deep learning

Ege Beyazit, Jonathan Kozaczuk, Bo Li, Vanessa Wallace, Bilal Fadlallah

RecSys 2023 Workshop on Causality, Counterfactuals & Sequential Decision-Making (CONSEQUENCES)

2023

Deep learning methods have achieved state-of-the-art performance in most modeling tasks involving images, text and audio, however, they typically underperform tree-based methods on tabular data. In this paper, we hypothesize that a significant contributor to this performance gap is the interaction between irregular target functions resulting from the heterogeneous nature of tabular feature spaces, and the

Machine learning

Varun Vallabhan

Applied Scientist

Offline recommender system evaluation under unobserved confounding

Olivier Jeunen, Ben London

2023

Off-Policy Estimation (OPE) methods allow us to learn and evaluate decision-making policies from logged data. This makes them an attractive choice for the offline evaluation of recommender systems, and several recent works have reported successful adoption of OPE methods to this end. An important assumption that makes this work, is the absence of unobserved confounders: random variables that influence both

Search and information retrieval

An H-field simulation method to solve wireless desensitization due to the DDR noise

Fu Luo-Larson, Amrithaa Seshadri, Akshay Mohan

EMC+SIPI 2023

2023

Most consumer electronics nowadays integrate multi-radios and high-speed memory interfaces into a very compact form-factor. High speed digital noise is one of common aggressors for desensitization. In this paper, a comprehensive EM simulation workflow is used to analyze the coupling mechanism from the DDR power plane, and optimize the decoupling capacitor value and location to minimize the desense to the

Operations research and optimization

Fu Luo-Larson

Sr. Wireless System Engineer

Amrithaa Seshadri

RF/EMC Systems Engineer II

Parameter-efficient cross-language transfer learning for a language-modular audiovisual speech recognition

Zhengyang Li, Thomas Graave, Jing Liu, Timo Lohrenz, Siegfried Kunzmann, Tim Fingscheidt

ASRU 2023

2023

In audiovisual speech recognition (AV-ASR), for many languages only few audiovisual data is available. Building upon an English model, in this work, we first apply and analyze various adapters for cross-language transfer learning to build a parameter-efficient and easy-to-extend AV-ASR in multiple languages. Fine-tuning only the bottleneck adapter with 4% of encoder’s parameters and the decoder shows comparable

Conversational AI / Natural-language processing

Vy Phan

Data Scientist

Mohammad Ebrahimpour

Data Scientist

Samarth Gupta

Amazon Postdoctoral Scientist

Shoufa Chen

Applied Scientist

Leian Chen

Applied Scientist

Theodore Rogers

Applied Scientist

Optimizing neural networks for special-purpose hardware

Sankalp Dayal, Mahdi Heydari

November 10, 2023

Curating the neural-architecture search space and taking advantage of human intuition reduces latency on real-world applications by up to 55%.

Cloud and systems

Enhancing user intent capture in session-based recommendation with attribute patterns

Xin Liu, Zheng Li, Yifan Gao, Jingfeng Yang, Tianyu Cao, Zhengyang Wang, Bing Yin, Yangqiu Song

2023

The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history. However, constructing global or local transition graphs to supplement session data can lead to noisy correlations and user intent vanishing. In this work, we propose the Frequent Attribute Pattern Augmented Transformer (FAPAT) that characterizes

Conversational AI / Natural-language processing

Multi-teacher distillation for multilingual spelling correction

Jingfen Zhang, Xuan Guo, Sravan Bodapati, Christopher Potts

Masha Belyi, Charlotte Dzialo, Chaitanya Dwivedi, Prajit Reddy Muppidi, Kanna Shimizu

2023

Accurate spelling correction is a critical step in modern search interfaces, especially in an era of mobile devices and speech-to-text inter-faces. For services that are deployed around the world, this poses a significant challenge for multilingual NLP: spelling errors need to be caught and corrected in all languages, and even in queries that use multiple languages. In this paper, we tackle this challenge

Conversational AI / Natural-language processing

Ryan Tibshirani recognized for contributions to statistics

John Roach

November 09, 2023

Ryan Tibshirani, right, an Amazon Scholar, is seen on stage receiving the Committee of Presidents of Statistical Societies (COPSS) Presidents’ Award at the Joint Statistical Meetings in Toronto in August. Handing him the award is Michael R. Kosorok, president of the Institute of Mathematical Statistics.

Amazon Scholar received Committee of Presidents of Statistical Societies Presidents’ Award for his achievements in statistics.

Jing Huang

Principal, Applied Scientist

Piyush Gupta

Data Scientist

Yun Luo

Applied Scientist

Personalized dense retrieval on global index for voice-enabled conversational systems

2023

Voice-controlled AI dialogue systems are susceptible to noise from phonetic variations and failure to resolve ambiguous entities. Typically, personalized entity resolution (ER) and/or query rewrites (QR) are deployed to recover from these error modes. Previous work in this field achieves personalization by constraining retrieval search space to personalized indices built from user’s historical interactions

Conversational AI / Natural-language processing

Pre-training intent-aware encoders for zero- and few-shot intent classification

Mujeen Sung, James Gung, Elman Mansimov, Nikolaos Pappas, Raphael Shu, Salvatore Romeo, Yi Zhang, Vittorio Castelli

2023

Intent classification (IC) plays an important role in task-oriented dialogue systems. However, IC models often generalize poorly when training without sufficient annotated examples for each user intent. We propose a novel pre-training method for text encoders that uses contrastive learning with intent pseudo-labels to produce embeddings that are well-suited for IC tasks, reducing the need for manual annotations

Conversational AI / Natural-language processing

Measuring and mitigating dialog-to-API constraint violations of in-context learning

Shufan Wang, Sebastien Jean, Sailik Sengupta, James Gung, Nikolaos Pappas, Yi Zhang

2023

In executable task-oriented semantic parsing, the system aims to translate users’ utterances in natural language to machine-interpretable programs (API calls) that can be executed according to pre-defined API specifications. With the popularity of Large Language Models (LLMs), in-context learning offers a strong baseline for such scenarios, especially in data-limited regimes (Hu et al., 2022; Shin et al

Conversational AI / Natural-language processing

CESAR: Automatic induction of compositional instructions for multi-turn dialogs

Taha Aksu, Devamanyu Hazarika, Shikib Mehri, Seokhwan Kim, Dilek Hakkani-Tür, Yang Liu, Mahdi Namazifar

2023

Instruction-based multitasking has played a critical role in the success of large language models (LLMs) in multi-turn dialog applications. While publicly available LLMs have shown promising performance, when exposed to complex instructions with multiple constraints, they lag against state-of-the-art models like Chat-GPT. In this work, we hypothesize that the availability of large-scale complex demonstrations

Conversational AI / Natural-language processing

A multi-modal multilingual benchmark for document image classification

Yoshinari Fujinuma, Siddharth Varia, Nishant Sankaran, Bonan Min, Srikar Appalaraju, Yogarshi Vyas

2023

Document image classification is different from plain-text document classification and consists of classifying a document by understanding the content and structure of documents such as forms, emails, and other such documents. We show that the only existing dataset for this task (Lewis et al., 2006) has several limitations and we introduce two newly curated multilingual datasets (WIKI-DOC and MULTIEURLEX

Conversational AI / Natural-language processing

Plan, verify and switch: Integrated reasoning with diverse x-of-thoughts

Tengxiao Liu, Qipeng Guo, Yuqing Yang, Xiangkun Hu, Yue Zhang, Xipeng Qiu, Zheng Zhang

2023

As large language models (LLMs) have shown effectiveness with different prompting methods, such as Chain of Thought, Program of Thought, we find that these methods have formed a great complementarity to each other on math reasoning tasks. In this work, we propose XoT, an integrated problem solving framework by prompting LLMs with diverse reasoning thoughts. For each question, XoT always begins with selecting

Conversational AI / Natural-language processing

Semantic matching for text classification with complex class descriptions

Brian de Silva, Kuan-Wen Huang, Gwang Lee, Karen Hovsepian, Yan Xu, Mingwei Shen

2023

Text classifiers are an indispensable tool for machine learning practitioners, but adapting them to new classes is expensive. To reduce the cost of new classes, previous work exploits class descriptions and/or labels from existing classes. However, these approaches leave a gap in the model development cycle as they support either zero- or few-shot learning but not both. Existing classifiers either do not

Conversational AI / Natural-language processing

End-to-end single-channel speaker-turn aware conversational speech translation

Juan Pablo Zuluaga Gomez, Zhaocheng Huang, Xing Niu, Rohit Paturi, Sundararajan Srinivasan, Prashant Mathur, Brian Thompson, Marcello Federico

Leonardo Ribeiro, Mohit Bansal, Markus Dreyer

2023

Conventional speech-to-text translation (ST) systems are trained on single-speaker utterances, and they may not generalize to real-life scenarios where the audio contains conversations by multiple speakers. In this paper, we tackle single-channel multi-speaker conversational ST with an end-to-end and multi-task training model, named Speaker-Turn Aware Conversational Speech Translation, that combines automatic

Conversational AI / Natural-language processing

Generating summaries with controllable readability levels

Sanjay Agrawal, Vivek Sembium, Ankith M S

2023

Readability refers to how easily a reader can understand a written text. Several factors affect the readability level, such as the complexity of the text, its subject matter, and the reader’s background knowledge. Generating summaries based on different readability levels is critical for enabling knowledge consumption by diverse audiences. However, current text generation approaches lack refined control

Conversational AI / Natural-language processing

KD-Boost: Boosting real-time semantic matching in e-commerce with knowledge distillation

2023

Real-time semantic matching is vital to web and product search. Transformer-based models have shown to be highly effective at encoding queries into an embedding space where semantically similar entities (queries or results) are in close proximity. However, the computational complexity of large transformer models limits their utilization for real-time matching. In this paper, we propose KD-Boost, a novel

Machine learning

NameGuess: Column name expansion for tabular data

Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Shen Wang, Huzefa Rangwala, George Karypis

2023

Recent advances in large language models have revolutionized many sectors, including the database industry. One common challenge when dealing with large volumes of tabular data is the pervasive use of abbreviated column names, which can negatively impact performance on various data search, access, and understanding tasks. To address this issue, we introduce a new task, called NameGuess, to expand column

Conversational AI / Natural-language processing

INVITE: A testbed of automatically generated invalid questions to evaluate large language models for hallucinations

Anil Ramakrishna, Rahul Gupta, Jens Lehmann, Morteza Ziyadi

Aditya Anantharaman, Kai Zhong

2023

Recent advancements in Large language models (LLMs) have enabled them to hold free-form conversations over multiple turns, but they exhibit a tendency to make unfounded and incorrect statements, commonly known as hallucinations. In particular, LLMs hallucinate frequently when given invalid questions, i.e. ones with incorrect assumptions. The most common approach to evaluate LLMs on hallucinations is to

Conversational AI / Natural-language processing

Wilko Schulz-Mahlendorf

Director, Economist

Ranjith Krishnamurthy

Applied Scientist

Nikhil Vichare

Applied Scientist

Hanliang Zhang

Applied Scientist

Emery Berger

Amazon Scholar

Rana Basheer

Applied Scientist

Simon Miller

Data Scientist

Yu Chen

Applied Scientist

Amazon and Howard University announce inaugural fellowship, gift project recipients

Staff writer

November 06, 2023

Howard fellowship gift project recipients 2023.png

One Howard student and five faculty members awarded for projects that elevate differentiated research.

Machine learning

Cyrus DSouza

Research Scientist

Amazon and UIUC announce inaugural slate of funded research projects

Staff writer

November 03, 2023

UIUC AICE research awards Lead Image_231103113005.png

Amazon-Illinois Center on Artificial Intelligence for Interactive Conversational Experiences also names first cohort of academic fellows.

Conversational AI / Natural-language processing

Sean McCracken

Data Scientist

Diego Mazariegos

Manager, Data Science

Mukund Ratan Agrawal

Applied Scientist

Using teacher knowledge at inference time to enhance student model

November 02, 2023

New method improves the state of the art in knowledge distillation by leveraging a knowledge base of teacher predictions.

Conversational AI / Natural-language processing

Putra Manggala

Applied Scientist

Shayan Jawed

Applied Scientist

Moritz Haas

Applied Scientist

Nicolas Ugrinovic Kehdy

Applied Scientist

Alice Bizeul

Applied Scientist

Jin Xu

Applied Scientist

Creating a public repository for joining private data

James Cook, Milind Shyani, Nina Mishra

2023

How can one publish a dataset with sensitive attributes in a way that both preserves privacy and enables joins with other datasets on those same sensitive attributes?This problem arises in many contexts, e.g., a hospital and an airline may want to jointly determine whether people who take long-haul flights are more likely to catch respiratory infections. If they join their data by a common keyed user identifier

Machine learning

Fortuna: A library for uncertainty quantification in deep learning

Gianluca Detommaso, Alberto Gasparin, Michele Donini, Matthias Seeger, Andrew Wilson, Cédric Archambeau

Journal of Machine Learning Research

2023

We present Fortuna, an open-source library for uncertainty quantification in deep learning. Fortuna supports a range of calibration techniques, such as conformal prediction that can be applied to any trained neural network to generate reliable uncertainty estimates, and scalable Bayesian inference methods that can be applied to deep neural networks trained from scratch for improved uncertainty quantification

Machine learning

Complex query answering on eventuality knowledge graph with implicit logical constraints

Jiaxin Bai, Xin Liu, Weiqi Wang, Chen Luo, Yangqiu Song

NeurIPS 2022 Workshop on Interactive Learning for NLP

2023

Querying incomplete knowledge graphs (KGs) using deep learning approaches can naturally leverage the reasoning and generalization ability to learn to infer better answers. Traditional neural complex query answering (CQA) approaches mostly work on entity-centric KGs. However, in the real world, we also need to make logical inferences about events, states, and activities (i.e., eventualities or situations

Information and knowledge management

An interactive framework for image retrieval

Nikhil Mehta, Fan Yang, Stephen Rawls, Loris Bazzani, Chengwei Su, Emre Barut

2023

While online shopping, customers often see a product that they have a preference for, but do not purchase it due to not liking a few aspects of the product (e.g., sleeve type or stripe colors on a shirt), and thus have to continue their search. Instead, if the customer were to select a preferred product and issue a modification query, and the system could find a similar product with the desired modification

Conversational AI / Natural-language processing

DeTiME: Diffusion-enhanced topic modeling using encoder-decoder based LLM

Weijie Xu, Wenxiang Hu, Fanyou Wu, Srinivasan Sengamedu, "SHS"

2023

In the burgeoning field of natural-language processing, Neural Topic Models (NTMs) and Large Language Models (LLMs) have emerged as areas of significant research interest. Despite this, NTMs have predominantly leveraged contextual embeddings from LLMs, neglecting the potential benefits of harnessing the overall structure. Our study addresses this gap by introducing a novel framework named Diffusion-Enhanced

Conversational AI / Natural-language processing

Rahul Bhatta

Economist

Dekang Lin

Senior Principal Scientist

Fiona Anting Tan

Applied Scientist

Nadav Borenstein

Applied Scientist

Saurav Manchanda

Applied Scientist

Muqsit Azeem

Applied Scientist

Jing Sun

Data Scientist

ZHENYU SONG

Applied Scientist

FMCAD 2023

Formal Methods in Computer-Aided Design provides a leading forum to researchers in academia and industry for presenting and discussing groundbreaking methods, technologies, theoretical results, and tools for reasoning formally about computing systems. FMCAD covers formal aspects of computer-aided system design including verification, specification, synthesis, and testing.

Automated reasoning

October 23 - 27, 2023

Ames, Iowa

Better foundation models for video representation

David Fan

October 27, 2023

Motion vectors — which are common in popular video formats — can be used to efficiently track regions of interest across multiple frames of video to generate motion-aware masks that improve video representation learning.

Computer vision

Coordinated replay sample selection for continual federated learning

Jack Good, Jimit Majmudar, Christophe Dupuy, Jixuan Wang, Charith Peris, Clement Chung, Richard Zemel, Rahul Gupta

2023

Continual Federated Learning (CFL) combines Federated Learning (FL), the decentralized learning of a central model on a number of client devices that may not communicate their data, and Continual Learning (CL), the learning of a model from a continual stream of data without keeping the entire history. In CL, the main challenge is forgetting what was learned from past data. While replay-based algorithms

Conversational AI / Natural-language processing

More-efficient recovery from failures during large-ML-model training

Zhuang Wang, Zhen Jia

October 25, 2023

Novel “checkpointing” scheme that uses CPU memory reduces the time wasted on failure recovery by more than 92%.

Cloud and systems

From aerospace to Amazon, Nia Jetter is blazing new paths

Staff writer

October 24, 2023

Jetter says her goals include lowering barriers to understanding technology and cultivating a more diverse workforce.

Robotics

Jianshu Chen

Principal, Applied Scientist

Maryam Mirzaei

Data Scientist

Kartik Gairola

Data Scientist

George Wang

Applied Scientist

Guangrui Li

Applied Scientist

Sudiksha Joshi

Economist

Henry Feng

Data Scientist

Gemini: Fast failure recovery in distributed training with in-memory checkpoints

Zhuang Wang, Zhen Jia, Shuai Zhang, Zhen Zhang, Mason Fu, T. S. Eugene Ng, Yida Wang

ACM 2023 Symposium on Operating Systems Principles (SOSP)

2023

Large deep learning models have recently garnered substantial attention from both academia and industry. Nonetheless, frequent failures are observed during large model training due to large-scale resources involved and extended training time. Existing solutions have significant failure recovery costs due to the severe restriction imposed by the bandwidth of remote storage in which they store checkpoints

Machine learning

Siming Mao

Data Scientist

Joyce Lu

Data Scientist

Geographical erasure in language generation

Pola Schwöbel, Jacek Golebiowski, Michele Donini, Cédric Archambeau, Danish Pruthi

2023

Large language models (LLMs) encode vast amounts of world knowledge. However, since these models are trained on large swaths of internet data, they are at risk of inordinately capturing information about dominant groups. This imbalance can propagate into generated language. In this work, we study and operationalise a form of geographical erasure, wherein language models underpredict certain countries. We

Machine learning

Influence scores at scale for efficient language data sampling

Nikhil Anand, Joshua Tan, Maria Minakova

Benjamin Kiesl-Reiter, Mike Whalen

2023

Modern ML systems ingest data aggregated from diverse sources, such as synthetic, human-annotated, and live customer traffic. Understanding which examples are important to the performance of a learning algorithm is crucial for efficient model training. Recently, a growing body of literature has given rise to various “influence scores,” which use training artifacts such as model confidence or checkpointed

Conversational AI / Natural-language processing

SAAML: A framework for semi-supervised affective adaptation via metric learning

Minh Tran, Yelin Kim, Che-Chun Su, Cheng-Hao Kuo, Mohammad Soleymani

ACMMM 2023

2023

Socially intelligent systems such as home robots should be able to perceive emotions and social behaviors. Affect recognition datasets have limited labeled data, and existing large unlabeled datasets, e.g., VoxCeleb2, suitable for pre-training, mostly contain neutral expressions, limiting their application to affective downstream tasks. We introduce a novel Semi-supervised Affective Adaptation framework

Computer vision

Proving that solutions to incremental satisfiability problems are correct

October 20, 2023

Method enables machine-checkable proofs of SAT solvers’ decisions on incremental SAT problems, in which problem constraints are gradually imposed over time.

Automated reasoning

Deep metric learning to hierarchically rank—An application in product retrieval

Kee Kiat Koo, Ashutosh Joshi, Nishaanth Reddy, Ismail Tutar, Vaclav Petricek, Changhe Yuan, Karim Bouyarmane

2023

Most e-commerce search engines use customer behavior signals to augment lexical matching and improve search relevance. Many ecommerce companies like Amazon, Alibaba, Ebay etc. operate in multiple countries with country specific stores. However, customer behavior data is sparse in newer stores. To compensate for sparsity of behavioral data in low traffic stores, search engines often use crosslisted products

Search and information retrieval

Nishaanth Reddy

Multi document summarization evaluation in the presence of damaging content

Avshalom Manevich, David Carmel, Nachshon Cohen, Elad Kravi, Ori Shapira

2023

In the Multi-document summarization (MDS) task, a summary is produced for a given set of documents. A recent line of research introduced the concept of damaging documents, denoting documents that should not be exposed to readers due to various reasons. In the presence of damaging documents, a summarizer is ideally expected to exclude damaging content in its output. Existing metrics evaluate a summary based

Conversational AI / Natural-language processing

Re-examining summarization evaluation across multiple quality criteria

Ori Ernst, Ori Shapira, Ido Dagan, Ran Levy

2023

The common practice for assessing automatic evaluation metrics is to measure the correlation between their induced system rankings and those obtained by reliable human evaluation, where a higher correlation indicates a better metric. Yet, an intricate setting arises when an NLP task is evaluated by multiple Quality Criteria (QCs), like for text summarization where prominent criteria include relevance, consistency

Conversational AI / Natural-language processing

Knowledge-selective pretraining for attribute value extraction

Hui Liu, Qingyu Yin, Zhengyang Wang, Chenwei Zhang, Haoming Jiang, Yifan Gao, Zheng Li, Xian Li, Chenwei Zhang, Bing Yin, William Wang, Xiaodan Zhu

2023

Attribute Value Extraction (AVE) aims to retrieve the values of attributes from the product profiles. The state-of-the-art methods tackle the AVE task through a question-answering (QA) paradigm, where the value is predicted from the context (i.e. product profile) given a query (i.e. attributes). Despite of the substantial advancements that have been made, the performance of existing methods on rare attributes

Information and knowledge management

Strong and efficient baselines for open domain conversational question answering

Andrei C. Coman, Gianni Barlacchi, Adrià de Gispert

2023

Unlike the Open Domain Question Answering (ODQA) setting, the conversational (ODConvQA) domain has received limited attention when it comes to reevaluating baselines for both efficiency and effectiveness. In this paper, we study the State-of-the-Art (SotA) Dense Passage Retrieval (DPR) retriever and Fusion-in-Decoder (FiD) reader pipeline, and show that it significantly underperforms when applied to ODConvQA

Conversational AI / Natural-language processing

Efficient long-range transformers: You need to attend more, but not necessarily at every layer

Qingru Zhang, Dhananjay Ram, Cole Hawkins, Sheng Zha, Tuo Zhao

Yang He, Emre Eftelioglu, Mohamed Moustafa, Amber Roy Chowdhury

2023

Pretrained transformer models have demonstrated remarkable performance across various natural language processing tasks. These models leverage the attention mechanism to capture long- and short-range dependencies in the sequence. However, the (full) attention mechanism incurs high computational cost — quadratic in the sequence length, which is not affordable in tasks with long sequences, e.g., inputs with

Conversational AI / Natural-language processing

A highly efficient and elective attribute learning framework for road graph from aerial imagery and GPS

ACM 2023 SIGSPATIAL Workshop on Analytics for Big Geospatial Data

2023

Road attributes play a pivotal role in digital maps, providing critical information for various routing and planning applications that aim to create a safe and efficient traffic environment. While some road attributes are available in existing map data such as OpenStreetMap [3], these sources may not cover all regions, meet highquality standards, or include specific attributes required for specialized applications

Machine learning

Improving consistency for text summarization with energy functions

Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang

Bryan Zhang, Salah Danial, Stephan Walter

2023

Current abstractive summarization models often generate inconsistent content, i.e. texts that are not directly inferable from the source document, are not consistent with respect to world knowledge, or are self-contradictory. These inconsistencies motivate a new consistency taxonomy that we define as faithfulness, factuality, and self-supportiveness. However, most recent work on reducing inconsistency in

Conversational AI / Natural-language processing

Enhancing Arabic machine translation for e-commerce product information: Data quality challenges and innovative selection approaches

EMNLP 2023 Workshop on Arabic Natural Language Processing (ArabicNLP)

2023

Product information in e-commerce is usually localized using machine translation (MT) systems. The Arabic language has rich morphology and dialectal variations, so Arabic MT in e-commerce training requires a larger volume of data from diverse data sources; Given the dynamic nature of e-commerce, such data needs to be acquired periodically to update the MT. Consequently, validating the quality of training

Machine learning

Tabletop transparent scene reconstruction via epipolar-guided optical flow with monocular depth completion prior

Xiaotong Chen, Zheming Zhou, Zhuo Deng, Omid Ghasemalizadeh, Min Sun, Cheng-Hao Kuo, Arnie Sen

IEEE RAS Humanoids 2023

2023

Reconstructing transparent objects using affordable RGB-D cameras is a persistent challenge in robotic perception due to inconsistent appearances across views in the RGB domain and inaccurate depth readings in each single-view. We introduce a two-stage pipeline for reconstructing transparent objects tailored for mobile platforms. In the first stage, off-theshelf monocular object segmentation and depth completion

Computer vision

Distributionally robust Bayesian optimization with ϕ-divergences

Hisham Husain, Vu Nguyen, Anton van den Hengel

NeurIPS 2023, NeurIPS 2022 Workshop on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems

2023

The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. [26], which bridges the existing literature

Machine learning

AdaBERT-CTC: Leveraging BERT-CTC for text-only domain adaptation in ASR

Tyler Vuong, Karel Mundnich, Dhanush Bekal, Veera Raghavendra Elluru, Srikanth Ronanki, Sravan Bodapati

Srinivasan Sengamedu, "SHS", Hien Pham

2023

End-to-end (E2E) automatic speech recognition (ASR) models are becoming increasingly popular in commercial applications, such as virtual assistants, closed captioning, and dictation systems. The accuracy of the ASR is crucial to their success. However, E2E models still struggle to recognize out-of-domain words such as proper nouns and domain-specific terms. In this paper we introduce AdaBERT-CTC, a domain

Conversational AI / Natural-language processing

Towards sequential counterfactual learning to rank

Tesi Xiao, Branislav Kveton, Sumeet Katariya, Tanmay Gangwani, Anshuka Rangi

SIGIR-AP 2023

2023

Counterfactual evaluation plays a crucial role in learning-to-rank problems, as it addresses the discrepancy between the data logging policy and the policy being evaluated, due to the presence of presentation bias. Existing counterfactual methods, which are based on the empirical risk minimization framework, aim to evaluate the ability of a ranking policy to produce optimal results for a single query using

Search and information retrieval

PhraseSumm: Abstractive short phrase summarization

Kasturi Bhattacharjee, Kathleen McKeown, Rashmi Gangadharaiah

IJCNLP-AACL 2023

2023

Prior work in the field of text summarization mostly focuses on generating summaries that are a sentence or two long. In this work, we introduce the task of abstractive short-phrase summarization (PhraseSumm), which aims at capturing the central theme of a document through a generated short phrase. We explore BART & T5-based neural summarization models, and measure their effectiveness for the task using

Conversational AI / Natural-language processing

FAIRLABEL: Correcting bias in labels

ICDM 2023

2023

There are several algorithms for measuring fairness of ML models. A fundamental assumption in these approaches is that the ground truth is fair or unbiased. In real-world datasets, however, the ground truth often contains data that is a result of historical and societal biases and discrimination. Models trained on these datasets will inherit and propagate the biases to the model outputs. We propose FAIRLABEL

Machine learning

Discriminative speech recognition rescoring with pre-trained language models

Prashanth Gurunath Shivakumar, Jari Kolehmainen, Yi Gu, Ankur Gandhe, Ariya Rastrow, Ivan Bulyko

ASRU 2023

2023

Second pass rescoring is a critical component of competitive automatic speech recognition (ASR) systems. Large language models have demonstrated their ability in using pre-trained information for better rescoring of ASR hypothesis. Discriminative training, directly optimizing the minimum word-errorrate (MWER) criterion typically improves rescoring. In this study, we propose and explore several discriminative

Machine learning

Tianqin Cai

Applied Scientist

INFORMS 2023

Held each fall, the annual meeting brings together over 6,500 people to the world's largest operations research and analytics conference. It features more than 800 sessions and presentations, opportunities to meet with leading companies, universities and other exhibitors, an onsite career fair connecting top talent with over 100 organizations at the forefront of O.R. and analytics application, and other networking and educational events.

Operations research and optimization

October 15 - 18, 2023

Phoenix, Arizona

Shweta Khare

Applied Scientist

Huirong Ai

Data Scientist

Johns Hopkins and Amazon announce new AI2AI research awards

Staff writer

October 17, 2023

A photo grid shows JHU research award recipients. Top row, left to right: Jiang Liu, Ambar Pal, Aniket Roy, and Xuan Zhang; second row, left to right: Rama Chellappa, Anjalie Field, Philipp Koehn, and Leibny Paola Garcia Perera; fourth row, left to right: Vishal Patel, Carey Priebe, Jan Trmal, and Masha Yarmohammadi.

Research award recipients named as part of the JHU + Amazon Initiative for Interactive AI (AI2AI), now in its second year.

Machine learning

Rajat Shrivastava

Data Scientist

Richard Yan

Economist

Anurag Kumar Mishra

Applied Scientist

Shadi Iskander

Applied Scientist

Yifan YU

Research Scientist

Meng-Jhang Fong

Amazon Postdoctoral Scientist

Jay Pillai

Senior Manager, Applied Science

Nadia El hanine

Applied Scientist

Hannah Sansford

Applied Scientist

Mary Beth Kurz

Amazon Scholar

Frank Tanner

Manager, Data Science

Yassine Nemmour

Applied Scientist

Pierre Osselin

Applied Scientist

Zach Jones

Data Scientist

Seonwoo Min

Applied Scientist

“I want to help people automate boring tasks”

Staff writer

October 16, 2023

Margarida Ferreira is seen sitting cross legged on an empty road on a sunny day, she is smiling and there is a snow covered mountain range and trees in the background

Former Amazon applied science intern Margarida Ferreira conducts research to make complex cloud resources easier to manage.

Cloud and systems

Prompt pre-training with twenty-thousand classes for open-vocabulary visual recognition

Shuhuai Ren, Aston Zhang, Yi Zhu, Shuai Zhang, Shuai Zheng, Mu Li, Alex Smola, Xu Sun

2023

This work proposes POMP, a prompt pre-training method for vision-language models. Being memory and computation efficient, POMP enables the learned prompt to condense semantic information for a rich set of visual concepts with over twenty-thousand classes. Once pre-trained, the prompt with a strong transferable ability can be directly plugged into a variety of visual recognition tasks including image classification

Computer vision

Enhancing abstractiveness of summarization models through calibrated distillation

Hwanjun Song, Igor Shalyminov, Hang Su, Siffi Singh, Kaisheng Yao, Saab Mansour

Veera Raghavendra Elluru, Devang Kulshreshtha, Rohit Paturi, Sravan Bodapati, Srikanth Ronanki

2023

Sequence-level knowledge distillation reduces the size of Seq2Seq models for more efficient abstractive summarization. However, it often leads to a loss of abstractiveness in summarization. In this paper, we propose a novel approach named DisCal to enhance the level of abstractiveness (measured by n-gram overlap) without sacrificing the informativeness (measured by ROUGE) of generated summaries. DisCal

Conversational AI / Natural-language processing

Saliency driven imagery preprocessing for efficient compression - Industrial paper

Justin Downes, Sam Saltwick, Anthony Chen

ACM 2023 SIGSPATIAL Workshop on Analytics for Big Geospatial Data

2023

The compression of satellite imagery remains an important research area as hundreds of terabytes of images are collected every day, which drives up storage and bandwidth costs. Although progress has been made in increasing the resolution of these satellite images, many downstream tasks are only interested in small regions of any given image. These areas of interest vary by task but, once known, can be used

Computer vision

Generalized zero-shot audio-to-intent classification

ASRU 2023

2023

Spoken language understanding systems using audio-only data are gaining popularity, yet their ability to handle unseen intents remains limited. In this study, we propose a generalized zero-shot audio-to-intent classification framework with only a few sample text sentences per intent. To achieve this, we first train a supervised audio-to-intent classifier by making use of a self-supervised pre-trained model

Conversational AI / Natural-language processing

Gerard Ion Gallego Olsina

Applied Scientist

Andrea Taverna

Applied Scientist

Egor Burlakov

Senior Manager, Data Science

2023 SCOT INFORMS scholarship recipients announced

Staff writer

October 13, 2023

2023 SCOT INFORMS recipients: Fernando Acosta-Perez, Adeola Adegbemijo, Nathan Adeyemi, Grace Babalola, Ayesha Farooq, Henry Ivuawuogu, Caroline Johnston, Jiayue-Sylvia Li, Ogechi Vivian Nwadiaru, Paula Penagos, Anastasia Rivera, Ryan Rodriguez, Austin Iglesias Saragih, Jessica Shi, and Morgan Wood

Program is aimed at expanding participation in operations research, management science, and analytics research for those from underrepresented backgrounds.

Operations research and optimization

Uncertainty quantification

Gianluca Detommaso, Alberto Gasparin, Oleg Smirnov, Thomas Pinder, Christian Leibig, Paul Scemama

2023

Proper estimation of predictive uncertainty is fundamental in applications that involve critical decisions. Uncertainty can be used to assess the reliability of model predictions, trigger human intervention, or decide whether a model can be safely deployed in the wild. We introduce Fortuna, an open-source library for uncertainty quantification. Fortuna provides calibration methods, such as conformal prediction

Machine learning

Learning to learn learning-rate schedules

Shreyas Subramanian, Vignesh Ganapathiraman

October 12, 2023

In a series of papers, Amazon researchers performed a theoretical analysis of a simplified problem that led to a learnable learning-rate scheduler, applied that scheduler to a more complex neural model, and distilled the results into a practical algorithm.

Machine learning

Charlie Bushrow

Data Scientist

USC and Amazon select six new research projects

Staff writer

October 10, 2023

Faculty and academic-fellow projects are focused on various aspects of trustworthy machine learning.

Security, privacy, and abuse prevention

How CloudTune generates forecasts for the Amazon Store

John Roach

October 10, 2023

Animation shows a flow of dots (historical data) flowing through a CloudTune forecasting icon to generate forecasts, it also includes some detailed shots of pretend peak event forecasts for the US and India.

The system has expanded from generating peak computation-load forecasts one year in advance to a series of forecasts that include per-minute forecasts several months into the future.

Economics

Anna Li

Data Scientist

Meng Jiang

Amazon Visiting Academic

Chenwei Ran

Applied Scientist

Yanbo Wang

Applied Scientist

Yifan Li

Applied Scientist

Thomas Palmeira Ferraz

Applied Scientist

Siva Somayyajula

Applied Scientist

Sangram Kadam

Manager, Data Science

Mrinal Mathur

Applied Scientist

Yuncong Chen

Applied Scientist

Rajat Koner

Applied Scientist

Sneha Rajan

Data Scientist

Dominik Winterer

Applied Scientist

Hierarchical attention-based contextual biasing for personalized speech recognition using neural transducers

Sibo Tong, Philip Harding, Simon Wiesler

ASRU 2023

2023

Although end-to-end (E2E) automatic speech recognition (ASR) systems excel in general tasks, they frequently struggle with accurately recognizing personal rare words. Leveraging contextual information to bias the internal states of E2E ASR model has proven to be an effective solution. However most existing work focuses on biasing for a single domain and it is still challenging to expand such contextualization

Conversational AI / Natural-language processing

Wenshuo Chen

Applied Scientist

ICCV 2023

ICCV is the premier international computer vision event comprising the main conference and several co-located workshops and tutorials.

Computer vision

October 2 - 6, 2023

Paris, France

Teaching household robots where to find requested objects

Gunnar A. Sigurdsson

October 06, 2023

Leveraging a large vision-language foundation model enables state-of-the-art performance in remote-object grounding.

Robotics

IROS 2023

The International Conference on Intelligent Robots and Systems (IROS) showcases leading-edge research in robotics. The conference focuses on future directions in robotics, and the latest approaches, designs, and outcomes. It also provides an opportunity to network with the world’s leading roboticists.

Robotics

October 1 - 5, 2023

Detroit, Michigan