Learning algorithms

How we add new skills to Alexa’s name-free skill selector

Young-Bum Kim

May 3, 2019

Cosine_normalization.jpg._CB465442559_.jpg

Using cosine similarity rather than dot product to compare vectors helps prevent "catastrophic forgetting".

Conversational AI

On the irrelevance of machine learning algorithms and the importance of relativity

Carlos Huertas, Qi Zhao

IEEE ICME 2023

2023

Information explosion has brought us a wide range of data formats and machine learning keeps in constant evolution to develop mechanisms to extract knowledge from them. Modern models in the Deep Learning space have proven to be very successful in multiple applications, yet in the tabular space they fail to provide consistent competitive performance. However, in this work we claim model selection can become

Machine learning

To correct imbalances in training data, don’t oversample: Cluster

Ming Sun

March 11, 2019

Model depicting the stages in the training of an embedding network

In experiments involving sound recognition, technique reduces error rate by 15% to 30%.

Machine learning

New method for compressing neural networks better preserves accuracy

Anish Acharya, Rahul Goel

January 15, 2019

Neural networks have been responsible for most of the top-performing AI systems of the past decade, but they tend to be big, which means they tend to be slow. That’s a problem for systems like Alexa, which depend on neural networks to process spoken requests in real time.

Conversational AI

Meta-learning the difference: Preparing large language models for efficient adaptation

Zejiang Hou, Julian Salazar, George Polovets

Transactions of the Association for Computational Linguistics

2022

Large pretrained language models (PLMs) are often domain- or task-adapted via finetuning or prompting. Finetuning requires modifying all of the parameters and having enough data to avoid overfitting while prompting requires no training and few examples but limits performance. Instead, we prepare PLMs for data- and parameter-efficient adaptation by learning to learn the difference between general and adapted

Conversational AI

Optimal continual learning has perfect memory and is NP-HARD

Jeremias Knoblauch, Hisham Husain, Tom Diethe

ICML 2020

2020

Continual Learning (CL) algorithms incrementally learn a predictor or representation across multiple sequentially observed tasks. Designing CL algorithms that perform reliably and avoid so-called catastrophic forgetting has proven a persistent challenge. The current paper develops a theoretical approach that explains why. In particular, we derive the computational properties which CL algorithms would have

Machine learning

Introducing Alexa for e-learning

Jinjin Zhao, Shreyansh Bhatt, Candace Thille, Dawn Zimmaro, Neelesh Gattani, Josh Walker

ACM L@S 2020

2020

E-learning is becoming popular as it provides learners the flexibility, targeted resources across the internet, personalized guidance, and immediate feedback during learning. However, lack of social interaction, an indispensable component in developing some skills, has been a pain point in e-learning. We propose using Alexa, a voice-controlled Intelligent Personal Assistants (IPA), in e-learning to provide

Conversational AI

FastPoint: Scalable deep point processes

Alex Smola, Caner Turkmen, Yuyang (Bernie) Wang

ECML-PKDD 2019

2019

We propose FastPoint, a novel multivariate point process that enables fast and accurate learning and inference. FastPoint uses deep recurrent neural networks to capture complex temporal dependency patterns among different marks, while self-excitation dynamics within each mark are modeled with Hawkes processes. This results in substantially more efficient learning and scales to millions of correlated marks

Information and knowledge management

Learning fashion traits with label uncertainty

Assaf Neuberger, Sharon Alpert, Eli Alshan, Nati Bubis, Eduard Oks

CVPR 2018

2018

We consider the task of predicting subjective fashion traits from images. Specifically, we are interested in understanding which outfit actually better suites the user. Since these traits are highly subjective, they tend to be noisier. One solution is to annotate each example several times, but this makes it hard to collect large amounts of data.

Machine learning

signSGD: compressed optimisation for non-convex problems

Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, Animashree Anandkumar

ICML 2018

2018

Training large neural networks requires distributing learning across multiple workers, where the cost of communicating gradients can be a significant bottleneck. SIGNSGD alleviates this problem by transmitting just the sign of each minibatch stochastic gradient. We prove that it can get the best of both worlds: compressed gradients and SGD-level convergence rate. The relative `1/`2 geometry of gradients

Machine learning

Learning algorithms

Work with us