Vancouver, Canada

3 important themes from Amazon's 2019 NeurIPS papers

Time series forecasting, bandit problems, and optimization are integral to Amazon's efforts to deliver better value for its customers.

Last year, the first 2,000-2,500 publicly released tickets to the Conference on Neural Information Processing Systems, or NeurIPS, sold out in 12 minutes.

This year, the conference organizers moved to a lottery system, allowing aspiring attendees to register in advance and randomly selecting invitees from the pool of registrants. But they also bumped the number of public-release tickets up from around 2,000 to 3,500, testifying to the conference’s continued popularity.

At NeurIPS this year, there are 26 papers with Amazon coauthors. They cover a wide range of topics, but surveying their titles, Alex Smola, a vice president and distinguished scientist in the Amazon Web Services organization, discerns three prominent themes, all tied to Amazon’s efforts to deliver better value for its customers.

Those three themes are time series forecasting (and causality), bandit problems, and optimization.

1. Time series forecasting

Time series forecasting involves measuring some quantity over time — such as the number of deliveries in a particular region in the past six months, or the number of cloud servers required to support a particular site over the past two years — and attempting to project that quantity into the future.

“That’s something that is very dear to Amazon’s heart,” Smola says. “For anything that Amazon does, it’s really beneficial to have a good estimate of what our customers will expect from us ahead of time. Only by being able to do that will we be able to satisfy customers’ demands, be it for products or services.”

A sequence of basis time series, forecast into the near future and summed together to approximate a new time series.
The paper “Think Globally, Act Locally” examines data sets with many correlated time series, such as the demand curves for millions of products sold online. The researchers describe a method for constructing a much smaller set of “basis time series”; the time series for any given product can be approximated by a weighted sum of the bases.
Courtesy of the researchers

The basic mathematical framework for time series forecasting is a century old, but the scale of modern forecasting problems calls for new analytic techniques, Smola says.

“Problems are nowadays highly multivariate,” Smola says. “If you look at the many millions of products that we offer, you want to be able to predict fairly well what will sell, where and to whom.

“You need to make reasonable assumptions on how this very large problem can be decomposed into smaller, more tractable pieces. You make structural approximations, and sometimes those structural approximations are what leads to very different algorithms.

“So you might, for instance, have a global model, and then you have local models that address the specific items or address the specific sales. If you look at ‘Think Globally, Act Locally’” — a NeurIPS paper whose first author is Rajat Sen, an applied scientist in the Amazon Search group — “it’s already in the title. Or look at ‘High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes’. In this case, you have a global structure, but it’s only in a small subspace where interesting things happen.”

Side-by-side images depict correlations between taxi traffic at different points in Manhattan at different times of day
The paper "High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes" describes a method for predicting correlations among many parallel time series. In one example, the researchers forecast correlations between the taxi traffic at different points in New York City at different times of day. Red lines indicate strong correlations; blue lines indicate strong negative correlations. Weekend midday traffic patterns (left) show negative correlations between locations near the Empire State Building, suggesting that taxis tend to prefer different routes depending on traffic conditions. Weekend evening traffic patterns show positive correlations between the vicinity of the Empire State Building and areas with high concentrations of hotels.
Courtesy of the researchers

An aspect of forecasting that has recently been drawing more attention, Smola says, is causality. Where traditional machine learning models merely infer statistical correlations between data points, “it is ultimately the causal relationship that matters,” Smola says.

“I think that causality is one of the most interesting conceptual developments affecting modern machine learning,” says Bernhard Schölkopf, like Smola a vice president and distinguished scientist in Amazon Web Services. “This is the main topic that I have been interested in for the last decade.”

Two of Schölkopf’s NeurIPS papers — “Perceiving the Arrow of Time in Autoregressive Motion” and “Selecting Causal Brain Features with a Single Conditional Independence Test per Feature” — address questions of causality, as does “Causal Regularization”, a paper by Dominik Janzing, a senior research scientist in Smola’s group.

“Normal machine learning builds on correlations of other statistical dependences,” Schölkopf explains. “This is fine as long as the source of the data doesn't change. For example, if in the training set of an image recognition system, all cows are standing on green pasture, then it is fine for an ML system to use the green as a useful feature in recognizing cows, as long as the test set looks the same. If in the test set, the cows are standing on the beach, then such a purely statistical system can fail.

“More generally: causal learning and inference attempts to understand how systems respond to interventions and other changes, and not just how to predict data that looks more or less the same as the training data.”

2. Bandit problems

The second major theme that Smola discerns in Amazon scientists’ NeurIPS papers is a concern with bandit problems, a phrase that shows up in the titles of Amazon papers such as “MaxGap Bandit: Adaptive Algorithms for Approximate Ranking” and “Low-Rank Bandit Methods for High-Dimensional Dynamic Pricing”. Bandit problems take their name from one-armed bandits, or slot machines.

“It used to be that those bandits were all mechanical, so there would be slight variations between them, and some would have maybe a slightly a higher return than others,” Smola explains. “I walk into a den of iniquity, and I want to find the one-armed bandit where I will lose the least money or maybe make some money. And the only feedback I have is that I pull arms, and I get money or lose money. These are very unreliable, noisy events.”

Bandit problems present what’s known as an explore-exploit trade-off. The gambler must simultaneously explore the environment — determine which machines pay out the most — and exploit the resulting knowledge — concentrate as much money as possible on the high-return machines. Early work on bandit problems concerned identifying the high-return machines with minimal outlays.

“That problem was solved about 20 years ago,” Smola says. “What hasn’t been solved — and this is where things get a lot more interesting — is once you start adding context. Imagine that I get to show you various results as you’re searching for your next ugly Christmas sweater. The unfortunate thing is that the creativity of sweater designers is larger than what you can fit on a page. Now the context is essentially, what time, where from, which user, all those things. We want to find and recommend the ugly Christmas sweater that works specifically for you. This is an example where context is immediately relevant.”

It’s really beneficial to have a good estimate of what our customers will expect from us ahead of time. Only by being able to do that will we be able to satisfy customers’ demands.
Alex Smola, VP and distinguished scientist, Amazon

In the bandit-problem framework, in other words, the high-payout machines change with every new interaction. But there may be external signals that indicate how they’re changing.

Distributed computing, which is inescapable for today’s large websites, changes the structure of the bandit problem, too.

“Say you go to a restaurant, and the cook wants to improve the menu,” Smola says. “You can try out lots of new menu items, and that’s a good way to improve the menu overall. But if you start offering a lot of undercooked dishes because you’re experimenting, then at some point your loyal customers will stay away.

“Now imagine you have 100 restaurants, and they all do the same thing at the same time. They can’t necessarily communicate at the per-second level; maybe every day or every week they chat with each other. Now this entire exploration problem becomes a little more challenging, because if two restaurants try out the same undercooked dish, you make the customer less happy than you could have.

“So how does this map back into Amazon land? Well, if you have many servers doing this recommendation, the explore-exploit trade-off might be too aggressive if every one of them works on their own.”

3. Optimization

Finally, Smola says, “There is a third category of results that has to do with making algorithms faster. If you look at ‘Primal-Dual Block Frank-Wolfe’, ‘Communication-Efficient Distributed SGD with Sketching’, ‘Qsparse-Local-SGD’ — those are the workhorses that run underneath all of this. Making them more efficient is obviously something that we care about, so we can respond to customer requests faster, train algorithms faster.”

Bird’s-eye view

NeurIPS is a huge conference, with more than 1,400 accepted papers that cover a bewildering variety of topics. Beyond the Amazon papers, Caltech professor and Amazon fellow Pietro Perona identifies three research areas as growing in popularity.

“One is understanding how deep networks work, so that we can better design architectures and optimization algorithms to train models,” Perona says. “Another is low-shot learning. Machines are still much less efficient than humans at learning, in that they need more training examples to achieve the same performance. And finally, AI and society — identifying opportunities for social good, sustainable development, and the like.”

NeurIPS is being held this year at the Vancouver Convention Center, and the main conference runs from Dec. 8 to Dec. 12. The Women in Machine Learning Workshop, for which Amazon is a gold-level sponsor, takes place on Dec. 9; the Third Conversational AI workshop, whose organizers include Alexa AI principal scientist Dilek Hakkani-Tür, will be held on Dec. 14.

Amazon's involvement at NeurIPS

Paper and presentation schedule

Tuesday, 12/10 | 10:45-12:45pm | East Exhibition Hall B&C

A Meta-MDP Approach to Exploration for Lifelong Reinforcement Learning | #192
Francisco Garcia (UMass Amherst/Amazon) · Philip Thomas (UMass Amherst)

Blocking Bandits | #17
Soumya Basu (UT Austin) · Rajat Sen (UT Austin/Amazon) · Sujay Sanghavi (UT Austin/Amazon) · Sanjay Shakkottai (UT Austin)

Causal Regularization | #180
Dominik Janzing (Amazon)

Communication-Efficient Distributed SGD with Sketching | #81
Nikita Ivkin (Amazon) · Daniel Rothchild (University of California, Berkeley) · Md Enayat Ullah (Johns Hopkins University) · Vladimir Braverman (Johns Hopkins University) · Ion Stoica (UC Berkeley) · Raman Arora (Johns Hopkins University)

Learning Distributions Generated by One-Layer ReLU Networks | #49
Shanshan Wu (UT Austin) ·Alexandros G. Dimakis (UT Austin) · Sujay Sanghavi (UT Austin/Amazon)

Tuesday, 12/10 | 5:30-7:30pm | East Exhibition Hall B&C

Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control | #195
Sai Qian Zhang (Harvard University) · Qi Zhang (Amazon) · Jieyu Lin (University of Toronto)

Extreme Classification in Log Memory using Count-Min Sketch: A Case Study of Amazon Search with 50M Products | #37
Tharun Kumar Reddy Medini (Rice University) · Qixuan Huang (Rice University) · Yiqiu Wang (Massachusetts Institute of Technology) · Vijai Mohan (Amazon) · Anshumali Shrivastava (Rice University/Amazon)

Iterative Least Trimmed Squares for Mixed Linear Regression | #50
Yanyao Shen (UT Austin) · Sujay Sanghavi (UT Austin/Amazon)

Meta-Surrogate Benchmarking for Hyperparameter Optimization | #6
Aaron Klein (Amazon) · Zhenwen Dai (Spotify) · Frank Hutter (University of Freiburg) · Neil Lawrence (University of Cambridge) · Javier Gonzalez (Amazon)

Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification and Local Computations | #32
Debraj Basu (Adobe) · Deepesh Data (UCLA) · Can Karakus (Amazon) · Suhas Diggavi (UCLA)

Selecting Causal Brain Features with a Single Conditional Independence Test per Feature | #139
Atalanti Mastakouri (Max Planck Institute for Intelligent Systems) · Bernhard Schölkopf (MPI for Intelligent Systems/Amazon) · Dominik Janzing (Amazon)

Wednesday, 12/11 | 10:45-12:45pm | East Exhibition Hall B&C

On Single Source Robustness in Deep Fusion Models | #93
Taewan Kim (Amazon) · Joydeep Ghosh (UT Austin)

Perceiving the Arrow of Time in Autoregressive Motion | #155
Kristof Meding (University Tübingen) · Dominik Janzing (Amazon) · Bernhard Schölkopf (MPI for Intelligent Systems/Amazon) · Felix A. Wichmann (University of Tübingen)

Wednesday, 12/11 | 5:00-7:00pm | East Exhibition Hall B&C

Compositional De-Attention Networks | #127
Yi Tay (Nanyang Technological University) · Anh Tuan Luu (MIT) · Aston Zhang (Amazon) · Shuohang Wang (Singapore Management University) · Siu Cheung Hui (Nanyang Technological University)

Low-Rank Bandit Methods for High-Dimensional Dynamic Pricing | #3
Jonas Mueller (Amazon) · Vasilis Syrgkanis (Microsoft Research) · Matt Taddy (Amazon)

MaxGap Bandit: Adaptive Algorithms for Approximate Ranking | #4
Sumeet Katariya (Amazon/University of Wisconsin-Madison) · Ardhendu Tripathy (UW Madison) · Robert Nowak (UW Madison)

Primal-Dual Block Generalized Frank-Wolfe | #165
Qi Lei (UT Austin) · Jiacheng Zhuo (UT Austin) · Constantine Caramanis (UT Austin) · Inderjit S Dhillon (Amazon/UT Austin) · Alexandros Dimakis (UT Austin)

Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling | #208
Tengyang Xie (University of Illinois at Urbana-Champaign) · Yifei Ma (Amazon) · Yu-Xiang Wang (UC Santa Barbara)

Thursday, 12/12 | 10:45-12:45pm | East Exhibition Hall B&C

AutoAssist: A Framework to Accelerate Training of Deep Neural Networks | #155
Jiong Zhang (UT Austin) · Hsiang-Fu Yu (Amazon) · Inderjit S Dhillon (UT Austin/Amazon)

Exponentially Convergent Stochastic k-PCA without Variance Reduction | #200 (oral, 10:05-10:20 W Ballroom C)
Cheng Tang (Amazon)

Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift | #54
Stephan Rabanser (Technical University of Munich/Amazon) · Stephan Günnemann (Technical University of Munich) · Zachary Lipton (Carnegie Mellon University/Amazon)

High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes | #107
David Salinas (Naverlabs) · Michael Bohlke-Schneider (Amazon) · Laurent Callot (Amazon) · Jan Gasthaus (Amazon) · Roberto Medico (Ghent University)

Learning Search Spaces for Bayesian Optimization: Another View of Hyperparameter Transfer Learning | #30
Valerio Perrone (Amazon) · Huibin Shen (Amazon) · Matthias Seeger (Amazon) · Cedric Archambeau (Amazon) · Rodolphe Jenatton (Amazon)

Mo’States Mo’Problems: Emergency Stop Mechanisms from Observation | #227
Samuel Ainsworth (University of Washington) · Matt Barnes (University of Washington) · Siddhartha Srinivasa (University of Washington/Amazon)

Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting | #113
Rajat Sen (Amazon) · Hsiang-Fu Yu (Amazon) · Inderjit S Dhillon (UT Austin/Amazon)

Thursday, 12/12 | 5:00-7:00pm | East Exhibition Hall B&C

Dynamic Local Regret for Non-Convex Online Forecasting | #20
Sergul Aydore (Stevens Institute of Technology) · Tianhao Zhu (Stevens Institute of Technology) · Dean Foster (Amazon)

Interaction Hard Thresholding: Consistent Sparse Quadratic Regression in Sub-quadratic Time and Space | #47
Suo Yang (UT Austin), Yanyao Shen (UT Austin), Sujay Sanghavi (UT Austin/Amazon)

Inverting Deep Generative Models, One Layer at a Time |#48
Qi Lei (University of Texas at Austin) · Ajil Jalal (UT Austin) · Inderjit S Dhillon (UT Austin/Amazon) · Alexandros Dimakis (UT Austin)

Provable Non-linear Inductive Matrix Completion| #215
Kai Zhong (Amazon) · Zhao Song (UT Austin) · Prateek Jain (Microsoft Research) · Inderjit S Dhillon (UT Austin/Amazon)

Amazon researchers on NeurIPS committees and boards

  • Bernhard Schölkopf – Advisory Board
  • Michael I. Jordan – Advisory Board
  • Thorsten Joachims – senior area chair
  • Anshumali Shrivastava – area chair
  • Cedric Archambeau – area chair
  • Peter Gehler – area chair
  • Sujay Sanghavi – committee member

Workshops

Learning with Rich Experience: Integration of Learning Paradigms

Paper: "Meta-Q-Learning" | Rasool Fakoor, Pratik Chaudhari, Stefano Soatto, Alexander J. Smola

Human-Centric Machine Learning

Paper: "Learning Fair and Transferable Representations" | Luco Oneto, Michele Donini, Andreas Maurer, Massimiliano Pontil

Bayesian Deep Learning

Paper: "Online Bayesian Learning for E-Commerce Query Reformulation" | Gaurush Hiranandani, Sumeet Katariya, Nikhil Rao, Karthik Subbian

Meta-Learning

Paper: "Constrained Bayesian Optimization with Max-Value Entropy Search" | Valerio Perrone, Iaroslav Shcherbatyi, Rodolphe Jenatton, Cedric Archambeau, Matthias Seeger

Paper: "A Quantile-Based Approach to Hyperparameter Transfer Learning" | David Salinas, Huibin Shen, Valerio Perrone

Paper: "A Baseline for Few-Shot Image Classification" | Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, Stefano Soatto

Conversational AI

Organizer: Dilek Hakkani-Tür

Paper: "The Eighth Dialog System Technology Challenge" | Seokhwan Kim, Michel Galley, Chulaka Gunasekara, Sungjin Lee, Adam Atkinson, Baolin Peng, Hannes Schulz, Jianfeng Gao, Jinchao Li, Mahmoud Adada, Minlie Huang, Luis Lastras, Jonathan K. Kummerfeld, Walter S. Lasecki, Chiori Hori, Anoop Cherian, Tim K. Marks, Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta

Paper: “Just Ask: An Interactive Learning Framework for Vision and Language Navigation” | Ta-Chung Chi, Minmin Shen, Mihail Eric, Seokhwan Kim, Dilek Hakkani-Tur

Paper: “MA-DST: Multi-Attention-Based Scalable Dialog State Tracking” | Adarsh Kumar, Peter Ku, Anuj Kumar Goyal, Angeliki Metallinou, Dilek Hakkani-Tür

Paper: “Investigation of Error Simulation Techniques for Learning Dialog Policies for Conversational Error Recovery” | Maryam Fazel-Zarandi, Longshaokan Wang, Aditya Tiwari, Spyros Matsoukas

Paper: “Towards Personalized Dialog Policies for Conversational Skill Discovery”| Maryam Fazel-Zarandi, Sampat Biswas, Ryan Summers, Ahmed Elmalt, Andy McCraw, Michael McPhillips, John Peach

Paper: “Conversation Quality Evaluation via User Satisfaction Estimation” | Praveen Kumar Bodigutla, Spyros Matsoukas, Lazaros Polymenakos

Paper: “Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering” | Li Zhou, Kevin Small

Science Meets Engineering of Deep Learning

Paper: "X-BERT: eXtreme Multi-label Text Classification using Bidirectional Encoder from Transformers" Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, Inderjit S. Dhillon

Machine Learning with Guarantees

Organizers: Ben London, Thorsten Joachims
Program Committee: Kevin Small, Shiva Kasiviswanathan, Ted Sandler

MLSys: Workshop on Systems for ML

Paper: "Block-Distributed Gradient Boosted Trees" | Theodore Vasiloudis, Hyunsu Cho, Henrik Boström

Women in Machine Learning

Gold sponsor: Amazon

About the Author
Larry Hardesty is a science writer at Amazon. Previously, he was managing editor of the Boston Book Review, a senior editor at MIT Technology Review, and the computer science writer at the MIT News Office.

Work with us

See More Jobs
AU, NSW, Sydney
DESCRIPTIONAmazon Web Services is the largest consumer cloud offering in the world, powering cutting-edge science, rapidly growing startups, and industry leaders. Our team here in Sydney is developing novel products focused on helping our customers achieve higher availability standards for the applications and workloads they build on AWS. We are building solutions that aim to learn from past incidents data to improve automatic behaviours and help investigate and resolve incidents faster in real time. We are working on complex problems including automatic incident investigation and root-causing, as well as other ideas for assisting our customers. This is an exciting opportunity if you love solving challenging problems and contributing to high impact customer features involving real-life data and Machine Learning methods, while working within a development team to see the results deployed in production, and being part of the large global community of Amazon scientists.We are looking for an Applied Scientist who is enthusiastic about diving deep into the data to analyse and uncover relevant patterns, and is excited to go beyond the standard toolbox when needed, to research and create novel methods for producing the best possible solutions to the problems our customers are facing. The methodology and algorithm development are highly coupled here with deep data analysis and understanding of the real-life data.Are you interested in solving complex real-life challenges which are not easily translatable to textbook examples, where the data is rarely clean or readily available, and its amount varies from immense to lacking? We need a practitioner who loves designing a new model or an algorithm, and also working with the rest of the development team to implement it efficiently in code and deploy to production.While working on these problems, you will help to grow the Machine Learning and Data Science team and the ML community in Sydney, collaborate with the global Amazon research community, and publish research results. You will engage with other teams across Amazon who are working on related problems, and communicate with product management and stakeholders to provide clear answers for business questions, present the findings and new proposals.In this role you will build something brand new that will positively affect every AWS customer and in turn their customers, while being at the forefront of using and developing novel Machine Learning techniques to deliver new functionality. Join us, and help us revolutionise how people approach availability!Responsibilities Summary· Design and implement predictive models based on past data, using either existing methods or developing new algorithms and tools when needed.· Collect and analyze data, develop hypotheses, design experiments, collaborate with engineering team to implement live tests and evaluate performance.· Answer complex business questions by using appropriate statistical techniques on available data or designing and running experiments to gather data.· Design, develop, and operate critical processes that are central to the delivery of accurate results (e.g. data pipelines).· Code in a high-level language (e.g. Python, Java) and work with software engineers to produce high-quality software in production, based on research models.· Communicate findings to managers and engineers, often through succinct written summaries of findings and code samples, as well as through verbal explanation.· Publish research results.
US, CA, San Francisco
The new Amazon Scholars program has broadened opportunities for academics to join Amazon in a flexible capacity, in particular part-time arrangements and sabbaticals.The Amazon Scholars program is designed for academics from universities around the globe who want to apply research methods in practice and help us solve hard technical challenges without leaving their academic institutions. We believe that Amazon is a unique place to measure the impact of new scientific ideas, given our scale and our ownership of both an information infrastructure and physical infrastructure. You will have a chance to have a ground-up impact on our systems, our business, and most importantly, our customers, through your expertise.Applications are accepted from academic experts in research areas including, but not limited to, the following: Artificial Intelligence, Avionics, Computer Vision, Data Science, Economics, Machine Learning, Optimization, Natural Language Processing, and Robotics.As an Amazon Scholar, your responsibilities may include 1) advising business leaders on strategic plans, 2) diving deep to solve a specific technical problem in an organization’s roadmap, and 3) advising junior researchers on methods.We’re working on the future here at Amazon. Come join us.
US, CA, San Francisco
Are you interested in building, leading and driving the machine learning technical vision, strategy and implementation of Personalized ML products for Customers? Build and lead an innovative team of scientists and engineers who will invent and apply advanced machine learning science and techniques to create the best customer experience on the earth.The ideal candidate will be an expert in the areas of reccomender systems, data science, machine learning and statistics. You will have hands-on experience leading multiple simultaneous product development initiatives as well as be able to balance technical leadership with strong business judgment to make the right decisions about technology, models and methodologies choices. You will strive for simplicity, and demonstrate significant creativity and high judgment backed by statistical proof.
US, WA, Bellevue
The Alexa Experience organization owns a number of programs and domains which drive high customer engagement and feature discovery, maintains customer trust, understands customers’ feedback, develops new features that provide utility value for customers with special needs, and develops locally relevant experiences. Our team develop and train natural language understanding models to improve these programs and to create new delighting Alexa experiences for customers.We’re looking for a passionate scientist to help design, develop and release to production NLU models that provide the best-possible experience for our customers. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to develop new features and predict key user behaviors, both offline and in real time.What You Will Do· Be the subject matter expert in NLU model authoring process, guiding multiple product teams toward delivering best customer experiences on Alexa.· Contribute to the design of new features by extending existing annotation schemas to cover new areas· Develop and release to production Alexa NLU models· Measure and analyze accuracy of models; develop solutions to improve performances· Contribute to the research vision of the team, develop and execute on a roadmap that addresses the technical challenges our product teams face· Raise the bar for research quality and impact. Stay up to date on research results both within and complementary to the Alexa technology space and helping your team and its partner to utilize tried patterns and state of the art results to work smarter and faster.· Propose new modeling solutions and run experiments to prove their efficacy· Deliver reports and/or models to influence technical directions.· Design processes and tools to simplify Alexa feature expansion for your team and partners.· Perform data analysis, A/B experimentation and modeling with large data sets to develop insights that increase device usage and customer experience· Collaborate with peers in data analysis and feature design efforts
US, CA, San Francisco
We are looking for a passionate and talented Applied Scientist who will collaborate with other scientists and engineers to leverage machine learning methods and algorithms for the modeling and analysis of data.You'll design and run experiments, research new algorithms, and find new ways of optimizing customer experience. Besides theoretical analysis and innovation, you will work closely with talented engineers to put your algorithms and models into practice. Your work will directly impact the trust customers place in Amazon.You are the ideal candidate if you are passionate about delivering experiences that delight customers and creating solutions that are robust. You thrive in ambiguous environments that require to find solutions to problems that have not been solved before. You leverage your exceptional technical expertise, a sound understanding of the fundamentals of Computer Science, and practical experience to creating reliable, scalable, and high performance products. Your strong communication skills enable you to work effectively with both business and technical partners. You have hands-on experience making the right decisions about technology, models and methodology choices. You strive for simplicity, and demonstrate significant creativity and high judgment backed by statistical proof.Responsibilities include:· Design, implement, test, deploy and maintain innovative data and machine learning solutions to further the customer experience.· Create experiments and prototype implementations of new learning algorithms and prediction techniques· Collaborate with scientists, engineers, product managers and business stakeholders to design and implement software solutions· Use machine learning best practices to ensure a high standard of quality for all of the team deliverables
US, WA, Seattle
Are you interested in building and driving the technical vision, strategy, and implementation for AWS Hardware? AWS Hardware is hiring a Senior Data Scientist to help lead the definition and prioritization of our customer focused technologies and services. AWS Hardware is responsible for designing, qualifying, and maintaining server solutions for AWS and its customers as well as developing new cloud focused hardware solutions. You will be a senior technical leader in the existing Data Sciences and Analytics Team, build, and drive the data science and analytics needed for our product development and operations.As a Senior Data Scientist, you are considered a data science leader on your team. You take the lead on large projects and drive solutions to complex or ambiguous problems. You design solutions that help us take advantage of business opportunities (new and existing). You may own the delivery of data science solutions for an entire business application. You are a key influencer of your team’s business strategy are able to influence the strategies of related teams. You work efficiently and routinely deliver the right things with limited guidance. Last, you are externally aware of the state-of-the-art in the data science field and are constantly focused on advancing the state-of-the-art for improving Amazon’s products and services.The ideal candidate will be able to solve complex or critical problems using data science principles that are highly accurate, stable, secure, scalable, and cost-effective. You will have hands-on experience leading multiple simultaneous product development initiatives as well as be able to balance technical leadership with strong business judgment to make the right decisions about technology, models and methodologies choices.You will strive for simplicity, and demonstrate significant creativity and high judgment backed by statistical proof. As a Senior Data Scientist, you will be responsible for developing KPIs, programs, tests and research in order to drive business solutions using computational techniques, and statistics to support strategic and tactical decision-making. The ideal candidate will have significant experience performing large-scale data analysis and reporting and will be responsible for developing, improving and simplifying our models. This person has the opportunity to leverage extensive datasets and provide meaningful insights to improve the customer experience as well as optimize business decisions for Amazon.This position requires a solution-oriented candidate with a combination of deep business acumen, expert knowledge of statistics and algorithm development, and an analytical mindset. The candidate must have the ability to work with diverse stakeholder groups to solve business problems and provide data solutions that are organized and simple to understand. This is a highly visible position that will interact at all levels of the business. Become part of this unique opportunity to make history defining the evolution of cloud computing.
LU, Luxembourg
Amazon is looking for a Data Scientist to provide statistical and analytical insights from the vast array of services powering one of the world’s largest Reliability, Maintenance Engineering (RME) platforms. As part of the Automation Engineering team, we support the business in all aspects of the traditional automation pyramid, and we provide our internal users information on all aspects of the status of control systems within Amazon’s EU Fulfillment Network. At a basic level, these systems link our low-level automation systems with the cloud and we work at the cutting edge of all aspects of the automation pyramid, from device level to the enterprise level. The more complex systems are leveraging Machine Learning and Big Data to drive predictive actions, preventing downtime or defects in the material handling equipment and improving the Overall Equipment Efficiency of the installations.This role requires an individual with excellent statistical and analytical abilities, professional experience applying data science methodologies and data engineering practices as well as outstanding business acumen and ability to work with various teams across Amazon. The successful candidate will be a self-starter comfortable with ambiguity, with strong attention to detail, an ability to work in a fast-paced and ever-changing environment, and driven by a desire to innovate in this space.Responsibilities:You are a significant and autonomous contributor. Your work is consistently of high quality. You are able to use a range of data science methodologies to conduct analysis for cases when the solution approach is unclear that relate to a portion of a business or business process. You apply a breadth of experience from practical application of data science techniques and tools to solve difficult business problems. Your work is focused on team-level goals, medium size projects, and small subsets of larger goals. You are able to ramp up quickly on new areas where colleagues identify established solutions. You are expected to consistently demonstrate a combination of the following:· You independently own and solve difficult business problems. These may be well-defined problems where the solution has not yet been outlined or approach to solve is unclear.· You deliver artifacts on medium size projects that affect important business decisions. You define the methodology and own the analysis.· You are able to gather and use complex data set across domains. You proactively gather data when it is not readily available.· You skilfully employ a range of data science methods, tools, and best practices. You are able to justify your approach.· You write clear and factually correct documents explaining technical concept to non-technical audience.· You have good working relationships with team-mates and peers working on related areas. You recognize discordant views and take part in constructive dialogue to resolve them.· You confidently train new team-mates about your customers, how your team’s solutions work, and how those solutions are reflected in the data.
IN, KA, Bangalore
Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.The Moderation and Relevance System (MARS) team, based in Bangalore, is responsible for ensuring that ads are relevant and is of good quality, leading to higher conversion for the sellers and providing a great experience for the customers. We deal with one of the world’s largest product catalog, handle billions of requests a day with plans to grow it by order of magnitude and use automated systems to validate tens of millions of offers submitted by thousands of merchants in multiple countries and languages. We are looking for a highly motivated, top notch applied scientist to build machine learning models at scale to enforce our policy guidelines. A successful candidate will have demonstrated experience in at least some of the following areas: NLP, Image Recognition and Classification, Video Recognition and Classification, Generative Models, Reinforcement Learning, Active Learning, Weak SupervisionYour areas of responsibility include:· · Designing and implementing new features and machine learned models, including the application of state-of-art deep learning to solve ad policy enforcement and creative intelligence, including NLP, deep image and video models, generative models · Perform analysis of data and metrics relevant to ad content generation and policing · Gathering ad policy related requirements from business owners, other tech teams, as well as by analyzing customer feedback and translate them into modeling problems · Integrate and productize ML models with overall engineering infrastructure to be made available at scaleAd Quality protects the customer experience and is a critical component of our business success. One of the earliest teams to be established in Amazon Bangalore, Ad Quality has both Operations and Development teams in Bangalore supporting multiple ad programs in markets around the world.sspajobs
IN, KA, Bangalore
Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.The Moderation and Relevance System (MARS) team, based in Bangalore, is responsible for ensuring that ads are relevant and is of good quality, leading to higher conversion for the sellers and providing a great experience for the customers. We deal with one of the world’s largest product catalog, handle billions of requests a day with plans to grow it by order of magnitude and use automated systems to validate tens of millions of offers submitted by thousands of merchants in multiple countries and languages. We are looking for a highly motivated, top notch applied scientist to build machine learning models at scale to enforce our policy guidelines. A successful candidate will have demonstrated experience in at least some of the following areas: NLP, Image Recognition and Classification, Video Recognition and Classification, Generative Models, Reinforcement Learning, Active Learning, Weak SupervisionYour areas of responsibility include:· · Designing and implementing new features and machine learned models, including the application of state-of-art deep learning to solve ad policy enforcement and creative intelligence, including NLP, deep image and video models, generative models · Perform analysis of data and metrics relevant to ad content generation and policing · Gathering ad policy related requirements from business owners, other tech teams, as well as by analyzing customer feedback and translate them into modeling problems · Integrate and productize ML models with overall engineering infrastructure to be made available at scaleAd Quality protects the customer experience and is a critical component of our business success. One of the earliest teams to be established in Amazon Bangalore, Ad Quality has both Operations and Development teams in Bangalore supporting multiple ad programs in markets around the world.sspajobs
IN, KA, Bangalore
Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.The Moderation and Relevance System (MARS) team, based in Bangalore, is responsible for ensuring that ads are relevant and is of good quality, leading to higher conversion for the sellers and providing a great experience for the customers. We deal with one of the world’s largest product catalog, handle billions of requests a day with plans to grow it by order of magnitude and use automated systems to validate tens of millions of offers submitted by thousands of merchants in multiple countries and languages. We are looking for a highly motivated, top notch applied scientist to build machine learning models at scale to enforce our policy guidelines. A successful candidate will have demonstrated experience in at least some of the following areas: NLP, Image Recognition and Classification, Video Recognition and Classification, Generative Models, Reinforcement Learning, Active Learning, Weak SupervisionYour areas of responsibility include:· · Designing and implementing new features and machine learned models, including the application of state-of-art deep learning to solve ad policy enforcement and creative intelligence, including NLP, deep image and video models, generative models · Perform analysis of data and metrics relevant to ad content generation and policing · Gathering ad policy related requirements from business owners, other tech teams, as well as by analyzing customer feedback and translate them into modeling problems · Integrate and productize ML models with overall engineering infrastructure to be made available at scaleAd Quality protects the customer experience and is a critical component of our business success. One of the earliest teams to be established in Amazon Bangalore, Ad Quality has both Operations and Development teams in Bangalore supporting multiple ad programs in markets around the world.sspajobs
US, WA, Seattle
Amazon is looking for a passionate, talented, and inventive Senior Scientist with a strong machine learning background to help build industry-leading Speech and Language technology. Our mission is to push the envelope in Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), and Audio Signal Processing, in order to provide the best-possible experience for our customers.As a Senior Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in spoken language understanding. Your work will directly impact our customers in the form of products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding.We are hiring in all areas of spoken language understanding: ASR, NLU, text-to-speech (TTS), and Dialog Management.
US, TN, Nashville
Workforce Staffing (WFS), a division of Amazon’s Worldwide Operations Human Resources, manages Amazon’s Tier 1 talent supply chain. We attract, hire, and onboard the associates who, by fulfilling orders at the frontlines of the company, make Amazon a global leader in delivery and logistics. The mission of the Candidate Experience Research Team (CER) is to drive positive change to Workforce Staffing’s hiring and onboarding experience. We conduct research and analyses on opportunities and successes for the business, and develop products that amplify the voice of our candidates.We are seeking a manager, data science, with a heavy focus on quantitative data analysis and evaluation, and a deep focus on understanding labor markets. You will be responsible for building a new team from the ground up, develop roadmaps, and drive business impact through your research at global scale.The ideal candidate should be well versed in quantitative methods, including classical statistics and machine learning approaches. Competitive candidates will be very comfortable with at least one computational language (e.g., R, Python). Candidates should be comfortable combing through computational models, machine learning algorithms, and analyzing their output.Candidates should have demonstrated experience leading data science projects related to labor market research and analysis, including research on wage sensitivity and elasticity, workforce relevance, and other factors.A customer-obsessed, relentless curiosity is a must, as is commitment to the highest standards of methodological rigor that a given study allows. This role provides opportunity for significant exposure to Amazon’s culture, leadership, and global businesses, and furthermore provides significant opportunity to influence how Workforce Staffing matches talent to business demand.This role will be located in Nashville.If you’re hungry to engage and empower Amazon Associates your expertise in mixed methods research, let's talk.
US, WA, Seattle
Workforce Staffing (WFS), a division of Amazon’s Worldwide Operations Human Resources, manages Amazon’s Tier 1 talent supply chain. We attract, hire, and onboard the associates who, by fulfilling orders at the frontlines of the company, make Amazon a global leader in delivery and logistics. The mission of the Candidate Experience Research Team (CER) is to drive positive change to Workforce Staffing’s hiring and onboarding experience. We conduct research and analyses on opportunities and successes for the business, and develop products that amplify the voice of our candidates.We are seeking a research scientist with deep expertise in diversity research, with a heavy focus on quantitative data analysis and evaluation.The ideal candidate should be well versed in quantitative methods, including classical statistics and machine learning approaches. Competitive candidates will be very comfortable with at least one computational language (e.g., R, Python). Candidates should be comfortable combing through computational models, machine learning algorithms, and analyzing their output.Candidates should have demonstrated experience leading research projects related to diversity, including program evaluation. The ideal candidate will have demonstrated experience auditing programs, and advocating for and implementing solutions.A customer-obsessed, relentless curiosity is a must, as is commitment to the highest standards of methodological rigor that a given study allows. This role provides opportunity for significant exposure to Amazon’s culture, leadership, and global businesses, and furthermore provides significant opportunity to influence how Workforce Staffing matches talent to business demand.If you’re hungry to engage and empower Amazon Associates your expertise in mixed methods research, let's talk.
US, WA, Seattle
Want to transform the way people enjoy music, video, and radio? Come join Alexa Entertainment, the team that made Amazon Music, Spotify, Hulu, Netflix, Pandora, and more available to Alexa customers. We are at the epicenter of the future of entertainment, innovating the way our customers interact with content at home and on the go.Alexa Entertainment is looking for a Senior Applied Scientist to lead a new team of talented and passionate scientists to intelligently arbitrate among competing possible actions to take on behalf of customers, as part of our wider spoken language understanding efforts. As a Senior Applied Scientist, you will lead the design, development, and evaluation of models and ML (machine learning) technology to enable a magical entertainment experience for Alexa customers. You will help lay the foundation to move from directed interactions to learned behaviors that enable Alexa to proactively take action on behalf of the customer. And, you will have the satisfaction of working on a product your friends and family can relate to, and want to use every day. Like the world of smart phones less than 10 years ago, this is a rare opportunity to have a giant impact on the way people live.
US, WA, Seattle
Are you passionate about building data-driven solutions to drive the profitability of the business? Are you excited about solving complex real world problems? Do you have proven analytical capabilities, exceptional communication, project management skills, and the ability to multi-task and thrive in a fast-paced environment? We need your skills and experience to help us grow Amazon Lending.Amazon Lending is an exciting program that provides businesses with click-to-cash access to funds that they need to grow their business with Amazon. We take pride in building scalable automated systems that make data-driven decisions to automate the entire lending process on a global scale.
US, WA, Seattle
Are you passionate about building data-driven solutions to drive the profitability of the business? Are you excited about solving complex real world problems? Do you have proven analytical capabilities, exceptional communication, project management skills, and the ability to multi-task and thrive in a fast-paced environment? We need your skills and experience to help us grow Amazon Lending.Amazon Lending is an exciting program that provides businesses with click-to-cash access to funds that they need to grow their business with Amazon. We take pride in building scalable automated systems that make data-driven decisions to automate the entire lending process on a global scale.
US, CA, Santa Clara
The Household Organization (HHO) team builds the conversations you and your family have with Alexa that help you get and stay organized. Millions of customers use our products to manage their time, get things done, and access and manage their personal information with their voice.We are making Alexa smarter and more personalized and are proud to do this in a privacy first way. We are looking to hire a machine learning scientist with expertise in NLP, differential privacy and data generation.
US, WA, Seattle
The IMDb TV team is shaping the future of digital video entertainment. Our mission is to build earth’s most customer centric ad supported premium free video service and make it trivially easy for hundreds of millions of customers to enjoy. Our team is re-inventing how to find content and building a new video destination.We need to make agile decisions based on what content creates the most value for our customers and pursue the most efficient content acquisition strategies for our desired outcomes. We are seeking an innovative Data Scientist to predict and measure the benefit of different IMDb TV titles and their impact on customer engagement as well as advertiser value. This position will be responsible for designing and building the suite of models that will predict content performance as well as the interface used by our content acquisition team to assess content for use in real time deal evaluations.This role requires a team member with strong quantitative modeling skills and the ability to apply statistical/machine learning methods to large amounts of customer and title level data. The candidate should have strong communication skills, be able to work closely with stakeholders and translate data-driven findings into actionable insights. The successful candidate will be a self-starter comfortable with ambiguity, with strong attention to detail and an ability to work in a fast-paced and ever-changing environment. This person will also be proficient in communicating their recommendations directly to advertising and video leadership.This position will be part of the Prime Video Content Research team, which includes a diverse scientific team of computer scientists and economists as well as other data scientists who build statistical models using world-class data systems and partner directly with the business to implement the solutions. We use detailed customer behavioral data (e.g. streaming history) and detailed information about content (e.g. IMDb-sourced characteristics) to predict and understand what customers like to watch. As a Data Scientist at Amazon, you will have the opportunity to work on one of the world's largest consumer data sets and influence the long term evolution of our analytics capability.This is very much Day 1 for IMDb TV. You would be joining an entrepreneurial and pioneering team working to reinvent ad supported television.
US, NY, New York
About Amazon Advertising:Amazon is building a world class advertising business and defining and delivering a collection of self-service performance advertising products that drive discovery and sales of merchandise. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products.About our team:Our team's mandate is to accelerate advertising in the Books category as it requires a differentiated customer and advertiser experience. We own end-to-end the advertising experience including placements, ad relevance, creative, ad serving, advertiser experience, and marketing. We are looking for entrepreneurial, innovative individuals who thrive on solving tough problems. We are investing in a deep science and technical team to pursue a transformation opportunity.About this role:We are seeking a talented, energetic, entrepreneurial, and self-driven Senior Data Scientist to join our team. Our team works on complex science, engineering, optimization, econometric, and user-experience problems in order to deliver relevant product ads on Amazon search and detail pages world-wide. Leveraging Amazon's massive data repository, our data scientists develop experiments, insights and optimizations that enable the monetization of Amazon online and mobile search properties while enhancing the experience of Amazon shoppers.In this role, you will:· Solve new and challenging problems using statistical, Machine Learning, or optimization approaches to create measurable customer facing impact.· Translate prototype models to production quality, scaling to millions of customers and across different languages.· Design experiments leveraging your models to measure effectiveness.· Work closely with product, business and engineering teams to execute on your project plans.· Communicate verbally and in writing to senior leaders with various levels of technical knowledge, educating them about your approach, as well as sharing insights and recommendations.· Drive best practices on the team; mentor and guide junior members to achieve their career growth potential.
US, MA, Cambridge
Alexa is the groundbreaking voice service that powers Echo and other devices designed around your voice. Our team is creating the science and technology behind Alexa. We’re working hard, having fun, and making history. Come join our team! You will have an enormous opportunity to impact the customer experience, design, architecture, and implementation of a cutting edge product used every day by people you know.We’re looking for a passionate, talented, and inventive scientist to help build industry-leading conversational technologies that customers love. Our team's mission is the enable Alexa to understand sounds and vocalization beyond speech. As a Research Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in speech and audio processing. Your work will directly impact our customers in the form of novel products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding. You will mentor junior scientists, create and drive new initiatives.