Amazon Scientists Use Transfer Learning to Accelerate Development of New Alexa Capabilities

Amazon scientists are continuously expanding Alexa’s natural-language-understanding (NLU) capabilities to make Alexa smarter, more useful, and more engaging.

We also enable external developers to build and deploy their own Alexa skills through the Alexa Skills Kit (ASK), allowing for an effectively unlimited expansion of Alexa’s NLU capabilities for our customers. (There are currently more than 40,000 Alexa skills.) In general, the machine learning models that provide NLU functionality require annotated data for training. Skill developers, and particularly those external ASK developers with limited resources or experience, would find skill development much more efficient if they could spend less time on data collection and annotation.

At the upcoming Human Language Technologies conference of the North American chapter of the Association for Computational Linguistics (NAACL 2018), we will present a technique that lets us scale Alexa’s NLU capabilities faster, with less data. Our paper is called “Fast and Scalable Expansion of Natural Language Understanding Functionality for Intelligent Agents”.

Our work uses transfer learning and deep learning for rapid development of NLU models in low-resource settings, by transferring knowledge from mature NLU functions. In this, we take advantage of linguistic patterns that are shared across domains. For example, named entities like city, time, and person and conversational phrases like ‘tell me about’, ‘what is’, ‘can you play’, etc., can appear across domains such as Music, Local Search, Video, Calendar, or Question Answering. In our work, we reuse large amounts of existing annotated data from mature domains such as Music or Weather to bootstrap accurate models for new domains or skills for which we have very little in-domain training data.

Our paper builds upon prior work in deep learning and transfer learning. Deep recurrent models and multitask learning architectures are state-of-the-art for many natural-language-processing (NLP) problems. Transfer learning is common in the computer vision community, where deep networks are typically trained on large sets of image data, and the lower layers of those networks learn task-independent features that transfer well to different tasks. We propose a similar approach for NLP, where such practices are relatively less explored. Within NLP, there is rich related work in the earlier domain adaptation literature and recent papers that use deep learning to transfer representations across domains and languages.

Our NLU system includes models for user intent classification (IC) and named-entity recognition (NER), which are used to classify both user intentions, such as “PlayMusic”, and named entities, such as people, dates, and times. Our methodology uses deep multitask bidirectional LSTM (bi-LSTM) models to jointly learn the IC and NER tasks. Our network (illustrated below) includes a common bi-LSTM layer and two stacked task-specific bi-LSTM layers for NER and IC respectively.

Transfer learning happens in two stages. First, we build generic multitask bi-LSTM models trained on millions of annotated data across existing, high-volume domains. This stage is also referred to as pretraining, where the lower bi-LSTM layers learn domain-independent embeddings for words and phrases. When a small amount of annotated data for the new domain becomes available, we begin the second, fine-tuning stage, where the generic model is adapted to the new data. First, we replace the upper network layers, resetting their weights and resizing them to fit the new problem, while the lower layers are retrained with the weights learned during the pretraining stage. Then we retrain the whole model on the available domain data. This allows us to effectively transfer knowledge from the large pool of annotated data from mature domains into the new domain.

Stacked multitask bi-LSTM model architecture
Stacked multitask bi-LSTM model architecture

We experimented with this approach on hundreds of domains, from both first-party, Amazon-developed Alexa functions and Alexa skills developed by third parties. ASK skills represent a very-low-resource setting: developers typically provide only a few hundred examples of training utterances for each of their skill models. We evaluated our methods on 200 skills and attained an average error reduction of 14% relative to strong baselines that do not use transfer learning. This allows us to significantly increase the accuracy of third-party developers’ NLU models without requiring them to annotate more data.

For domains developed by Amazon scientists and engineers, we simulated the early stages of domain development, when training data is gradually becoming available through internal data collection. Below we illustrate the combined NER and IC error at various stages for both a strong baseline that does not use transfer learning and our proposed transfer learning model. Our model consistently outperforms the baseline, with the greatest gains at the earliest and lowest-resource stages of domain development (up to 7% relative error reduction). As a result, we can reach high NLU accuracy with less data, reducing the need for data collection and enabling us to more rapidly deliver new functionality to Alexa users.

NLU error for proposed modes vs. baseline,through early stages of domain development
NLU error for proposed modes vs. baseline,through early stages of domain development

Finally, we found that, in addition to providing higher accuracy, the fine-tuning stage of our pretrained models converges about five times as fast as bi-LSTMs trained "from scratch" (i.e., without transfer learning). This enables faster turnaround time when training bi-LSTM models for new functionality.

Paper: "Fast and Scalable Expansion of Natural Language Understanding Functionality for Intelligent Agents"

Acknowledgments: Anuj Goyal, Spyros Matsoukas

About the Author
Angeliki Metallinou is an Alexa senior speech scientist.

Related content

Amazon Science Newsletter Project Kuiper.jpg
Get more from Amazon Science
Sign up for our monthly newsletter

Work with us

See More Jobs
US, WA, Bellevue
Come join the Alexa team, building the speech and language solutions behind Amazon Echo and other Amazon products and services! You will help us invent the future.As a Data Scientist, you will design, evangelize, and implement state-of-the-art solutions for never-before-solved problems, helping Alexa to provide customer great products. This role will be a key member of a Alexa Data Service Science team based in Bellevue, WA. You will work closely with other research scientists, machine learning experts, engineers to design and run experiments, research new algorithms, and find new ways to improve Alexa Data Service products. You will partner with technology and product leaders to solve business and technology problems using scientific approaches to build new services that surprise and delight our customers. Our scientists work closely with software engineers to put algorithms into practice. They also work on cross-disciplinary efforts with other scientists within Amazon.The key responsibility for this role include:· Define proper output business Metrics, and build input models to identify patterns and drivers of the output.· Drive actions at scale using scientifically-based methods and decision making.· Design and develop complex mathematical, statistical, simulation and optimization models and apply them to define strategic and tactical needs and drive the appropriate business and technical solutions· Design experiments, test hypotheses, and build actionable models· Prototype these models by using modeling languages such as R or in software languages such as Python.· Work with software engineering teams to drive scalable, real-time implementations· Utilizing Amazon systems and tools to effectively work with terabytes of data
US, WA, Bellevue
Come join the Alexa team, building the speech and language solutions behind Amazon Echo and other Amazon products and services! You will help us invent the future.As a Senior Data Scientist, you will design, evangelize, and implement state-of-the-art solutions for never-before-solved problems, helping Alexa to provide customer great products. This role will be a key member of a Alexa Data Service Science team based in Bellevue, WA. You will work closely with other research scientists, machine learning experts, engineers to design and run experiments, research new algorithms, and find new ways to improve Alexa Data Service products. You will partner with technology and product leaders to solve business and technology problems using scientific approaches to build new services that surprise and delight our customers. Our scientists work closely with software engineers to put algorithms into practice. They also work on cross-disciplinary efforts with other scientists within Amazon.The key responsibility for this role include:· Define proper output business Metrics, and build input models to identify patterns and drivers of the output.· Drive actions at scale using scientifically-based methods and decision making.· Design and develop complex mathematical, statistical, simulation and optimization models and apply them to define strategic and tactical needs and drive the appropriate business and technical solutions· Design experiments, test hypotheses, and build actionable models· Prototype these models by using modeling languages such as R or in software languages such as Python.· Work with software engineering teams to drive scalable, real-time implementations· Utilizing Amazon systems and tools to effectively work with terabytes of data
IN, KA, Bangalore
Are you interested in applying your background in machine learning to solve key business problems that impact the experience of millions of Amazon customers? Do you like to own and solve complex, ambiguous and hard business problems that impact the growth and profitability of company? If yes, then you may be a great fit to join the Retail Business Services (RBS) Tech group at Amazon.As an Applied Scientist, you will be responsible to design and deploy scalable NLP and Computer Vision solutions that will impact the content visible to millions of customer and solve key customer experience issues. You will develop novel statistical models or deep learning techniques for text processing, image processing, pattern recognition, and anomaly detection problems. You will define the research and experiments strategy with an iterative execution approach to create the machine learning models and progressively improve the results over time. You will work closely with engineers, other scientists and product managers to come up with novel solutions for some hard business critical problems.
US, WA, Seattle
Prime Video is changing traditional media with an ever-increasing selection of movies, TV shows, Emmy Award winning original content, add-on subscriptions including HBO, and live events like Thursday Night Football. Our architecture operates at exabyte-scale, engineered for reliability, scalability, and performance. Prime Video runs on thousands of device types in over 200 territories worldwide.The Playback Optimization team designs, implements and operates the services and systems responsible for Optimizing and delivering video content to all Prime video customers. Our services and systems are invoked every time our customers click play on the thousands of device types supported by Prime Video in over 240 territories worldwide. We direct gigabits of video data across the Internet every second and ensure that our customers receive the best Optimized playback experience possible.As a Senior applied scientist on our team you will wear many hats and work in a highly collaborative environment that’s more startup than big company. We’ll need to tackle complex problems that span a variety of domains: Machine learning, Artificial intelligence, Natural Language processing, real-time and distributed systems and help us build services and systems from ground up which scale and serve billions of requests per day, with obsessively high reliability and low operational overhead. You will have an opportunity to explore, innovate, invent and simplify various Playback state-of-art Optimization services and algorithms which leverage both custom and Industry proven Machine Learning, Natural Language processing and Artificial Intelligence technologies. Your work will focus on training and evaluating models and deploying them to production where we continuously monitor and evaluate. You will work on large engineering efforts that solve significantly complex problems facing global customers. You will be trusted to operate with independence and are often assigned to focus on areas with significant impact on audience satisfaction. You must be equally comfortable digging in to customer requirements as you are drilling into design with development teams and developing production ready learning models. You consistently bring strong, data-driven business and technical judgment to decisions. We experiment a lot and it is a must to learn and be curios. You will be encouraged to see the big picture, be innovative, and positively impact millions of customers.Successful candidates have a background in modern programming languages, distributed system design, service-oriented architecture, and high scalability. Experience in advanced machine learning technologies and Artificial intelligence is a big plus. Equally important is the ability to multi-task, invent, create reliable and maintainable code, and find creative, scalable solutions to difficult problems.You'll work with experienced managers who'll care for you. We'll guide you on your career growth path and there's no shortage of technical challenges.We're building a new and ground-breaking technology from the ground-up to deliver spectacular Optimized video Playback experience to our customers for big live events such as English Premier League and the US Open.
US, WA, Seattle
Amazon Care is looking for a Senior UX Research Scientistto join an exciting new project team. We are an innovative, cross-platform, customer-obsessed team, working on big-bets in an innovative environment. Our organization rewards intellectual curiosity while maintaining the direct to market product focus. Our mission is to invent and simplify large-scale solutions and bring the future to Amazon customers.This role conducts UX research and data synthesis across a growing product & clinical health team. You’ll work cross-functionally and collaborate with product managers, designers, clinicians and engineering teams to proactively ingest research data, organize and synthesize reports, and strategize on new insights that translate into new product roadmap initiatives and service improvements.What you will do· Help UX and product team leadership identify the right research questions to ask, creating long-term research roadmaps and opportunity frameworks that align to and inform business strategy and goals.· Own, Evolve & extend the existing VOC program to connect new sources of data while extending the impact of the program across internal audiences of business, product, UX and clinicians.· Define and execute (and, as necessary, invent) research methods appropriate to the questions at hand, including but not limited to surveys, A-B tests/experiments, and some remote testing.· Synthesize a wide range of primary and secondary data types to perform quick and thorough analyses that leads to focused, insightful, and actionable findings.· Successfully persuade decision makers at all levels of the organization to take aligned action on research findings – inspire them and help them internalize opportunities to delight customers and differentiate our service.· Translate insights into measurable customer experience outcomes aligned to business goals.· Own and synthesize insights for MBR and QBR reporting.· Own the insights and UX data generated by emerging payer audience.· Work with a wide range of business partners, particularly in analytics and product management, to develop innovative end-to-end solutions that address top outcomes, and iteratively design and test these solutions until outcomes are achieved or hypotheses are proven wrong.As a Senior UX Research Scientist, you will be part of a greenfield effort to build a truly innovative product from the ground up. You will have a deep understanding of user research methods and best practices, along with a creative and pragmatic approach on how to connect new and existing data sources, synthesize and report them. You will design and implement research activities that help us gain valuable insights into our customers and inform product direction, concepts and designs. To succeed in this role, you genuinely be a self-starter, able to thrive in a fast-paced start-up like environment with considerable product ambiguity, welcoming the opportunity to extend your influence to the entire team.You will manage a VOC (Voice of the Customer) research program that will grow in both intake, synthesis and reporting impact. You will work with other UXR team members to synthesize data from both clinician and customer insights. You have the ability to work efficiently and routinely deliver the right things with limited guidance. Your work will be focused on medium-to-large goals and research projects that span multiple customer segments and/or teams. Once completed, your research will be able to distill clear customer needs from large, diverse data sets (e.g., behavioral/telemetry, survey, qualitative interviews/usability, etc.) to provide inputs that solve complex business problems. You will own the delivery of customer insights for multiple teams at the product level. You will proactively identify and inspect existing mechanisms for sharing user insights to increase their adoption and measurability. You will ensure your program stays aligned with organizational goals and you should have fun in the process while building a world class consumer healthcare product.Responsibilities:· Employ the best research method(s) based on research questions, timeline, and resourcing· Must illustrate skills in navigating topics and sensitive data environments including PHI· Ability to develop metrics that appropriately reflect the user experience quality of a feature, product or system· Ability to Work closely with product, UX design, engineering, marketing, and sales teams to identify research topics and critical research questions, create detailed plans and build a research roadmap; communicate and refresh on a regular basis to ensure impact and relevancy· Define and execute a comprehensive research plan, using the best method or mix of quantitative and qualitative methods based on research questions, timeline, and resourcing.· Perform hands-on data analysis and modeling with very large data sets to develop insights into different aspects of our business and user experience.· Communicate results and opportunities in clear, concise, and compelling ways to all levels of management· Drive impact, as measured by changes to the product design and development roadmap· Understand different data types from different sources and perform quick but thorough data analysis in order to create insightful and actionable findings· Create a deep understanding and empathy for our target customers through primary field research, behavioral analysis, and/or secondary research· Act as a representative within the larger Amazon user researcher community. Collect and synthesize useful research from across the organization, bringing insights back to your team· Eagerness to continually learn, grow and improve the craft of UX research
US, WA, Seattle
The Amazon Web Services (AWS) Training and Certification (T&C) organization educates customers, partners, and AWS Employees globally on AWS products, solutions, and best practices. We are seeking a talented and experienced Data Scientist to perform data analytics, create forecasting processes to influence business strategies, and analyze customer trends to make recommendations to improve their training & certification journey.As part of the AWS Training and Certification Business Intelligence and Data Analytics team you will analyze data to provide insights using statistical methods and processes. We are looking for someone who is comfortable getting into data and will uncover patterns and performance that can be used to influence business strategies. The right candidate will be passionate about working with large datasets and should be someone who loves to bring data together to answer complex business questions that deepen our understanding of our business drivers. As a Data Scientist within AWS, you will have the exciting opportunity to help analyze and deliver data driven insights that will broaden AWS’s penetration in the cloud computing market.The ideal candidate will have a background in analyzing and deriving insights from data by using SQL (e.g. Redshift), analytics (e.g. R, Python), statistical processes (e.g. machine learning, forecasting, and predictive analytics), and business intelligence tools (e.g. Tableau). A successful candidate should have the technical proficiency to develop code to produce processes that analyze our data and the business proficiency to summarize their findings and report to the AWS Training and Certification leadership team. This role will work closely with Business Intelligence Engineers, Business Analyst, Economists, Program Managers, and AWS T&C Leadership.Responsibilities:· Serve as a key member of the AWS Training & Certification Business Intelligence and Data Analytics team by turning data into actionable insights and providing support for strategic initiatives.· · Conduct analysis of existing metrics and create new metrics that will be shared with the Global Operations Team and executive stakeholders.· · Recommend, develop, and manage machine learning/statistical processes and tools.· · Conducts data analyses with the appropriate statistical approach (regressions, linear and non-linear correlation, entropy, t-test, F-test, etc.)· · Identify needed data for projects and understand underlying structure (distribution, variable independence, outliers, etc.), and develop process to help improve data quality· · Influence the decisions of senior business leaders through effective verbal and written communication and logical reasoning.· · Able to discuss and present data science processes to business leaders and provide support to help with understanding of processes to non-tech individuals.· · Manage numerous requests concurrently and strategically, prioritizing when necessary.· · Drive and support projects that help build AWS into the most customer-centric technology platform
US, MA, Cambridge
The Alexa Translations team is looking for an experienced Applied Science Manager to build industry-leading technologies in speech translation. Alexa is the voice activated digital assistant powering devices like Amazon Echo, Echo Dot and Fire TV. Our team's mission is to enable Alexa to break down language barriers for our customers.As an Applied Science Manager, you will lead a team of exceptional scientists to develop novel algorithms and modeling techniques to advance the state of the art in speech translation. You will work in a hybrid, fast-paced organization where scientists, engineers and product managers work together to build novel customer facing experiences. You will collaborate with and mentor other scientists to raise the bar of scientific research in Amazon. Your work will directly impact our customers in the form of products and services that make use of speech and language technology.We are looking for a leader with strong technical experience, demonstrated progression of management scope, and a passion for managing Science talent in a fast-paced environment. In addition to technical depth, you must possess exceptional project management and communication skills, and understand how to coach a team. As a Science leader you will:· Manage and mentor other scientists and engineers, review and guide their work, help develop roadmaps for the team and provide coaching for career development· Contribute directly to our growth by hiring smart and motivated Scientists and managers to establish teams that can deliver swiftly and predictably, adjusting in an agile fashion to deliver what our customers need.· Work closely with other teams across Alexa to deliver platform features that require cross-team leadership.· Represent your business and operations to the highest level of leadership within Amazon.If you are looking for a challenging and innovative role where you can solve important problems while growing as a leader, this may be the place for you.
US, MA, Boston
Amazon Elastic File System (EFS) https://aws.amazon.com/efs/ is looking for a Data Scientist to dive deep into the vast data generated by a rapidly growing AWS storage service and applying current data analysis methods to produce insights that will improve customer experience, operational effectiveness, and business value. As a member of the Amazon EFS team, you’ll work closely with outstanding engineers and product managers to work hard, have fun, and create the future of cloud storage.Building a High-Performing & Inclusive Team CultureYou should be passionate about working with a world-class team that welcomes, celebrates, and leverages a diverse set of backgrounds and skillsets to deliver results. Driving results is your primary responsibility, and doing so in a way that builds on our inclusive culture is key to our long term success.Work/Life BalanceEFS values work-life balance. On normal days, our entire team is co-located in the Boston office, but we’re also flexible when people occasionally need to work from home. We generally keep core available hours from 10am to 4pm. Some of the team is available earlier and the rest of us work a little later.Energizing and Interesting Technical ProblemsYou will work in partnership with engineers on the team to build and operate large scale systems that move and transform customer volume data and accelerate access to their data. You’ll be working to provide solutions to both internal and external customers and engage deeply with other teams within EFS, S3, EC2, and many other services. It’s humbling and energizing to provide data movement solutions to customers at AWS scale.Mentorship & Career GrowthWe’re committed to the growth and development of every member of EFS, and that includes our engineers. You will have the opportunity to contribute to the culture and direction of the entire EFS org and deliver initiatives that will improve the life of all of our teams.As a Data Scientist on EFS you will be curious and dive deep into performance, business, and operational metrics. You’re excited about designing solutions that scale while also engaging individual customers in understanding their applications. You’re able to think about business opportunities, operational issues, and architectural diagrams in the course of a single conversation. You’re looking for a team of bright, capable engineers to work with directly in implementing your vision while also collaborating with other data scientists across AWS.
US, WA, Bellevue
ML scientists at Amazon participate in the design, development, evaluation, deployment and updating of data-driven models and analytical solutions for machine learning (ML) and time-series forecasting applications. Develop and/or apply statistical modeling techniques (e.g. Bayesian models and deep neural networks), optimization methods, and other ML techniques to different applications in business and engineering. Routinely build and deploy ML models on available data. Research and implement novel ML and statistical approaches to add value to the business. Mentor junior engineers and scientists.The AWS Demand Forecasting and Planning team is responsible for growing the world's largest Cloud. We forecast customer demand, build ML systems that understand customer needs, and drive utilization improvement for all AWS services.What we own:· Building a world-class forecasting platform that scales to handling billions of time series data in real time.· Developing predictive customer analytics models and recommendation engines.· Expanding inventory replenishment models and systems for each AWS service in the fast-growing AWS product portfolio.· Finding out the optimal tradeoff between AWS service availability and fleet utilization.· Driving fleet utilization improvement where each 1% means tens of millions of additional free cash flow.· Automating tactical and strategic capacity planning tools to optimize for service availability and infrastructure cost.What you will learn:· State of the art forecasting methodologies.· Application of machine learning to large-scale customer analytics.· Inventory management and supply chain management for the Cloud.· Resource management and admission control for the Cloud.· The internals of all AWS services.Note that location is flexible between our Bellevue and Seattle offices.Keywords:Forecasting, Statistics, Machine Learning, Optimization, Inventory Management, Supply Chain Management, AWS, Cloud, Cloud Computing, EC2, S3, EBS, DynamoDB, CloudFront, Java, C++, Object Oriented, R, Distributed Systems, High Availability, Scalability, Concurrent
RO
The Amazon Devices team designs and engineers consumer electronics, including the best-selling Kindle family of products, Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.As an Applied Scientist, you will participate in the design, development, and evaluation of models and machine learning (ML) technology to delight our customers.You will be part of a team delivering features that are well received by our customers.
RO
The Amazon Devices team designs and engineers consumer electronics, including the best-selling Kindle family of products, Fire tablets, Fire TV, Amazon Dash, and Amazon Echo.As an Applied Scientist, you will participate in the design, development, and evaluation of models and machine learning (ML) technology to delight our customers.You will be part of a team delivering features that are well received by our customers.
US, WA, Seattle
Workforce Staffing (WFS) supports Amazon Operations by hiring the hourly associates that staff our operational buildings. WFS is quickly becoming one of the world’s largest staffing organizations, forecasted to hire over one million hourly associates across North America and the European Union this year alone. Currently, we hire full time, part time, flex time and seasonal hires across Fulfillment Centers, Sort Centers, Amazon Logistics, Whole Foods, Amazon Air, Prime Now, Amazon Fresh, and emerging business lines. Interested in the businesses that Amazon creates and grows? Here’s your opportunity to be a part of this journey.The Workforce Intelligence team was created in 2018 to support the massive growth in scale and scope that WFS has experienced. The team has continued to grow rapidly in order to meet the expanding needs of the business, including: big data and machine learning solutions, innovative approaches to complex HR problems, and data-driven recommendations during a time of rapid change.Here’s where you come in:As a Research Scientist in Workforce Intelligence, your work is focused on research to deeply understand the people that make up our hourly workforce and help others do the same. You understand that even when hiring hundreds of thousands of hourly associates across multiple types of roles and businesses, the experience of each candidate matters.You use your deep expertise in surveys and statistics (regressions, multilevel models, etc.) to define and answer nebulous problems. You use experimental, quasi-experimental, and RCT methods to understand our candidates and influence critical business decisions. You relentlessly obsess over understanding our candidates and lead our survey program that seeks to amplify the voice of our candidates. You work with colleagues across Research, Data Science, Business Intelligence and related teams to enable Amazon find and hire the right candidates for the right roles at an unprecedented scale.This will be a highly visible role across multiple key deliverables for our global organization. If you are passionate and curious about data, obsess over customers, love questioning the status quo, and want to make the world a better place, let’s chat.
US, WA, Seattle
Amazon Science gives you insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Our scientists continue to publish, teach, and engage with the academic community, in addition to utilizing our working backwards method to enrich the way we live and work.Please visit https://www.amazon.science for more information.At Alexa Shopping, we strive to enable shopping in everyday life. We allow customers to instantly order whatever they need, by simply interacting with their Smart Devices such as Amazon Show, Spot, Echo, Dot or Tap. Our Services allow you to shop, no matter where you are or what you are doing, you can go from 'I want that' to 'that's on the way' in a matter of seconds. We are seeking the industry's best to help us create new ways to interact, search and shop. Join us, and you'll be taking part in changing the future of everyday life.What you will do: You will lead a team of talented and experienced scientists and engineers that implement solutions for natural language understanding of Alexa Shopping customers: this involves taking the outputs from automated speech recognition (ASR) component and producing a representation of its meaning. Additionally, your team will build Alexa Shopping Automated CX quality metrics and provide analytics. This involves exploring, developing, socializing, and implementing mechanisms for tracking automated CX quality across customer’s journey with Alexa Shopping to improve Alexa Shopping CX. And finally, you will have the satisfaction of being able to look back and say you were a key contributor to something special from its earliest stages. You will be working closely with executive leadership, multiple product managers and leaders from partner teams in Amazon Retail, Alexa, and Speech Recognition teams.What we are looking for: We are looking for a talented Data Science Manager with a strong technical background and solid people management skills to build, manage and develop a highly-talented and experienced data science team. We are seeking leaders that can guide technical and product innovation in the areas of voice experiences, machine learning models and the distributed systems to bring our vision together. Strong judgment and communication skills, long term technical vision, and continuous focus on engineering and operational excellence are essential for the success in this role.
DE, BE, Berlin
As a Senior Applied Scientist on this growing team, you will take on a key role in improving the NLP and ranking capabilities of the Amazon product search engine. Our ultimate goal is to help customers find the products they are searching for, and discover new products they would be interested in. We do so by developing NLP components that cover a wide range of languages, not only English and major languages of Europe, but also Turkish, Arabic, Japanese, and more. The team plays a central role in search query understanding, product indexing, and representations/embeddings of queries and products, all of which aid in improving the ranking and relevance of search results.This is a rewarding role where you will be able to draw a clear connection between your work and how it improves the experience of millions of Amazon customers across the globe every day. You will propose and explore publication-worthy innovation in NLP and IR to build ML models trained on terabytes of product and traffic data, which are evaluated using both offline metrics as well as online metrics from A/B testing. You will then integrate these models into the production search engine that serves customers, closing the loop through data, modeling, application, and customer feedback. The chosen approaches for model architecture will balance business-defined performance metrics with the needs of millisecond response times.Your responsibilities include:· Analyze the data and metrics resulting from traffic into Amazon's product search service· Design, build, and deploy effective and innovative ML solutions to improve various components of the search stack, such as indexing, ranking, and query understanding· Evaluate the proposed solutions via offline benchmark tests as well as online A/B tests in production· Publish and present your work at internal and external scientific venues in the fields of ML/NLP/IRYour benefits include:· Working on a high-impact, high-visibility product, with your work improving the experience of millions of customers· The opportunity to use (and innovate) state-of-the-art ML methods to solve real-world problems· Being part of a growing team where you can influence the team's mission, direction, and how we achieve our goals· Excellent opportunities, and ample support, for career growth, development, and mentorship· Competitive compensation, including relocation support (for both domestic and international candidates)
US, WA, Seattle
Business/Team IntroductionThe Supply Chain Optimization Technologies (SCOT) team builds technology to automate and optimize Amazon’s supply chain of physical goods. We seek a Data Scientist with strong analytical and communication skills to join our team. SCOT manages Amazon's inventory under uncertainty of demand, pricing, promotions, supply, vendor lead times, and product life cycle. We optimize complex trade-offs between customer experience, inventory costs, fulfillment costs, fulfillment center capacity, etc. We develop sophisticated algorithms that involve learning from large amounts of data such as prices, promotions, similar products, and other data from our product catalog in order to automatically act on millions of dollars’ worth of inventory weekly and establish plans for tens of thousands of employees. As a Data Scientist, you will contribute to the research community, by working with other scientists across Amazon and our Supply Chain, as well as collaborating with academic researchers and publishing papers. SCOT also engages in cutting edge research that we try to share with the community. Recent work from SCOT includes papers presented at the NIPS 2017 Time Series Workshop, SSRN, KDD 2018 Time Series Workshop, and ICML 2018 Deep Generative Models Workshop.Data Scientist ResponsibilitiesAs a Data Scientist in SCOT, will be tasked to understand and work with bleeding edge research to enable the implementation of sophisticated models on big data. As a successful data scientist in the SCOT team, you are an analytical problem solver who enjoys diving into data from various businesses, is excited about investigations and algorithms, can multi-task, and can credibly interface between scientists, engineers and business stakeholders. Your expertise in synthesizing and communicating insights and recommendations to audiences of varying levels of technical sophistication will enable you to answer specific business questions and innovate for the future.Major responsibilities include:· Analysis of large amounts of data from different parts of the supply chain and their associated business functions· Improving upon existing machine learning methodologies by developing new data sources, developing and testing model enhancements, running computational experiments, and fine-tuning model parameters for new models· Formalizing assumptions about how models are expected to behave, creating definitions of outliers, developing methods to systematically identify these outliers, and explaining why they are reasonable or identifying fixes for them· Communicating verbally and in writing to business customers with various levels of technical knowledge, educating them about our research, as well as sharing insights and recommendations· Utilizing code (Python, R, Scala, etc.) for analyzing data and building statistical and machine learning models and algorithms
US, WA, Seattle
Global Talent Management (GTM) is centrally responsible for creating and evolving Amazon’s human capital and talent programs and processes.People Science Team within GTM is a growing start-up team with direct impact on Amazonians across all of our businesses and locations around the world. We play a crucial role in ensuring top notch data products and insights facilitate our growth and development of talent in intelligent and curious ways. We regularly use data to pitch ideas and drive conversations with Amazon’s Senior Vice President of HR and other executives about how to improve existing talent programs to solve organizational problems focused on (but not limited to) talent differentiation, talent movement, employee-role matching, product integration, promotion practices, organization design and succession planning, and diversity and inclusion, or invent new ones that address the evolving needs of our diverse employee base.We are looking for a self-driven Economist to help shape analytics and research roadmap and enable data-driven innovation that fuel our rapidly scaling talent management mission. You will build econometric models, using our world class data systems, and apply economic theory to solve business problems in a fast moving environment. Economists at GTM will be expected to develop new techniques to process large data sets, apply a causal lens to the framework, address ambiguous business problems, and contribute to design of automated systems around the company.You will partner closely with product and program owners, as well as scientists and engineers from other disciplines (e.g. data science, software engineers, data engineering) with a clear path to business impact. You develop innovative and even frighteningly bold plans and ideas to discover new ways to advance our goals. You will be expected to be a thought leader as we chart new courses with our rapidly growing employee populations, and lead the way in experimenting new ideas that have not yet been explored.Key Responsibilities:· Participate in scoping and planning of GTM’s Science roadmap· Uncover drivers, impacts, and key influences on talent outcomes· Build new econometric models to improve existing talent products or those that make the case for new products· Bring a causal lens to questions in human resources employing either experiments or non-experimental approaches· Develop predictive and optimization models for key applications· Navigate a variety of data sources, such as enterprise data, customize surveys, focus groups, and/or external data sources· Ability to distill informal customer requirements into problem definitions, dealing with ambiguity and competing objectives· Work in expert cross-functional teams delivering on demanding projects
US, CA, Virtual Location - California
Amazon.com strives to be Earth's most customer-centric company where people can find and discover anything they want to buy online. We hire the world's brightest minds, offering them a fast paced, technologically sophisticated and friendly work environment.Economists at Amazon will be expected to work directly with senior management on key business problems faced in retail, international retail, cloud computing, third party merchants, search, Kindle, streaming video, and operations. Amazon economists will apply the frontier of economic thinking to market design, pricing, forecasting, program evaluation, online advertising and other areas. You will build econometric models, using our world class data systems, and apply economic theory to solve business problems in a fast moving environment. Economists at Amazon will be expected to develop new techniques to process large data sets, address quantitative problems, and contribute to design of automated systems around the company.
US, WA, Seattle
The AWS Central Economics team is looking for a PhD economist. The ideal candidate will have experience with time-series forecasting.You will learn about cloud products, including compute, storage, and databases. You will work on analytic projects requested by senior leadership. You will get the opportunity to learn new techniques. You will be a part of a team with many experienced economists.
US, VA, Arlington
Amazon’s Talent Assessment team designs and implements groundbreaking hiring solutions for one of the world’s fastest growing companies. We work in a fast-paced, global environment where we must solve complex problems (ranging from research-based to technical) and deliver solutions that have impact.We are seeking personnel selection researchers with a strong foundation in the development of pre-hire selection assessments, traditional and alternative legally defensible assessment validation approaches, research methodology, and data analysis. We are looking for equal parts researchers and consultant/thought leaders who are highly adaptable and continual learners who thrive in a fast paced environment.You will work closely with global teams to design and experiment new hiring solutions that predict success for highly complex roles (technical and non-technical) that have great impact on Amazon globally.What you’ll do:· Lead the tactical development and execution of large scale, highly visible personnel selection research projects· Develop and iterate on experimental research studies to optimize qualitative and quantitative hiring strategies· · Develop and innovate on new pre-hire test assessment design, validation, and implementation· · Partner with internal and external technology teams· Influence executive project sponsors and multiple business and development teams across the company· Drive effective teamwork, communication, and collaboration across multiple stakeholder groups
US, VA, Arlington
Amazon’s Talent Assessment team designs, implements, and optimizes hiring systems for one of the world’s fastest growing companies. We work in a data-focused, global environment solving complex problems with deep thought, large-sample research, and advanced quantitative methods to deliver practical solutions that make all aspects of hiring more fair, accurate, and efficient.We're looking for a curious data scientist interested in working on a multi-disciplinary team of applied scientists, psychologists, data engineers, business analysts, and program managers. In this role, you will apply your modeling skills to bust myths, create insights, and produce recommendations to help Amazon evaluate millions of potential new hires per year. You'll be involved in all phases of research and experiment design and analysis, including defining research questions, designing experiments, identifying data requirements, conducting statistical or machine learning-based modeling, and communicating insights and recommendations. You'll also be expected to continuously learn new systems, tools, and industry best practices to analyze big data and enhance our analytics.