How We Add New Skills to Alexa’s Name-Free Skill Selector

In the past year, we’ve introduced what we call name-free skill interaction for Alexa. In countries where the service has rolled out, a customer who wants to, say, order a car can just say, “Alexa, get me a car”, instead of having to specify the name of a ride-sharing provider.

Underlying this service is a neural network that maps utterances to skills, and expanding the service to new skills means updating the network. The optimal way to do that would be to re-train the network from scratch, on all of its original training data, plus data corresponding to any new skills. If the network requires regular updates, however, this is impractical — and Alexa has added tens of thousands of new skills in the past year alone.

At this year’s meeting of the North American Chapter of the Association for Computational Linguistics, we present a way to effectively and efficiently update our system to accommodate new skills. Essentially, we freeze all the settings of the neural network and add a few new network nodes to accommodate the new skills, then train the added nodes on just the data pertaining to the new skills.

If this is done naively, however, the network’s performance on existing skills craters: in our experiments, its accuracy fell to less than 5%. With a few modifications to the network and the training mechanism, however, we were able to preserve an accuracy of 88% on 900 existing skills while achieving almost 96% accuracy on 100 new skills.

Cosine_normalization.jpg._CB465442559_.jpg
An example of the technique we use to mitigate “catastrophic forgetting” when updating our model. A network previously trained on skills including “Weather” (red vector) is re-trained on the new skill “Uber” (blue vector). The network can ensure good performance on the new training data by simply assigning a large magnitude (length) to the new skill’s representation vector (a). Using cosine similarity (b) rather than dot product to compare skill representation vectors to a new input vector (h) ensures that vector magnitude does not overwhelm the more relevant information about vector angle.

We described the basic architecture of our skill selection network — which we call Shortlister — in a paper we presented last year at the Association for Computational Linguistics’ annual conference.

Neural networks usually have multiple layers, each consisting of simple processing nodes. Connections between nodes have associated “weights”, which indicate how big a role one node’s output should play in the next node’s computation. Training a neural network is largely a matter of adjusting those weights.

Like most natural-language-understanding networks, ours relies on embeddings. An embedding represents a data item as a vector — a sequence of coordinates — of fixed size. The coordinates define points in a multidimensional space, and data items with similar properties are grouped near each other.

Shortlister has three modules. The first produces a vector representing the customer’s utterance. The second uses embeddings to represent all the skills that the customer has explicitly enabled — usually around 10. It then produces a single summary vector of the enabled skills, with some skills receiving extra emphasis, on the basis of the utterance vector.

Finally, the third module takes as input both the utterance vector and the skill summary vector and produces a shortlist of skills, ranked according to the likelihood that they should execute the customer’s request. (A second network, which we call HypRank, for hypothesis ranker, refines that list on the basis of finer-grained contextual information.)

Embeddings are typically produced by neural networks, which learn during training how best to group data. For efficiency, however, we store the skill embeddings in a large lookup table, and simply load the relevant embeddings at run time.

When we add a new skill to Shortlister, then, our first modification is to add a row to the embedding table. All the other embeddings remain the same; we do not re-train the embedding network as a whole.

Similarly, Shortlister’s output layer consists of a row of nodes, each of which corresponds to a single skill. For each skill we add, we extend that row by one node. Each added node is connected to all the nodes in the layer beneath it.

Next, we freeze the weights of all the connections in the network, except those of the output node corresponding to the new skill. Then we train the new embedding and the new output node on just the data corresponding to the new skill.

By itself, this approach leads to what computer scientists call “catastrophic forgetting”. The network can ensure strong performance on the new data by funneling almost all inputs toward the new skill.

In Shortlister’s third module — the classifier — we correct this problem by using cosine similarity, rather than dot product, to gauge vector similarity.

The classifier works by mapping inputs (customer utterances, combined with enabled-skill information) and outputs (skill assignments) to the same vector space and finding the output vector that best approximates the input vector. A vector can be thought of as a point in space, but it can also be thought of us a line segment stretching from the origin through that point.

Usually, neural networks use dot products to gauge vector similarity. Dot products compare vectors by both length and angle, and they’re very easy to calculate. The network’s training process essentially normalizes the lengths of the vectors, so that the dot product is mostly an indicator of angle

But when the network is re-trained on new data, the new vectors don’t go through this normalization process. As in the figure above, the re-trained network can ensure good performance on the new training data by simply making its vector longer than all the others. Using cosine similarity to compare vectors mitigates this problem.

Similarly, when the network learns the embedding for a new skill, it can improve performance by making the new skill’s vector longer than other skills’. We correct this problem by modifying the training mechanism. During training, the new skill’s embedding is evaluated not just on how well the network as a whole classifies the new data, but on how consistent it is with the existing embeddings.

We used one other technique to prevent catastrophic forgetting. In addition to re-training the network on data from the new skill, we also used small samples of data from each of the existing skills, chosen because they were most representative of their respective data sets.

In experiments, we first trained a network on 900 skills, then re-trained it, sequentially, on each of 100 new skills. We tested six versions of the network: two baselines that don’t use cosine similarity and four versions that implemented various combinations of our three modifications.

For comparison, we also evaluated a network that was trained entirely from scratch on both old and new skills. The naïve baselines exhibited catastrophic forgetting. The best-performing network used all three of our modifications, and its accuracy of 88% on existing skills was only 3.6% lower than that of the model re-trained from scratch.

Overall_performance_(1).jpg._CB465110660_.jpg
Comparative performance of the models we tested. "Upperbound" is a network re-trained from scratch on all the available data. "Cos+der+ns" is the network that uses all three of our modifications.

Acknowledgments: Han Li, Jihwan Lee, Sidharth Mudgal, Ruhi Sarikaya

About the Author
Young-Bum Kim
Young-Bum Kim is an Amazon science leader in the Alexa AI organization.

Related content

Work with us

See more jobs
US, CA, Santa Monica
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our businesses driving long term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day!The Campaign and Creative Services (CCS) organization's mission is to drive advertiser success. Specifically, CCS is responsible for defining and managing customer-obsessed ad policies across the entire advertising business, generating bar-raising creatives, and ensuring timely and flawless campaign execution and support. CCS plays an important role in educating and guiding advertisers toward the best standard or custom solution to meet or exceed their objectives, including assistance navigating the Amazon consumer ecosystem. The Moderation Automation and Relevance Systems (MARS) team programmatically assists advertisers by providing fast and relevant creative feedback frictionlessly. When an advertiser submits a campaign (or content), it flows through certain intermediate upstream systems and then enters the MARS Automation Pipeline i.e. Lestrade. This pipeline is set up such that, it has several automated checks to review an incoming campaign (or any other content), and flag any violations in policies identified for that specific type of content.As an Applied Scientist on this team, you will:· Build ML models to address content intelligence problems· Build advanced algorithms to generate AI powered multi-modal content.· These models will rely on a variety of visual, textual and multi-modal features requiring expertise in these domains.· These models need to scale to multiple languages and countries. You will collaborate with engineers and other scientists to build, train and deploy these models.· Propose hypotheses, validate these offline and run A/B tests to validate them online.· As part of these activities, you will develop production level code that enables high-quality content generation and ranking.· Use your expertise to generate different aspects of creatives for advertisements including — image-based content creation, script generation and content summarization, realistic text-to-speech/voice-over generation, AI-powered music generation and video synthesis· Work closely with software engineers on detailed requirements to productionize the ML models you build.· Run A/B experiments that affect hundreds of millions of customers, evaluate the impact of your optimizations and communicate your results to various business stakeholders.· Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving.· Research new innovate machine learning approaches.Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate.Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.Team video https://youtu.be/zD_6Lzw8raE
US, WA, Seattle
Job summaryJob summaryAs the Applied Science Manager within AWS Global Lead Management & Operations, you will help build a team that leads creation and innovation on the next generation of Machine Learning (ML) technologies on Native AWS Lead Management System. Our team owns creating ML solutions that understand long-term leads converting patterns, modeling the patterns into algorithms and applying machine learning techniques operating at world-class scale, we focus on prioritizing and routing the right potential customers to the right sellers at the right time.Key job responsibilitiesAs the Applied Science Manager for the NAWS lead management system, you will be responsible for leading and managing a team of scientists and analysts to support the ML solution and application for global marketing team’s strategic projects and future state initiatives.This role will develop innovative solutions to implement ML capability into marketing and sales workflows, deliver data-driven methodology in enterprise scale, to enable lead management enhancements, enhance data hygiene across systems, and lead the analytics used to consistently refine and improve sales and marketing workflow processes. You will provide technical leadership on analytics and ML programs for partner engineering team to design the overarching architecture on data ingestion, model processing pipeline and test/validation framework.This role is in Seattle but also open to other locations with AWS offices. Relocation within the US is only available to Seattle.
US, WA, Seattle
Job summaryAmazon Prime is looking for a talented Senior Economist Manager to help manage our ever growing information needs and support analysis of increasingly complex business questions. At Amazon Prime, understanding customer data is paramount to our success in providing customers with relevant and enticing benefits such as fast free shipping, instant videos and music, in an expanding number of international marketplaces. At Amazon you will be working in one of the world's largest and most exciting big-data environments. The Senior Economist Manager role occupies a unique space at the intersection of technology, machine-learning, econometrics, large-scale scientific computing, social science, and product management.As a Senior Economist Manager within Amazon Prime, you will manage a team of economists, applied scientists, and engineers, working closely with our world-class business and software development teams to propose and estimate novel statistical and econometric models to directly inform strategic decisions about characteristics of the Amazon Prime membership. These include what membership prices, benefits, and benefit content deliver the most value for our customers around the world. Your team will solve these problems using structural econometrics, causal inference, machine learning, and massively parallelized scientific computing, and work closely with our software development team to automate these models at scale in distributed computing infrastructures such as Apache Spark. This position is unique in its exposure to senior members of the Prime team and other Amazon business units.The successful candidate will have demonstrated their own capacity for building, estimating, and defending causal statistical models using software such as R, Python, or STATA, with a willingness to learn causal inference and structural econometrics and creating production software. Knowledge of SQL, machine learning, and large-scale scientific computing using distributed computing infrastructures such as Spark-Scala or PySpark would be a plus. Additionally, this candidate will show a track record of managing others to perform these tasks, creating a strategic vision for teams and managing team workflows, at-scale and on time. The role will provide the opportunity to have a large strategic, world-wide impact on the customer experiences of Prime members.Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, visit https://www.amazon.jobs/en/disability/us
AT, Graz
Job summaryWe’re working on the future. If you are seeking an iterative fast-paced environment where you can drive innovation, apply state-of-the-art technologies to solve extreme-scale real world delivery challenges, and provide visible benefit to end-users, this is your opportunity.Come work on the Amazon Prime Air!We're looking for outstanding scientists and engineers who combine superb technical, research and analytical capabilities with a demonstrated ability to get complex hardware, software, embedded, mobile and mission-critical systems up and running quickly and effectively. This person must be comfortable working with a team of top-notch software, hardware and applied science Engineers .We’re looking for people who innovate and love solving hard problems. You will work hard, have fun, and of course, make history!Export License ControlThis position may require a deemed export control license for compliance with applicable laws and regulations. Placement is contingent on Amazon’s ability to apply for and obtain an export control license on your behalf.
US, WA, Seattle
Job summaryJoin the Amazon Web Services (AWS) Finance, Analytics, and Science Team! AWS is one of Amazon’s fastest growing businesses, serving millions of customers in more than 190 countries. We are reshaping the way global enterprises use information technology and we are powering the next generation of global business leaders. We are looking for entrepreneurial, analytical, creative, flexible leaders to help us redefine the information technology industry. If you want to join a fast-paced, innovative team that is making history, this is the place for you.As a Data Scientist, you will be expected to work directly with senior management on key business problems. Scientists at Amazon use statistical and ML models to address complex problems, design and test hypotheses, and contribute to the design of automated systems. We are looking for a results-driven scientist to join our team in Seattle. The right candidate will work closely with other scientists, business leaders, business intelligence engineers, and data engineers.The key strategic objectives for this role include:· Work with the AWS Partner Organization to provide data- and model-driven guidance and recommendations on strategic questions posed by leadership.· Identify opportunities to measure and help improve productivity with the end goal of driving growth.· Conduct, direct, and coordinate all phases of data science projects, demonstrating skill in defining key research questions, working with multiple data sources, evaluating competing models, and interpreting and communicating results.If you have an entrepreneurial spirit, you know how to deliver results fast, and you have a deeply quantitative, highly innovative approach to solving problems, we want to talk to you.This role can be based in Seattle, WA, Herndon, VA, or Jersey City, NJ.
US, NY, New York
Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.Sponsored Products help merchants, retail vendors, and brand owners succeed via native advertising that grows incremental sales of their products sold through Amazon. The Sponsored Products organization optimizes the systems and ad placements to match advertiser demand with publisher supply using a combination of machine learning, big data analytics, ultra-low latency high-volume engineering systems, and quantitative product focus. Our goals are to help buyers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and to build a major, sustainable business that helps Amazon continuously innovate on behalf of all customers.As the Data Science Manager on this team, you will:· Lead and manage a team of scientists, business intelligence engieers (etc) on solving science problems with a high degree of complexity and ambiguity.· Develop science roadmaps, run annual planning, and foster cross-team collaboration to execute complex projects.· Perform hands-on data analysis, build machine-learning models, run regular A/B tests, and communicate the impact to senior management.· Hire and develop top talent, provide technical and career development guidance to scientists and engineers in the organization.· Analyze historical data to identify trends and support optimal decision making.· Apply statistical and machine learning knowledge to specific business problems and data.· Formalize assumptions about how our systems should work, create statistical definitions of outliers, and develop methods to systematically identify outliers. Work out why such examples are outliers and define if any actions needed.· Given anecdotes about anomalies or generate automatic scripts to define anomalies, deep dive to explain why they happen, and identify fixes.· Build decision-making models and propose effective solutions for the business problems you define.· Conduct written and verbal presentations to share insights to audiences of varying levels of technical sophistication.Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate.Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.Team video https://youtu.be/zD_6Lzw8raE
US, WA, Bellevue
Job summaryExcited about robotics, and how it will be used in our everyday lives? The Amazon Consumer Robotics group is pushing the boundaries of what’s possible, making the future closer than ever.Astro is Amazon’s first household robot to use AI and Intelligent Motion to gracefully and intuitively interact with people and spaces. It is a different kind of robot, one that’s designed to help customers with a range of tasks like home monitoring and keeping in touch with family. It brings together new advancements in artificial intelligence, computer vision, sensor technology, voice and edge computing in a package that’s designed to be helpful and convenient.The Role:We are a smart team of doers that work passionately to apply cutting-edge advances in Computer Vision and Robotics to solve real-world challenges that will transform our customers’ experiences in ways we can’t even imagine yet. We are looking for an intrepid problem-solver to dive into an ambiguous problem space and design and deliver innovative Computer Vision solutions, and to collaborate with cross-functional engineering teams, to put the concepts you develop into production. You will work alongside applied scientist, engineering (software/hardware), product management and design to define and build key features and functionality for a truly unique product. As Applied Scientist Manager, you will be responsible for building and leading a team of applied scientists and engineers to architect, design, and develop cutting edge CV/ML/DL technologies and services. This leader will own critical platform components spanning across device and cloud, and will work with other leaders to solve difficult technical problems. The ideal candidate will have an established background building best-of-class applied science teams and a proven track record of solving hard technical problems. This person must also be a strong leader who can drive consensus across cross-functional business and software development teams and be an excellent communicator who can articulate technical design tradeoffs with senior technical leaders.
IL, Haifa
Job summaryYou: Alexa, I am looking for a role in which I could learn, research, and innovate in AI and, most of all, impact the life of millions of customers worldwide. What do you suggest?Alexa: The Alexa Shopping team is looking for research engineers to help me become the best personal shopping assistant. Do you want to hear more?You: Yes, please!Alexa: As a research engineer, you will work with top researchers and engineers, both locally and abroad, to explore and develop new AI technologies helping me in my journey to become the ultimate shopping assistant for millions of customers around the world. You should have strong computer science foundations, excellent development skills, and some experience with research methodology. You also preferably have some applied or research expertise in at least one of the following fields: Web search and mining, Machine Learning, Natural Language Processing, Computer Vision, Speech Processing, or Artificial Intelligence.BASIC QUALIFICATIONSMSc in Computer Science, or related field5+ years of industrial experienceStrong computer science foundations (data structures and algorithms)Excellent coding and design skills, expert in C++/C#, Java, or PythonBasic research methodology skillsGood writing and verbal English skillsStrong communication and collaboration skillsPREFERRED QUALIFICATIONS•One or more publications at peer-reviewed research conferences•Solid knowledge of AI and/or Data Science technologies (Machine Learning, Deep Learning, Natural-Language Processing, Computer Vision, etc.), big data and cloud technologies (e.g., Spark, AWS, etc.)• Industrial experience in building and launching AI-related features, products or systemsAmazon Science (www.amazon.science) gives you insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Our scientists continue to publish, teach, and engage with the academic community, in addition to utilizing our working backwards method to enrich the way we live and work.#scienceemea
DE, BE, Berlin
Job summaryAre you interested in delighting Alexa customers around the world? Come join us and help shape Alexa's future. We are the Alexa Natural Language Understanding (NLU) Non-English eXpansion Team (NEXT) team in Berlin! We are tackling some of the most interesting problems on the leading edge of Machine Learning and NLU for German and French Alexa.We are seeking a creative Applied Scientist who has profound knowledge and experience in machine learning / data science and is passionate about working with language data and model. Our ongoing challenges include various aspects of NLU modelling, such as model accuracy and robustness, efficient model training and testing. You will be responsible for delivering insightful data analytics to optimize accuracy and efficiency of our NLU models, where your work will have a direct impact on our customers. You should have extensive experience in research as well as business analytics and have the aptitude to incorporate new approaches and methodologies while dealing with ambiguities in sourcing processes. You should have a demonstrated ability to think strategically and analytically about business, product and technical challenges. This role requires a strong passion for customers, a high level of comfort navigating through ambiguity and a keen sense of ownership and drive to deliver results.Key job responsibilities* Ensure data quality throughout all stages of processing, including such areas as data sourcing/collection, ground truth generation, normalization, transformation, cross-lingual alignment/mapping, etc.* Create experiments and prototype implementations of new learning algorithms and prediction techniques* Continuously evaluate data analysis tools and processes and offer solutions to ensure they are efficient, high-quality and scalable* Lead high impact programs collaborating with various teams across Amazon (from science, engineering and business backgrounds)* Lead and test tooling developments and pilot processes to support expansion to new data areas* Present proposals and results in a clear manner backed by data and coupled with actionable conclusions
US, CA, San Diego
Job summaryAmazon Private Brands is looking for a Data Scientist to join our Private Brands Intelligence (PBI) team in building Machine Learning solutions at scale. PBI applies Machine Learning, Statistics, and Econometrics/Economics to derive actionable insights about the complex economy of Amazon’s retail business. We also develop statistical models and algorithms to drive strategic business decisions and improve operations. We are an interdisciplinary team of Economists, Engineers, and Scientists incubating and building day one solutions using cutting-edge technology, to solve some of the toughest business problems at Amazon.You will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed services. You will invent and implement scalable ML and econometric models while building tools to help our customers gain and apply insights. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale economic problems, enable measurable actions on the consumer economy, and work closely with scientists and economists. We are particularly interested in candidates with experience building predictive models and working with distributed systems.As a Data Scientist, you bring business and industry context to science and technology decisions. You tackle intrinsically hard problems, acquiring expertise as needed. You decompose complex problems into straightforward solutions.
US, Virtual
Job summaryAmazon Care is seeking a talented Economist to apply expertise in economic thinking and theory to key business problems faced in healthcare delivery, patient access, cost and ROI performance, and reaching positive per unit economics at scale. You will work directly with a cross-functional team comprised of clinical, operations, and care delivery members. You will build econometric models, using our world class data systems, and apply economic theory to solve business problems in a fast moving environment and inform future and strategic direction. You will use those models and data to determine the best way to deliver efficient, cost-effective services. At Amazon Care, we are a passionate team working to build a best-in-class healthcare product designed to make high-quality healthcare easy to access.The ideal candidate will be able to: (1) use economic theories to structure ambiguous problems; (2) assess and interpret empirical relationships using large data sets; and (3) design long-term causal estimation models with scalability.Key job responsibilities· Work with Clinical Operations, Capacity Planning, Product, Finance, Data Science, and others across Amazon Care to deliver data-driven insights to efficiently and cost effectively deliver care.· Influence key business decisions in potentially adjusting staffing models, shift structures, care delivery organization, vendor makeup, and balancing cost vs ROI to improve Amazon Care’s at scale economics.· Partner with Operations and Finance teams to build models and decision frameworks related to Care’s unique business problems.· Own the development of economic models and manage the data analysis, modeling, and experimentation necessary to validate your model.· Effectively communicate econometric models to business teams and incorporate feedback into project analysis/modeling.A day in the lifeInclusive Team CultureHere at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. We care about your career growth and strive to assign opportunities based on what will help each team member develop into a better-rounded contributor.
US, WA, Bellevue
Job summaryHave you ever ordered a product on Amazon, and when that box-with-the-smile arrives, you wonder how it got to you so fast? Wondered where it came from and how much it would have cost Amazon? If so, Amazon's Technologies (SCOT) team is for you. We build systems to peer into the future and estimate the distribution of tens of millions of products every week to Amazon's warehouses in the most cost-effective way. This team focuses on saving hundreds of millions of dollars using cutting edge science, , and scalable distributed software on the Cloud that automates and optimizes and shipments to customers under the uncertainty of demand, , and . Watch this short video for more on SCOT: http://bit.ly/amazon-scotWithin SCOT, Amazon's Fulfillment Network Planning (FNP) team focuses on generating free cash flow and fast customer deliveries by designing Amazon's near-term network. Amazon's network involves thousands of trucks and dozens of planes every day to move items that customers have ordered from our Fulfillment Centers to our Sort Centers and Delivery Stations. A quality network design saves Amazon tens of millions of dollars a year, improves the promised delivery time we can offer our customers, and reduces the possibility of missing a promised delivery date.FNP is building the software solution to automate the Network design of the outbound network. Automation saves time and serves as a foundational requirement for global network across tens of thousands of configurations with trillions of potential combinations.
US, WA, Bellevue
Job summarySort Tech Science is looking for a motivated Principal Research Scientist with strong modeling and analytical skills to join our team. As a scientist on our team, you will play an integral role in developing analysis approaches, strategies, and models that improve the efficiency and cost effectiveness of our global network of sortation facilities. Our scientists work with a mix of planning, scheduling, forecasting, and operational decision making models. These models present a range of interesting challenges such as: (i) how do we express our problems in a way that can be solved and what are the appropriate solution techniques (ii) how do we quantify the impact of our decisions on operations (iii) how do we improve our model inputs (forecasts, etc) and make our models robust to errors in the same (iv) how do we guarantee that our models can produce solutions fast enough to be practically usable, without having to sacrifice output quality (v) and many more.Key job responsibilities· Prototyping, analyzing, and maintaining models which are crucial components of our sortation strategy.· Working closely with software development teams as well as business partners to guarantee that models are both implemented correctly and understood by the wider business.· Driving model improvements to assure these models become the de-facto standard across Amazon’s sortation network.· Identifying and evaluating opportunities to reduce costs and maximize efficiency with new models and techniques.· Gathering, analyzing, and presenting data to support various business initiatives. Proposing new metrics if the necessary data is not available.· Helping guide the long term vision of our team, through effective communication with senior management as well with colleagues from computer science, operations research, and business backgrounds.A day in the lifeExploring new opportunities, methods, models, and algorithms to delight our customers by improving sortation technologies and influencing future business requirements.About the teamSort Tech supports under-the-roof (UTR) operations for post-SLAM packages at Outbound Dock of Fulfillment Centers (FC OB Docks), Sort Centers (SCs), AIR Hubs & Gateways, as well as First Mile Operations, plus dock capabilities for Inbound Cross Docks (IXDs) and Delivery Stations (DS). As the newest vertical in Middle Mile Planning, Research, and Optimization Science (mmPROS), Sort Tech Science's vision is to bring modeling, optimization, analysis, and automation to Middle Mile package and transportation flow, to improve safety, reduce variable costs, and accelerate delivery speeds.
US, CA, Santa Clara
Job summaryWho are we and what we do ?Amazon Forecast, an applied research team situated within the larger AWS AI organization called AWS AI Labs, is looking to hire a senior applied machine learning (ML) scientist to work on a variety of important applied ML problems in the area of time series modeling. Our applied ML/AI group is entrusted with developing state-of-the-art generalized deep learning based time series models for time series forecasting and anomaly detection.Who is an ideal candidate ?You are a technical leader of the science team. You work efficiently and routinely develop the right things with limited guidance. Your work focuses on ambiguous problem areas in existing or new ML initiatives. You take a long-term view of your team's ML solutions and how it fits into the production environment. You understand the business impact of your solutions and you show extreme good judgement when making technical trade-offs between short term technology/operational needs and long term business needs.As a technical leader of Amazon Forecast science team, you are expected be an expert in the area of time series modeling (forecasting, anomaly detection, change point detection, reinforcement learning). You are expected to maintain an understanding of industry and technology trends in the area of deep learning, temporal point processes, time series modeling. You will be expected to contribute to the larger science community by either giving presentations at workshops, summits, conferences and/or publishing in top-tier conferences.Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.
US, CA, San Francisco
Job summaryMachine learning (ML) has been strategic to Amazon from the early years. We are pioneers in areas such as recommendation engines, product search, eCommerce fraud detection, and large-scale optimization of fulfillment center operations.The ML team within AWS provides opportunities to innovate in a fast-paced organization that contributes to game-changing projects and technologies that get deployed on devices and in the cloud. As an Applied Science Manager in the ML Solutions Lab team, you'll partner with technology and business teams to build new solutions that delight our customers. You will be responsible for leading a team of data/research/applied scientists, deep learning architects, and ML engineers to build machine learning and deep learning models and pipelines, and deliver state-of-the-art solutions to customer’s business and mission problems. Your team will be working with terabytes of text, images, and other types of data to address real-world problems. You'll design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. You will do all that while yourself maintaining an active personal engagement in the same kind of work at the same kind of high level that you expect from your team.The successful candidate will possess both technical and customer-facing skills that will allow you to be the technical “face” of AWS within our solution providers’ ecosystem/environment as well as directly to end customers. You will be able to drive discussions with senior technical and management personnel within customers and partners, as well as the technical background that enables them to interact with and give guidance to data/research/applied scientists and software developers.The ideal candidate will also have a demonstrated ability to think strategically about business, product, and technical challenges. Finally, and of critical importance, the candidate will be an excellent technical team manager, someone who knows how to hire, develop, and retain high quality technical talent.The primary responsibilities of this position are to:· Interact with customers directly to understand the business problem· Lead a team of scientists and engineers, oversee development and research projects at various stages ranging from initial exploration to fully functional and scalable solutions.· Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, making use of both quantitative and business judgment.· Work closely with account teams, science teams and product engineering teams to drive model implementations· Drive new business and product initiatives, championing ideas that move our business forward· Employ a data driven approach to ensuring the success and outcomes of our ML engagementsThis position requires travel of up to 25%. Role can be based in the greater US West region, including San Francisco, San Diego, Los Angeles, Portland, Seattle, Salt Lake City, Phoenix, and Tempe.About UsInclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and we host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.
US, CA, Santa Monica
Job summaryThe Fashion Innovation Tech team within Amazon Fashion is looking for Applied Scientist to help redefine and build new, exciting experiences for customers shopping for apparel and related fashion products. We own services that enable personalized customer experiences for our customers while pushing the envelope through novel scientific research, advancing the state of art in the field of Computer Vision and Machine Learning. We’re looking for a thought leader to lead the charge with cutting edge research and application of machine learning to address the unique and ambiguous problems in our space. You will work with talented peers in a team that provides opportunities to innovate in a start-up mode, partner with business teams to deliver customer experiences that disrupt status quo.The ideal candidate is a strong, creative and highly-motivated Scientist with hands-on experience in leading multiple research and engineering initiatives. You balance technical leadership with strong business judgment to make the right decisions about technology, tools, and methodologies. You excel in translating broader business objective into Machine Learning science formulations, research for potential solutions or invent new solutions for the objective. You strive for simplicity, demonstrate high judgment backed by sound statistical reasoning and robust machine learning models to deliver creative solutions. You do independent research and develop non-trivial Machine Learning solutions. You mentor and lead scientists and engineers, contribute to Amazon's Intellectual Property through patents and/or external publications.Position Responsibilities: - Participate in the design, development, evaluation, deployment and updating of data-driven models and analytical solutions for machine learning (ML) applications.· Develop and/or apply Machine Learning models (e.g. Deep Neural Networks), optimization methods, and other ML techniques· Build and deploy ML models on available data.· Research and implement novel ML and statistical approaches to add value to the business.· · Mentor junior engineers and scientists.To know more about Amazon science, please visit https://www.amazon.scienceAmazon.com is an Equal Opportunity Employer – Minority / Women / Disability / Veteran / Gender Identity / Sexual Orientation / Age.
US, VA, Arlington
Job summaryThe AWS Infrastructure Planning group is responsible for planning and coordinating a complex, multi-tier supply chain that delivers capacity for all AWS services. We are responsible for ensuring that the AWS cloud remains elastic for its customers by taking care of all of the back-end complexity, enabling our infrastructure to stay ahead of our rapid growth.As part of the Applied Research team, you’ll partner closely with other software development engineers and product managers with a clear path to business impact. Working alongside fellow applied scientists, you’ll leverage your expertise to develop new machine learning techniques and develop statistical models for demand forecasting at different levels of aggregation, lead time and supply predictions, order promising, supply chain risk detection and simulation of complex systems.Successful candidates will have a deep knowledge of statistical and machine learning methods, the ability to map models into scalable production systems, the communication skills necessary to explain complex technical approaches to a variety of stakeholders and customers, and the ability to take an iterative approach to tackle ambiguous long-term problems.Key job responsibilities· Understand business requirements and existing challenges and map them to the right scientific solution.· Research, prototype, experiment, implement and launch machine learning models to solve problems that matter to our customers.· Lead project plans from a scientific perspective by managing product features, technical risks, milestones and launch plans· Create, enhance, and maintain technical documentation, and present to other Scientists, Product, and Engineering teams.· Influence the organization's long-term roadmap and resourcing, onboard new technologies onto our team's toolbox· Develop the right set of metrics to evaluate efficiency/accuracy of the algorithms.· Mentor and develop the scientist community across the organization.About the teamThe AWS Infrastructure Supply Chain Technology Applied Research team drives technological advances and develops machine learning, optimization and simulation products to address complex challenges in the AWS Infrastructure supply chain.We sit at the intersection of science and engineering, and innovate on behalf of customers to solve ambiguous problems that haven’t quite been fully articulated or even anticipated.
IN, KA, Bangalore
Job summaryWould you like to experience what it would have felt like to join Amazon in 1995? Amazon.in, Amazon's marketplace in India, is building a team to take it to the next level by building capabilities that are relevant for customers in India . Our Development team plays a pivotal role in this program, with the mission to build a comprehensive solution for the India business. This is a rare opportunity to be part of a team that will be responsible for building a successful, sustainable and strategic business for Amazon, from the ground up!Beauty and Fashion team's mission is to improve customer experiences for these category items and personalize the recommendations. We build mobile first features. Every page on the shopping funnel has the opportunity for recommendations that are driven with algorithms aided by regional and logistics driven algorithms and strategies. It is a great opportunity to innovate with billions of records of data for the customers. We use data mining and usability data to develop new features and test them through hundreds of A/B experiments a year. This immense amount of data challenges to create highly scalable and low latency solutions across a wide variety of categories, apparel, shoes, watches, luggage, jewelry.On a day-to-day basis, you'll be part of a small, close-knit team of engineers that are agile, data driven, and highly collaborative. You'll help analyze customer behavior, propose ideas and solutions during sprint planning with your team, implement big ideas, and then measure the results. Engineers on our team have proposed ideas that have impacted millions of customers and generated millions of dollars in revenue.You will be instrumental in shaping the product direction and will be actively involved in defining key product features that impact the business. You will work with Principal Engineers at Amazon to evolve the design and architecture of the products owned by this team. You will be responsible to set up and hold a high software quality bar besides providing technical direction to a highly technical team of Software Engineers. You should be a well-rounded software engineer, with expertise in building large-scale web applications and/or low-latency services, optimized for performance. You should love challenges and working on large-scale, customer facing projects.Responsibilities:· Design, implementation, and deployment of applications that impact the business with an emphasis on Payments, Customer Account Management, Merchant platform and e-Commerce website development.· Expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices.· Functionally decompose complex problems into simple, straight-forward solutions· Work extensively with cross-functional teams across Amazon’s website, ecommerce and fulfillment platform on the design and development of core platform functionality.· Work with the business team and project managers to convert functional requirements into detailed technical specifications.· Work with engineers both onsite and offsite to define technical tasks and build detailed implementation plans.· Participate in and provide design inputs towards the Platform initiative for streamlining international expansions.· The ideal candidate will be a leader, builder and operator.· He/she should be able to operate in a very fast paced environment where time to hit market is super critical. They would need to also balance technical leadership with savvy business judgment to make the right decisions about technology choices.
US, WA, Seattle
Job summaryDo you want the excitement of experimenting with cutting edge machine learning, natural language processing, computer vision, and active learning models to solve real world problems at scale? Imagine experimenting with Deep Neural Networks as your daily job and imagine using your model outputs to affect the product discovery of the biggest e-tailer in the world. Imagine leading research inside of an Amazon team that is always looking to deploy creative solutions to real world problems in product discovery. Your research findings are directly related to Amazon’s Browse experience and impact millions of customers, ingesting images, text and all the structured and unstructured attributes in the Amazon catalog to drive true understanding of products at scale.The Amazon Product Knowledge Classification team is seeking an Applied Scientist for developing ML systems that can help classify Amazon products into our catalog and build new experiences for improving customer discovery of products. You will be part of Product Knowledge Classification Science team consisting of experienced Applied Scientists working on a new set of initiatives, building models and delivering them into the Amazon production ecosystem. Your efforts will build a robust ensemble of ML systems that can drive classification of products with a high precision and recall, and scale to new marketplaces and languages. This problem is challenging due to sheer scale (billions of products in the catalog), diversity (products ranging from electronics to groceries to instant video across multiple languages) and multitude of input sources (millions of sellers contributing product data with different quality).We are looking for an experienced Scientist who can develop best in class solutions. Your primary customers are Amazon shoppers who would thank you for correctly identifying products in our catalogs across countries and languages.The ideal Applied Scientist candidate has deep expertise in one or several of the following fields: Web search, Applied/Theoretical Machine Learning, Deep Neural Networks, Classification Systems, Clustering, Label Propagation, Natural Language Processing, Computer Vision, Active learning, and Artificial Intelligence. S/he has a strong publication record at top relevant academic venues and experience in launching products/features in the industry.Amazon Science gives you insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Our scientists continue to publish, teach, and engage with the academic community, in addition to utilizing our working backwards method to enrich the way we live and work.Please visit https://www.amazon.science for more information.
US, CA, Santa Monica
Job summaryThe New Programs Group within Amazon Fashion is looking for Applied Scientist to help redefine and build new, exciting experiences for customers shopping for apparel and related fashion products. We own services that enable personalized customer experiences for our customers while pushing the envelope through novel scientific research, advancing the state of art in the field of Computer Vision and Machine Learning. We’re looking for a thought leader to lead the charge with cutting edge research and application of machine learning to address the unique and ambiguous problems in our space. You will work with talented peers in a team that provides opportunities to innovate in a start-up mode, partner with business teams to deliver customer experiences that disrupt status quo.The ideal candidate is a strong, creative and highly-motivated Scientist with hands-on experience in leading multiple research and engineering initiatives. You balance technical leadership with strong business judgment to make the right decisions about technology, tools, and methodologies. You excel in translating broader business objective into Machine Learning science formulations, research for potential solutions or invent new solutions for the objective. You strive for simplicity, demonstrate high judgment backed by sound statistical reasoning and robust machine learning models to deliver creative solutions. You do independent research and develop non-trivial Machine Learning solutions. You mentor and lead scientists and engineers, contribute to Amazon's Intellectual Property through patents and/or external publications.Position Responsibilities:· Participate in the design, development, evaluation, deployment and updating of data-driven models and analytical solutions for applications of machine learning (Deep Learning, etc), optimization, simulation or visualization techniques.· Build and deploy scientific models on available data.· Research and implement novel approaches to add value to the business.· · Mentor junior engineers and scientists.To know more about Amazon science, Please visit https://www.amazon.science