How Voice and Graphics Working Together Enhance the Alexa Experience

Last week, Amazon announced the release of both a redesigned Echo Show with a bigger screen and the Alexa Presentation Language, which enables third-party developers to build “multimodal” skills that coordinate Alexa’s natural-language-understanding systems with on-screen graphics.

Echo in use

One way that multimodal interaction can improve Alexa customers’ experiences is by helping resolve ambiguous requests. If a customer says, “Alexa, play Harry Potter”, the Echo Show screen could display separate graphics representing a Harry Potter audiobook, a movie, and a soundtrack. If the customer follows up by saying “the last one”, the system must determine whether that means the last item in the on-screen list, the last Harry Potter movie, or something else.

Alexa’s ability to handle these types of interactions derives in part from research that my colleagues and I presented earlier this year at the annual meeting of the Association for the Advancement of Artificial Intelligence. In our paper, we consider three different neural-network designs that treat query resolution as an integrated problem involving both on-screen data and natural-language understanding.

We find that they consistently outperform a natural-language-understanding network that uses hand-coded rules to factor in on-screen data. And on inputs that consist of voice only, their performance is comparable to that of a system trained exclusively on speech inputs. That means that extending the network to consider on-screen data does not degrade accuracy for voice-only inputs.

The other models we investigated are derivatives of the voice-only model, so I’ll describe it first.

All of our networks were trained to classify utterances according to two criteria, intent and slot. An intent is the action that the customer wants Alexa to perform, such as PlayAction<Movie>. Slot values designate the entities on which the intents act, such as ‘Harry Potter’->Movie.name. We have found, empirically, that training a single network to perform both classifications works better than training a separate network for each.

As inputs to the network, we use two different embeddings of each utterance. Embeddings represent words as points in a geometric space, such that strings with similar meanings (or functional roles) are clustered together. Our network learns one embedding from the data on which it is trained, so it is specifically tailored to typical Alexa commands. We also use a standard embedding, based on a much larger corpus of texts, which groups words together according to the words they co-occur with.

The embeddings pass to a bidirectional long short-term memory network. A long short-term memory (LSTM) network processes inputs in order, and its judgment about any given input reflects its judgments about the preceding inputs. LSTMs are widely used in both speech recognition and natural-language processing because they can use context to resolve ambiguities. A bidirectional LSTM (bi-LSTM) is a pair of LSTMs that process an input utterance both backward and forward.

Intent classification is based on the final outputs of the forward and backward LSTMs, since the networks’ confidence in their intent classifications should increase the more of the utterance they see. Slot classification is based on the total output of the LSTMs, since the relevant slot values can occur anywhere in the utterance.

A diagram describing the architectures of all four neural models we evaluated
A diagram describing the architectures of all four neural models we evaluated. The baseline system,which doesn’t use screen information, received only the (a) inputs. The three multimodal neuralsystems received, respectively, (a) and (b); (a), (b), and (c); and (a), (b), and (d).

The data on which we trained all our networks was annotated using the Alexa Meaning Representation Language, a formal language that captures more sophisticated relationships between the parts of an input sentence than earlier methods did. A team of Amazon researchers presented a paper describing the language earlier this year at the annual meeting of the North American chapter of the Association for Computational Linguistics.

The other four models we investigated factored in on-screen content in various ways. The first was a benchmark system that modifies the outputs of the voice-only network according to hand-coded rules.

If, for instance, a customer says, “Play Harry Potter,” the voice-only classifier, absent any other information, might estimate a 50% probability that the customer means the audiobook, a 40% probability that she means the movie, and a 10% probability that she means the soundtrack. If, however, the screen is displaying only movies, our rules would boost the probability that the customer wants the movie.

The factors by which our rules increase or decrease probabilities were determined by a “grid search” on a subset of the training data, in which an algorithm automatically swept through a range of possible modifications to find those that yielded the most accurate results.

The first of our experimental neural models takes as input both the embeddings of the customer’s utterances and a vector representing the types of data displayed on-screen, such as Onscreen_Movie or Onscreen_Book. We assume a fixed number of data types, so the input is a “one-hot” vector, with a bit for each type. If data of a particular type is currently displayed on-screen, its bit is set to 1; otherwise, its bit is set to 0.

The next neural model takes as additional input not only the type of data displayed on-screen but the specific name of each data item — so not just Onscreen_Movie but also ‘Harry Potter’ or ‘The Black Panther’. Those names, too, undergo an embedding, which the network learns to perform during training.

Our third and final neural model factors in the names of on-screen data items as well, but in a more complex way. During training, it uses convolutional filters to, essentially, identify the separate contribution that each name on the screen makes toward the accuracy of the final classification. During operation, it thus bases each of its classifications on the single most relevant name on-screen, rather than all the names at once.

So, in all, we built, trained, and evaluated five different networks: the voice-only network; the voice-only network with hand-coded rules; the voice-and-data-type network; the voice, data type, and data name network; and the voice, data type, and convolutional-filter network.

We tested each of the five networks on four different data sets: slots with and without screen information and intents with and without screen information.

We evaluated performance according to two different metrics, micro-F1 and macro-F1. Micro-F1 scores the networks’ performance separately on each intent and slot, then averages the results. Macro-F1, by contrast, pools the scores across intents and slots and then averages. Micro-F1 gives more weight to intents and slots that are underrepresented in the data, macro-F1 less.

According to micro-F1, all three multimodal neural nets outperformed both the voice-only and the rule-based system across the board. The difference was dramatic on the test sets that included screen information, as might be expected, but the neural nets even had a slight edge on voice-only test sets. On all four test sets, the voice, data type, and data name network achieved the best results.

According to macro-F1, the neural nets generally outperformed the baseline systems, although the voice, data type, and data name network lagged slightly behind the baselines on voice-only slot classification. There was more variation in the top-performing system, too, with each of the three neural nets achieving the highest score on at least one test. Again, however, the neural nets dramatically outperformed the baseline systems on test sets that included screen information.

Acknowledgments: Angeliki Metallinou, Rahul Goel

About the Author
Vishal Naik is an applied scientist in the Alexa AI group.

Work with us

See More Jobs
US, CA, Palo Alto
Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.We are looking for experienced Applied Scientists who have a deep passion for building machine-learning solutions, ability to communicate data insights and scientific vision, and execute strategic projects.As an Applied Scientist in Machine Learning, you will:· Build machine learning models and utilize data analysis to deliver scalable solutions to business problems.· Run A/B experiments, gather data, and perform statistical analysis.· Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving.· Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production· Research new machine learning approaches.
US, CA, Palo Alto
Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.We are looking for experienced Applied Scientists who have a deep passion for building machine-learning solutions, ability to communicate data insights and scientific vision, and execute strategic projects.As an Applied Scientist in Machine Learning, you will:· Build machine learning models and utilize data analysis to deliver scalable solutions to business problems.· Run A/B experiments, gather data, and perform statistical analysis.· Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving.· Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production· Research new machine learning approaches.
US, CA, Sunnyvale
Amazon Lab126 is an inventive research and development company that designs and engineers high-profile consumer electronics. Lab126 began in 2004 as a subsidiary of Amazon.com, Inc., originally creating the best-selling Kindle family of products. Since then, we have produced groundbreaking devices like Fire tablets, Fire TV, and Amazon Echo. What will you help us create?The Role:We are a smart team of doers that work passionately to apply cutting-edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even imagine yet. Key responsibilities will be to conduct research and development in computer vision and related disciplines, and to collaborate with cross-functional engineering teams, including Amazon Robotics, to put the concepts you develop into production. You will determine where commercially available solution and academic research can be applied to solve Amazon business problems, as well as identify opportunities for innovation. You will use a large amount of data to test algorithms to bring them up to production level qualityIf this describes you, come join our team at Lab126 in the heart of Silicon Valley. The team is using computer vision, sensor fusion, real-time and distributed systems to convert requirements into concrete deliverables. A Researcher on this team will translate business and functional requirements into working code. Comfort with a high degree of ambiguity and ability to solve problems that haven’t been solved to scale before are essential.Key responsibilities will be to conduct research and development in computer vision and related disciplines, and to collaborate with cross-functional engineering teams to put the concepts you develop into production. You will determine where commercially available solution and academic research can be applied to solve Amazon business problems, as well as identify opportunities for innovation. You will use a large amount of data to train and test algorithms to bring them up to production level quality. In this role, you will:· Research, design, implement and evaluate novel algorithms· Work on large-scale datasets, focusing on creating scalable and accurate computer vision systems in versatile application fields· Collaborate closely with team members on developing systems from prototyping to production level· Collaborate with teams spread all over the world· Work closely with software engineering teams to drive scalable, real-time implementations· Track general business activity and provide clear, compelling management reports on a regular basis
US, CA, Sunnyvale
Amazon Lab126 is an inventive research and development company that designs and engineers high-profile consumer electronics. Lab126 began in 2004 as a subsidiary of Amazon.com, Inc., originally creating the best-selling Kindle family of products. Since then, we have produced groundbreaking devices like Fire tablets, Fire TV, and Amazon Echo. What will you help us create?The Role:“If you do not work on an important problem, it's unlikely you'll do important work.” – Richard HammingWe have important problems to solve. There are great, world-changing products that should exist, but do not, because the technology to enable them does not exist. Yet. That’s where you come in.We are a smart team of doers that work passionately to apply cutting-edge advances in and to solve real-world challenges that will transform our customers’ experiences in ways we can’t even imagine yet. Key responsibilities will be to conduct research and development in algorithms and to collaborate with cross-functional engineering teams, including Amazon Robotics, to put the concepts you develop into production. You will determine where commercially available solution and academic research can be applied to solve Amazon business problems, as well as identify opportunities for innovation. You will use a large amount of data to train and test algorithms to bring them up to production level quality.If this describes you, come join our team at Lab126 in the heart of Silicon Valley. The team is using, image recognition, real-time and distributed systems to convert requirements into concrete deliverables. A Researcher on this team will translate business and functional requirements into working code. Comfort with a high degree of ambiguity and ability to solve problems that haven’t been solved to scale before are essential.· Research, design, implement and evaluate novel algorithms· Work on large-scale datasets, focusing on creating robust, scalable and accurate systems in versatile application fields· Collaborate closely with team members on developing systems from prototyping to production level· Collaborate with teams spread all over the world· Work closely with engineering teams to drive scalable, real-time implementations
US, WA, Seattle
Are you passionate about leveraging your data science skills to make impact at scale? Do you enjoy developing innovative algorithms, optimization and predictive models to generate insights and recommendations that will be used by millions of Amazon selling partners and FBA operation teams to drive customer impact?Over 2 million Sellers in 10 countries list their products for sale on the Amazon Marketplace. To meet our sellers’ needs, our smart and customer-obsessed employees are constantly innovating and building on new ideas. Fulfillment by Amazon (FBA) is an Amazon service for our sellers. FBA Inbound analytics and data science team partners with FBA inbound product management team to optimize supply chain cost and lead time variability by influencing right trade-offs among cost, speed and FBA supply network capacity.We are looking for a motivated Data Scientist to build, optimize and productionize cutting edge machine learning models. A successful candidate will have strong quantitative, data mining, statistical modeling, machine learning skills and is comfortable facilitating ideation and working from concept through to execution. The position will partner with Product Management, Engineering, Supply Chain optimization and Finance teams to enhance short term and long term business use cases that leverage a range of data science methodologies to solve complex problems for the global FBA Inbound network.A qualified candidate must have demonstrated ability to develop and manage medium to large-scale models and methodologies that are statistically grounded but also functional and practical. Must possess strong written and verbal communication skills, proven ability to engage and collaborate with customers to drive improvements. Possess high intellectual curiosity with ability to quickly learn new concepts/frameworks, algorithms and technology.Key responsibilities of FBA Inbound data scientist include the following:· Research machine learning algorithms and implement by tailoring to FBA business problems· Manipulate/mine data from large databases (Redshift, SQL Server) and create automated pipeline for model training data sets· Collaborate with BI/Data Engineering teams and drive the collection of new data and the refinement of existing data sources to continually improve data quality· Improve model usability by analyzing customer behavior and by gathering requirements from business owners and other tech teams. Incorporate new data sources and implement creative methodologies to improve model performance· Help build production grade systems to support decision making with an objective of optimizing the FBA Inbound business goals to improve customer experience and grow the Amazon business· Create and track accuracy and performance of model predictions/recommendations. Retrain models and research processes optimization· Foster culture of continuous engineering improvement through mentoring, feedback, and analysis.· Lead setting up of machine learning infrastructure and processes for team to collaborate and share code at scaleTo help describe some of our challenges, we created a short video about Supply Chain Optimization at Amazon - http://bit.ly/amazon-scotAmazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation / Age
US, WA, Seattle
Amazon.com strives to be Earth's most customer-centric company where people can find and discover anything they want to buy online. We hire the world's brightest minds, offering them a fast paced, technologically and friendly work environment.Amazon’s Digital Content and Commerce Services team (DCCS) powers ordering, subscriptions and vendor payments for all Amazon’s digital and subscription businesses including Prime, Amazon Video, Music, Alexa and Kindle. We process billions of digital transactions every year and enable our digital businesses to grow their businesses worldwide. We are seeking an experienced, results-driven economist to join our team and help build our next generation of digital solutions and offerings to delight Amazon customers.DCCS owns key customer facing experiences including content selection, subscriptions, pricing, availability, customer experience etc. The Economist will work with leaders Amazon to apply the frontier of economic thinking to market design, pricing, forecasting, program evaluation, and other areas. You will build econometric models using our world class data systems and apply economic theory to solve business problems in a fast moving environment. Economists at Amazon will be expected to develop new techniques to process large data sets, address quantitative problems, and contribute to design of automated systems around the company.Economists will work closely with other research scientists, machine learning experts, and economists globally to design new frameworks that systematically identify low touch machine driven recommendations that propel customer growth while creating a meaningful economic impact for Amazon. Research science at Amazon is a highly experimental activity, although theoretical analysis and innovation are also welcome. Our economists and scientists work closely with software engineers to put algorithms into practice. They also work on cross-disciplinary efforts with other scientists within Amazon.The key strategic objectives for this role include:-Provide data-driven guidance and recommendations on strategic initiatives- Working with senior leaders to identify opportunities/tests that drive product discovery, new customer acquisition to drive growth.-Conduct, direct, and coordinate all phases of research projects, demonstrating skill in all stages of the analysis process, including defining key research questions, recommending measures, working with multiple data sources, evaluating methodology and design, executing analysis plans, interpreting and communicating results.
US, WA, Seattle
Amazon Payments Services build systems that process payments at an unprecedented scale, with accuracy, speed, and mission-critical availability. We process millions of transactions every day worldwide across various payment methods. Over 100 million customers and merchants send hundreds of billions of dollars moving at light-speed through our systems annually. We are growing a new team to innovate and provide a seamless payment experience to our customers across all platforms like mobile, web, voice etc.We are looking for a highly skilled, experienced, and motivated applied scientist to join our product security team. This team owns all aspects of security for the features in the next generation of Amazon's core payments platformAs a senior leader, you will· secure the design and implementation of the most critical systems within Amazon's core payments platform· solve security challenges at a massive scale in a security-first, infrastructure as code, 100% AWS environment· partner closely with our development teams to produce innovative and secure solutions,· proactively perform security assessments to prevent security vulnerabilities,· perform security testing before release,· conducting trainings and hackathons to raise developer awareness of security best practices,· be expected to be strong in multiple domains and will be sought out for advice on technical issues,· have excellent time management skills along with the ability to deliver results in the face of uncertainty, and· foster strong team culture.A successful candidate will be a deeply curious individual who brings technical and leadership expertise, and ability to work within a fast-paced startup culture in a large company that has broad business impact. This is a rare opportunity to get in on the ground floor and start on the Amazon Payments innovation as a security leader.
US, WA, Seattle
Amazon Alexa AI is looking for a passionate, talented, and inventive Senior Applied Scientist with a strong machine learning background to help build industry-leading Speech and Language technology. Our mission is to push the envelope in Natural Language Understanding (NLU), Audio Signal Processing, text-to-speech (TTS), and Dialog Management, in order to provide the best-possible experience for our customers.As an Senior Applied Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in spoken language understanding. Your work will directly impact our customers in the form of products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding.
US, NY, New York
Amazon is investing heavily in building a world class advertising business. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.Job Responsibilities:· Design, develop, and deploy machine learning solutions, and communicating science insights, and executing strategic projects.· Work with business partners to design, implement and launch machine learning solutions. Innovate machine learning models and NLP techniques to connect shoppers with brand ads via shopper intent understanding as well as brand ads interpreting.· Ownership over end-to-end machine learning solutions.Impact and Career Growth:As an Applied Scientist for the Sponsored Products Allocation and Pricing team, you will own systems which make the final decision on which ads to show, where to place them on the page and how many ads to place. This is a challenging technical and business problem, which requires us to balance the interests of advertisers, shoppers, and Amazon. You'll develop a data-driven product strategy to define the right quantitative measures of shopper impact, using this to evaluate decisions and opportunities. You'll balance a portfolio of pragmatic and long-term investments to drive long term growth of the ads and retail businesses. You'll develop real-time algorithms to allocate billions of ads per day in advertising auctionsWhy you love this opportunity:Sponsored Products (SP) is Amazon's largest and fastest growing ad business. SP ads are shown prominently throughout search and detail pages and allow shoppers to seamlessly discover products sold on Amazon. These are native ads that appear visually similar to other content on the page, which presents a huge opportunity for growth and impact, but also a significant responsibility to protect shopper experience.
US, WA, Seattle
The Worldwide Defect Elimination (WWDE) organization in Customer Service drives Amazon towards a defect-free customer experience (CX) by building technology that rapidly identifies and remediates defects, associates them with the information required to resolve the root cause, and prioritizes multitude of improvement opportunities based on customer and business needs. Defect elimination is the upstream solution to achieve perfect CX. A defect when it occurs, causes customer frustration and makes it Amazon’s responsibility to offer appropriate remediation to earn the customer’s trust back leaving little to no margin for error. Finding the right balance between the Concessions offered to revive the CX, compliance to fiscal responsibility and increasing loyalty is a hard problem to solve requiring extensive qualitative analysis and complex computational science. The true goal of WWDE is to eliminate such defects. To achieve this, we must understand what our customers are saying. Our customer feedback is often in free form language and unstructured including Customer Service contacts, product reviews, and social media posts, requiring advanced engineering and science to derive insights. We are looking for a science leader to lead identification and attribution of defects as well as data driven optimization of remediation to earn the customer trust back. You will identify research directions, create roadmaps for forward-looking research and communicate them to senior leadership, and work closely with engineering teams to bring research to production. You will attract top talent, build and lead a high functioning science organization. In this role, you will partner with engineering, product and economist teams to build state of the art ML techniques to improve CX. You will experiment with massive data, work on challenging problems, implement and deploy your ideas at scale, and understand whether they succeed via statistically relevant experiments. You will foster an open and continuous learning environment where your team can pursue ambitious, long-term research, challenge assumptions, work on problems that haven’t been solved before, and publish research while delivering results
US, NY, New York
Amazon Advertising is looking for a motivated, analytical, self-starter to help pave the path for the next generation of insights and advertising products.You will take consumer insights, advertising effectiveness knowledge and business information needs of our advertising clients to envision new advertising products and tools. You will facilitate innovation on behalf of our customers through prototyping and delivering fully functional proof of concepts (POCs). You will partner with our engineering and platform teams to develop and scale successful POCs.This role requires strong hands-on skills in terms of effectively working with data and coding. However, the ideal candidate will also bring strong interpersonal and communication skills as the incumbent is expected to directly engage with advertising clients to validate the need and impact of prototypes, as well as to generally stay connected to insights needs of account teams and advertisers.This is a truly exciting and versatile position in that it allows you to apply and develop your hands-on data modelling and coding skills, to directly engage with stakeholders at major brands while at the same time offering opportunities to work with engineering teams to scale successful innovations.Specific Responsibilities· Create new analytical products from conception to prototyping. Scale the product through to production· Leverage Amazon's data to conduct research on consumer behaviour (in the context of media planning). Make findings available via publishing learning white papers· Communicate with clients to understand their needs· Represent the "voice of the customer", identify repeatable solutions that can be built to scale to serve more advertisers. Act as a liaison to product teams to help productize these solutions· Work closely with the Engineering team· Advocate and educate clients and internal teams on Amazon's ad platform products· Work closely with sales, retail and account management to make data-driven decisions around media planning and strategy· Utilize Amazon's marketing measurement and data platforms to help our advertisers understand the impact of their marketing investments with Amazon
US, NY, New York
Amazon’s Advertising Audience and Media Insights team is looking for a motivated and experienced Applied Scientist to deliver predictive models that will support actionable recommendations to optimize advertising campaigns. Our team owns the product, science, technology and deployment roadmap for advanced analytics and media planning products across our Insights and Performance Team. Analytics and media planning is core to Amazon’s growth, as it helps our suppliers drive awareness, consideration, and purchase of their products by hundreds of millions of consumers around the world, and generates revenue which helps us lower prices and invest in improvements to our customer experience. We are a highly motivated, collaborative and fun-loving team with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.You will be counted on to help the expansion of our Media Insights Science program so our models enable our customers (i.e., advertisers) to achieve the best return on their advertising investment through recommending the best advertising package for their product. You are customer obsessed and have sharp business acumen to work side by side with Software Development Engineers and contribute to a roadmap that contributes to the success of our advertisers. You experiment, develop proof of concept, and ship software. You are an owner and are responsible for the day to day maintenance and evolution of systems that serve the needs of our large and growing base of advertisers. We are looking for contributors who can thrive in a startup environment, think big, can move fast, and wants to change the way customers use data to drive business success. If you excel at proposing and developing unique analytical insights and are passionate about creating the future, come join us as we work hard, have fun, and make history.You will:* Create new analytical products from conception to prototyping. Scale the product through to production* Identify and mitigate risks, work within project schedules, and prepare status reports.* Utilize Amazon's marketing measurement and data platforms to help our advertisers understand the impact of their marketing investments with Amazon* Contribute in a variety of ways from developing scientific models to leading technical discussions, recruiting, helping to refine processes, and so on.* Collaborate on product direction: You’ll build and maintain strong relationships between your team and partner disciplines (Product, Software Development, User Experience, QA) to ensure that we're focused on delivering the right product for customers.You are:* Pragmatic and iterative in developing models and building software: you have an ability to simplify and get things done quickly with a demonstrated track record of building and delivering software and working effectively with external and internal teams.* Highly analytical and data-driven: You solve problems in ways that can be backed up with verifiable data. You focus on driving processes, tools, and statistical methods which support rational decision-making.* Technically fearless: You aren't satisfied by performing 'as expected' and push the limits past conventional boundaries.* Comfortable with ambiguity: You're able to explore new problem spaces with unique constraints and thus non-obvious solutions to deliver value to customers.
US, MA, North Reading
Are you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart team of doers that work passionately to apply cutting edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even image yet. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling, and fun.POSITION OVERVIEWThe Technical Operations Manufacturing team at Amazon Robotics is looking for an experienced, results-oriented Data Scientist to solve difficult problems in manufacturing. The data ranges from mechanical dimensional data to results of complex electromechanical tests, with a little bit of everything in between. Our team is looking for a data scientist to derive insights from this data and communicate the results though white papers, presentations, and production ready models. This role will interface with engineers in software, hardware test, manufacturing, and design. Our methods include design of experiments, statistical modeling, machine learning, financial analysis, and data visualization in a combined team of software engineering and data science.A successful candidate should have a background in business analytics, data science, and data visualization. We are looking for jack-of-all-trades that is able to select and use the right tools to bring clarity to complex questions and help drive decisions. While we don’t assume mastery in all areas, successful team members can speak and work competently in multiple domains.The key strategic objectives for this role include:· Understanding drivers, impacts, and key influences on manufacturing failures.· Optimizing identification of defective product to improve our customer satisfaction.· Drive actions at scale to improve manufacturing yields using scientifically-based methods and decision-making.· Helping to build production systems that take inputs from multiple models and make decisions in real-time.· Automating feedback loops for algorithms in production.· Utilizing Amazon systems and tools to effectively work with terabytes of data.
US, WA, Seattle
Prime Video is an industry leading, high-growth business and a critical driver of Amazon Prime subscriptions, which contribute to customer loyalty and lifetime value. Prime Video is used daily by a massive audience on Amazon's websites and through a variety of devices including the Kindle Fire, game consoles, smart TVs and Blu-ray players. Prime Video is a digital video streaming and download service that offers Amazon customers the ability to rent, purchase or subscribe to a huge catalog of videos. Movies range from new releases to classic favorites, and Prime Video offers major television episodes, entire seasons, or even day after air TV.The Prime Video Content Analytics team uses machine learning, econometrics, and data science to optimize Amazon’s streaming-video catalogue, driving customer engagement and Prime member acquisition. We generate insights to guide Amazon’s digital-video strategy, and we provide direct support to the content-acquisition process. We use detailed customer behavioral data (e.g. streaming history) and detailed information about content (e.g. IMDb-sourced characteristics) to predict and understand what customers like to watch.We are looking for an applied scientist to build innovative systems for measuring the impact of engagement with Prime Video content. Key responsibilities of Prime Video data scientists include the following:· Optimizing the digital video catalog to improve customer experience and grow the Amazon business.· Improving model usability by analyzing customer behavior and by gathering requirements from business owners and other tech teams. Incorporating new data sources and implementing creative methodology innovations to improve model performance.· Creating and tracking accuracy and performance metrics.· Helping build production systems that take inputs from multiple models and support decision making.
US, WA, Seattle
The Team: Amazon Go is a new kind of store with no lines and no checkout—you just grab and go! Customers simply use the Amazon Go app to enter the store, take what they want from our selection of fresh, delicious meals and grocery essentials, and go!Our checkout-free shopping experience is made possible by our Just Walk Out Technology, which automatically detects when products are taken from or returned to the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. Shortly after, we’ll charge your Amazon account and send you a receipt. Check it out at amazon.com/go. Designed and custom-built by Amazonians, our Just Walk Out Technology uses a variety of technologies including computer vision, sensor fusion, and advanced machine learning. Innovation is part of our DNA! Our goal is to be Earths’ most customer centric company and we are just getting started. We need people who want to join an ambitious program that continues to push the state of the art in computer vision, machine learning, distributed systems and hardware design.The Role: Everyone on the team needs to be entrepreneurial, wear many hats and work in a highly collaborative environment that’s more startup than big company. We’ll need to tackle problems that span a variety of domains: computer vision, image recognition, machine learning, real-time and distributed systems. Do you want to join a team of scientists who use computer vision and machine learning to keep Amazon at the frontiers of AI? Are you excited by the prospect of analyzing and modeling terabytes of data and create state-of-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join our Research group.Technical Responsibilities:· Lead the development and implementation of computer vision and machine learning models and algorithms· Applies advanced mathematical theory, statistical analysis and machine learning techniques to improve existing approaches and designs.· Designs solutions for complex business problems while keeping a company-wide perspective using quantitative methods· Executes highly complex projects and plays a role in key aspects of a strategic project.· Establishing efficient, automated processes for large data analyses, model development, model validation and model implementation.· Plan, design, execute and measure controlled experiments to understand impact of our actionsLeadership Responsibilities:· Understand high-level business objectives and continually align work with those objectives to meet needs of business (ROI, cost-benefit analysis).· Manage, mentor and coach researchers· Effectively advocate technical solutions to business stakeholders, engineering teams, as well as executive level decision makers.· Design strategies for effective use of model output.· Maintain a culture of innovation via rapid prototyping and research to leverage new data sources and techniques.· Leads report-writing efforts to communicate analysis and proposed solutions to stakeholders.
US, WA, Seattle
The Team: Amazon Go is a new kind of store with no lines and no checkout—you just grab and go! Customers simply use the Amazon Go app to enter the store, take what they want from our selection of fresh, delicious meals and grocery essentials, and go!Our checkout-free shopping experience is made possible by our Just Walk Out Technology, which automatically detects when products are taken from or returned to the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. Shortly after, we’ll charge your Amazon account and send you a receipt. Check it out at amazon.com/go. Designed and custom-built by Amazonians, our Just Walk Out Technology uses a variety of technologies including computer vision, sensor fusion, and advanced machine learning. Innovation is part of our DNA! Our goal is to be Earths’ most customer centric company and we are just getting started. We need people who want to join an ambitious program that continues to push the state of the art in computer vision, machine learning, distributed systems and hardware design.The Role: Everyone on the team needs to be entrepreneurial, wear many hats and work in a highly collaborative environment that’s more startup than big company. We’ll need to tackle problems that span a variety of domains: computer vision, image recognition, machine learning, real-time and distributed systems.As an Applied Scientist, you will help solve a variety of technical challenges and mentor other scientists and engineers. Given that this is an early-stage initiative, you will play an active role in translating business and functional requirements into concrete deliverables and build quick prototypes or proofs of concept in partnership with other technology leaders within the team. You will tackle challenging, novel situations every day and given the size of this initiative, you’ll have the opportunity to work with multiple technical teams across Amazon. You should be comfortable with a degree of ambiguity that’s higher than most projects and relish the idea of solving problems that, haven’t been solved at scale before - anywhere. Along the way, we guarantee that you’ll learn a ton, have fun and make a positive impact on millions of people.
US, NY, New York
Want to help the largest global enterprises derive business value through the adoption of Artificial Intelligence (AI) and Machine Learning (ML)? Excited by using massive amounts of disparate data to develop ML models? Eager to learn to apply ML to a diverse array of enterprise use cases? Thrilled to be a part of Amazon who has been pioneering and shaping the world’s AI/ML technology for decades?At Amazon Web Services (AWS), we are helping large enterprises build ML models on the AWS Cloud. We are applying predictive technology to large volumes of data and against a wide spectrum of problems. AWS Professional Services works together with AWS customers to address their business needs using AI solutions.AWS Professional Services is a unique consulting team. We pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers. If you have experience with AI, including building ML models, we’d like to have you join our team. You will get to work with an innovative company, with great teammates, and have a lot of fun helping our customers. A successful candidate will be a person who enjoys diving deep into data, doing analysis, discovering root causes, and designing long-term solutions.This is a customer-facing role and you will be required to travel to client locations and deliver professional services as needed.Major responsibilities include:· Assist customers by being able to deliver a ML project from beginning to end, including understanding the business need, aggregating data, exploring data, building & validating predictive models, and deploying completed models with concept-drift monitoring and retraining to deliver business impact to the organization· Use AWS AI services (e.g., Personalize), ML platforms (SageMaker), and frameworks (e.g., MXNet, TensorFlow, PyTorch, SparkML, scikit-learn) to help our customers build ML models· Research and implement novel ML approaches, including hardware optimizations on platforms such as AWS Inferentia· Work with our other Professional Services consultants (Big Data, IoT, HPC) to analyze, extract, normalize, and label relevant data, and with our Professional Services engineers to operationalize customers’ models after they are prototyped
US, CA, Sunnyvale
Amazon Alexa AI is looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading Speech and Language technology. Our mission is to push the envelope in Natural Language Understanding (NLU), Audio Signal Processing, text-to-speech (TTS), and Dialog Management, in order to provide the best-possible experience for our customers.As an Applied Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in spoken language understanding. Your work will directly impact our customers in the form of products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding.
US
Are you interested in building, leading, and driving a machine learning vision, strategy and execution plan for process improvement efforts centered on Operations? WW Ops Solutions is hiring a Data Scientist (DS) to play a leadership role in prototyping data science solutions that auto-generate business insights to help drive continuous improvement efforts across our network. Our goal is to apply innovative data solutions to process improvement efforts across Amazon in areas such as artificial intelligence, machine learning, computer vision, systems monitoring & alerting, and other capacities to advance Amazon’s performance.The ideal candidate will be an expert in the areas of data science, machine learning and statistics, having hands-on experience with multiple improvement initiatives as well as balancing technical and business judgment to make the right decisions about technology, models and methodologies. The candidate needs experience with data science / business intelligence, analytics, and reporting systems while striving for simplicity, and demonstrating significant creativity and high judgment backed by statistical proof. The successful candidate will be a recognized expert for their analytical and leadership abilities.
US, WA, Seattle
Amazon's Sponsored Ads is one of the fastest growing business domains and we are looking senior level scientists with experience building complex machine learning models and writing elegant prototype code. We are still in Day 1 and there is an abundance of opportunities that are yet to be explored. We are a team of highly motivated and collaborative applied scientists and software development engineers with an entrepreneurial spirit and bias for action. With the team growing at an unprecedented rate, there is broad mandate to experiment and innovate!Our systems and algorithms operate on one of the world's largest product catalogs, matching shoppers with products - with a high relevance bar and strict latency constraints. The ads sourcing works across the spectrum of ad serving including coverage expansion, increasing utilization of tail detail pages, ad relevance, ad quality, collaborative filtering, and much more. Our technology enables thousands of brands, sellers and authors to drive discovery and sales of their products at Amazon by millions of customers.We are looking for an Applied Scientists, with a background in Machine Learning to optimize serving ads on billions of product pages. The solutions you create/deploy would ensure relevant ads are served to Amazon's customers. You will directly impact the shopping experience while helping our advertisers get the maximum ROI. You will be expected to demonstrate strong ownership and should be curious to learn and leverage the rich textual, image, and other contextual signals.This role will challenge you to utilize cutting-edge machine learning techniques in the domain of predictive modeling, natural language processing (NLP), deep learning, and image recognition to deliver significant impact for the business. Ideal candidates will be able to work cross functionally across multiple stakeholders, synthesize the science needs of our business partners, develop models to solve business needs, and implement solutions in production.