New Way to Annotate Training Data Should Enable More Sophisticated Alexa Interactions

Developing a new Alexa skill typically means training a machine-learning system with annotated data, and the skill’s ability to “understand” natural-language requests is limited by the expressivity of the semantic representation used to do the annotation. So far, the techniques used to represent natural language have been fairly simple, so Alexa has been able to handle only relatively simple requests.

At the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) that begins this weekend, we will present a new, more sophisticated semantic-representation language that we call the Alexa Meaning Representation Language. Data annotated in the language should enable Alexa skills to handle much more complex conversational interactions and to process simple interactions more accurately. This new representation now powers the library of built-in “intents” — actions that skills can perform — available to skill developers to help them bootstrap their natural-language-understanding (NLU) systems.

Flat semantic representation
Flat semantic representation

Traditionally, data used to train Alexa skills has been annotated using the flat semantic representation of “domain”, “intent” and “slot”. “Domain” describes the class of skill that the utterance is meant to invoke, such as MusicApp or HomeAutomation. (Each domain may have multiple associated skills. For instance, the skills ClassicMusic and PopMusic might both fall under the MusicApp domain.) “Intent” describes the action that the skill is being asked to perform, such as PlayTune or ActivateAppliance. And “slot” describes the entities and classes of entities on which the action is to operate, such as “song,” “‘Thriller,’” and “Michael Jackson” in the command “Play ‘Thriller’ by Michael Jackson.”

Alexa’s popularity attests to the success of this relatively simple annotation scheme. But to realize the goal of seamless conversational interaction, Alexa skills must be able to both interpret more complex linguistic structures and distinguish between competing interpretations of simple ones.

For instance, Alexa should, ideally, be able to handle utterances like “Alexa, find me a restaurant near the Mariners game,” which spans two domains, local businesses and sporting events. Conversely, Alexa should be able to resolve utterances with similar structures into different domains, as occasion warrants — for instance, “Alexa, order me a cab,” versus “Alexa, order me an Echo Dot.” With the existing, flat annotation scheme, training a machine-learning system to better handle one of these instances will weaken its ability to handle the other. Distinguishing the two use cases in the training data would require such overspecification of intents and slots that the system would lose the ability to exploit the general form of the sentence.

The Alexa Meaning Representation Language (AMRL) addresses these problems. We build on previous work on graph-based semantic representations and adapt them to conversational systems. Our solution consists of two key components:

  1. A large hierarchical ontology of types, roles, actions, and operators: actions represent a predicate that determines what the agent should do; roles express the arguments to an action; types categorize textual mentions; and properties are relations between type mentions
  2. A set of conventions that map natural language to a graph-based, domain- and language-agnostic representation suited for conversational agents such as Alexa

AMRL leverages the type hierarchy by allowing fine-grained type and action annotations. When more than one fine-grained type is possible — as in the “order me a cab/Dot” example — the annotation backs off to a coarser-grained type or action in the hierarchy. For example, in the utterance “play ‘Thriller’”, both “MusicRecording” and “MusicVideo” are possible as types, so the annotation guideline prefers the closest common ancestor “MusicCreativeWork”.

Graph representation in Alexa Meaning Representation Language
Graph representation in Alexa Meaning Representation Language

A graph is a mathematical structure consisting of nodes — typically depicted as circles — and edges — typically depicted as lines connecting the nodes. In AMRL, nodes are types and edges are properties. So, for instance, the node “MusicRecording” would be connected to the node “Musician” by the edge “byArtist”.

Our paper presents detailed explanations and use cases for handling more complex cross-domain linguistic phenomena such as conjunctions, disjunctions, and negations; handling anaphora (the use of generic pronouns like “that” or “here” to refer to previously used words); handling multiple intents; and conditional statements. We hope these design decisions on how to annotate short text in conversational systems will spur further research in the area of semantic meaning representation for action-oriented systems.

Paper: "The Alexa Meaning Representation Language"

Alexa science

Acknowledgements: Thomas Kollar, Danielle Berry, Lauren Stuart, Karolina Owczarzak, Tagyoung Chung, Michael Kayser, Bradford Snow, Spyros Matsoukas

About the Author

Related content

View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Sign up for our monthly newsletter

Work with us

See more jobs
CA, ON, Toronto
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. As a core product offering within our advertising portfolio, Sponsored Products (SP) helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The SP team's primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day!As a Data Scientist on this team you will:· Solve real world problems by analyzing large amounts of business data, diving deep to identify business insights and opportunities, designing simulations and experiments, developing statistical and ML models by tailoring to business needs, and collaborating with Scientists, Engineers, BIE's, and Product Managers.· Translate business questions and concerns into specific quantitative questions that can be answered with available data using sound methodologies. In cases where questions cannot be answered with available data, work with engineers to produce the required data.· Deliver with independence on challenging large scale problems with ambiguity.· Manage and drive the technical and analytical aspects of Advertiser segmentation; continually advance approach and methods.· Retrieve, synthesize, and present critical data in a format that is immediately useful to answering specific questions or improving system performance.· Analyze historical data to identify trends and support decision making.· Improve upon existing methodologies by developing new data sources, testing model enhancements, and fine-tuning model parameters.· Provide requirements to develop analytic capabilities, platforms, and pipelines.· Apply statistical or machine learning knowledge to specific business problems and data.· Formalize assumptions about how our systems are expected to work, create statistical definition of the outlier, and develop methods to systematically identify these outliers. Work out why such examples are outliers and define if any actions needed.· Given anecdotes about anomalies or generate automatic scripts to define anomalies, deep dive to explain why they happen, and identify fixes.· Build decision-making models and propose solution for the business problem you defined· Conduct written and verbal presentation to share insights and recommendations to audiences of varying levels of technical sophistication.· Utilize code (python or another object oriented language) for data analyzing and modeling algorithms.Why you love this opportunityAmazon is investing heavily in building a world-class advertising business. This team is responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate.Impact and Career GrowthYou will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven fundamentally from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.Team video https://youtu.be/zD_6Lzw8raE
US, WA, Virtual Location - Washington
The intern will be placed in a large economic team with 10+ full time economists and 3 other interns in the cohort.A day in the lifeTalk with business people to understand business context.Brainstorm with peer economists.Network with other economists and scientists on the team.About the hiring groupThe Amazon Economics Team is hiring Interns in Economics. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Some knowledge of econometrics, as well as basic familiarity with Stata or R is necessary, and experience with SQL, UNIX, and Sawtooth would be a plus.Job responsibilitiesThe Amazon Economics Team is hiring Interns in Economics. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Some knowledge of econometrics, as well as basic familiarity with Stata or R is necessary, and experience with SQL, UNIX, and Sawtooth would be a plus.These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, data scientists and MBAʼs. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement.Roughly 50% of research assistants from previous cohorts have converted to full time data science or economics employment at Amazon.Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
US, CA, Palo Alto
As the Director of AI Data Labeling you will lead a geographically distributed, multi-disciplinary team that is responsible for building and operating a scalable, world-class, multi-tenant, high-throughput, high-fidelity data labeling infrastructure. This role requires the ability to lead a team of machine learning (ML) applied scientists, software engineers, statisticians, economists, in-house labelers, product, and program managers. The team builds new software systems, explores new scientific approaches, defines and improves operational processes, and optimizes significant resource investments (people, vendor and compute budgets). The role requires strong operational skills to manage programs and guide product management to consolidate different labeling systems across Search, create a product roadmap that meets the needs of Search and other Amazon Consumer teams, and scale the product while meeting all internal customer needs for labeled data. This role is an opportunity to lead a fast growing area delivering labels for ML applications through a mixture of ML algorithms and human judges. It is an active area of research that will continue to evolve quickly over the next few years and we are looking for a leader at the forefront of this exciting opportunity.
US, VA, Arlington
Are you customer-obsessed, data oriented, technologically curious, and eager to propose opportunities to improve the customer experience of using technical Amazon services? We are seeking an experienced, talented, and motivated Data Scientist to join our Technical Customer Experience Strategy team. Our team is dedicated to improving Amazon’s technical products through evaluating the end-to-end customer experience using both internal and external processes and technology. Our team’s mission is to constantly challenge the status quo and drive innovation to improve customer experience. This role specifically focuses on measuring and enhancing the quality of AWS AI services.You will own and drive the roadmap for studying AI quality and build out scalable methods, mechanisms, and tooling to evaluate and deep dive into the quality of AI services. You will engage with AWS AI service teams and leaders to develop scoring methods, identify datasets, evaluate performance, and influence change.Responsibilities:· Ensure data quality throughout all stages of processing, including such areas as data sourcing/collection, ground truth generation, normalization, transformation, cross-lingual alignment/mapping, etc.· Continuously evaluate data analysis tools and processes and offer solutions to ensure they are efficient, high-quality and scalable.· Design, build, and deploy effective and innovative ML solutions to improve various components of the performance measuring stack, such as scoring, trend/pattern analysis, and deep dives.· Develop and improve statistical analyses that provide insights into the business· Communicate analyses and techniques, both verbally and in writing, to peers and business partners.· Evaluate the proposed solutions via offline benchmark tests as well as online A/B tests in production.· Publish and present your work at internal and external scientific venues in the fields of ML.· Present proposals and results in a clear manner backed by data and coupled with actionable conclusions.· Report on the status of projects in a timely and accurate fashion using available metrics.
US, WA, Seattle
Amazon is focused on protecting the health and safety of our employees while continuing to serve people who need our services more than ever. Regular testing on a global scale across all industries would both help keep people safe and help get the economy back up and running. But, for this to work, we as a society would need vastly more testing capacity than is currently available. Unfortunately, today we live in a world of scarcity where COVID-19 testing is heavily rationed.To help solve this problem, Amazon has begun building incremental testing capacity and conducting research and development to apply cutting-edge technology advances to solve testing bottlenecks. We are seeking a Bioinformatics Lead to help define new approaches and execute against scaling them. You will work with a unique and gifted team developing these and collaborate with cross-functional teams. Our team rewards intellectual curiosity while maintaining a laser-focus in bringing products to market. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the cutting edge of both academic and applied research in this product area, you have the opportunity to work together with some of the most talented scientists, engineers, and product managers in the field.
US, MA, Cambridge
Want to transform the way people enjoy music, video, and radio? Come join the team that made Amazon Music, Spotify, Hulu, Netflix, Pandora, available to Alexa customers. We are innovating the way our customers interact with entertainment in the living room, on the go, and in the car. We are at the epicenter of the future of entertainment.Alexa Entertainment is looking for an Applied Scientist as we build a team of talented and passionate scientists for ASR (automatic speech recognition) and NLU (natural language understanding). As an Applied Scientist, you will participate in the design, development, and evaluation of models and ML (machine learning) technology so that customers have the magical experience of entertainment via Alexa. You will help lay the foundation to move from directed interactions to learned behaviors that enable Alexa to proactively take action on behalf of the customer. And, you will have the satisfaction of working on a product your friends and family can relate to, and want to use every day. Like the world of smart phones less than 10 years ago, this is a rare opportunity to have a giant impact on the way people live.You will be part of a team delivering features that are highly anticipated by media and well received by our customers.
US, WA, Seattle
Are you passionate about leveraging your economics/econometrics skills and data to deliver actionable insights that impact daily business decisions at Amazon? Do you like getting "scrappy" with data and applied economics tools to answer challenging product and customer behavior questions? Do you enjoy building flexible, performant, and global solutions for complex financial, risk, and causal inference problems? If so, here is a great opportunity to consider!A day in the lifeAmazon B2B Payments is seeking a thoughtful Economist who combines their causal inference expertise with business intuition to generate critical insights that will set the strategic direction of the business. As an Economist on the Credit Systems Research team, you will design and build systems/solutions that support financial products, working closely with business partners, product partners, and engineers. Your work will focus on Amazon Lending, which offers loan products to help small and medium-sized businesses grow on Amazon. You will utilize causal inference and experimentation methodologies, data and coding skills, problem solving and analytical skills, and excellent communication to deliver customer value.About the hiring groupThe mission of B2B Payments is to create trusted paying experiences and financial services that empower any business to engage with Amazon from anywhere in any way. Within B2B Payments, the Credit Systems team builds models and systems to maximize business growth and customer experience/value while achieving risk constraints.Job responsibilitiesYou will support our causal analysis and experimentation workstreams. You will use novel and advanced causal economics/econometrics techniques to answer questions on a number of challenging business questions related to credit systems products, focusing on Amazon Lending. You will be responsible for researching insights as well as educating the business, product, marketing, and business development teams on the implementation of those insights to enable data-driven decision making. You will partner with our product management, finance, marketing, engineering, and operations teams to:· Contribute to the development and enhancement of business payment products and features.· Use observational as well as experimental data to develop empirical econometric analysis and models to drive the business and improve our causal inference workstream.· Understand business and product strategies, goals, and objectives and help set the causal and experimental roadmap.· Analyze and solve problems at their root, stepping back to understand the broader context.Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
US, CA, Glendale
The Amazon Web Services (AWS) Center for Quantum Computing in Pasadena, CA, is looking to hire a Quantum Research Scientist with expertise in implementing, characterizing and validating coherentcontrol techniques for quantum information processing. Working alongside physicists and engineers, you will use your coherent control knowledge to help develop and implement novel multi-qubit control protocols. You will then use your experimental experience as you characterize, verify and validate their performance on qubit processors. Responsibilities include, but are not limited to:· using control theory, device physics, and quantum information concepts as necessary to develop and debug control/characterization protocols· numerical simulation of protocols on multi-qubit systems to inform experiments· converting protocols to experimental reality using a mixture of commercial and custom hardware· contributing experimental routines and data analysis to the control software codebase· developing statistically valid data analysis routines to distill the data into plainly communicable results to a multi-disciplinary teamWork/Life BalanceAt the AWS CQC, we understand that developing quantum computing technology is a marathon, not a sprint. Mental and physical wellness is encouraged within our team and throughout AWS. The work/life integration within Amazon encourages a culture where employees work hard and have ownership over their downtime. We are exploring more structured wellness elements including meditation scheduling, running group meet-ups, and a culture of sharing wellness tips.Mentorship and Career GrowthWe are committed to the growth and development of every member of the Center for Quantum Computing. You will receive career-growth-minded management and mentorship from a software and science team and also have the opportunity to participate in Amazon's mentorship programs. You will work closely with quantum research scientists and have opportunities to learn about quantum computing technology and contribute to the development of scientific software for quantum computing at AWS.Inclusive and Diverse CultureThe AWS CQC is intentional about attracting, developing, and retaining amazing talent from diverse backgrounds to build a world class team. We’re looking for a new teammate who is enthusiastic, empathetic, curious, motivated, reliable, and able to work effectively with their peers. With quantum computing being a new and growing initiative within AWS, you would have an opportunity to make an impact on our budding team culture.Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation
US, NY, New York
The Automated Reasoning Group in AWS Platform is looking for an Applied Scientist with experience in building secure, scalable solutions that delight customers. You will be part of a world-class team building the next generation of automated reasoning tools and services. You will apply your knowledge to propose solutions, create software prototypes, and productize prototypes into production systems using software development tools and methodologies. In addition, you will support and scale your solutions to meet the ever-growing demand of customer use. You have strong verbal and written communication skills, are self-driven and deliver high quality results in a fast-paced environment.Each day, hundreds of thousands of developers make billions of transactions worldwide on AWS. They harness the power of the cloud to enable innovative applications, websites, and businesses. Using automated reasoning technology and mathematical proofs, AWS allows customers to answer questions about security, availability, durability, and functional correctness. We call this provable security, absolute assurance in security of the cloud and in the cloud. https://aws.amazon.com/security/provable-security/As an applied scientist on team, you will play a pivotal role in shaping the definition, vision, design, roadmap and development of product features from beginning to end. You will:· Research algorithms in code-level automated reasoning involving the Rust and C languages to scale automated proofs of AWS code· Design, implement, test, deploy and maintain innovative software solutions to transform service performance, durability, cost, and security· Use software engineering best practices to ensure a high standard of quality for all of the team deliverables· Prove properties of business-critical software and systems software· Work in an agile, startup-like development environment, where you are always working on the most important stuff· Work with the team to help drive business decisionsAWS has the most services and more features within those services, than any other cloud provider–from infrastructure technologies like compute, storage, and databases–to emerging technologies, such as machine learning and artificial intelligence, data lakes and analytics, and Internet of Things. AWS Platform is the glue that holds the AWS ecosystem together. Whether its Identity features such as access management and sign on, cryptography, console, builder & developer tools, and even projects like automating all of our contractual billing systems, AWS Platform is always innovating with the customer in mind. The AWS Platform team sustains over 750 million transactions per second.This role can be based in Boston, NYC, Seattle, Bay Area, and other major cities.Mentorship & Career GrowthWe have a formal mentor search application that lets you find a mentor that works best for you based on location, job family, job level etc. Your manager can also help you find a mentor or two, because two is better than one. In addition to formal mentors, we work and train together so that we are always learning from one another, and we celebrate and support the career progression of our team members.Inclusion and DiversityOur team is diverse! We drive towards an inclusive culture and work environment. We are intentional about attracting, developing, and retaining amazing talent from diverse backgrounds. Team members are active in Amazon’s 10+ affinity groups, sometimes known as employee resource groups, which bring employees together across businesses and locations around the world. These range from groups such as the Black Employee Network, Latinos at Amazon, Indigenous at Amazon, Families at Amazon, Amazon Women and Engineering, LGBTQ+, Warriors at Amazon (Military), Amazon People With Disabilities, and more.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.This position involves on-call responsibilities, typically for one week every two months. We don’t like getting paged in the middle of the night or on the weekend, so we work to ensure that our systems are fault tolerant. When we do get paged, we work together to resolve the root cause so that we don’t get paged for the same issue twice.
US, WA, Seattle
Amazon is looking for an outstanding Data Scientist to join the AWS Product Analytics team. This is your opportunity to be a core part of the team that has direct impact on affecting the long term roadmap of the AWS EC2 Product Team. This role is within a larger Data science, Business Intelligence & Data engineering team that focuses on broad data exploration, quantitative methodology, and statistical modeling to drive actionable data intelligence in AWS.Since early 2006, AWS has provided companies of all sizes with an infrastructure platform in the cloud. AWS is a high-growth, fast-moving division within Amazon with a start-up mentality where new and diverse challenges arise every day. On the AWS Product Analytics team you will be surrounded by people that are exceptionally talented, bright, and driven, and believe that world class Data Science is critical to our success. To help build this growing team, you must be highly analytical and possess a strong passion for analytics and accountability, set high standards with a focus on superior business success. We take working hard, having fun, and making history seriously. AWS sets the standard for functionality, cost, and performance for many cloud based services, but it’s still early days for cloud computing, and there are boundless opportunities to continue to redefine the world of cloud computing - come help us make history!As a Data Scientist, you will discover and solve real world problems by analyzing large amounts of business data, defining new metrics and business cases, designing simulations and experiments, creating models, and collaborating with colleagues in business, software, and research. You will get the exciting opportunity to work on some of the world’s largest and diverse datasets. The successful candidate will have a strong quantitative background and can thrive in an environment that leverages statistics, machine learning, operations research, econometrics, and business analysis.Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.
US, NH, Virtual Location - New Hampshi
The Amazon Device and Services Design Group (DDG) creates revolutionary products used by millions of customers around the world every day - including Echo, Fire TV, Alexa Communications, and more. Within DDG, the DDG Research team is one of the behind-the-scenes teams that enables Amazon’s Devices’ leaders to make informed decisions based on sound mixed-method research and analytics. We are seeking an exceptional Research Scientist to join our team to help us derive insights that serve our customers. If you are inquisitive, passionate about how research can improve decision making, enjoy weaving data into a telling story, and like your research to have real world impact, we want you on our team.As a successful Research Scientist, you:· Are enthusiastic about collaborating with a cross-functional team of UX designers, researchers (qualitative and quantitative), business partners, design technologists, and developers to provide actionable insights.· Demonstrate excellent statistical skills and are excited about learning new methods.· Are naturally curious about people, especially their motivations, attitudes, and behaviors.· Excel when interacting with business and technical partners whether you are chatting, sending a written message, writing a technical report, or conducting a presentation.· Enjoy working with stakeholders to define their key business needs and deliver on those commitments.· Innovate on behalf of your customers by proactively implementing improvements and taking steps to address challenges that they had not yet identified.· Are drawn to ambiguous, complex problems.· Surface and question both your own and others’ underlying assumptions.· Persuade decision makers at all levels of the organization to take aligned action on research findings – inspire and help them internalize opportunities to delight our customers and differentiate our service.· Define, execute, and invent research methods appropriate to the questions at hand, including but not limited to field research, surveys, lab studies, remote testing, and A-B tests.
US, PA, Pennsylvania
The Amazon Economics Team is hiring Interns in Economics. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Some knowledge of econometrics, as well as basic familiarity with Stata or R is necessary, and experience with SQL, UNIX, and Sawtooth would be a plus.These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, data scientists and MBAʼs. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement.Roughly 50% of research assistants from previous cohorts have converted to full time data science or economics employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com.Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation
US, WA, Seattle
The Amazon Economics Team is hiring Interns in Economics. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Some knowledge of econometrics, as well as basic familiarity with Stata or R is necessary, and experience with SQL, UNIX, and Sawtooth would be a plus.These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, data scientists and MBAʼs. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement.Roughly 50% of research assistants from previous cohorts have converted to full time data science or economics employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com.Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation
US, WA, Seattle
Do you want to help guide the future of Amazon’s next generation of fulfilment services to create the best customer experience by pushing the boundaries of economic modeling and experiments? The Multi Channel Fulfillment team is looking for an Economist to play a key role in guiding the next fulfillment delivery platforms to serve the fast growing needs of our merchants and Customers. Economists at Amazon are expected to develop new techniques to process large data sets, address challenging quantitative problems, guide business decisions, and contribute to the design of automated systems around the company. Upon joining the team, you will guide our businesses by building econometric models, using our world class data systems and applying economic theory to solve problems in a fast moving environment.We are looking for an Economist who will lead the development of econometric models, experiments, and algorithms that will enhance our understanding of merchant and Customer behavior, and the impact of fee and policy changes globally. The ideal candidate will have experience with large scale marketplace behavior modeling, forecasting, outstanding technical abilities, and strong business acumen. They will be comfortable with ambiguity and will enjoy working in a fast-paced dynamic environment. The position will work closely with engineers, scientists, financial analysts, and product managers.Responsibilities:· Design and develop complex and innovative models to predict the impact of fees on third party seller’s behavior, growth, and commitment· Design and conduct experiments to validate theories, and improve understanding of Amazon’s third party ecosystem and the impact changes have on the customer experience· Refine fee structures to improve the benefits for customers and sellers· Collaborate with product managers, data scientists and software developers to incorporate models into production processes and influence senior leaders· Explain complex technical approaches to a variety of stakeholders, including senior leadership, software development teams, business/product teams, and cross-functional experts· Serve as subject matter expert for economics related issues within the Multi Channel Fulfillment team
MX, MEX, Mexico City
Excited by using massive amounts of data to develop Machine Learning (ML) and Deep Learning (DL) models? Want to help the largest global enterprises derive business value through the adoption of Artificial Intelligence (AI)? Eager to learn from many different enterprise’s use cases of AWS ML and DL? Thrilled to be key part of Amazon, who has been investing in Machine Learning for decades, pioneering and shaping the world’s AI technology?At Amazon Web Services (AWS), we are helping large enterprises build ML and DL models on the AWS Cloud. We are applying predictive technology to large volumes of data and against a wide spectrum of problems. Our Professional Services organization works together with our AWS customers to address their business needs using AI.AWS Professional Services is a unique consulting team. We pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers. If you have experience with AI, including building ML or DL models, we’d like to have you join our team. You will get to work with an innovative company, with great teammates, and have a lot of fun helping our customers.This role will focus specifically on AWS’ most complex and largest customers in the world to help solve a wide range of business problems. Consultants will provide deep and broad insight to customers and partners to help remove constraints that prevent them from leveraging AWS services to create strategic value.A successful candidate will be a person who enjoys diving deep into data, doing analysis, discovering root causes, and designing long-term solutions. It will be a person who likes to have fun, loves to learn, and wants to innovate in the world of AI. Major responsibilities include:· Understand the customer’s business need and guide them to a solution using our AWS AI Services, AWS AI Platforms, AWS AI Frameworks, and AWS AI EC2 Instances .· Assist customers by being able to deliver a ML / DL project from beginning to end, including understanding the business need, aggregating data, exploring data, building & validating predictive models, and deploying completed models to deliver business impact to the organization.· Use Deep Learning frameworks like MXNet, Caffe 2, Tensorflow, Theano, CNTK, and Keras to help our customers build DL models.· Use SparkML and Amazon Machine Learning (AML) to help our customers build ML models.· Work with our Professional Services Big Data consultants to analyze, extract, normalize, and label relevant data.· Work with our Professional Services DevOps consultants to help our customers operationalize models after they are built.· Assist customers with identifying model drift and retraining models.· Research and implement novel ML and DL approaches, including using FPGA.This is a customer facing role. You will be required to travel to client locations and deliver professional services when needed.
US, WA, Seattle
At Alexa Shopping, we strive to enable shopping in everyday life. We allow customers to instantly order whatever they need, by simply interacting with their Smart Devices such as Amazon Show, Spot, Echo, Dot or Tap. Our Services allow you to shop, no matter where you are or what you are doing, you can go from 'I want that' to 'that's on the way' in a matter of seconds. We are seeking the industry's best to help us create new ways to interact, search and shop. Join us, and you'll be taking part in changing the future of everyday lifeWe are seeking a Data Scientist to be part of the Correction and Automated Recovery (CARe) team for Alexa Shopping. Our team focuses on solving the hard problem of recovering from shopping errors in the Alexa pipeline and improve customer CX. The solutions will help address unique shopping challenges and maximize the self-learning benefits for shopping. This is a strategic role to shape and deliver our science strategy in developing and deploying Machine Learning solutions to our hardest customer facing problems. The initiatives are at the heart of the organization and recognized as the innovations that are ground breaking and will allow us to exceed customer expectations. If this role seems like a good fit, please reach out, we'd love to talk to you.This role requires working closely with business, engineering and other scientists within Alexa, Shopping and across Amazon to deliver ground breaking features. You will lead high visibility and high impact programs collaborating with various teams across Amazon. You will work with a team of SDEs and Scientists to launch new customer facing features and improve the current features.
US, WA, Seattle
We are looking for an Economist to join our fast paced, start-up environment to help invent the future of product economics. We solve significant business problems in the devices and retail spaces by understanding customer behavior and developing business decision-making frameworks. You will build econometric models for causal inference and prediction, using our world class data systems, and apply economic theory to solve business problems in a fast-moving environment. This involves analyzing Amazon Devices customer behavior, and measuring and predicting the lifetime value of existing and future products. We build scalable systems to ensure that our models have broad applicability and large impact.A day in the lifeEconomists closely work with our Finance stakeholders to ensure analyses are optimized for specific decision-making use cases. Economists define the specific research question based on stakeholder needs, propose methodologies, prototype models and explain their findings. For generalized solutions, we build econometric products, many of which are self-service, to continually generate value for stakeholders.About the hiring groupWe focus on guiding the business to make decisions based on causal inference metrics and providing decision-making frameworks. To ensure adoption of our work, we focus on transparency of our models. We also strive to explain heterogenous treatment effects, as average treatment effects can mask the most business relevant variation.Job responsibilitiesThe Device Economics team is hiring Interns in Economics. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Advanced knowledge of causal econometrics, as well as basic familiarity with Python, R or Stata is necessary, and experience with SQL would be a plus.These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, data scientists and MBAs. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement.Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
US, VA, Arlington
Global Talent Management (GTM) Science is an innovative organization that exists to propel Amazon HR towards being the most scientific HR organization on earth. The GTM Science mission is to use Science to assist and measurably improve every talent decision made at Amazon. We do this by discovering signals in workforce data, deploying statistical models into Amazon’s talent products, and guiding the broader GTM team to pursue high-impact opportunities with tangible returns. This multi-disciplinary approach spans capabilities, including: data engineering, reporting and analytics, research and behavioral sciences, and applied sciences such as economics and machine learning.We are seeking a Research Scientist with expertise in mixed-methods research in social science, public health, or similar research, including development and evaluation of theoretical frameworks, qualitative data collection and analysis methods in a variety of settings (e.g. focus groups, field studies, surveys, observational studies, “found data”, quantitative analytics), and statistics. The ideal candidate will be equally comfortable with qualitative and quantitative methods, though candidates with greater exposure to and familiarity with qualitative methods will be considered if a solid understanding of the quantitative methods described above exists. This person will possess strong experience with managing project deliverables for diverse stakeholders and thrive in a fast paced work environment. In this role you will:· Design, develop, and execute quantitative and qualitative data collection methods in Future of Work (FoW), Diversity Equity & Inclusion (DEI), and related talent management efforts· Conduct quantitative analyses of talent management data and trends· Conduct qualitative data collection and analysis· Partner closely and drive effective collaborations across multi-disciplinary research and product teams· Consult on appropriate analytic methodologies and scope research requests
US, VA, Arlington
AWS Outcome Driven Engineering (ODE) is a new AWS engineering organization chartered to build new AWS products by applying Amazon’s innovation mechanisms along with AWS digital technologies to real world industry problems. We dive deep with industry leaders to solve problems and unblock industries, enabling them to capitalize on new digital business models. Simply put, our goal is to use the skill and scale of AWS to make the benefits of a connected world achievable for all businesses. Our team is focused on saving hundreds of millions of dollars using cutting edge science, machine learning, and scalable distributed software on the Cloud that automates and optimizes inventory and shipments to customers under the uncertainty of demand, pricing and supply.We are looking for an experienced, passionate, hardworking and analytical researcher to work with our partners and build new AWS products. As a Data Scientist on the Outcome Development Engineering team, you will collaborate directly with economists and statisticians to produce modeling solutions, you will partner with software developers and data engineers to build end-to-end data pipelines and production code, and you will have exposure to senior leadership as we communicate results and provide scientific guidance to the business. You will analyze large amounts of business data, automate and scale the analysis, and develop metrics that will enable us to continually delight our customers worldwide. As a successful data scientist, you are an analytical problem solver who enjoys diving into data, is excited about investigations and algorithms, can multi-task, and can credibly interface between technical teams and business stakeholders. Your analytical abilities, business understanding, and technical savvy will be used to identify specific and actionable opportunities to solve existing business problems and look around corners for future opportunities. Your expertise in synthesizing and communicating insights and recommendations to audiences of varying levels of technical sophistication will enable you to answer specific business questions and innovate for the future.A day in the lifeAs a Sr. Data Scientist, you will solve real world problems by analyzing large amounts of business data, defining new metrics and business cases, designing simulations and experiments, creating ML models, and collaborating with teammates in business, software, and research. The successful candidate will have a strong quantitative background and can thrive in an environment that leverages statistics, machine learning, operations research, econometrics, and business analysis.Job responsibilities· Working with product managers, software engineers, data engineers, other data scientists, applied scientists to design, develop, and evaluate highly innovative statistics and ML models to drive efficiency through demand sensing, inventory optimization and the design of new policies and incentives.· Guide and establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation· Proactively seek to identify business opportunities and insights and provide solutions to automate and optimize key business processes and policies based on a broad and deep knowledge of data, industry best-practices, and work done by other teams.· Collaborating with our dedicated software team to create production implementations for large-scale data analysis and/or ML models.· Developing and owning key business metrics / KPIs and providing clear, compelling analysis that shapes the direction of our businessInclusive Team CultureOur team has a developed a reputation for attracting, developing, and retaining amazing talent from diverse backgrounds. Yes, we do get to build really cool services and work closely with customers, but we also think a big reason for our diversity is the inclusive and welcoming culture we try to cultivate every day. We’re looking for a new teammate who is enthusiastic, empathetic, curious, motivated, reliable, and able to work effectively with a diverse team of peers; someone who will help us amplify the positive & inclusive team culture we’ve been building.In addition to Seattle - Palo Alto, Dallas, Atlanta, the Boston Metro area, and other East Coast locations in North America will also be given consideration.
US, VA, Virtual Location - Virginia
AWS Professional Services is a unique consulting team. We pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers. We are looking for a passionate and talented Data Scientist who will collaborate with other scientists and engineers to develop computer vision and machine learning methods and algorithms to address real-world customer use-cases. You'll design and run experiments, research new algorithms, and work closely with talented engineers to put your algorithms and models into practice to help solve our customers' most challenging problems.This position requires that the candidate selected be a U.S. citizen and obtain and maintain an active TS/SCI security clearance.The primary responsibilities of this role are to:· Research, design, implement and evaluate novel computer vision algorithms.· Work on large-scale datasets, creating scalable, robust and accurate computer vision systems in versatile application fields.· Work closely with account team, research scientist teams and product engineering teams to drive model implementations and new algorithms· Interact with customer directly to understand the business problems and aid them in implementation of their ML solutionsHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and we host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.We're dedicated to supporting new team members. Our team has a broad mix of experience levels and Amazon tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our team also puts a high value on work-life balance. Striking a healthy balance between your personal and professional life is crucial to your happiness and success here, which is why we aren’t focused on how many hours you spend at work or online. Instead, we’re happy to offer a flexible schedule so you can have a more productive and well-balanced life—both in and outside of work.**If you have questions, wish to apply or refer someone contact Josh May directly - joshmy@amazon.com**