Prem Natarajan, Alexa AI vice president of natural understanding
Prem Natarajan, Alexa AI vice president of natural understanding
Credit: Micron Technology, Inc.

3 questions: Prem Natarajan on issues of AI fairness and bias

Alexa AI vice president of natural understanding Prem Natarajan discusses the upcoming cycle for the National Science Foundation collaboration on fairness in AI, his participation on the Partnership on AI board, and issues related to bias in natural language processing.

A year ago, Amazon and the National Science Foundation (NSF) announced a $20 million collaboration to fund academic research on fairness in AI over a three-year period. Recently, Erwin Gianchandani, deputy assistant director for Computer and Information Science and Engineering at NSF, discussed the work of the first ten recipients of the program’s grants. Here, Prem Natarajan, Alexa AI vice president of natural understanding, and the Amazon executive who helped launch the collaboration with NSF, discusses the next cycle of upcoming proposals from academic researchers, his work with the Partnership on AI, and what can be done to address bias in natural language processing models.

The 2020 award cycle for the Fairness in AI program in conjunction with the NSF recently launched. Full proposals are due by July 13th. What are you hoping to see in the next round of proposals?

We collaborated with the NSF to launch the Fairness in AI program with the goal of promoting academic research in this important aspect of AI. Our primary objective for engaging with academia on issues related to fairness and transparency in AI is to get many different and diverse perspectives focused on the challenge. The teams selected by NSF in the first round are addressing a variety of topics – from principled frameworks for developing and certifying fair AI, to domain-focused applications such as fair recommender systems for foster care services. To that end, I hope that the second round will build upon the success of the first round by bringing an even greater diversity of perspectives on definitions and perceptions of fairness. Without such diversity the entire field of research into fair AI will become a self-defeating exercise.

Another hope I have for the second round, and indeed for all rounds of this program, is that it will drive the creation of a portfolio of open-source artifacts – such as data sets, metrics, tools, and testing methodologies – which all stakeholders in AI can use to promote the use of fair AI. Such readily available artifacts will make it easier for the community to learn from one another, promote the replication of research results, and, ultimately, advance the state of the art more rapidly. Put differently, we hope that open access to the research under this program will form a rising tide that lifts all boats. It also seems natural that methodologies for fairness will benefit from broad and inclusive discussion across relevant academic and scientific communities.

The deadline for this next round of proposal submissions is July 13th. We hope that the response to this round will be even stronger than for the first. NSF selects the recipients, and I am sure NSF’s reviewers are looking forward to a summer of interesting reading!

You are Amazon’s representative on the Partnership on AI (PAI) board of directors. This unique organization has thematic pillars related to safety-critical AI; fair, transparent and accountable AI; AI labor and the economy; collaborations between AI systems and people; social and societal influences of AI; and AI and social good. It’s an ambitious, broad agenda. You’re fairly new in your role with PAI; what most excites you about the work being done there?

The most exciting aspect of the Partnership on AI is that it is a unique multi-sector forum where I get to listen to and learn from the incredible diversity of perspectives – from industry, academia, non-profits, and social justice groups. PAI today counts amongst its members about 59 non-profits, 24 academic institutions, and 18 industrial organizations. While I joined the board just a few months ago, I have already attended several meetings and participated in discussions with other PAI members as well as PAI staff. While every member has their own unique perspective on AI, it’s been really interesting and encouraging to see that we all share the same values and many of the same concerns. It should be of no surprise that the issue of equity is top of mind with a concomitant focus on fairness considerations.

Alexa & Friends Twitch show features Prem Natarajan

Earlier this month, Alexa evangelist Jeff Blankenburg interviewed Prem Natarajan live on the 'Alexa & Friends' Twitch show. In the video, they discuss recent advances in natural understanding , and how those advancements translate into better experiences for customers, developers and third-party device manufacturers.

From a technical perspective, I am excited by the number and quality of research initiatives underway at PAI. Many of these initiatives are of critical importance to the future development of the field of AI. Let me give you a couple of examples.

One is the area of fairness, accountability and transparency. There are several projects underway in this area, but I will mention one that to me exemplifies the kind of work that an organization like PAI can do. PAI researchers interviewed practitioners at twenty different organizations and performed an in-depth case study of how explainable AI is used today. This kind of research is very important to AI practitioners because it gives them a referential basis to assess their own work and to identify useful areas for future contributions.

Another example is ABOUT ML, which is focused on developing and sharing best practices as well as on advancing public understanding of AI. A couple of years ago some researchers had proposed the development of an AI model scorecard, along the lines of the nutritional information you get on the back of most food items we buy today. The scorecard would describe the attributes of the data used to train the models, the way in which it was tested, etc. The motivation behind the scorecard is to give other developers or model builders a sense of the strengths and limitations of the model, so they can better estimate and address potential weaknesses in the model for their target use cases. ABOUT ML goes well beyond such a scorecard, focusing on documentation, provenance of data and code artifacts, and other critical attributes of the model development process. Ultimately, only multisector organizations like PAI can successfully drive this kind of initiative, bringing together people across organizations and sectors.

Lastly, there’s an education role that PAI serves that I believe is unique, serving as the bridge between AI technologists and other stakeholders within society, making sure AI technologists are appropriately factoring in the perspectives and concerns of the other stakeholders within society. Some examples here include PAI’s collaborative work with First Draft, a PAI Partner, to help technologists and journalists at digital platforms address growing issues around manipulated media. PAI also helps those stakeholders understand more about how AI technology works, its strengths and its limitations.

You oversee Alexa’s natural understanding team. Natural language processing models have drawn criticism for capturing common social biases with respect to gender and race. A large body of work is emerging related to bias in word embedding and classifiers, and there are many proposals for countermeasures. Can you describe the challenge of bias in NLP models, and give us insight into some of the countermeasures you think are, or could be, effective?

A word embedding is a vector of real numbers representing that word; the core idea is that words with similar meanings map to vectors that are “close” to each other. Word embeddings have become a central feature of modern NLP. While embeddings can be computed using a variety of different techniques, deep learning techniques have proven to be tremendously effective at numerically representing the semantics of a word and concepts, etc. Today, deep learning based embeddings are used for all kinds of processing, from named entity recognition, to question answering, and natural language generation. As a result, the semantics that these embeddings encode greatly influence how we interpret text, the accuracy of those interpretations, and the actions we take in response to those interpretations.

Bias can also manifest in other ways because any system that is based on data can exhibit a majoritarian bias to it.
Prem Natarajan, Alexa AI VP of natural understanding

As word embeddings became prevalent, researchers naturally started looking into their fragilities and shortcomings. One of those fragilities is that the embeddings derive and encode meaning from context, which means that the meaning of a word is largely controlled by the different contexts in which that word is observed in the training data. While that seems like a reasonable basis for inferring meaning, it leads to undesirable consequences. My friend Kai-Wei Chang at UCLA is one of the early investigators of bias in NLP and he uses the following example: take the vector for doctor and you subtract the vector for man; when you add the vector for woman, you should in principle get the vector for doctor again, or a female doctor. But instead the resulting vector is close to the vector for ‘nurse.’ What this example shows is that the latent biases in human-generated text get encoded into the embeddings. One example of a system that is affected by these biases is natural language generation. Many studies have shown that such biases can result in the generation of text that exhibits the same biases and prejudices as humans, sometimes in an amplified manner. Left unmitigated, such systems could reinforce human biases and stereotypes.

Bias can also manifest in other ways because any system that is based on data can exhibit a majoritarian bias to it. So, for example, different groups in different parts of the world may speak the same language with different dialects, but the most frequent dialect will likely see the best performance only because it forms the major proportion of the training data. But we don’t want dialect or accent to determine how well the system will work for an individual. We want our systems to work equally well for everyone, regardless of geography, dialect, gender, or any other irrelevant factor.

Methodologically, we counter the impact of bias by using a principled approach to characterize the dimensions of bias and associated impact, and by developing techniques that are robust to these biasing factors. For example, it stands to reason that speech recognition systems should ignore parts of the signal that are not useful for recognizing the words that were spoken. It shouldn’t really matter whether the voice is male or female, only the actual words should. Similarly for natural language understanding, we want to be able to understand the queries of different groups of people regardless of the stylistic or syntactic variations of the language used. Scientists at Amazon and elsewhere are exploring a broad variety of approaches such as de-biasing techniques, adversarial invariance, active learning, and selective sampling. Personally, I find the adversarial approaches to both testing and to generating bias or nuisance invariant representations most appealing because of their scalability, but in the next few years, we will all find out what works best for different problems!

Amazon Science Newsletter Project Kuiper.jpg
Get more from Amazon Science
Sign up for our monthly newsletter

Work with us

See More Jobs
US, WA, Seattle
Why this job is awesome?· This is SUPER high-visibility work: Our mission is to provide consistent, accurate, and relevant delivery, customer service, and product safety information to every single page on every Amazon-owned site.· MILLIONS of customers will be impacted by your contributions: The changes we make directly impact the customer experience on every Amazon site. This is a great position for someone who likes to leverage Machine learning technologies to solve the real customer problems, and also wants to see and measure their direct impact on customers.· We are a cross-functional team that owns the ENTIRE delivery experience for customers: From the business requirements to the technical systems that allow us to directly affect the on-site experience from a central service, business and technical team members are integrated so everyone is involved through the entire development process.- Do you want to join an innovative team of scientists and engineers who use machine learning and statistical techniques to deliver the best delivery experience, product safety, and customer service on every Amazon-owned site?- Are you excited by the prospect of analyzing and modeling terabytes of data on the cloud and create state-of-art algorithms to solve real world problems?- Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company?- Do you like to innovate and simplify?If yes, then you may be a great fit to join one of the Operations Technology Machine Learning teams.Major responsibilities:· Research and implement machine learning and statistical techniques to create scalable and effective models in Delivery, Customer Service, and Product Security systems.· Deep data analysis to solve business problems and to identify business opportunities to provide the best delivery experience on all Amazon-owned sites.· Design, development and evaluation of highly innovative machine learning models for big data.· Analyzing and understanding large amounts of Amazon’s historical business data to detect patterns, to analyze trends and to identify correlations and causalities· Working closely with other software engineering teams to drive real-time model implementations and new feature creations· Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation
US, WA, Seattle
Compensation Science is building economic models and algorithms from the ground up to design and scale pay for hundreds of thousands of Amazon employees worldwide. The fast-growing, interdisciplinary team is working at the intersection of economics, machine learning, and product development.We are looking for an outstanding, end-to-end economist who is able to provide structure to ambiguous, complex business problems, conduct causal econometric analysis, work with engineers to launch science models in product, and measure the impact of policy changes.This role will build and operationalize econometric models for new compensation products owned by the team and initiate high impact projects in the compensation and benefits space. Economists at Amazon have access to engineering and econometric tools and opportunities to learn new econometric and machine learning methods from world-class scholars.Responsibilities:· Own the development of economic models and econometric analysis· Assist in the delivery of automated, scalable analytic models· Work collaboratively with economists, scientists, and engineers on the team and across the HR organization· Interpret and communicate results to global business stakeholders
DE, BY, Munich
As a Research Scientist you will use your experience to develop new strategies to improve the performance of Amazon’s systems and networks. Working closely with fellow research scientists and product managers, you will use your experience in modeling, statistics, and simulation to design models of new policies, simulate their performance, and evaluate their benefits and impacts to cost, reliability, and speed of our fulfillment network.Our teams are looking for experience in network and combinatorial optimization, algorithms, data structures, statistics, and/or machine learning. This position requires superior analytical thinking, and ability to apply their technical and statistical knowledge to identify opportunities for industrial research.You should be able to mine and analyze large data, and be able to use necessary programming and statistical analysis software/tools to do so.
GB, London
As a Research Scientist you will use your experience to develop new strategies to improve the performance of Amazon’s systems and networks. Working closely with fellow research scientists and product managers, you will use your experience in modeling, statistics, and simulation to design models of new policies, simulate their performance, and evaluate their benefits and impacts to cost, reliability, and speed of our fulfillment network.Our teams are looking for experience in network and combinatorial optimization, algorithms, data structures, statistics, and/or machine learning. This position requires superior analytical thinking, and ability to apply their technical and statistical knowledge to identify opportunities for industrial research.You should be able to mine and analyze large data, and be able to use necessary programming and statistical analysis software/tools to do so.
RO, Iasi
We are looking for motivated data scientists with excellent leadership skills, and the ability to develop, automate, and run analytical models of our systems. You will have strong modeling skills and are comfortable owning data and working from concept through to execution. This role will also build tools and support structures needed to analyze data, dive deep into data to resolve root cause of systems errors and changes, and present findings to business partners to drive improvements.Applicants have a demonstrated ability to manage medium-scale modeling projects, identify requirements, and build methodology and tools that are statistically grounded. You will have experience collaborating across organizational boundaries.
GB, London
We are looking for motivated data scientists with excellent leadership skills, and the ability to develop, automate, and run analytical models of our systems. You will have strong modeling skills and are comfortable owning data and working from concept through to execution. This role will also build tools and support structures needed to analyze data, dive deep into data to resolve root cause of systems errors and changes, and present findings to business partners to drive improvements.Applicants have a demonstrated ability to manage medium-scale modeling projects, identify requirements, and build methodology and tools that are statistically grounded. You will have experience collaborating across organizational boundaries.
US, NY, New York
Amazon Web Services (AWS) is building a world-class marketing organization, and we are looking for an Economist to apply the frontier of economic thinking to experimental design, measurement, forecasting, program evaluation, online advertising and other areas. As a key member on the Marketing Data: Science & Engineering (D:SE) team you will partner with a team of experts across the fields of Product Management, Data Science, Machine Learning and Data Engineering to develop new and innovative solutions to some of the hardest challenges in Marketing. In this role, you will work on the key initiatives and partner with business stakeholders, to build solutions and recommendations to drive key business outcomes for the AWS business.About our team:We refuse to accept constraints, internal or external, and have a strong bias for action. We love data and believe that we can use it to deliver epic experiences for our customers. We work across all areas of AWS marketing including our core marketing data solution, insights and reporting, targeting and personalization models, marketing measurement and the operational systems to support each of these areas that power interactions with millions of customers every day. As a multi-functional team of experts, we deliver scaled solutions in use globally across AWS marketing.About you:Economists at Amazon will be expected to work directly with the chief economist and senior management on key business problems faced in retail, international retail, cloud computing, third party merchants, search, Kindle, streaming video, and operations. You will build econometric models, using our world class data systems, and apply economic theory to solve business problems in a fast moving environment. Economists at Amazon will be expected to develop new techniques to process large data sets, address quantitative problems, and contribute to design of automated systems around the company. You have strong leadership qualities, great judgment, clear communication skills, and a track record of shipping great products.
US, VA, Arlington
Amazon’s Global Accounts Receivable team is looking for a Data Scientist to join our fast paced, stimulating environment to help invent the future of Accounts Receivable with technology, and to turn big data into actionable insights.Our team charter is to optimize credit risks, cash flow, customer satisfaction and internal efficiency. We provide insights and recommendations to senior business leaders in terms of policies, process and systems. We build large-scale models that help our global teams manage their receivables portfolios, run their operations to maximum effect and foresee future trends. We contribute algorithms to O2C systems towards effective credit management.We are seeking to hire a Data Scientist with strong scientific acumen, technical skills and communication to join our team.The role will help build global-scale components of our economics and statistical toolkit, initially focusing on trend and regression analysis, machine learning, and more. They will discover and define problems; and find the right quantitative solutions. They will measurably impact the success of the major receivables processes in Amazon's core businesses, including credit and risk management, as well as dunning and collections strategies.The role will actively interact with business in translating requirements into Data Science problem statements; following through modelling and deployment; and driving continuous improvement and learning. The role will work hand in hand with software engineers, business intelligence engineers and business teams towards implementation at scale.Responsibilities· Apply judgement to identify and develop science solutions· Design and develop models to predict process behaviour and outcomes· Apply advanced statistical and/or machine learning know-how e.g. to optimise predictive abilities· Develop new data sources to enable statistical modelling and learning; continuously fine-tune data models· Design and utilise code (Python, R, Scala, etc.) as required· Formulate experiments to assess AR process strategies· Collaborate with engineering to build data, algorithms and models· Communicate scientific solutions and insights effectively to a senior leadership and non-scientific audience
US, CA, Sunnyvale
Interested in making Amazon Echo more intuitive? Help us make Alexa personalized to each of our customers. We’re building the speech and language solutions behind Amazon Echo and other Amazon products and services. Come join us!Alexa is the groundbreaking cloud-based voice service that powers Amazon Echo and other devices designed around your voice. Our mission is to push the envelope in Natural Language Understanding (NLU), Machine Learning (ML), Automatic Speech Recognition (ASR), and Speaker Recognition, in order to provide the best-possible experience for our customers. We’re looking for an Applied Scientist to help build industry-leading speaker recognition technologies and machine learning systems that customers love.The Speaker ID (Voice Recognition) team enables Alexa to provide personalized experiences to millions of Alexa customers. Our mission is to make Alexa your best friend, recognizing you by your voice with confidence. At the core, we use both statistical, deep learning, and neural network models to make the magic happen. We are the brains behind “Alexa, who am I?”, “Echo, call my mom.” and more. We provide millions of Alexa customers personalized experiences 24 hours a day, 7 days a week.As an Applied Scientist for the Alexa Engine team focused on Speaker Recognition, you will be responsible for building industry-leading intelligent offerings that customers love. Our mission is to apply Artificial Intelligence (AI) and Machine Learning (ML), in order to reduce users cognitive load, reduce friction in their day-to-day activities and finally, inspire our customers by enabling serendipitous discovery of experience.We are looking for top Applied Scientists who can build new product and/or help us take our products to the next level who has deep passion for building machine-learning solutions; ability to communicate data insights and scientific vision, and has a proven track record of execute complex projects. As an Applied Scientist in Machine Learning, you will:· Use machine learning and data analysis to deliver scalable solutions to business problems· Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production· Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving· Research new machine learning approaches to all aspects of the voice recognition, personalization and ASR.You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in artificial intelligence. Your work will directly impact our customers in the form of novel products and services that make use of speech and language technology.
US, MA, North Reading
Working at Amazon RoboticsAre you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart team of doers that work passionately to apply cutting edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even image yet. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling and fun.Position OverviewThe Amazon Robotics (AR) Virtual Systems Profiling team builds models, runs simulation experiments and delivers analyses that are central to understanding performance of the entire AR system, e.g. operational and software scaling characteristics, bottlenecks, robustness to “chaos monkey” stresses -- we inform critical engineering and business decisions about Amazon’s approach to robotic fulfillment.We seek a talented and motivated engineer to tackle broad challenges in system-level analysis. You will work in a small team to quantify system performance at scale and to expand the breadth and depth of our analysis (e.g. increase the range of software components and warehouse processes covered by our models, develop our library of key performance indicators, construct experiments that efficiently root cause emergent behaviors). You will engage with growing teams of software development and warehouse design engineers to drive evolution of the AR system and of the simulation engine that supports our work.
US, WA, Seattle
Amazon is looking for an outstanding applied scientist to help build next generation selection/assortment systems. On the Core Selection Team within the Supply Chain Optimization Technologies (SCOT) organization, we own the selection for WW Amazon Marketplaces. We build tools and systems that enable our partners and business owners to scale themselves by leveraging our problem domain expertise, focusing instead on introspecting our outputs and iteratively helping us improve our ML models rather than hand-managing their assortment.As an Applied Scientist, you will work with software engineers, product managers and business teams to understand the requirements/current challenges, distill that understanding to elegantly define the problem, and develop innovative solutions to address those problems using techniques in machine learning and optimization.You will work with a team of engineers and scientists who are passionate about using machine learning to build automated systems and solve problems that matter to our customers. Your work will directly impact our customers in form of selection we offer them.Responsibilities:· Research and implement machine learning and optimization models to solve problems that matter to our customers· Understand business requirements and existing challenges and map them to the right scientific solution· Own end-to-end solution in terms of research, prototyping, experimentation to eventual roll-out· Develop the right set of metrics to evaluate efficiency/accuracy of the algorithms· Mentor and develop the scientist community across the organizationTo help describe some of our challenges, we created a short video about Supply Chain Optimization at Amazon - is an Equal Opportunity Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.
US, NJ, Newark
Are you excited about an opportunity to apply your experience and passion for Machine Learning (ML) and/or Deep Learning (DL) to improving customers product experience? Are you a data scientist who dreams of building scalable solutions and innovations that enhances and enriches millions of customers lives every day? If so, this may be a great fit for you!Audible Product Data Science team partners with technology and product leaders to solve business and technology problems using scientific approaches to build product and services that surprise and delight our customers. Improving Search and Content discovery experience is our key focus. We employ cutting-edge ML, deep learning techniques and Natural Language Processing (NLP) knowledge to improve the relevance of search results, query intents understanding, and recommendation system, etc. We operate in an agile environment in which we own and collaborate the life cycle of research, design, and model development of relevant projects.As a data scientist, you will be responsible for driving projects through their entire lifecycle from idea creation through implementation, experimentation and finally, deployment. You will be working with other data scientists, ML experts, engineers as well as product teams locally and abroad, and on cross-disciplinary efforts with other scientist within Amazon.We are looking for motivated, results-oriented Data Scientist with strong rigor and demonstrable skills in information retrieval, DL, ML, NLP, data mining and/or large-scale distributed computation.
US, WA, Seattle
We’re working on the future. If you are seeking an iterative fast-paced environment where you can drive innovation, apply state-of-the-art technologies to solve extreme-scale real world challenges, and provide visible benefit to end-users, this is your opportunity.Come work on the Prime Air team!We're looking for an outstanding applied scientist who combines superb technical, research and analytical capabilities with a demonstrated ability to get the right things done quickly and effectively. This person must be comfortable working with a team of top-notch applied scientists. We’re looking for people who innovate and love solving hard problems. You will work hard, have fun, and of course, make history!Export License: This position may require a deemed export license for compliance with applicable laws and regulations. Placement is contingent on ’s ability to apply for and obtain an export license on your behalf
US, WA, Seattle
Do you want to join an innovative team of scientists who use machine learning, NLP and statistical techniques to provide the best customer experience on the earth? Do you want to change the way that people work with customer service? Our team wants to lead the technical innovations in these spaces and set the bar for every other company that exists. We love data, and we have lots of it. We're looking for data scientist to own end-to-end business problems and metrics which would have a direct impact on the bottom line of our business while improving customer experience.If you see how big data and cutting-edge technology can be used to improve customer experience, if you love to innovate, if you love to discover knowledge from big structured and unstructured data and if you deliver results, then we want you to be in our team.Major responsibilities· Analyze and extract relevant information from large amounts of both structured and unstructured data to help automate and optimize key processes· Design structured, multi-source data solutions to deliver the dashboards and reports that make data actionable· Drive the collection of new data and the refinement of existing data sources to continually improve data quality· Support data analysts and product managers by turning business requirements into functional specifications and then executing delivery· Lead the technical lifecycle of data presentation from data sourcing to transforming into user-facing metrics· Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation
US, WA, Seattle
Amazon Go is a new kind of store with no lines and no checkout—you just grab and go! Customers simply use the Amazon Go app to enter the store, take what they want from our selection of fresh, delicious meals and grocery essentials, and go! Our checkout-free shopping experience is made possible by our Just-Walk-Out-Technology, which automatically detects when products are taken from the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. Shortly after, we’ll charge your Amazon account and send you a receipt. Check it out at are looking for a seasoned leader to charter continued innovation on behalf of the research science and analytics team at Amazon Go. This role is highly strategic, and this leader will interact with all levels across a broad range of teams across the company.You will lead a team of data scientists, research scientists, applied scientists, economists and engineers in the development of state-of-the-art models to influence a wide range of decisions within Amazon Go spanning across stochastic optimization (i.e., its application in inventory planning, supply chain management, vendor selection) and beyond! You will oversee a team of Business Intelligence Engineers (BIEs) who provide data, self-service tools and analytics to support Amazon Go.The ideal candidate:· Has a track record for developing and applying robust research to business problems· Is entrepreneurial and innovative, and thrives on solving challenging, ambiguous problems· Has built a strong followership, (e.g., a reputation for hiring and developing strong teams)· Is adept at delivering short-term results while also building towards the future· Can work fluently across the research, technical and business aspects of the job· Excels at leadership and stakeholder managementAs the leader of this team, you will:· Work closely with product, business and engineering teams to set the vision and roadmap for your team.· Lead scientists and engineers in the development novel machine learning and optimization models, in addition to their deployment in production systems.· Represent your business and operations to executive leadership across functions.· Identify new mathematical modeling opportunities, and make business cases for resources to pursue the best of them.· Hire, develop, and retain top science and analytic talent.
DE, BY, Munich
Are you a MS or PhD student interested in a 2021 Applied Science Internship in the fields of Speech, Robotics, Computer Vision, or Machine Learning/Deep Learning?Do you enjoy diving deep into hard technical problems and coming up with solutions that enable successful products that improve the lives of people in a meaningful way?If this describes you, come join our research teams at Amazon. As an Applied Science Intern, you will have access to large datasets with billions of images and video to build large-scale machine learning systems. Additionally, you will analyze and model terabytes of text, images, and other types of data to solve real-world problems and translate business and functional requirements into quick prototypes or proofs of concept.We are looking for smart scientists capable of using a variety of domain expertise combined with machine learning and statistical techniques to invent, design, evangelize, and implement state-of-the-art solutions for never-before-solved problems.Machine Learning Science:Amazon has multiple positions available for Machine Learning Scientists in Berlin, Cambridge, Edinburgh, Iasi and Tuebingen.A few of the teams that are hiring include:- Core AI- Amazon Search- AWS AI- Advertising Technologies- Community Shopping- Prime VideoSpeech and Language Technology:We are hiring in all areas of spoken language understanding: ASR, NLP, NLU, text-to-speech (TTS), and Dialog Management. Amazon has multiple positions available for Speech Scientists in Aachen, Barcelona, Berlin, Cambridge, Edinburgh, Gdansk, Haifa, Tel Aviv and Turin.A few of the teams that are hiring currently include:- Alexa ML- Alexa Brain- Alexa Shopping- Amazon Search- CS TechnologyComputer Vision:Amazon has multiple positions available for Computer Vision Scientists in locations such as Berlin, Barcelona, Tuebingen, Haifa and Tel Aviv.We are currently hiring for multiple teams including:· Visual Search· Amazon AI (AWS Rekognition)· Amazon Go· Lab126
DE, BW, Tuebingen
Are you interested in working on fascinating scientific and engineering challenges of modern information technology? Would you like to contribute to the development of the future generation of cloud computing at Amazon Web Services?As a Sr. Applied Scientist, you will be working on cutting edge projects in the intersection of causal inference, machine learning, and high-dimensional statistics. You will be part of an ambitious team of scientists and software engineers that is together developing novel software products for world-wide use.The AWS Causality Lab is located at the Tübingen site in Germany. Our goal is to enable our customers to improve confidence in their data science conclusions by making the underlying cause-effect relationships explicit. Going beyond mere correlational analysis, we quantify the causes of observations, and provide actionable insights based on data-driven what-if predictions.Our mission is to provide automated software for causal inference to our customers which builds on formalisms, algorithms, and statistical guarantees.As a Sr. Applied Scientist in the Causality Lab, you will be responsible for:· research and development of algorithms in causal inference· analyzing different data types, including time series, textual data sources and graphs· infering causal relationships between these inputs, and discriminating these from coincidental correlations· identifying the causes of particularities in data and quantify their specific contributions to downstream metrics· infering interventional and counterfactual analysis· collaborating product and development teams across AWS and Amazon as well as directly with customers· engaging in the interview process and otherwise developing, growing, and mentoring junior scientistsWe at AWS value individual expression, respect different opinions, and work together to create a culture where each of us is able to contribute fully. Our unique backgrounds and perspectives strengthen our ability to achieve Amazon's mission of being Earth's most customer-centric company.
US, NY, New York
Amazon Advertising operates at the intersection of eCommerce and advertising, offering a rich array of digital display advertising solutions with the goal of helping our customers find and discover anything they want to buy. We help advertisers reach Amazon customers on, across our other owned and operated sites, on other high quality sites across the web, and on millions of Kindles, tablets, and mobile devices. We start with the customer and work backwards in everything we do, including advertising. If you’re interested in joining a rapidly growing team working to build a unique, world-class advertising group with a relentless focus on the customer, you’ve come to the right place.In this role you will develop and evaluate machine learning models using large data-sets, cloud services and customer behavior insights to improve our customer’s experience. Working closely with best-in-class engineers you will have the opportunity to apply a variety of machine learning algorithms, including deep learning, and work on one of the world's largest data sets to influence the long term evolution of our technology roadmap.Impact and Career Growth:· Innovate and define the new pricing model for Sponsored Brand across many creative types.· · Addressing principles of allocation function and pricing in ad marketplace auctions;· · Developing efficient algorithms for multi-objective optimization and AI control methods to find operating points for the ad marketplace auctions and to evolve them· · Opportunity to grow and broaden your machine learning skills a make impact – the work you deliver directly impacts customers and revenue!· · Work in an environment that thrives on creativity, experimentation, and product innovation.· · Have the ability to experiment autonomously with meaningful projects.· · Mentor others Applied Scientists.Why you love this opportunity:Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.
US, NJ, Newark
Audible is searching for an exceptional data scientist to join our economics team and drive the development of models at the intersection of machine learning and econometrics at scale. The Audible economics organization works across the business to measure and maximize the value Audible delivers to customers, creators, and communities globally. In this role, there will be a focus on partnering with our content and strategy teams to build a groundbreaking catalog of audiobooks and spoken-word entertainment.We are looking for someone experienced in building ML models at scale for complex prediction and optimization problems, who also has a background (or burgeoning interest!) in causal inference or interpretable machine learning. In addition to working with our staff economists, you will also collaborate closely with scientists across Audible and partner teams at Amazon on problems pertinent to subscription businesses and the production of original media content.KEY RESPONSIBILITIES· Work with leadership in our content and strategy organizations to identify key analytical problems and opportunities – your work is expected to be a key input to our future content strategy.· Develop and maintain scalable, innovative data science and machine learning models that deliver actionable insights and results.· Collaborate with other data scientists, economists, and analysts at Audible to build data-driven solutions to key business problems.
US, NJ, Newark
Audible is looking for a talented teammate to join our economics team as senior manager of economic insights. The Audible Economics organization works across the business to measure and maximize the value Audible delivers to customers, creators, and communities globally. In this role, there is a focus on partnering with our economists and data scientists to craft analyses that solve crucial questions for external content stakeholders, including publishers, authors, and the broader creative community of spoken-word entertainment and audiobooks. In addition to helping shape analyses, you will be responsible for translating findings into key, actionable insights, and communicating them clearly to a range of audiences.We are searching for someone who has exceptional presentation, writing, and data visualization skills; a thorough knowledge of analytical methods used in economics, statistics, and data science; and an exemplary ability to tell simple, compelling stories with data. As Audible continues to redefine the ways people access, discover, and share stories, you will be at the forefront of discussions with our content team and creators on how to evolve the industry to discover new value for customers and storytellers alike.KEY RESPONSIBILITIES· Partner with content and strategy leadership to develop a roadmap of key analytical questions for creators.· Conceptualize and deliver analytical, data-driven solutions to those key questions, in original analyses and in partnership with staff economists and data scientists.· Produce data-driven recommendations to augment our content strategy and maximize value for customers and creators.· Develop innovative partnership structures and deal making strategies with existing and new creative partners.