Michele Donini, left, a senior applied scientist with Amazon Web Services, and Luca Oneto, associate professor of computer engineering at University of Genoa
Michele Donini, left, a senior applied scientist with Amazon Web Services, and Luca Oneto, associate professor of computer engineering at University of Genoa, review different approaches that can make data-driven predictions fairer for underrepresented groups.
Credit: Glynis Condon

Working toward fairer machine learning

Exploring and analyzing possible techniques to make ML algorithms capable of learning fairer models by utilizing empirical risk minimization theory.

Editor’s note: Michele Donini is a senior applied scientist with Amazon Web Services (AWS). He and his co-author, Luca Oneto, associate professor of computer engineering at University of Genoa, have written about how different approaches can make data-driven predictions fairer for underrepresented groups. Oneto also won a 2019 Machine Learning Research award for his work on algorithmic fairness. In this article, Donini and Oneto explore the research they and other collaborators have published related to designing machine learning (ML) models from a human-centered perspective, and building responsible AI.

What is fairness?

Fairness can be defined in many different ways, and many different formal notions exist, such as demographic parity, equal opportunity, and equal odds.

Graphic shows a model of an unfair outcome on the left and a fair outcome on the right
Algorithmic fairness is a topic of great importance, with impact on many applications. The issue requires much further research; even the definition of what “being fair” means for an ML model is still an open research question.

Nevertheless, the basic and common idea behind notions of fairness is that the learned ML model should behave equivalently, or at least similarly, no matter whether it is applied to one subgroup of the population (e.g., males) or to another one (e.g., females).

For example, demographic parity, which arguably is the most common notion of fairness, implies that the probability of a certain output of an ML model (e.g., deciding to make a loan) should not depend on the value of specific demographic attributes (e.g., gender, race, or age).

Moving toward fairer models

Broadly speaking, we can group current literature on algorithmic fairness into three main approaches:

  • The first approach consists of pre-processing the data to remove historical biases and then feeding this data to classical ML models.
  • The second approach consists of post-processing an already learned ML model. This approach is useful when very complex ML models need to be made fairer without touching their inner structure or when re-training them is unfeasible (due to computational cost, or time requirements).
  • The third approach, called in-processing, consists of enforcing fairness notions by imposing specific statistical constraints during the learning phase of the model. This is the most natural approach, but so far, it has required ad hoc solutions tailored to specific tasks and data sets.
Fair Models.png
Broadly speaking, current literature on algorithmic fairness falls into three main approaches: pre-processing data; post-processing an already learned ML model; and in-processing, which consists of enforcing fairness notions by imposing specific statistical constraints during the learning phase of the model.

We decided to explore and analyze possible techniques to make ML algorithms capable of learning fairer models.

We started from the base concepts of statistical learning theory — a mathematical framework for describing machine learning — and, in particular, from empirical risk minimization theory. The core concept of empirical risk minimization is that a model’s performance on test data may not accurately predict its performance on real-world data, as the real-world data may have a different probability distribution.

Empirical-risk-minimization theory provides a way to estimate the “true risk” of a model from its “empirical risk”, which can be computed from the available data. We extended this concept to the true and empirical fairness risk of ML models.

Below is a summary of three papers we’ve published related to these topics.

Empirical risk minimization under fairness constraints

This paper presents a new in-processing method, meaning that we incorporate a fairness constraint into the learning problem. We derive theoretical guarantees on both the accuracy and fairness of the resulting models, and we show how to apply our method to a large family of machine learning algorithms, including linear models and support vector machines for classification (a widely used supervised-learning method).

We observe that, in practice, we can meet our fairness constraint simply by requiring that a scalar product between two vectors remains small (an orthogonality constraint between the vector of the weights describing our model and the vector describing the discrimination between the different subgroups). We further observe that, for linear models, this requirement translates into a simple pre-processing method. Experiments indicate that our approach is empirically effective and performs favorably against state-of-the-art approaches.

Fair regression with Wasserstein barycenters

In this paper, we consider the case in which the ML model learns a regression function (as opposed to a classification task). We propose a post-processing method for transforming a real-valued regression function — the ML model — into one that satisfies the demographic-parity constraint (i.e., the probability of getting a positive outcome should be virtually the same for different subgroups). In particular, the new regression function is as good an approximation of the original as is possible while still satisfying the constraint, making it an optimal fair predictor.

Fair Representation.png
In “Fair regression with Wasserstein barycenters”, we consider the case in which the ML model learns a regression function and propose a post-processing method for transforming a real-valued regression function — the ML model — into one that satisfies the demographic-parity constraint.

We assume that the sensitive attribute — the demographic attribute that should not bias outcome — is available to the ML model at inference time and not only during training. We establish a connection between learning a fair model for regression and optimal transport theory, which describes how to measure distances among probability distributions. On that basis, we derive a closed-form expression for the optimal fair predictor.

Specifically, under the unfair regression function, different populations have different probability distributions; the function skews the probabilities for the population with the sensitive attribute. The difference between subgroups’ distributions can be calculated using the Wasserstein distance. We show that the mean of the distribution of the optimal fair predictor is the mean of the different subgroups’ distributions, as calculated using Wasserstein distance. This mean is known as the Wasserstein barycenter.

This result offers an intuitive interpretation of optimal fair prediction and suggests a simple post-processing algorithm to achieve fairness. We establish fairness-risk guarantees for this procedure. Numerical experiments indicate that our method is very effective in learning fair models, with a relative increase in error rate that is smaller than the relative gain in fairness.

"Exploiting MMD and Sinkhorn divergences for fair and transferable representation learning

Where the first paper described a general learning method, and the second a regression method, this paper concerns deep learning. We show how to improve demographic parity in the multitask-learning setting, in which a deep-learning model learns a single representation of the input data that is useful for multiple tasks. We derive theoretical guarantees on the learned model, establishing that the representation will still reduce bias even when transferred to novel tasks.

We propose a learning algorithm that imposes constraints based on two different ways of measuring distances between probability distributions, maximum mean discrepancy and Sinkhorn divergence. Keeping this distance small ensures that we represent similar inputs in a similar way when they differ only on the sensitive attribute. We present experiments on three real-world datasets, showing that the proposed method outperforms state-of-the-art approaches by a significant margin.

Algorithmic fairness is a topic of great importance, with impact on many applications. In our work, we have attempted to take a small step forward, but the issue requires much further research; even the definition of what “being fair” means for an ML model is still an open research question.

It’s also becoming clearer that we need to keep humans in the loop during the lifecycle of ML models, to evaluate whether the models are acting as we would like them to. In this sense, it is important to note that many other research subjects – such as the explainability, interpretability, and privacy of ML models – are deeply connected to algorithmic fairness. They can work in synergy, with the common goal of increasing the trustworthiness of ML models.

View from space of a connected network around planet Earth representing the Internet of Things.
Sign up for our newsletter

Research areas

Work with us

See more jobs
US, CA, San Francisco
Job summaryWe are seeking a DSP Analytics scientist to further the development and application of analytics methods to examine the complex data flows of the DSP, and to translate deep-dives into actionable insights for our product teams. In this role you will develop new tools to analyze our advertising data to help improve the performance of our bidding algorithms, targeting and relevance systems, help advance our 3rd party and O&O supply strategy, and evaluate the adoption and impact of feature releases.Key job responsibilities• Analyze data trends regarding supply, optimization, ad load, and advertising mix effects that affect advertiser performance and contribute to achieving advertiser goals.• Present papers to senior leaders on issues like feature development impact on identity recognition rates, and changes of ad selection systems to improve fill rate highlighting insights that will inform our business development and engineering roadmaps.• Formalize our analytics approach to the DSP auctions by analyzing bid spreads, auction depth, and simulating impacts of potential auction structure changes.• Identify, standardize, and operationalize KPIs to effectively measure the performance of all systems involved in ad serving, and use trend insights to inform business priorities.• Partner with engineering teams to define data logging requirements and getting these prioritized in engineering roadmaps.• Validate financial models through analysis• Develop and own ad revenue and supply intelligence analytics decks that provide ongoing deep-divesA day in the lifeThe DSP Analytics Leader will work closely with business leaders and engineers on developing common data architecture that will optimize our data logging at different grains, and will allow data interoperability from bid flow to optimization to campaign delivery. The candidate will then analyze the data and present papers and ongoing reports on actionable insights.About the teamThe DSP Ads Product Analytics Team's Mission: Work alongside those who need product data to apply objective perspective and business logic to uncover insights, advise strategic decisions, and adjust to industry changes
US, CA, San Francisco
Job summaryWe are seeking a DSP Analytics scientist to further the development and application of analytics methods to examine the complex data flows of the DSP, and to translate deep-dives into actionable insights for our product teams. In this role you will develop new tools to analyze our advertising data to help improve the performance of our bidding algorithms, targeting and relevance systems, help advance our 3rd party and O&O supply strategy, and evaluate the adoption and impact of feature releases.Key job responsibilities• Analyze data trends regarding supply, optimization, ad load, and advertising mix effects that affect advertiser performance and contribute to achieving advertiser goals.• Present papers to senior leaders on issues like feature development impact on identity recognition rates, and changes of ad selection systems to improve fill rate highlighting insights that will inform our business development and engineering roadmaps.• Formalize our analytics approach to the DSP auctions by analyzing bid spreads, auction depth, and simulating impacts of potential auction structure changes.• Identify, standardize, and operationalize KPIs to effectively measure the performance of all systems involved in ad serving, and use trend insights to inform business priorities.• Partner with engineering teams to define data logging requirements and getting these prioritized in engineering roadmaps.• Validate financial models through analysis• Develop and own ad revenue and supply intelligence analytics decks that provide ongoing deep-divesA day in the lifeThe DSP Analytics Leader will work closely with business leaders and engineers on developing common data architecture that will optimize our data logging at different grains, and will allow data interoperability from bid flow to optimization to campaign delivery. The candidate will then analyze the data and present papers and ongoing reports on actionable insights.About the teamThe DSP Ads Product Analytics Team's Mission: Work alongside those who need product data to apply objective perspective and business logic to uncover insights, advise strategic decisions, and adjust to industry changes
GB, London
Job summaryWe are seeking a Senior DSP Analytics Scientist to further the development and application of analytics methods to examine the complex data flows of the DSP, and to translate deep-dives into actionable insights for our product teams. In this role you will develop new tools to analyze our advertising data to help improve the performance of our bidding algorithms, targeting and relevance systems, help advance our 3rd party and O&O supply strategy, and evaluate the adoption and impact of feature releases.Key job responsibilities• Analyze data trends regarding supply, optimization, ad load, and advertising mix effects that affect advertiser performance and contribute to achieving advertiser goals.• Present papers to senior leaders on issues like feature development impact on identity recognition rates, and changes of ad selection systems to improve fill rate highlighting insights that will inform our business development and engineering roadmaps.• Formalize our analytics approach to the DSP auctions by analyzing bid spreads, auction depth, and simulating impacts of potential auction structure changes.• Identify, standardize, and operationalize KPIs to effectively measure the performance of all systems involved in ad serving, and use trend insights to inform business priorities.• Partner with engineering teams to define data logging requirements and getting these prioritized in engineering roadmaps.• Validate financial models through analysis• Develop and own ad revenue and supply intelligence analytics decks that provide ongoing deep-divesA day in the lifeThe Senior DSP Analytics Scientist will work closely with business leaders and engineers on developing common data architecture that will optimize our data logging at different grains, and will allow data interoperability from bid flow to optimization to campaign delivery. The candidate will then analyze the data and present papers and ongoing reports on actionable insights. In addition we will be looking for the candidate to attract and develop a high-impact data analytics team.About the teamThe DSP Ads Product Analytics Team's Mission: Work alongside those who need product data to apply objective perspective and business logic to uncover insights, advise strategic decisions, and adjust to industry changes
US, VA, Virtual Location - Virginia
Amazon Web Services are looking for a passionate and talented Applied Scientist who will collaborate with other scientists and engineers to develop computer vision and machine learning methods and algorithms to address real-world customer use-cases. You'll design and run experiments, research and develop new algorithms, and put your algorithms and models into practice to help solve our customers' most challenging problems. This role resides in AWS Professional Services, a unique consulting team where we pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers.If you do not live in a market where we have an open Applied Scientist position, please feel free to apply. Our Applied Scientists can live in any location (D.C, Maryland, Virginia, Illinois, Pennsylvania, New York, New Jersey, Denver) where we have a WWPS Professional Service office.The primary responsibilities of this role are to:* Research, design, implement and evaluate new machine learning models, including the application of state-of-art computer vision algorithms to solve object detection and tracking problems.* Work on large-scale datasets, creating scalable, robust and accurate computer vision systems in various application fields.* Communicate with senior management, research scientist teams, and product engineering teams to drive model implementations and new algorithms.* Interact with customers directly to understand their business problems and aid them in implementation of their ML solutions.* Provide technical and scientific guidance to your team members.
US, VA, Virtual Location - Virginia
Amazon Web Services are looking for a passionate and talented Applied Scientist who will collaborate with other scientists and engineers to develop computer vision and machine learning methods and algorithms to address real-world customer use-cases. You'll design and run experiments, research and develop new algorithms, and put your algorithms and models into practice to help solve our customers' most challenging problems. This role resides in AWS Professional Services, a unique consulting team where we pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers.If you do not live in a market where we have an open Applied Scientist position, please feel free to apply. Our Applied Scientists can live in any location (D.C, Maryland, Virginia, Illinois, Pennsylvania, New York, New Jersey, Denver) where we have a WWPS Professional Service office.The primary responsibilities of this role are to:* Research, design, implement and evaluate new machine learning models, including the application of state-of-art computer vision algorithms to solve object detection and tracking problems.* Work on large-scale datasets, creating scalable, robust and accurate computer vision systems in various application fields.* Communicate with senior management, research scientist teams, and product engineering teams to drive model implementations and new algorithms.* Interact with customers directly to understand their business problems and aid them in implementation of their ML solutions.* Provide technical and scientific guidance to your team members.This position requires that the candidate selected be a US Citizen and obtain and maintain an active TS/SCI security clearance.
US, VA, Virtual Location - Virginia
Amazon Web Services are looking for a passionate and talented Applied Scientist who will collaborate with other scientists and engineers to develop computer vision and machine learning methods and algorithms to address real-world customer use-cases. You'll design and run experiments, research and develop new algorithms, and put your algorithms and models into practice to help solve our customers' most challenging problems. This role resides in AWS Professional Services, a unique consulting team where we pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers.If you do not live in a market where we have an open Applied Scientist position, please feel free to apply. Our Applied Scientists can live in any location (D.C, Maryland, Virginia, Illinois, Pennsylvania, New York, New Jersey, Denver) where we have a WWPS Professional Service office.The primary responsibilities of this role are to:· Research, design, implement and evaluate new machine learning models, including the application of state-of-art computer vision algorithms to solve object detection and tracking problems.· Work on large-scale datasets, creating scalable, robust and accurate computer vision systems in various application fields.· Communicate with senior management, research scientist teams, and product engineering teams to drive model implementations and new algorithms.· Interact with customers directly to understand their business problems and aid them in implementation of their ML solutions.· Provide technical and scientific guidance to your team members.This position requires that the candidate selected be a US Citizen and obtain and maintain an active TS/SCI security clearance.This position can have periods of up to 10% travel.accommodation, visit https://www.amazon.jobs/en/disability/us
US, VA, Virtual Location - Virginia
Amazon Web Services are looking for a passionate and talented Data Scientist who will collaborate with other scientists and engineers to develop computer vision and machine learning methods and algorithms to address real-world customer use-cases. You'll design and run experiments, research new algorithms, and work closely with talented engineers to put your algorithms and models into practice to help solve our customers' most challenging problems. This role resides in AWS Professional Services, a unique consulting team where we pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers.If you do not live in a market where we have an open Data Scientist position, please feel free to apply. Our Data Scientists can live in any location (D.C, Maryland, Virginia, Illinois, Pennsylvania, New York, New Jersey, Denver) where we have a WWPS Professional Service office.The primary responsibilities of this role are to:* Research, design, implement and evaluate novel computer vision algorithms.* Work on large-scale datasets, creating scalable, robust and accurate computer vision systems in versatile application fields.* Work closely with account team, research scientist teams and product engineering teams to drive model implementations and new algorithms.* Interact with customer directly to understand the business problems and aid them in implementation of their ML solutions.This position can have periods of up to 10% travel.*Inclusive Team Culture*Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and we host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.*Work/Life Balance*Our team also puts a high value on work-life balance. Striking a healthy balance between your personal and professional life is crucial to your happiness and success here, which is why we aren’t focused on how many hours you spend at work or online. Instead, we’re happy to offer a flexible schedule so you can have a more productive and well-balanced life—both in and outside of work.*Mentorship & Career Growth*Our team is dedicated to supporting new team members. Our team has a broad mix of experience levels and Amazon tenures, and we’re building an environment that celebrates knowledge sharing and mentorshipAmazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, visit https://www.amazon.jobs/en/disability/usA day in the lifeAbout the hiring groupJob responsibilities
US, CA, Palo Alto
Job summaryAmazon Search is looking for a senior applied scientist to build the next-generation Autocomplete and Spelling Correction experience. Autocomplete provides real-time suggestions as customers type in the search box. It serves as a virtual assistant on customer’s shopping journey. Our suggestions complete customers’ thoughts and help them to explore Amazon’s vast product offering. Spelling correction recovers the spelling errors in the queries, allowing customers to search effortlessly in any language they prefer, without worrying about making spelling mistakes. You will have the opportunities to leverage the latest developments in Natural Language Processing, Deep Learning, and Reinforcement Learning to re-invent customers’ shopping experience. Your wok will touch the life of hundreds of millions customers and deliver billion-dollar business impact.You will love the job if you enjoy brining the cutting-edge research to people’s (and yours) day-to-day life. It’s a great feeling to realize the model you deployed yesterday has helped your shopping journey today. You will work closely with scientists and engineers to do fast-pace model development and experiment iterations. You will have access to Amazon’s rich datasets, AWS’s massive computation power, and science and engineering leaders across the company.As a senior applied scientist and a technical leader, you will· Drive the science vision and roadmap.· Develop data-driven solutions for the real-world, large scale problems.· Deliver and maintain software and models in the production environment.· Lead cross-functional collaborations between product, design, and engineering.
US, WA, Seattle
Job summaryDo you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating models using state-of-the-art machine learning algorithms to solve real world problems? Do you enjoy collaborating in a diverse team environment? How about owning end-to-end business problems/metrics and directly impact the profitability of the company?If yes, then you may be a great fit to join the Amazon Buyer Risk Prevention (BRP) Machine Learning group. We are looking for a talented scientist who is passionate to build advanced algorithmic systems that help manage the safety of millions of transactions every day.Major responsibilities· Use statistical and machine learning techniques to create scalable risk management systems· Learning and understanding large amounts of Amazon’s historical business data for specific instances of risk or broader risk trends· Design, development and evaluation of highly innovative models for risk management· Working closely with software engineering teams to drive real-time model implementations and new feature creations· Working closely with operations staff to optimize risk management operations,· Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation· Tracking general business activity and providing clear, compelling management reporting on a regular basis· Research and implement novel machine learning and statistical approachesPlease visit https://www.amazon.science for more information
US, WA, Seattle
Job summaryThe Global Specialty Fulfillment (GSF) team that delights millions of customers around the world by delivering our most complex product lines at speeds that redefine the bar. The GSF Strategies Team is a collaborative group of Business Analysts, Business Intelligence Engineers, Data Engineers, Data Scientists, Economists, Product Managers, and Program Managers that creates people-centric innovations this business and their workforce of over a hundred thousand valued associates and leaders.We are now recruiting for an exceptional Economist, Global OperationsThe ideal candidate will be:· A Well-Rounded Athlete – You understand that we succeed or fail as a team. You are always ready to step up beyond your core responsibilities and go the extra mile for the project and your team. You nimbly overcome barriers to deliver the best products more quickly than expected.· A Perpetual Student – You seek knowledge and insight. You turn moments into master’s classes. Whether closing a gap, developing a new skill, or staying ahead of your industry, you revel in the joy of learning and growing.· A Skilled Communicator – You excel when interacting with business and technical partners whether you are chatting, sending a written message, or conducting a presentation.· A Trusted Advisor – You work closely with stakeholders to define key business needs and deliver on commitments. You enable effective decision making by retrieving and aggregating data from multiple sources and compiling it into a digestible and actionable format.· An Inventor at Heart – You innovate on behalf of your customer by proactively implementing improvements, enhancements, and customizations. Your customers marvel at your creative solutions to their unidentified needs.· A Fearless Explorer – You are drawn to take on the hardest problems, navigate ambiguity, and see possibility in what others view with skepticism. You never settle, even in the face of overwhelming obstacles.You will:· Collaborate with business intel and data engineering teams to collect new data and refine of existing data sources to continually improve solutions· Test hypotheses in a high-ambiguity environment making use of qualitative data, judgment, and customer feedback.· Utilize code (Python, R, Scala, etc.) to design, build, and manage scientifically-sound, production-grade models and hands-off-the-wheel solutions that solve specific business problems· Advocate for your customer and align your stakeholders to address our most pressing needs· Distill informal customer requirements into problem definitions· Manage and quantify improvement in customer experience or value for the business resulting from research outcomes· Convey rigorous mathematical concepts and considerations to non-expertsAmazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.
US, MA, Cambridge
Job summaryThe Amazon Alexa Artificial Intelligence (AI) team is looking for a passionate, talented, and inventive Applied Scientist to help build industry-leading technologies in Natural Language Processing and Entity Resolution.Key job responsibilitiesAs an Applied Scientist with the Alexa AI team, you will be responsible for developing novel algorithms that advance the state-of-the-art in language processing and entity resolution, driving model and algorithmic improvements, formulating evaluation methodologies and for influencing design and architecture choices. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to build novel products and services that make use of speech and language technology. You will work in a hybrid, fast-paced organization where scientists and engineers work together and drive improvements to production. You will collaborate with and mentor other scientists to raise the bar of scientific research in Amazon.
US, NY, New York City
Job summaryAre you seeking an environment where you can drive innovation? Do you want to apply techniques and advanced mathematical modeling to solve real world problems? Do you want to play a key role in the future of Amazon's Retail business? Come and join us!The Amazon Demand team seeks an Applied Scientist to join our team. Scientists in Demand lead the development of novel algorithmic architectures, toward the ultimate goal of accurately predicting customer demand for millions of products world-wide. This drives down costs and enables the offer of lower prices and better in-stock selection for our customers.Working collaboratively, you will develop solutions to complex problems, such as designing the next generation of algorithms to improve the supply chain efficiency. Our science community values teamwork, supports continued , and recognizes the need to take chances and try new ideas that may fail. Furthermore, our builder culture means that Scientists and Software Development Engineers work closely together to invent and construct at a massive scale. Your work can be part of Amazon production system and result in concrete business impact. Recent work from Demand includes papers presented at the NIPS 2017 Time Series Workshop, KDD 2018 Time Series Workshop, and ICML 2018 Generative Models Workshop.Responsibilities:· Collaborative research and development of demand forecasting solutions· Prototype and implement new algorithms and prediction techniques· Collaborate with product managers and engineering teams to design and implement software solutions for Amazon problems· Contribute to progress of the Amazon and broader research communities by producing publications
US, WA, Bellevue
Job summaryAmazon’s Modeling and Optimization team (MOP) is looking for a Research Scientist to analyze and optimize the most complex logistics systems in the world. Academic and/or practical background in Operations Research, Industrial Engineering, or System Engineering and Optimization are particularly relevant for this position. Experience in model-based engineering and/or multidisciplinary analysis & optimization is also a plus.Amazon’s extensive logistics system comprises thousands of fixed infrastructure nodes with millions of possible connections between them. Billions of packages flow through this network on a yearly basis, making the impact of optimal improvements truly unparalleled. This magnificent challenge is a terrific opportunity to analyze Amazon data and understand, model, simulate, optimize, and reshape one of the world's most complex systems.Key job responsibilitiesYour main focus will be on developing model-based optimization tools, at various levels of fidelity, for improving our transportation network under the uncertainty of demand and supply. You will make the real complexity of our logistics system visible, tangible, and manageable using cutting edge analytical methods. You will use modeling and simulation to validate assumptions on the intricate interactions among different elements of our system. You will identify and evaluate opportunities to improve customer experience, network speed, cost, and the efficiency of capital investment. You will develop system components using optimization, simulation, and machine learning techniques. You will quantify the improvements resulting from the application of these tools and you will evaluate trade-offs between competing outputs of the system.This position requires drive and self-motivation, superior analytical thinking, data-driven disposition, application of technical knowledge to a business context, effective collaboration with fellow research scientists, software development engineers, and product managers, effective communication of technical designs to technical and non-technical audiences, and close partnership with many stakeholders from operations, finance, IT, and business leadership
GB, London
Job summaryDo you want to join Amazon Studios AI team – the team that is at the forefront of inventing new visual media technologies within the entertainment and streaming industries to provide incredible experiences to audiences around the world? Do you want to invent and leverage cutting-edge deep-learning and machine-learning algorithms to build innovative solutions most people cannot even yet imagine?If your answers to these questions are “yes”, then come and join us at Amazon Studios AI. We are leveraging the latest breakthroughs in artificial intelligence to create a new way of watching streaming video content. We expect our solutions to disrupt the media and entertainment industry and delight audiences around the world. We are looking for talented and ambitious teammates who will Think Big with us. If you have a passion for groundbreaking technology, please join us and be part of this exciting, newly funded and fast-growing project.On our team, you will partner with top AI scientists, software engineers, product managers, and others. You will be joining on the ground floor and will grow with our team all the way through to our expected launch. You will be expected to:· Build state-of-the art computer vision solutions using approaches such as CNN, LSTM, GAN, etc.· Build 2D/3D face/body models in time series domain.· Research, invent and implement novel deep learning architectures/algorithms as part of building a disruptive product.· Work in a collaborative environment with other scientists, engineers, product managers and others.· Mentor and develop junior scientists in the team.Key job responsibilitiesPhD, CNN, LSTM, GAN, Computer Vision, Machine Learning.
US, CA, Irvine
Job summaryWould you like to work on a greenfield project that'll help improve the shopping experience of millions of Amazon customers? Want to help invent the next generation technologies in recommender and content optimization systems? We’ve got the perfect job for you.We are a team in a fast-paced organization with a huge impact on hundreds of millions of customers. We innovate at the intersection of customer experience, deep learning, and high-scale machine-learning systems.We are looking for Applied Scientists who love big data, and are capable of inventing and applying Machine Learning, Natural Language Processing, Image processing, Data Mining, Classification and Clustering techniques to solve real world problems and build novel customer facing innovations on Amazon. Our applied scientists work closely with software engineers to put algorithms into practice. They also work in partnership with teams across Amazon to create enormous benefits for our customers.As a member of the our Content Optimization team, you would be expected to move fast, have good judgment on what is and what is not worth exploring, create simple and scalable solutions and identify correct problem sets. You will be surrounded by thought-leaders in the Personalization space who are patent-leaders within Amazon. You will keep the team up-to date with latest academic research in relevant fields.About our team: Our team has the autonomy to decide where we can have the most impact and get down to experimenting. We love metrics and the fast pace. We analyze data to uncover potential opportunities, generate hypotheses, and test them. We refuse to accept constraints, internal or external, and have a strong bias for action. We imagine, build prototypes, validate ideas, and launch follow-up experiments from the successful ones.About our organization: Consider the following problem: every day, millions of customers with unique interests and needs come to Amazon looking for products out of a catalog of over a billion items. Not only do we need to decide what content would be most helpful to customers, we also need to present it in an inspiring manner. The Personalization organization within Amazon is responsible for the secret sauce that not only made Amazon the industry pioneer in building recommender systems at scale, but is also continuing to help raise the bar for building delightful and highly personalized shopping experiences.About you: You are an Applied Scientist with an interest in machine learning, data science, search, or recommendation systems. You have great problem solving skills. You love keeping abreast of the latest technology and use it to help you innovate. You have strong leadership qualities, great judgment, clear communication skills, and a track record of delivering great products. You enjoy working hard, having fun, and making history!
US, CA, Irvine
Job summaryWould you like to work on a greenfield project that'll help improve the shopping experience of millions of Amazon customers? Want to help invent the next generation technologies in recommender and content optimization systems? We’ve got the perfect job for you.We are a team in a fast-paced organization with a huge impact on hundreds of millions of customers. We innovate at the intersection of customer experience, deep learning, and high-scale machine-learning systems.We are looking for Applied Scientists who love big data, and are capable of inventing and applying Machine Learning, Natural Language Processing, Image processing, Data Mining, Classification and Clustering techniques to solve real world problems and build novel customer facing innovations on Amazon. Our applied scientists work closely with software engineers to put algorithms into practice. They also work in partnership with teams across Amazon to create enormous benefits for our customers.As a member of the our Content Optimization team, you would be expected to move fast, have good judgment on what is and what is not worth exploring, create simple and scalable solutions and identify correct problem sets. You will be surrounded by thought-leaders in the Personalization space who are patent-leaders within Amazon. You will keep the team up-to date with latest academic research in relevant fields.About our team: Our team has the autonomy to decide where we can have the most impact and get down to experimenting. We love metrics and the fast pace. We analyze data to uncover potential opportunities, generate hypotheses, and test them. We refuse to accept constraints, internal or external, and have a strong bias for action. We imagine, build prototypes, validate ideas, and launch follow-up experiments from the successful ones.About our organization: Consider the following problem: every day, millions of customers with unique interests and needs come to Amazon looking for products out of a catalog of over a billion items. Not only do we need to decide what content would be most helpful to customers, we also need to present it in an inspiring manner. The Personalization organization within Amazon is responsible for the secret sauce that not only made Amazon the industry pioneer in building recommender systems at scale, but is also continuing to help raise the bar for building delightful and highly personalized shopping experiences.About you: You are an Applied Scientist with an interest in machine learning, data science, search, or recommendation systems. You have great problem solving skills. You love keeping abreast of the latest technology and use it to help you innovate. You have strong leadership qualities, great judgment, clear communication skills, and a track record of delivering great products. You enjoy working hard, having fun, and making history!
US, WA, Seattle
Job summaryAre you excited about using econometrics to make multi-million dollar decisions more Science and Data Driven? Are you interested in supporting Consumer Hardware device concepts from innovative idea inception to launch? Do you want to work on a Economics and Data Science team focused on tackling some of the hardest business questions within the Devices business at Amazon and then scaling those Statistics and Econometrics solutions via internal to Amazon tools? Then this could be the role for you!Amazon.com strives to be Earth's most customer-centric company where people can find and discover anything they want to buy online. We hire the world's brightest minds, offering them a fast paced, technologically sophisticated and friendly work environment. Amazon Devices and Services team is the area of Amazon focused on inventing platforms that delight customers by eliminating friction they have in supplying, entertaining, and managing the home and beyond.The Device Economics team owns demand estimates and pricing recommendations of concept devices before customers know they exist. We support over 100 device-specific analyses a year on hardware and services ranging from Echo Frames to Kindle Paperwhite to Blink Video Camera subscriptions to the Amazon Smart Plug…all prior to launch.. We are a cross-functional Product team working to scale Econometrics through Amazon and beyond by incorporating Science into internal facing tools and making it easier for others to do so as well.In this role, you will support up to senior leadership decision meetings around approving confidential funding requests (PRFAQs) for brand new devices and services, build decisions around how many hardware devices to manufacture prior to receiving any customer signal, and pricing decisions around how to price and promote products and services. You will leverage Science and Tools produced by the Device Economics team such as conjoint demand models to produce these recommendations. As part of the stakeholder-facing arm of the team, you will own relationships with decision makers to help improve the end-customer experience by making the decisions that impact those end-customers more data and Science-driven. In parallel, you will work with Scientists, Economists, Product Managers, and Software Developers to provide meaningful feedback about stakeholder problems to inform business solutions and increase the velocity, quality, and scope behind our recommendations. You will own projects to make progress on Decision Science itself. Through this all, we will invest in your development to pursue your career goals.We are willing to consider L5 candidates across the BA/BIE job families where we'll bar raise your Science skills.
US, WA, Seattle
Job summaryAlexa Smart Home Research is looking for a brilliant quantitative researcher to drive a new program to ensure Alexa always delivers a four star experience. In this role, you will define the roadmap for the SH segmentation program, create experiments to evaluate customer behavior and sentiment that drive these higher quality experiences for our target customers. These insights help Alexa Smart Home Marketing and Product teams make data driven decisions about our marketing and product strategies ensuring products are accurately conveyed, appropriately priced and designed, and with each launch we are moving the needle for customers to help them accomplish their ideal smart home.Key job responsibilities· Identify and propose key opportunities for improving the product development and marketing strategy for Alexa products· Develop and execute research projects, including leading all project phases: methodology and study design, data gathering and manipulation, analysis, interpretation and presentation of results· Lead and execute validation and impact studies· Define project requirements, document business and functional specifications, map current and future state business processes· Build automated mechanisms for evaluating, measuring, and deploying the algorithms and/or models you develop.· Bring a deep level of expertise in one of the Research Marketing disciplines (e.g. Statistics)A day in the lifeAs part of your work, you will lead quantitative research projects that build our understand of smart home customers, identifying what works well and areas of improvement for Alexa Smart Home that will ensure we continue to delight our customers. Excellent business and communication skills are a must to develop and define key business questions and to build data sets that answer those questions. You should be able to work with business customers in understanding the business requirements and research impact.About the teamWe are responsible for UX and market research (foundational, market fit, usability and concept testing), Beta launch readiness and voice of the customer. These services product org-wide customer insights that help SH teams connect directly with customers daily, supporting the end-to-end product readiness, and look around the corner to understand customer and competitor trends.
US, CA, Irvine
Job summaryWould you like to work on a greenfield project that'll help improve the shopping experience of millions of Amazon customers? Want to help invent the next generation technologies in recommender and content optimization systems? We’ve got the perfect job for you.We are a team in a fast-paced organization with a huge impact on hundreds of millions of customers. We innovate at the intersection of customer experience, deep learning, and high-scale machine-learning systems.We are looking for Applied Scientists who love big data, and are capable of inventing and applying Machine Learning, Natural Language Processing, Image processing, Data Mining, Classification and Clustering techniques to solve real world problems and build novel customer facing innovations on Amazon. Our applied scientists work closely with software engineers to put algorithms into practice. They also work in partnership with teams across Amazon to create enormous benefits for our customers.As a member of the our Content Optimization team, you would be expected to move fast, have good judgment on what is and what is not worth exploring, create simple and scalable solutions and identify correct problem sets. You will be surrounded by thought-leaders in the Personalization space who are patent-leaders within Amazon. You will keep the team up-to date with latest academic research in relevant fields.About our team: Our team has the autonomy to decide where we can have the most impact and get down to experimenting. We love metrics and the fast pace. We analyze data to uncover potential opportunities, generate hypotheses, and test them. We refuse to accept constraints, internal or external, and have a strong bias for action. We imagine, build prototypes, validate ideas, and launch follow-up experiments from the successful ones.About our organization: Consider the following problem: every day, millions of customers with unique interests and needs come to Amazon looking for products out of a catalog of over a billion items. Not only do we need to decide what content would be most helpful to customers, we also need to present it in an inspiring manner. The Personalization organization within Amazon is responsible for the secret sauce that not only made Amazon the industry pioneer in building recommender systems at scale, but is also continuing to help raise the bar for building delightful and highly personalized shopping experiences.About you: You are an Applied Scientist with an interest in machine learning, data science, search, or recommendation systems. You have great problem solving skills. You love keeping abreast of the latest technology and use it to help you innovate. You have strong leadership qualities, great judgment, clear communication skills, and a track record of delivering great products. You enjoy working hard, having fun, and making history!
US, WA, Seattle
Job summaryAre you excited about cutting-edge deep-learning NLP, NLU, and Conversational AI? If so, then come and join the Alexa Artificial Intelligence (AI) team. We are the science team behind Amazon’s intelligence voice assistance system and are responsible for the deep learning technology that is central to the automated ranking and arbitration to optimize for end-to-end customer satisfaction.Key job responsibilitiesAs an Applied Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in spoken language understanding. Your work will directly impact our customers in the form of products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding.A day in the life· Design, build, test and release predictive ML models· Ensure data quality throughout all stages of acquisition and processing, including such areas as data sourcing/collection, ground truth generation, normalization, and transformation.· Collaborate with colleagues from science, engineering and business backgrounds.· Present proposals and results to partner teams in a clear manner backed by data and coupled with actionable conclusions· Work with engineers to develop efficient data querying and inference infrastructure for both offline and online use casesAbout the teamWe are a science and engineering team part of Alexa AI organization. Our mission is to help Alexa decide which action to take in response to customer requests, incorporating a variety of contextual signals including both direct and indirect customer feedback to provide the best response to the customer. Our work directly contributes to improvement in Alexa business and customer metrics.