A screenshot from SageMaker Clarify
SageMaker Clarify is integrated with Amazon SageMaker Data Wrangler, making it easier to identify bias during data preparation. You specify attributes of interest, such as gender or age, and SageMaker Clarify runs a set of algorithms to detect any presence of bias in those attributes.
Credit: AWS

How Clarify helps machine learning developers detect unintended bias

Learn why the science team behind Clarify turned to a concept from 1951 to address a modern complexity.

In his machine learning keynote at re:Invent on Tuesday, Swami Sivasubramanian, vice president of machine learning, Amazon Web Services (AWS), announced Amazon SageMaker Clarify, a new service that helps customers detect statistical bias in their data and machine learning models, and helps explain why their models are making specific predictions. Clarify saves developers time and effort by providing them the ability to better understand and explain how their machine learning models arrive at their predictions.

Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and explaining

Developers today contend with both increasingly large volumes of data, as well as more complex machine learning models. In order to detect bias in those complex models and data sets, developers must rely on open-source libraries replete with custom code recipes that are inconsistent across machine learning frameworks. This tedious approach requires a lot of manual effort and often arrives too late to correct unintended bias.

“If you care about this stuff, it's pretty much a roll-your-own situation right now,” said University of Pennsylvania computer science professor and Amazon Scholar Michael Kearns, who provided guidance to the team of scientists that developed SageMaker Clarify. “If you want to do some practical bias detection, you either need to implement it yourself or go to one of the open-source libraries, which vary in quality. They're frequently not well-maintained or documented. In many cases, it's just, ‘Here is the code we used to run our experiments for this academic paper, good luck.’”

SageMaker Clarify helps address the challenges of relying on multiple open-source libraries by offering robust, reliable code in an integrated, cloud-based framework.

Increasingly complex networks

The efficacy of machine learning models depends in part on understanding how much influence a given input has on the output.

AWS on Air 2020: AWS What’s Next ft. Amazon SageMaker Clarify

“A lending model for consumer loans might include credit history, employment history, and how long someone has lived at their current address,” Kearns explained. “It might also utilize variables that aren't specifically financial, such as demographic variables. One thing you might naturally want to know is which of these variables is more important in the model’s predictions, which may be used in lending decisions, and which are less important.”

With linear models, each variable is assigned some weight, positive or negative, and the overall decision is a sum of those weighted inputs. In those cases, the inputs with the bigger weights clearly have more influence on the output.

However, that approach falls short with neural networks or more complicated, non-linear models. “When you get to models like neural networks, it's no longer a simple matter of determining or measuring the influence of an input on the output,” Kearns said.

To help account for the growing complexity of modern machine learning models, the Amazon science team looked to the past — specifically to an idea from 1951.

Shapley values

The team wanted to design a solution to help machine learning pros be able to better explain their models’ decisions in the face of growing complexity. They found inspiration in a popular scientific method called Shapley values.

Shapley values were named in honor of Lloyd Shapley, who introduced the idea in 1951 and who won the Nobel Prize in Economics for it in 2012. The Shapley value approach, which is rooted in game theory, considers a wide range of possible inputs and outputs and offers “the average marginal contribution of a feature value across all possible coalitions”.  The comprehensive nature of the approach means it can help provide a framework for understanding the relative weight of a set of inputs, even across complex models and multiple inputs.

“SageMaker Clarify utilizes Shapley values to essentially take your model and run a number of experiments on it or on your data set,” Kearns said. “It then uses that to help come up with a visualization and quantification of which of those inputs is more or less important.”

Nor does it matter which kind of model a developer uses. “One of the nice things about this approach is it is model agnostic,” Kearns said. “It performs input-output experiments and gives you some sense of the relative importance of the different inputs to the output decision.”

The science team also worked to be certain SageMaker Clarify had a comprehensive view. They designed it so everyday developers and data scientists can detect bias across the entire machine learning workflow — including data preparation, training, and inference. SageMaker Clarify is able to achieve that comprehensive view, Kearns explained, because (again) it is model agnostic. “Each of these steps has been designed to avoid making strong assumptions about the type of model that the user is building.”

Bias detection and explainability

Model builders who learn that their models are making predictions that are strongly correlated to a specific input may find those predictions fall short of their definition of fairness. Kearns offered the example of a lending company that discovers its model’s predictions are skewed. “That company will want to understand why its model is making predictions that might lead to decisions to give loans at a lower rate to group A than to group B, even if they're equally credit worthy.”

SageMaker Clarify can examine tabular data and help the modelers spot where gaps might exist. “This company would upload a spreadsheet of data showing who they gave loans to, what they knew about them, et cetera,” Kearns said. “What the data bias detection part does is say, ‘For these columns, there may be over or underrepresentation of certain features, which could lead to a discriminatory outcome if not addressed.’”

A screenshot from SageMaker Clarify
SageMaker Clarify is integrated with SageMaker Model Monitor, enabling you to configure alerting systems like Amazon CloudWatch to notify you if your model exceeds certain bias metric thresholds. 
Credit: AWS

That can be influenced by a number of factors, including simply lacking the correct data to build accurate predictions. For example, SageMaker Clarify can indicate whether modelers have enough data on certain groups of applicants to expect an accurate prediction. The metrics provided by SageMaker Clarify can then be used to correct unintended bias in machine learning models, and automatically monitor model predictions in production to help ensure they are not trending toward biased outcomes.

Future applications

The SageMaker Clarify science team is already looking to the future.

Their research areas include algorithmic fairness and machine learning, as well as explainable AI. Team members have published widely in the academic literature on these topics, and worked hard in the development of SageMaker Clarify to balance the science of fairness with engineering solutions and practical product design. Their approaches are both statistical and causal, and focus not only on bias measurement in trained models, but also bias mitigation. It is that last part that has Kearns particularly excited about the future.

“The ability to not just identify problems in your models, but also have the tools to train them in a different way would go a long way toward mitigating that bias,” he said. “It’s good to know that you have a problem, but it's even better to have a solution to your problem.”

Best practices

The notions of bias and fairness are highly application dependent and the choice of the attributes for which bias is to be measured, as well as the choice of the bias metrics, may need to be guided by social, legal, and other non-technical considerations,” said principal applied scientist Krishnaram Kenthapadi, who led the scientific effort behind SageMaker Clarify. “For successful adoption of fairness-aware machine learning and explainable AI approaches in practice, it’s important to build consensus and achieve collaboration across key stakeholders such as product, policy, legal, engineering, and AI/ML teams, as well as end users and communities,” he said. “Further, it’s good to take into account fairness and explainability considerations during each stage of the ML lifecycle, for example, Problem Formation, Dataset Construction, Algorithm Selection, Model Training Process, Testing Process, Deployment, and Monitoring/Feedback.

Find more best practices on the AWS website.

Research areas

Related content

IL, Tel Aviv
This role is available across multiple locations in Tel-Aviv and Haifa. Amazon Internships have start dates throughout the year and can vary in length from 3-6 months for full-time. Please note these are not remote internships. Are you a MS or PhD student interested in a 2024 Internship in the field of Applied Sciences? Are you interested in machine learning, deep learning, automated reasoning, speech, robotics, computer vision, optimization, or quantum computing? If so, we want to hear from you! We are looking for scientists with interest in using a variety of domain expertise to invent, design and implement state-of-the-art solutions for never-before-solved problems. At Amazon, you will grow into the high impact, visionary person you know you’re ready to be. Every day will be filled with developing new skills and achieving personal growth. How often can you say that your work changes the world? At Amazon, you’ll say it often. Join us and define tomorrow. For more information on the Amazon Science community please visit the Science hub page: https://www.amazon.science/ For more information on the Amazon Science EMEA internship program or to join us for one of our upcoming informational webinars please visit this page: https://amazonscienceopportunitiesemea.splashthat.com/ Key job responsibilities As an Applied Science Intern, you will own the design and development of end-to-end systems. You’ll have the opportunity to write technical white papers, create technical roadmaps and drive production level projects that will support Amazon Science. You will work closely with Amazon scientists, and other science interns to develop solutions and deploy them into production. You will have the opportunity to design new algorithms, models, or other technical solutions whilst experiencing Amazon’s customer focused culture. The ideal scientist must have the ability to work with diverse groups of people and cross-functional teams to solve complex business problems. We are open to hiring candidates to work out of one of the following locations: Haifa, ISR | Tel Aviv, ISR
GB, London
Have you ever wished to build high standard Operations Research and Machine Learning algorithms to optimize one of the most complex logistics network? Have you ever ordered a product on Amazon websites and wondered how it got delivered to you so fast, and what kinds of algorithms & processes are running behind the scenes to power the whole operation? If so, this role is for you. The team: Global transportation services, Research and applied science - Operations is at the heart of the Amazon customer experience. Each action we undertake is on behalf of our customers, as surpassing their expectations is our passion. We improve customer experience through continuously optimizing the complex movements of goods from vendors to customers throughout Europe. - Global transportation analytical teams are transversal centers of expertise, composed of engineers, analysts, scientists, technical program managers and developers. We are focused on Amazon most complex problems, processes and decisions. We work with fulfillment centers, transportation, software developers, finance and retail teams across the world, to improve our logistic infrastructure and algorithms. - GTS RAS is one of those Global transportation scientific team. We are obsessed by delivering state of the art OR and ML tools to support the rethinking of our advanced end-to-end supply chain. Our overall mission is simple: we want to implement the best logistics network, so Amazon can be the place where our customers can be delivered the next-day. The role: Applied scientist, speed and long term network design The person in this role will have end-to-end ownership on augmenting RAS Operation Research and Machine Learning modeling tools. They will help understand where are the constraints in our transportation network, and how we can remove them to make faster deliveries at a lower cost. Concretely, you will be responsible for designing and implementing state-of-the-art algorithmic in transportation planning and network design, to expand the scope of our Operations Research and Machine Learning tools, to reflect the constantly evolving constraints in our network. You will enable the creation of a product that drives ever-greater automation, scalability and optimization of every aspect of transportation, planning the best network and modeling the constraints that prevent us from offering more speed to our customer, to maximize the utilization of the associated resources. The impact of your work will be in the Amazon EU global network. The product you will build will span across multiple organizations that play a role in Amazon’s operations and transportation and the shopping experience we deliver to customer. Those stakeholders include fulfilment operations and transportation teams; scientists and developers, and product managers. You will understand those teams constraints, to include them in your product; you will discuss with technical teams across the organization to understand the existing tools and assess the opportunity to integrate them in your product. You will also be challenged to think several steps ahead so that the solutions you are building today will scale well with future growth and objective (e.g.: sustainability). You will engage with fellow scientists across the globe, to discuss the solutions they have implemented and share your peculiar expertise with them. This is a critical role and will require an aptitude for independent initiative and the ability to drive innovation in transportation planning and network design. Successful candidates should be able to design and implement high quality algorithm solutions, using state-of-the art Operations Research and Machine Learning techniques. You will have the opportunity to thrive in a highly collaborative, creative, analytical, and fast-paced environment oriented around building the world’s most flexible and effective transportation planning and network design management technology. Key job responsibilities - Engage with stakeholders to understand what prevents them to build a better transportation network for Amazon - Review literature to identify similar problems, or new solving techniques - Build the mathematical model representing your problem - Implement light version of the model, to gather early feed-back from your stakeholders and fellow scientists - Implement the final product, leveraging the highest development standards - Share your work in internal and external conferences - Train on the newest techniques available in your field, to ensure the team stays at the highest bar About the team GTS Research and Applied Science is a team of 15 scientists and engineers whom mission is to build the best decision support tools for strategic decisions. We model and optimize Amazon end-to-end operations. The team is composed of enthusiastic members, that love to discuss any scientific problem, foster new ideas and think out of the box. We are eager to support each others and share our unique knowledge to our colleagues. We are open to hiring candidates to work out of one of the following locations: London, GBR
US, WA, Seattle
Amazon’s Global Media and Entertainment (GME) organization is creating the future of entertainment where creative content, innovation, and commerce come together. We leverage Amazon’s unique expertise across video, music, gaming, and more to create a truly immersive entertainment experience. Our team, GME Economics, is focused on building strategic science tools to optimize Amazon’s entertainment offerings, so that we can provide a great customer experience while operating as a sustainable and profitable business. This role will build and expand long-term measurement models serving all GME businesses, making strong causal inference skills essential. The GME business context is complex, ambiguous, and can change quickly. You'll need to be adaptable and resourceful, converting complex business problems into estimable econometric problems, and building solutions that anticipate future business needs. Once the model is built, you'll partner hands-on with engineers to productionize it, ensuring the output is high quality and that computation is efficient. Our team values excellent written and verbal communication with partners of all levels, with technical and nontechnical backgrounds. You'll need to translate data and findings into actionable insights for leaders with the same skill that you explain the nuances of your identification assumptions to scientists, or describe the logical flow of your estimations to engineers. Our healthy team culture is supportive and fast-paced, and we prioritize learning, growth, and helping each other to continuously raise the bar. We think big, taking on high-risk, high-reward projects based on novel scientific approaches. As a central science team, we work hard to build trust with partners, who always have fresh problems for us to tackle. Impact and Career Growth: In today’s entertainment landscape, critical decisions are made with data and economic models. You’ll help leaders ask the right questions, and then deliver data-driven answers, creating the future of GME at Amazon. You’ll help define a long-term science vision and translate it into an actionable roadmap. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding – a perfect recipe for career growth as an economist in tech. Key job responsibilities • Design and build econometric models, especially causal models, to measure the value of the business and its many features • Develop science products from concept to prototype to production, incorporating feedback from scientists and business partners • Independently identify and pursue new opportunities to leverage economic insights across GME businesses • Collaborate with PMs to build product roadmaps, ensuring our work meets the needs of our stakeholders and then communicating progress and findings to them • Write business and technical documents communicating business context, methods, and results to business leadership and other scientists • Work with engineers within and outside our org to productionize science models • Serve as a technical reviewer for our team and related teams, including document and code reviews We are open to hiring candidates to work out of one of the following locations: Los Angeles, CA, USA | New York, NY, USA | Seattle, WA, USA
US, WA, Seattle
We are designing the future. If you are in quest of an iterative fast-paced environment, where you can drive innovation through scientific inquiry, and provide tangible benefit to hundreds of thousands of our associates worldwide, this is your opportunity. Come work on the Amazon Worldwide Fulfillment Design & Engineering Team! We are looking for an experienced and Research Scientist with background in Ergonomics and Industrial Human Factors, someone that is excited to work on complex real-world challenges for which a comprehensive scientific approach is necessary to drive solutions. Your investigations will define human factor / ergonomic thresholds resulting in design and implementation of safe and efficient workspaces and processes for our associates. Your role will entail assessment and design of manual material handling tasks throughout the entire Amazon network. You will identify fundamental questions pertaining to the human capabilities and tolerances in a myriad of work environments, and will initiate and lead studies that will drive decision making on an extreme scale. .You will provide definitive human factors/ ergonomics input and participate in design with every single design group in our network, including Amazon Robotics, Engineering R&D, and Operations Engineering. You will work closely with our Worldwide Health and Safety organization to gain feedback on designs and work tenaciously to continuously improve our associate’s experience. Key job responsibilities - Collaborating and designing work processes and workspaces that adhere to human factors/ ergonomics standards worldwide. - Producing comprehensive and assessments of workstations and processes covering biomechanical, physiological, and psychophysical demands. - Effectively communicate your design rationale to multiple engineering and operations entities. - Identifying gaps in current human factors standards and guidelines, and lead comprehensive studies to redefine “industry best practices” based on solid scientific foundations. - Continuously strive to gain in-depth knowledge of your profession, as well as branch out to learn about intersecting fields, such as robotics and mechatronics. - Travelling to our various sites to perform thorough assessments and gain in-depth operational feedback, approximately 25%-50% of the time. We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
DE, BE, Berlin
Have you ever wondered how Amazon delivers timely and reliably hundreds of millions of packages to customer’s doorsteps? Are you passionate about data and mathematics, and hope to impact the experience of millions of customers? Are you obsessed with designing simple algorithmic solutions to very challenging problems? If so, we look forward to hearing from you! Amazon Transportation Services is seeking Applied (or Research) Scientists. As a key member of the central Research Science Team of ATS operations, these persons will be responsible for designing algorithmic solutions based on data and mathematics for optimizing the middle-mile Amazon transportation network. The job is opened in the EU Headquarters in Luxembourg (alternatively: Barcelona, Berlin or London), designed to maximize interaction with the team and stakeholders, but we will consider applicants with remote work requirements as well. Basic qualifications * PhD in Operations Research, Machine Learning, Statistics, Applied Mathematics, Computer Science or other field related to algorithms and data (or equivalent experience). * Excellent written and verbal communication skills. * Experience with some programming language (Java/python/C++) * Research experience in one or more: *Combinatorial optimization problems (e.g., scheduling, vehicle routing, facility location). *Continuous optimization problems (e.g., linear programming, convex programming, non-convex programming). *Predictive analytics (e.g., forecasting, time-series, neural networks) *Prescriptive analytics (e.g., stochastic optimization, bandits, reinforcement learning). Preferred qualifications * Experience from working in a fast-paced applied research environment. * Ability to handle ambiguity. * Top tier publications pertinent to the field of study. Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice (https://www.amazon.jobs/en/privacy_page) to know more about how we collect, use and transfer the personal data of our candidates. Key job responsibilities Solve complex optimization and machine learning problems using scalable algorithmic techniques. Design and develop efficient research prototypes that address real-world problems in the middle-mile operations of Amazon. Lead complex time-bound, long-term as well as ad-hoc analyses to assist decision making. Communicate to leadership results from business analysis, strategies and tactics. A day in the life You will be brainstorming algorithmic approaches with team-mates to solve challenging problems for the middle-mile operations of Amazon. You will be developing and testing prototype solutions with above algorithmic techniques. You will be scavenging information from the sea of Amazon data to improve these solutions. You will be meeting with other scientists, engineers, stakeholders and customers to enhance the solutions and get them adopted. About the team The Science and Tech team of ATS EU is looking for candidates who are looking to impact the world with their mathematical and data-driven skills. ATS stands for Amazon Transportation Service, we are the middle-mile planners: we carry the packages from the warehouses to the cities in a limited amount of time to enable the “Amazon experience”. As the core research team, we grow with ATS business to support decision making in an increasingly complex ecosystem of a data-driven supply chain and e-commerce giant. We schedule more than 1 million trucks with Amazon shipments annually; our algorithms are key to reducing CO2 emissions, protecting sites from being overwhelmed during peak days, and ensuring a smile on Amazon’s customer lips. Our mathematical algorithms provide confidence in leadership to invest in programs of several hundreds millions euros every year. Above all, we are having fun solving real-world problems, in real-world speed, while failing & learning along the way. We use modular algorithmic designs in the domain of combinatorial optimization, solving complicated generalizations of core OR problems with the right level of decomposition, employing parallelization and approximation algorithms. We use deep learning, bandits, and reinforcement learning to put data into the loop of decision making. We like to learn new techniques to surprise business stakeholders by making possible what they cannot anticipate. For this reason, we work closely with Amazon scholars and experts from Academic institutions. We code our prototypes to be production-ready We prefer provably optimal solutions than heuristics, though we settle for heuristics when performance dictates it. Overall, we appreciate the value of correct modeling. We are open to hiring candidates to work out of one of the following locations: Berlin, BE, DEU
US, WA, Bellevue
We are seeking a passionate, talented, and inventive individual to join the Applied AI team and help build industry-leading technologies that customers will love. This team offers a unique opportunity to make a significant impact on the customer experience and contribute to the design, architecture, and implementation of a cutting-edge product. The mission of the Applied AI team is to enable organizations within Worldwide Amazon.com Stores to accelerate the adoption of AI technologies across various parts of our business. We are looking for a Senior Applied Scientist to join our Applied AI team to work on LLM-based solutions. On our team you will push the boundaries of ML and Generative AI techniques to scale the inputs for hundreds of billions of dollars of annual revenue for our eCommerce business. If you have a passion for AI technologies, a drive to innovate and a desire to make a meaningful impact, we invite you to become a valued member of our team. You will be responsible for developing and maintaining the systems and tools that enable us to accelerate knowledge operations and work in the intersection of Science and Engineering. You will push the boundaries of ML and Generative AI techniques to scale the inputs for hundreds of billions of dollars of annual revenue for our eCommerce business. If you have a passion for AI technologies, a drive to innovate and a desire to make a meaningful impact, we invite you to become a valued member of our team. We are seeking an experienced Scientist who combines superb technical, research, analytical and leadership capabilities with a demonstrated ability to get the right things done quickly and effectively. This person must be comfortable working with a team of top-notch developers and collaborating with our research teams. We’re looking for someone who innovates, and loves solving hard problems. You will be expected to have an established background in building highly scalable systems and system design, excellent project management skills, great communication skills, and a motivation to achieve results in a fast-paced environment. You should be somebody who enjoys working on complex problems, is customer-centric, and feels strongly about building good software as well as making that software achieve its operational goals. We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA
US, WA, Seattle
Have you ever wanted to solve a mystery or be part of solving a case? Are you fascinated by detective stories or crime shows on TV? Do you love to catch bad actors, build ML models and solve complex problems. If so, working on the Loss Prevention Tech team as a Sr Applied Scientist is the place for you! We detect theft, fraud and organized crime happening across our global supply chain and operations for millions of items, for hundreds of product lines worth billions of dollars of inventory world-wide. We foster new game-changing ideas, creating ever more intelligent and self-learning systems to maximize the cost savings of Amazon's inventory losses. The primary role of a Sr Applied Scientist within Amazon is to address business challenges through building a compelling case, and using data to influence change across the organization. This individual will be given responsibility on their first day to own those business challenges and the autonomy to think strategically and make data driven decisions. Decisions and tools made in this role will have significant impact to the customer experience, as it will have a major impact on all the fraud investigations happening across Amazon operations. Ideal candidates will be a high potential, strategic and analytic graduate with a PhD in ( Research, Statistics, Engineering, and Supply Chain) ready for challenging opportunities in the core of our world class operations space. Great candidates have a history of building fraud detections, detecting organized crime and the ability to use data and research to make changes. This individual will need to be able to work with a team, but also be comfortable making decisions independently, in what is often times an ambiguous environment. Key job responsibilities - Own KPIs that measure fraud management performance and efficiencies. - Detect and automate theft, fraud MOs - Detect organized crime rings and bad actor clusters - Build data or computer vision based ML models - Perform end to end evaluation of operational defects, system gaps, and scaling challenges (both system and operational). - Contribute to the overall fraud management and product development strategies. - Present key learnings and vision to stakeholders and leadership. - Integrate ML detection models via software applications About the team We believe that building a culture that is welcoming and inclusive is integral to people doing their best work and is essential to what we can achieve as a company. We actively recruit people from diverse backgrounds to build a supportive and inclusive workplace. Our team puts a high value on work-live balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
US, WA, Seattle
Are you interested in working with top talent in Optimization, Operations Research and Supply Chain to help Amazon to efficiently match our Devices with worldwide customers? We have challenging problems and need your innovative solutions to make tremendous financial impacts! The Amazon Devices Science team is looking for a Research Scientist with background in Operations Research, Optimization, Supply Chain and/or Simulation to support science efforts to integrate across inventory management functionalities. Our team is responsible for science models (both deterministic and stochastic) that power world-wide inventory allocation for Amazon Devices business that includes Echo, Kindle, Fire Tablets, Amazon TVs, Amazon Fire TV sticks, Ring, and other smart home devices. We formulate and solve challenging large-scale financially-based optimization problems which ingest demand forecasts and produce optimal procurement, production, distribution, and inventory management plans. In addition, we also work closely with demand forecasting, material procurement, production planning, finance, and logistics teams to co-optimize the inventory management and supply chain for Amazon Devices given operational constraints. Key job responsibilities The successful candidate will be a self-starter, comfortable with ambiguity, with strong attention to detail, and ability to work in a fast-paced and ever-changing environment and a desire to help shape the overall business. Job responsibilities include: - Design and develop advanced mathematical, simulation, and optimization models and apply them to define strategic and tactical needs and drive appropriate business and technical solutions in the areas of inventory management and distribution, network flow, supply chain optimization, and demand planning - Apply mathematical optimization techniques (linear, quadratic, SOCP, robust, stochastic, dynamic, mixed-integer programming, network flows, nonlinear, nonconvex programming) and algorithms to design optimal or near optimal solution methodologies to be used by in-house decision support tools and software - Research, prototype and experiment with these models by using modeling languages such as Python; participate in the production level deployment - Create, enhance, and maintain technical documentation, and present to other Scientists, Product, and Engineering teams - Support project plans from a scientific perspective by managing product features, technical risks, milestones and launch plans - Influence organization's long-term roadmap and resourcing, and onboard new technologies onto the Science team's toolbox We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
US, WA, Bellevue
Inventory Planning and Control (IPC) is seeking an experienced senior data scientist to join its central science team. Our team owns the core decision models in the space of Buying, Placement, and Capacity Control. Our models decide when, where, and how much we should buy, flow, and hold inventories in our global fulfillment network to meet Amazon’s business goals and to make our customers happy. We do this for hundreds of millions of items and hundreds of product lines worth billions of dollars of world-wide for both our Retail and third-party seller business. Our systems are built entirely in-house, for which we constantly develop new technologies in automated inventory planning, prediction, optimization and simulation. Our systems operate at various scales, from real-time decision system that completes thousands of transactions per seconds, to large scale distributed system that optimizes the inventory decisions over millions of products simultaneously. IPC is also unique in that we are simultaneously developing the science and software of inventory optimization and solving some of the toughest computational/operational challenges in production. Our team members have an opportunity to be on the forefront of supply chain thought leadership by working on some of the most difficult problems in the industry with some of the best product managers, scientists, and software engineers in the industry. Key job responsibilities Candidates will be responsible for developing causal, machine learning and data driven models to enhance the various inventory optimization engines that the team owns. The successful candidate should have solid hands-on experience in applying machine learning or causal inference models. They will also be responsible for conducting data driven analysis to facilitate strategic decisions. They require superior logical thinkers who are able to quickly approach large ambiguous problems and develop a practical plan to tackle. Successful candidates must thrive in fast-paced environments, which encourage collaborative and creative problem solving. They are able to measure and estimate risks, and constructively critique peer research. As a senior scientist, you will also help coach/mentor junior scientists in the team. A day in the life The IPC science team contains a large group of scientists with different technical expertise, who will help and collaborate with you on your projects. In this role, you will also work with our internal customers from the Retail, third-party seller and operations departments worldwide. You will understand their challenges and pain points, and help develop data driven solutions that improve how Amazon manages inventory in our global supply chain. You will work closely with the product managers, engineers and other scientists to turn science proposals into production implementation. About the team We are a team of scientists, product managers and engineers focusing on innovation. We promote experimentation and learn by building. We often tackle the hardest problem in the organization and work cross-functionally. We are at the center of developing inventory solutions to support the rapid growth of Amazon's store business. We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA
US, WA, Seattle
Amazon Prime is looking for an ambitious Economist to help create econometric insights for world-wide Prime. Prime is Amazon's premiere membership program, with over 200M members world-wide. This role is at the center of many major company decisions that impact Amazon's customers. These decisions span a variety of industries, each reflecting the diversity of Prime benefits. These range from fast-free e-commerce shipping, digital content (e.g., exclusive streaming video, music, gaming, photos), and grocery offerings. Prime Science creates insights that power these decisions. As an economist in this role, you will create statistical tools that embed causal interpretations. You will utilize massive data, state-of-the-art scientific computing, econometrics (causal, counterfactual/structural, time-series forecasting, experimentation), and machine-learning, to do so. Some of the science you create will be publishable in internal or external scientific journals and conferences. You will work closely with a team of economists, applied scientists, data professionals (business analysts, business intelligence engineers), product managers, and software engineers. You will create insights from descriptive statistics, as well as from novel statistical and econometric models. You will create internal-to-Amazon-facing automated scientific data products to power company decisions. You will write strategic documents explaining how senior company leaders should utilize these insights to create sustainable value for customers. These leaders will often include the senior-most leaders at Amazon. The team is unique in its exposure to company-wide strategies as well as senior leadership. It operates at the cutting-edge of utilizing data, econometrics, artificial intelligence, and machine-learning to form business strategies. A successful candidate will have demonstrated a capacity for building, estimating, and defending statistical models (e.g., causal, counterfactual, time-series, machine-learning) using software such as R, Python, or STATA. They will have a willingness to learn and apply a broad set of statistical and computational techniques to supplement deep-training in one area of econometrics. For example, many applications on the team use structural econometrics, machine-learning, and time-series forecasting. They rely on building scalable production software, which involves a broad set of world-class software-building skills often learned on-the-job. As a consequence, already-obtained knowledge of SQL, machine learning, and large-scale scientific computing using distributed computing infrastructures such as Spark-Scala or PySpark would be a plus. Additionally, this candidate will show a track-record of delivering projects well and on-time, preferably in collaboration with other team members (e.g. co-authors). Candidates must have very strong writing and emotional intelligence skills (for collaborative teamwork, often with colleagues in different functional roles), a growth mindset, and a capacity for dealing with a high-level of ambiguity. Endowed with these traits and on-the-job-growth, the role will provide the opportunity to have a large strategic, world-wide impact on the customer experiences of Prime members. Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, visit https://www.amazon.jobs/en/disability/us We are open to hiring candidates to work out of one of the following locations: Arlington, VA, USA | Chicago, IL, USA | Seattle, WA, USA