An Amazon employee is seen making a delivery while an electric delivery van is parked behind him on a residential street in Los Angeles
When Amazon announced it would purchase 100,000 custom electric delivery vehicles, a team of scientists within the Amazon Logistics Research organization took on the challenge of determining the best strategy for deploying them.
About Amazon

The science of operations planning under uncertainty

How the Amazon Logistics Research Science team guides important decisions related to last-mile delivery.

When Amazon announced it would purchase 100,000 custom electric delivery vehicles as part of The Climate Pledge, a team of scientists within the Amazon Logistics (AMZL) Research organization took on the challenge of determining the best strategy for deploying them. Based on sophisticated models that simulate Amazon’s shipments and external parameters like power availability in each city, the team is developing a plan to gradually electrify Amazon’s entire fleet.

This is just one of many projects the AMZL Research Science team is tackling related to last-mile delivery. Last mile, as the name implies, is the last leg of the journey of a product to a customer’s doorstep. The team develops models to predict shipments per route (SPR) and distribution, which is the average number of packages delivered by a single driver in a given city on a given day (weeks to years in the future). These models help to predict the number and the different sizes of vans the company should purchase to meet the predicted demands.

“With these complex models we develop, we have been influencing the company’s investment in vehicles, Delivery Service Partners, and their drivers,” says Rohit Malshe, a principal research scientist at Amazon.

How to forecast when everything is changing

There are multiple scientific challenges involved in developing these models given the dynamic nature of Amazon’s operations.

“One of these challenges is that our volume keeps growing. In general, as the volume grows, the shipments per route will also increase, but not linearly,” explains Abhilasha Katariya, a senior research scientist on the team. New delivery stations are frequently launched, leading to several changes in the geographical area that each station covers. Stations may incorporate different types of vehicles and modify their operation hours, which also impacts how much they can deliver. Additionally, road networks are subject to alterations as well, impacting driving time.

Left to right, Rohit Malshe, principal research scientist; Abhilasha Katariya, senior research scientist; and Natarajan Gautam, an Amazon Scholar and a professor at Texas A&M University, are all part of the Amazon Logistics Research Science team.
Left to right, Rohit Malshe, principal research scientist; Abhilasha Katariya, senior research scientist; and Natarajan Gautam, an Amazon Scholar and a professor at Texas A&M University, are all part of the Amazon Logistics Research Science team.

The team’s scientists must develop models that can handle the variability and complexity. To do that, they use a bottoms-up approach that starts at the zip code level. “This creates a foundation where any changes in the stations’ jurisdiction can be taken into consideration directly,” says Katariya.

Pure machine-learning approaches are not adequate because the team must frequently make predictions based on new scenarios, for which there is no training data available. To compensate for the lack of training data, the team develops models that combine machine learning and physics-based models that have an optimization component which helps to take into account new variables.

For example, if a large van is added to an Amazon station that previously only worked with small and medium vans, there is no training data to inform the model. “But because the core of the model uses analytical and optimization components, we can still predict the shipments per route for a larger van,” says Katariya.

“If you think about a machine learning model, typically interpolating is very easy. But, in our case, we typically want to extrapolate because we're always getting more volume,” says Natarajan Gautam, an Amazon Scholar and a professor at Texas A&M University. “Using historical data to extrapolate is generally not recommended in machine learning, because you haven’t seen those things in the past.”

This is where the physics-based model comes in handy, although a pure physics-based model also wouldn’t work, notes Gautam, because there are so many simplifying assumptions that need to be made to obtain an analytically tractable model. “We want to get the best of both worlds, in some sense. We all want something that adequately represents what is observed, but we also want to be able to extrapolate when not observed.”

Another strategy the team employs to deal with situations where the parameters are constantly changing is to run the same model over and over again to do a type of course correction. “Just run the model every month, so that all the parameters that are changing are learned by the model, and then you are always getting the latest and greatest picture you should expect. This way you have a good model that handles all types of situations, even the ones where no data exists,” says Malshe.

The science team works very closely with people on the ground, both in station and on the road, to perfect these models. They frequently visit the delivery stations and interview the drivers whenever an opportunity arrives. “We make visits to stations and do ride-alongs so that we stay connected with how the business is evolving,” says Katariya.  

In one of these meetings, Gautam says, station employees said their results were different from what the models were predicting. “We went back to the drawing board, looked at the code and the data they were getting ,and took a deep dive to find what was causing the problem”.

They realized the station started delivering to a new zip code, but it didn’t perform the same way the previous station did. That explained the difference between what the model was observing and the real-life data. Having a close connection with operations allowed them to identify the problem and adjust their model.

Dealing with COVID-19 disruptions

For big decisions like vehicle purchases, the AMZL Research Science team forecasts on a 16-month horizon. However, when the team predicted the number of vans needed for 2020, their model didn’t consider the COVID-19 pandemic. “Suddenly there was so much more package demand that all our forecasts were basically incorrect,” says Malshe.

An Amazon employee loads an electric delivery van inside a delivery station in Los Angeles.
For big decisions like vehicle purchases, the AMZL Research Science team forecasts on a 16-month horizon.
About Amazon

He says, when situations like these arise, the first thing the team does is to upgrade the forecasts to incorporate the additional volume. They also perform scenario analyses to check, for example, if the vehicles that had already been budgeted and procured would serve the purpose. Fortunately, in this case, because these decisions are made so far in advance, the team intentionally overbudgeted to account for uncertainties. “Luckily enough, the previous year, we had spent a lot of money on bigger vehicles, and they were able to absorb the additional package volume. So, when we ran these forecasts, we figured out we were in a good spot to be able to handle such changes,” says Malshe.

“Another risk mitigation lever we applied is to make sure there is enough storage space in the delivery stations,” says Malshe. “We made sure we looked into every possible parameter to optimize for vehicles and their placement in various cities, and their deployment to various Delivery Service Partner companies so that they are utilized to the best of our capabilities.”

‘Many challenges and interesting solutions’

The electrification of Amazon’s fleet presents its own set of challenges. Some of these include how to make sure batteries in the vehicles don’t run out of charge on the road; how to optimize electricity and power consumption; and how to account for extreme weather, long trips and hilly areas. “We will keep learning on all of these items as we go forward, and each year we will come up with more innovations to overcome any barriers,” says Malshe.  

For Malshe, the diversity of the scientists working on the team – which includes people with various backgrounds, industries, educations, and skill sets – is what contributes to its success in tackling these unresolved challenges.

"We have people on our team who are extremely data savvy.  We have team members who know  SQL coding in depth and some are extremely good in Python coding. Other team members have expertise in areas like machine learning, optimization, pure modeling, Monte Carlo simulations and what not," says Malshe, who is himself a chemical engineer with experience in logistics.

 “Usually two to three people are working on every project. It divides and conquers various tasks and ultimately gives everyone an opportunity to do valuable work,” he says.

In addition to the team’s range of expertise, Katariya says another team success factor is its ability to collaborate on a wide range of problems. “Each problem has a different challenge, some have a very simple mathematical solution, but are very heavy on the implementation side, and others may require more complex models from a mathematical perspective, but are easier to implement.”

And there are many more challenges to be tackled. In fact, Gautam says, some of his peers have yet to fully grasp the challenges involved in this field of research.

“A lot of people think of last mile as solving a vehicle routing problem. But we do a lot more than that,” he says. “There are so many challenges and interesting solutions that you just can’t take it off the shelf, you really have to invent as you go along. There are tremendous opportunities to do that here and the range of challenges we get to address is what makes being involved with this team so professionally rewarding.”

The team is currently hiring research and data scientists and is looking for experienced researchers to consider applying.

Related content

US, WA, Bellevue
Conversational AI ModEling and Learning (CAMEL) team is part of Amazon Devices organization where our mission is to build a best-in-class Conversational AI that is intuitive, intelligent, and responsive, by developing superior Large Language Models (LLM) solutions and services which increase the capabilities built into the model and which enable utilizing thousands of APIs and external knowledge sources to provide the best experience for each request across millions of customers and endpoints. We are looking for a passionate, talented, and resourceful Senior Applied Scientist in the field of LLM, Artificial Intelligence (AI), Natural Language Processing (NLP), Recommender Systems and/or Information Retrieval, to invent and build scalable solutions for a state-of-the-art context-aware conversational AI. A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, enjoy operating in dynamic environments, be self-motivated to take on challenging problems to deliver big customer impact, moving fast to ship solutions and then iterating on user feedback and interactions. Key job responsibilities As a Senior Applied Scientist, you will leverage your technical expertise and experience to demonstrate leadership in tackling large complex problems, setting the direction and collaborating with other talented applied scientists and engineers to research and develop LLM modeling and engineering techniques to reduce friction and enable natural and contextual conversations. You will analyze, understand and improve user experiences by leveraging Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in artificial intelligence. You will work on core LLM technologies, including Prompt Engineering, Model Fine-Tuning, Reinforcement Learning from Human Feedback (RLHF), Evaluation, etc. Your work will directly impact our customers in the form of novel products and services .
US, CA, Pasadena
The Amazon Web Services (AWS) Center for Quantum Computing (CQC) is a multi-disciplinary team of scientists, engineers, and technicians, on a mission to develop a fault-tolerant quantum computer. We are looking to hire a Research Scientist with fabrication and data analysis experience working on all elements of a superconducting circuit. The position is on-site at our lab, located on the in Pasadena, CA. The ideal candidate will have had prior experience building software tools for data analysis and visualization to enable deep diving into fabrication details, electrical test data. We are looking for candidates with strong engineering principles, resourcefulness and data science experience. Organization and communication skills are essential. Key job responsibilities * Develop and automate data pipeline pertinent to superconducting device fabrication. * Develop analytical tools to uncover new information about established and new processes. * Develop new or contribute to modifying existing data visualization tools. * Utilize machine learning to enable better deeper dives into fabrication and related data. * Interface with various software, design, fabrication and electrical test teams to enable new functionalities. A day in the life The role will be vital to the fabrication team and quantum computing device integration mechanism. The candidate will develop software based analytical tools to enable data driven decisions across projects related to fabrication and supporting infrastructure. Each fabrication run delivers additional data. The candidate will stay close to the details of fabrication providing data analysis and quick feedback to key stakeholders. At the end of fabrication runs custom and standardized reports will be generated by the candidate to provide insights into data generated from the run. This position may require occasional weekend work. About the team AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices.
US, WA, Seattle
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Buyer Risk Prevention (BRP) Machine Learning group. We are looking for a talented scientist who is passionate to build advanced algorithmic systems that help manage safety of millions of transactions every day. Key job responsibilities Use machine learning and statistical techniques to create scalable risk management systems Learning and understanding large amounts of Amazon’s historical business data for specific instances of risk or broader risk trends Design, development and evaluation of highly innovative models for risk management Working closely with software engineering teams to drive real-time model implementations and new feature creations Working closely with operations staff to optimize risk management operations, Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation Tracking general business activity and providing clear, compelling management reporting on a regular basis Research and implement novel machine learning and statistical approaches
CA, ON, Toronto
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! As an Applied Scientist on this team, you will: - Drive end-to-end Machine Learning projects that have a high degree of ambiguity, scale, complexity. - Perform hands-on analysis and modeling of enormous data sets to develop insights that increase traffic monetization and merchandise sales, without compromising the shopper experience. - Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production; work closely with software engineers to assist in productionizing your ML models. - Run A/B experiments, gather data, and perform statistical analysis. - Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. - Research new and innovative machine learning approaches. - Recruit Applied Scientists to the team and provide mentorship. Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Team video https://youtu.be/zD_6Lzw8raE
US, WA, Seattle
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! As the Data Science Manager on this team, you will: - Lead of team of scientists, business intelligence engineers, etc., on solving science problems with a high degree of complexity and ambiguity. - Develop science roadmaps, run annual planning, and foster cross-team collaboration to execute complex projects. - Perform hands-on data analysis, build machine-learning models, run regular A/B tests, and communicate the impact to senior management. - Hire and develop top talent, provide technical and career development guidance to scientists and engineers in the organization. - Analyze historical data to identify trends and support optimal decision making. - Apply statistical and machine learning knowledge to specific business problems and data. - Formalize assumptions about how our systems should work, create statistical definitions of outliers, and develop methods to systematically identify outliers. Work out why such examples are outliers and define if any actions needed. - Given anecdotes about anomalies or generate automatic scripts to define anomalies, deep dive to explain why they happen, and identify fixes. - Build decision-making models and propose effective solutions for the business problems you define. - Conduct written and verbal presentations to share insights to audiences of varying levels of technical sophistication. Why you will love this opportunity: Amazon has invested heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Team video ~ https://youtu.be/zD_6Lzw8raE
US, WA, Seattle
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! As an Applied Science Manager in Machine Learning, you will: - Directly manage and lead a cross-functional team of Applied Scientists, Data Scientists, Economists, and Business Intelligence Engineers. - Develop and manage a research agenda that balances short term deliverables with measurable business impact as well as long term investments. - Lead marketplace design and development based on economic theory and data analysis. - Provide technical and scientific guidance to team members. - Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, making use of both quantitative and business judgment - Advance the team's engineering craftsmanship and drive continued scientific innovation as a thought leader and practitioner. - Develop science and engineering roadmaps, run annual planning, and foster cross-team collaboration to execute complex projects. - Perform hands-on data analysis, build machine-learning models, run regular A/B tests, and communicate the impact to senior management. - Collaborate with business and software teams across Amazon Ads. - Stay up to date with recent scientific publications relevant to the team. - Hire and develop top talent, provide technical and career development guidance to scientists and engineers within and across the organization. Why you will love this opportunity: Amazon is investing heavily in building a world-class advertising business. This team defines and delivers a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are a highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth: You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Team video ~ https://youtu.be/zD_6Lzw8raE
US, WA, Seattle
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Buyer Risk Prevention (BRP) Machine Learning group. We are looking for a talented scientist who is passionate to build advanced algorithmic systems that help manage safety of millions of transactions every day. Key job responsibilities Use machine learning and statistical techniques to create scalable risk management systems Learning and understanding large amounts of Amazon’s historical business data for specific instances of risk or broader risk trends Design, development and evaluation of highly innovative models for risk management Working closely with software engineering teams to drive real-time model implementations and new feature creations Working closely with operations staff to optimize risk management operations, Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation Tracking general business activity and providing clear, compelling management reporting on a regular basis Research and implement novel machine learning and statistical approaches
US, CA, San Diego
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Buyer Risk Prevention (BRP) Machine Learning group. We are looking for a talented scientist who is passionate to build advanced algorithmic systems that help manage safety of millions of transactions every day. Key job responsibilities Use machine learning and statistical techniques to create scalable risk management systems Learning and understanding large amounts of Amazon’s historical business data for specific instances of risk or broader risk trends Design, development and evaluation of highly innovative models for risk management Working closely with software engineering teams to drive real-time model implementations and new feature creations Working closely with operations staff to optimize risk management operations, Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation Tracking general business activity and providing clear, compelling management reporting on a regular basis Research and implement novel machine learning and statistical approaches
US, WA, Seattle
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Buyer Risk Prevention (BRP) Machine Learning group. We are looking for a talented scientist who is passionate to build advanced algorithmic systems that help manage safety of millions of transactions every day. Key job responsibilities Use machine learning and statistical techniques to create scalable risk management systems Learning and understanding large amounts of Amazon’s historical business data for specific instances of risk or broader risk trends Design, development and evaluation of highly innovative models for risk management Working closely with software engineering teams to drive real-time model implementations and new feature creations Working closely with operations staff to optimize risk management operations, Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation Tracking general business activity and providing clear, compelling management reporting on a regular basis Research and implement novel machine learning and statistical approaches
US, CA, San Diego
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Buyer Risk Prevention (BRP) Machine Learning group. We are looking for a talented scientist who is passionate to build advanced algorithmic systems that help manage safety of millions of transactions every day. Key job responsibilities Use machine learning and statistical techniques to create scalable risk management systems Learning and understanding large amounts of Amazon’s historical business data for specific instances of risk or broader risk trends Design, development and evaluation of highly innovative models for risk management Working closely with software engineering teams to drive real-time model implementations and new feature creations Working closely with operations staff to optimize risk management operations, Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation Tracking general business activity and providing clear, compelling management reporting on a regular basis Research and implement novel machine learning and statistical approaches