An overhead shot inside an Amazon fulfillment center shows hundreds of boxes on conveyor belts along with people monitoring the flow of those packages
Amazon's scale makes picking the right package for each product a challenge. Fortunately, machine learning approaches — particularly deep learning — thrive on big data and massive scale. These tools have helped Amazon reduce per-shipment packaging weight by 36% and eliminate more than a million tons of packaging.

How pioneering deep learning is reducing Amazon’s packaging waste

A combination of deep learning, natural language processing, and computer vision enables Amazon to hone in on the right amount of packaging for each product.

Finding the right amount of packaging to ship an item can be challenging — and at Amazon, an ever-changing catalog of hundreds of millions of products makes it an ongoing challenge. In addition, Amazon’s scale also means it is impossible to solve this challenge using manual inspection to choose packaging for each and every item. For the same reason, general packaging rules and run-of-the-mill logic just won’t cut it. What’s required is a cutting-edge-smart automated mechanism that can adapt on the fly to changing circumstances.

Prasanth Meiyappan, top right, an applied scientist, and Matthew Bales, a research science manager, authored "Reducing Amazon’s packaging waste using multimodal deep learning". Their position paper was one of the 10 most read research papers on Amazon Science in 2021.

Fortunately, machine learning approaches — particularly deep learning — thrive on big data and massive scale, and a pioneering combination of natural language processing and computer vision is enabling Amazon to hone in on using the right amount of packaging. These tools have helped Amazon drive change over the past six years, reducing per-shipment packaging weight by 36% and eliminating more than a million tons of packaging, equivalent to more than 2 billion shipping boxes.

“When I started at Amazon in 2017, we had a lot of physical testing of products going on, but not a scalable mechanism that could assess hundreds of millions of products to identify the optimal packaging type for each product,” says research science manager Matthew Bales. Bales, who is also a physicist, heads up machine learning within Amazon’s Customer Packaging Experience team.

“Statistical tests were the first piece, but they are essentially only useful when products have already been shipped in more than one package type. We wanted the capability to predict how a product would fare in a less-protective, lighter, and more sustainable package type. And once you're in that predictive space, you need machine learning,” Bales explains.

The power of customer feedback

To make a prediction about whether a given product could be safely shipped in a particular package type, Bales and his colleagues built a ML model based largely on the text-based data that customers find on the Amazon Store — the item name, description, price, package dimensions, and so on.

Related content
As office buildings become smarter, it is easier to configure them with sustainability management in mind.

The model was trained on millions of examples of products successfully delivered in various packaging types, and on examples of products that arrived damaged in given packaging types. Amazon has access to almost real-time feedback when a product is not sufficiently protected by its packaging, because customers report it via the Online Returns Center and other forms of feedback, including product reviews.

“Customer feedback is paramount,” says Bales. “It powers all of our statistical testing.”

The model learned that certain keywords were particularly important when making packaging decisions. For example, keywords that indicated that a padded mailer would not be the right packaging included “ceramic”, “grocery”, “mug” and “glass”. These products were better shipped in a box. Keywords that suggested mailers were the right choice included “multipack” and “bag.” Those indicated the product might already have some form of protective packaging.

“The portion of the model that's learning from the Amazon Store has learned really well what the product is, and about its dimensions,” says Bales.

Reducing Amazon’s packaging waste using multimodal deep learning

It’s an important step in the journey, but automatically learning what a product is represents only half the battle. Equally important is how the vendor packaged the product before sending it to a fulfillment center. For example, a ceramic mug may be packaged in clear plastic bag, or in a sturdy box.

To identify product packaging at scale, computer vision needed to be deployed. The ML team already knew that the product images on the Amazon Store weren’t helpful when selecting packaging. For example, a multipack of LED bulbs might be illustrated by a picture of a single, unpacked bulb, suggesting it is fragile, yet the multipack is, in fact, safely packaged by the vendor and doesn’t require additional packaging. It is best shipped in its own container.

Bales’s team addressed this challenge by using Amazon’s own image data. When products are delivered to fulfillment centers, many are sent via conveyor belt through special computer-vision tunnels equipped with cameras that capture images of the products from multiple angles. These tunnels are used for many things, including ascertaining product dimensions and spotting defects.

Prasanth Meiyappan, an Amazon applied scientist, expanded the training of the team’s ML model to include these standardized product images in addition to the text classifiers from the catalog — a multimodal approach.

Our model detects the packaging edges to determine shape, identifies a perforation, a bag around the product, or light shining through a glass bottle.
Prasanth Meiyappan

“Our model detects the packaging edges to determine shape, identifies a perforation, a bag around the product, or light shining through a glass bottle.” Meiyappan explains. But to some extent, how the model makes its judgement about what it detects in images is hard for a human to discern, because the product features identified and weighted by the model tend to be complex.

“The important thing,” Bales notes, “is that the packaging decisions generated by the model are empirically accurate.”

Incorporating both text-based and visual data improved the ML model’s performance by as much as 30%, compared with using text-based data alone. Bales and Meiyappan have produced a position paper describing their work.

“When the model is certain of the best package type for a given product, we allow it to auto-certify it for that pack type,” says Bales. “When the model is less certain, it flags a product and its packaging for testing by a human.” The technology is currently being applied to product lines across North America, Europe, and Japan — automatically reducing waste at a growing scale.

“It’s a triple win,” says Bales. “Reduced waste, increased customer satisfaction, and lower costs.”

Balancing act

To arrive at this triple win, though, the team also had to take on a thorny challenge encountered frequently in the ML domain: class imbalance. In a nutshell, the problem is this: if you want an ML model to learn effectively, you ideally provide it with as many examples of failures as successes, so it can learn to differentiate effectively between the two.

The data used to train the model had many millions of examples of product/package pairings, yet depending on the package type, as little as 1% of those examples were for packages that turned out to be unsuitable in some way for the product within.

The machine learning literature to do with packaging is pretty sparse. Not many people deal with the kind of datasets we are dealing with in the packaging domain.
Prasanth Meiyappan

“Prior to implementing ML, we’ve shipped some product in envelopes and mailers for some time,” says Bales. “So, we had loads of examples of things that were good in mailers, but didn't have a lot of examples of things that were bad in mailers. ML models have problems with this kind of overwhelming imbalance.”

“The machine learning literature to do with packaging is pretty sparse,” Meiyappan says. “Not many people deal with the kind of datasets we are dealing with in the packaging domain. How effective a technique is in dealing with dataset imbalance is both domain and dataset specific.”

Thus the team’s approach to the class imbalance problem was primarily experimental. And of the six approaches they applied — four data based, two algorithm based — the clear winner produced a marked improvement in model accuracy. That was a data-based approach called two-phase learning with random under sampling which focuses the model on the minority class in the first phase of training and then on all of the data in the second. “In our position paper we share that knowledge with the ML community,” says Bales, “so that anyone who encounters a similar problem might choose to try this approach for themselves, to see if it also works in their problem space.”

What’s next

The team said they are eager to expand the use of this tool by training the model to understand all Amazon’s customers languages while also incorporating the unique aspects of fulfilment in each country.

Read the Amazon Sustainability Report

Amazon is committed to building a sustainable business for customers and the planet. Learn more about Amazon's goals, strategies, and policies in the Amazon Sustainability Report.

While Amazon scientists continue to research other ways to utilize machine learning to eliminate waste, the company is also working to reduce packaging waste throughout the e-commerce supply chain. Amazon is, for example, increasingly incentivizing its vendors to create optimized e-commerce packaging for themselves that saves space and materials without compromising product protection.

The company’s Shipment Zero goal is to deliver 50% of shipments with net-zero carbon by 2030, which from a packaging perspective means shipping products without added Amazon packaging or in carbon-neutral packaging. This is part of the Amazon’s wider Climate Pledge — a commitment to reach net-zero carbon by 2040, a decade earlier than the 2050 emissions target of the Paris Agreement.

Related content

CN, Shanghai
亚马逊云科技上海人工智能实验室OpenSearch 研发团队正在招募应用科学家实习,方向是机器学习。OpenSearch是一个开源的搜索和数据分析套件, 它旨在为数据密集型应用构建解决方案,内置高性能、开发者友好的工具,并集成了强大的机器学习、数据处理功能,可以为客户提供灵活的数据探索、丰富和可视化功能,帮助客户从复杂的数据中发现有价值的信息。OpenSearch是现有AWS托管服务(AWS OpenSearch)的基础,OpenSearch核心团队负责维护OpenSearch代码库,他们的目标是使OpenSearch安全、高效、可扩展、可扩展并永远开源。这是一个为期3个月到6个月的实习机会,旨在让你真正体验软件开发的全流程,提升实际工作能力。如果你对这个职位感兴趣,欢迎投递简历! 该实习有转正机会。 点击下方链接查看申请手册获得更多信息: https://amazonexteu.qualtrics.com/CP/File.php?F=F_55YI0e7rNdeoB6e Key job responsibilities 在这个实习期间,你将有机会: 1. 应用先进的人工智能和机器学习技术提升用户体验。 2.研发先进的机器学习检索算法,了解机器学习算法如何与工程结合。 3. 学习亚马逊云上的各种云服务。 4. 参与产品需求讨论,提出技术实现方案。 5. 与国内外杰出的开发团队紧密合作,学习代码开发和审查的流程。
US, WA, Seattle
The Worldwide Defect Elimination (WWDE) Team is seeking a highly skilled economist to estimate the customer impact of each Customer Service action. Your analysis will assist teams across Amazon to prioritize defect elimination efforts and optimize how we respond to customer contacts. You will partner closely with our product, program, and engineering teams to deliver your findings to users via systems and dashboards that guide Customer Service planning and policies. Key job responsibilities - Develop causal, economic, and machine learning models at scale. - Engage in economic analysis; raise the bar for research. - Inform strategic discussions with senior leaders across the company to guide policies. A day in the life We thrive on solving challenging problems to innovate for our customers. By pushing the boundaries of technology, we create unparalleled experiences that enable us to rapidly adapt in a dynamic environment. Our decisions are guided by data, and we collaborate with engineering, science, and product teams to foster an innovative learning environment. If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: * Medical, Dental, and Vision Coverage * Maternity and Parental Leave Options * Paid Time Off (PTO) * 401(k) Plan About the team The WWDE team's mission is to understand and resolve all issues impacting customers and connect all organizations in Amazon to customer experiences. Our vision is to be the ultimate steward of the Voice of the Customer (VoC), empowering CS and Amazon teams to easily measure, listen, and act on customer feedback. The team broadly supports defect detection, root cause identification, and resolution to earn customer trust. The Customer Service Economics & Optimization team is a force multiplier within this group. Through causal analysis, we estimate the effectiveness of our efforts to delight the customer
AE, Dubai
Amazon launched the Generative AI Innovation Center (GenAIIC) in June 2023 to help AWS customers accelerate the use of generative AI to solve business and operational problems and promote innovation in their organization. This is a team of strategists, data scientists, engineers, and solution architects working step-by-step with customers to build bespoke solutions that harness the power of generative AI. (https://press.aboutamazon.com/2023/6/aws-announces- generative-ai-innovation-center). We’re looking for Data Scientists to use generative AI and other techniques to design, evangelize, and implement state-of-the-art solutions for never-before-solved problems. You will work directly with customers and innovate in a fast-paced organization that contributes to game-changing projects and technologies. You will design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. As an early-in-career joiner, you will initially join our A2C (Associate to Consultant) program for intensive training on AWS technology and delivery approach. Emirati nationality is required. Key job responsibilities As a Data Scientist, you will - Collaborate with AI/ML scientists, engineers, and architects to Research, design, develop, and evaluate cutting-edge generative AI algorithms to address real-world challenges - Interact with customers directly to understand the business problem, help and aid them in implementation of generative AI solutions, deliver briefing and deep dive sessions to customers and guide customer on adoption patterns and paths to production - Create and deliver best practice recommendations, tutorials, blog posts, sample code, and presentations adapted to technical, business, and executive stakeholder - Provide customer and market feedback to Product and Engineering teams to help define product direction About the team The team helps customers imagine and scope the use cases that will create the greatest value for their businesses, select and train or fine tune the right models, define paths to navigate technical or business challenges, develop proof-of-concepts, and make plans for launching solutions at scale. The Generative AI Innovation Center team provides guidance on best practices for applying generative AI responsibly and cost efficiently. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
US, CA, Sunnyvale
The Amazon Artificial General Intelligence (AGI) Personalization team is looking for a passionate, highly skilled and inventive Applied Scientist with strong machine learning background to build state-of-the-art ML systems for personalizing large-scale, high-quality conversational assistant systems. As a Applied Scientist, you will play a critical role in driving the development of personalization techniques enabling conversational systems, in particular those based on large language models, information retrieval, recommender systems and knowledge graph, to be tailored to customer needs. You will handle Amazon-scale use cases with significant impact on our customers' experiences. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized assistant systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for contextual knowledge extraction and information retrieval, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, compute, latency and quality - Research in advanced customer understanding and behavior modeling techniques - Collaborate with cross-functional teams of scientists, engineers, and product managers to identify and solve complex problems in personal knowledge aggregation, processing, modeling, and verification - Design and execute experiments to evaluate the performance of state-of-the-art algorithms and models, and iterate quickly to improve results - Think Big on conversational assistant system personalization over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports About the team The AGI Personalization org uses various contextual signals to personalize Large Language Model output for our customers while maintaining privacy and security of customer data. We work across multiple Amazon products, including Alexa, to enhance the user experience by bringing more personal context and relevance to customer interactions.
US, WA, Seattle
If you are excited about applying your science and engineering skills in business problems in the space of risk measurement, quantification, and mitigation, we invite you to consider this Applied Scientist opportunity within Amazon B2B Payment and Lending (ABPL). ABPL is seeking an Applied Scientist who combines their scientific and technical expertise with business intuition to build flexible, performant, and global solutions for complex financial and risk problems. You will develop and deploy production models to enhance our product features & processes that will delight our customers. Key job responsibilities - Apply advanced machine learning, deep learning and other analytical/scientific techniques to enable and improve Credit Management decisions - Source and assess various structured and unstructured data and leverage automated modeling framework to streamline data evaluation and integration - Spearhead leader to research and adopt State-of-the-Art AI/ML techniques and define the roadmap to revolutionize underwriting models leveraging adaptive modelling methods, Large Language Models(LLM), etc. - Bar-raising the design and implementation of production model pipelines(real time and batch) , lead design and code reviews to insist on high bar of engineering excellence and ensure high performance of the models - Collaborate effectively with Credit Strategy, Operations, Product, data and engineering teams. You will be advising and educating the leadership and stakeholders of the models and strategic decision making. - Understand business and product strategies, goals and objectives. Make recommendations for new techniques/strategies to improve customer outcomes. A day in the life As an Applied Scientist, you will design and build systems that support financial products. You will work closely with business partners, software and data engineers to build and deploy scalable solutions that deliver exceptional value for our customers. You will utilize intellectual and technical capabilities, problem solving and analytical skills, and excellent communication to deliver customer value. You will partner with product and operations management to launch new, or improve existing, financial products within Amazon.
US, WA, Seattle
Do you want to join an innovative team of scientists who use deep learning, natural language processing, large language models to help Amazon provide the best seller experience across the entire Seller life cycle, including recruitment, growth, support and provide the best customer and seller experience by automatically mitigating risk? Do you want to build advanced algorithmic systems that help manage the trust and safety of millions of customer interactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Are you excited by the opportunity to leverage GenAI and innovate on top of the state-of-the-art large language models to improve customer and seller experience? Do you like to build end-to-end business solutions and directly impact the profitability of the company? Do you like to innovate and simplify processes? If yes, then you may be a great fit to join the Machine Learning Accelerator team in the Amazon Selling Partner Services (SPS) group. Key job responsibilities The scope of an Applied Scientist II in the SPS Machine Learning Accelerator (MLA) team is to research and prototype Machine Learning applications that solve strategic business problems across SPS domains. Additionally, the scientist collaborates with project leaders, engineers and business partners to design and implement solutions at scale. The scientist focuses on components of large-scale projects, systems and products and can work independently and with the team to deliver successful solutions with medium to large business impact. The scientist helps our team evolve by actively participating in discussions, team planning, and by staying current on the latest techniques arising from both the scientist community in SPS, the larger Amazon-wide community, and beyond. The scientist develops and introduces tools and practices that streamline the work of the team, and he mentors junior team members and participates in hiring.
US, MA, Boston
The Artificial General Intelligence (AGI) Information team is looking for a passionate, and talented Senior Applied Scientist with a strong experience in cutting-edge LLM technologies. We are a hybrid team of scientists and engineers focused on building visuals (composed of graphics, images, text, etc.) that are integrated into conversational question answering experiences. Key research areas for the team include agentic workflows and code generation capabilities of large language models (LLMs) for generation of visual responses to questions. If you are deeply familiar with LLMs, natural language processing, and machine learning and have experience working in high-performing research teams, this may be the right opportunity for you. Our fast-paced environment requires a high degree of independence in making decisions and driving ambitious research agendas all the way to production. You will work with other science and engineering teams as well as business stakeholders to maximize the velocity and impact of your contributions. Key job responsibilities - Leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in generative artificial intelligence (GenAI). - Work with talented peers to lead the development of novel algorithms and modeling techniques to advance the state of the art with LLMs. - Collaborate with other science and engineering teams as well as business stakeholders to maximize the velocity and impact of your contributions. About the team It's an exciting time to be a leader in AI research. In Amazon's AGI Information team, you can make your mark by improving information-driven experiences of Amazon customers worldwide. Your work will directly impact our customers in the form of products and services that make use of language and multimodal technology!
US, WA, Seattle
The PXT (People Experience and Technology) Analytics team for AIGC (Ads, IMDb and Grand Challenge) is seeking a highly skilled and motivated Research Scientist to join our team. You will be an integral part of the Research Science space to support the AIGC PXT org initiatives. If you enjoy innovating, thinking big and want to contribute directly to the success of a growing team, you may be a prime candidate for this position. Key job responsibilities Design experiments, test hypotheses, and build actionable models. Conduct quantitative analyses of talent management data and trends. Conduct qualitative data collection and analysis. Partner closely and drive effective collaborations across multi-disciplinary research and product teams. Consult on appropriate analytic methodologies and scope research requests. Write comprehensive reports that summarize research methodology, results, and insights for both business and technical audiences.
US, CA, Sunnyvale
Amazon's AGI Web & Knowledge Services group is seeking a passionate, talented, and inventive Applied Scientist to lead the development of industry-leading Information retrieval systems. As part of our cutting-edge AGI-IR team, you will play a pivotal role in developing efficient AI solutions for a multi-modal future at scale. In this role, you will work alongside renowned researchers and engineers to enable our customers to seamlessly interact with unstructured and semi-structured content through advanced capabilities like question answering, contextual search, and multi-turn dialogues. Your work will directly impact our customers in the form of products and services that make use of various machine learning, deep learning and language model technologies. Key job responsibilities As an Applied Scientist, you will leverage your technical expertise and experience to demonstrate leadership in tackling large complex problems, setting the direction and collaborating with applied scientists and engineers to develop novel algorithms and modeling techniques to enable timely, relevant and delightful conversations. - Leverage Amazon's large-scale data and computing resources to accelerate advances in the state of the art. - Work backwards from customer needs and use that information to make trade-offs between different modeling approaches - Collaborate with software engineering teams to integrate successful experimental results into complex Amazon production systems - Report results to technical and business audiences in a manner that is statistically rigorous and compellingly relevant, exemplifying good scientific practice in a business environment - Drive best practices, helping to set high scientific and engineering standards on the team - Promote the culture of experimentation and applied science at Amazon
US, WA, Seattle
Amazon Prime’s Science organization leads the Research & Development towards innovation for Amazon Prime. Amazon Prime is the backbone of Amazon’s consumer business and aspires to be the world’s most engaging, satisfying, and loved membership program, driving growth and profitability. The program serves over 200 million members across 25 countries and is key to Amazon’s customer growth and engagement. Prime Science innovates in Artificial Intelligence and Economics, to develop algorithms and systems for automated marketing, personalization, targeting, and decisioning. To lead Prime’s growth and impact through science and innovation, we are seeking a Principal Applied Scientist, Prime Science, to create and deliver Machine Learning innovation both for customer facing and internal solutions. To be successful in this role you should be recognized as an industry expert who has experience defining a long-term science vision, driven fundamentally from the needs of customers, translating that direction into specific plans for research scientists, applied scientists, engineering, and product teams. This is a role that combines science leadership, organizational ability, technical expertise, product focus, and business understanding. Prime is committed to diversity, equity, and inclusion and recognizes that our ability to serve our members is contingent on having diverse backgrounds, ideas, and ways of working on our team. This leader plays a key role as a member of the Prime worldwide leadership team to set the right organizational vision, standards, and mechanisms to ensure Prime is the best place for top talent to thrive. Key job responsibilities - Own and drive the most complex and strategic solutions across the business; responsible for many billions in revenue. - Act as a thought leader and forward thinker, anticipating obstacles to success and helping avoid common failure modes. - Research, build, and deploy innovative machine learning solutions; working across all technical disciplines. - Partner with science, engineering, and product leaders across the business to identify and deliver innovative machine solutions; identify new opportunities, define the vision and execute solutions that continually delight customers. - Hire, mentor, and guide senior scientists; partner with engineering leaders to build efficient and scalable solutions.