This image is overlaid with graphics and labels showing an example of instance segmentation as it applies to people eating at a barbecue, there are labels for person, bowl, cup, and knife
Object instance segmentation, a research field embraced by ARA recipient Yong Jae Lee, is the ability of a CV model to not only detect that there are objects in an image, but also to accurately locate and classify each object of interest, such as a person, bowl, cup, or knife.
Courtesy of Yong Jae Lee

How Yong Jae Lee is advancing the cutting edge of computer vision research

University of Wisconsin-Madison associate professor and Amazon Research Award recipient has authored a series of pioneering papers on real-time object instance segmentation.

Making sense of our kaleidoscopic visual world has been a decades-long grand challenge for computer scientists. That’s because there’s so much more to vision than mere seeing. To make the most out of machines, and ultimately have them move usefully and safely among us, they must understand what is happening around them with a superhuman degree of confidence.

The knowledge humans bring to every scene we encounter is what imbues that scene with meaning and enables us to respond appropriately. In the early days of computer vision (CV), artificial intelligence systems could only learn to discern via training on huge numbers of example images painstakingly annotated by humans — a process known as supervised learning.

Yong Jae Lee, associate professor at the University of Wisconsin-Madison, is seen standing outside on a sunny day, smiling into the camera -- there are trees and plants in the background
Yong Jae Lee, associate professor at the University of Wisconsin-Madison, received a 2019 ARA award for his research into real-time object instance segmentation.
Courtesy of Yong Jae Lee

When electrical engineering undergrad Yong Jae Lee first got hooked on the CV challenge, about 15 years ago, supervised learning reigned supreme. Back then, to teach a CV system how to spot a cat, you had to show it thousands of pictures of cats, with a box painstakingly drawn around each feline and labelled “cat”.

In this way, it could learn the constellation of features that makes felines uniquely identifiable. The idea that a CV system could learn to pick out the many important features of the visual world with little or no help from pre-labelled data felt so distant and difficult, even attempting it felt borderline pointless to many in the field.

Computer vision and the natural world
Amazon Machine Learning Research Award recipient utilizes a combination of people and machine learning models to illuminate the planet's incredible biodiversity.

But Lee, now an associate professor at the University of Wisconsin-Madison, felt strongly even back then that the future of CV lay in unsupervised, or weakly supervised learning.

The idea for this form of machine learning (ML) is that a CV model takes in large amounts of largely unlabelled images and works out for itself how to distinguish between many different classes of objects contained within them, from cats, dogs and fleas, to people, cars and trees.

Computer vision at Amazon
Why multimodal identification is a crucial step in automating item identification at Amazon scale.

“Back then, unsupervised learning was not popular, but I had no doubt it was the right problem to work on,” says Lee. “Now, I think almost the entire community believes in this direction. Huge progress is being made.”

This shift towards unsupervised (aka self-supervised) learning was brought about by the deep learning revolution, says Lee. In this paradigm, ML algorithms have been developed that can extract pertinent information from enormous amounts of raw, unlabelled data. This learning has been likened to how babies learn about the world, albeit on digital timescales.

The blistering rate of success of deep learning means the content of Lee’s graduate teaching evolves from one semester to the next.

“The state of the art this month will no longer be so next month,” he says. “There are frequent surprises, and paradigm shifts every few years. It’s a lot to navigate, but an exciting time for students.”

This image is overlaid with graphics and labels showing an example of instance segmentation as it applies to cars and trucks on a road, there are cones and there is a person, also labeled, in the foreground directing traffic
With instance segmentation, the model differentiates between objects of the same class, eg cars or trucks, by clearly segmenting each “instance” of that class of object.
Courtesy of Yong Jae Lee

When he’s not teaching, Lee is pushing the boundaries of both supervised and self-supervised approaches to CV. In 2019 he received an Amazon Machine Learning Research Award (now known as Amazon Research Awards), in part to support a series of pioneering papers on real-time object instance segmentation.

Object instance segmentation goes a lot further than visual object detection: it is the ability of a CV model to not only detect that there are objects somewhere in an image, but also to accurately locate and classify each object of interest — be that a chair, human, or plant — and delineate its visual boundary within the image.

With instance segmentation, not only is every pixel in an image attributed to a class of object, the model also differentiates between two objects of the same class by clearly segmenting each “instance” of that class of object.

The challenge in 2019: although this instance segmentation task could be done to a high standard when applied to individual images, no system could yet hit high-accuracy benchmarks when applied to real-time streaming video (defined as 30 frames per second or above).

Yong Jae Lee at CVPR 2019

It is important for CV systems to comprehend visual scenes at speed because a range of burgeoning technologies depend on such an ability, from driverless cars to autonomous warehouse robots.

Lee, then at the University of California, Davis, and his students Daniel Bolya, Chong Zhou, and Fanyi Xiao, not only developed the first model to attain such accuracy at speed, but also managed achieve it by training their model on just one GPU.

Their supervised system, called YOLACT (You Only Look At CoefficienTs), was lean and mean. It was fast because the researchers had developed a novel way to run aspects of the instance segmentation task in parallel rather than relying on slower, sequential processing. YOLACT won the Most Innovative Award at the COCO Object Detection Challenge at the International Conference on Computer Vision in 2019.

Since then, Lee’s team has gone on to markedly improve the efficiency and performance of the system, and the latest version of YOLACT called YolactEdge (built with students Haotian Liu, Rafael Rivera-Soto, and Fanyi Xiao) can be carried in a device no bigger than your hand. And by making the YOLACT code available on GitHub, Lee has put the system into many people’s hands.

YOLACT: Real-Time Instance Segmentation [ICCV Trailer]

“It’s had a big impact. I know there are a lot of people using YOLACT, and at least one start-up,” says Lee. “This is not some intellectual exercise. We’re creating systems with real-world value. For me, that’s a tremendously exciting feeling.”

In another branch of Lee’s work, also supported by his Amazon award, he pioneers new approaches to ML-based image generation. One example of another research first is MixNMatch, a minimal-supervision model that, when supplied with many real images, teaches itself to differentiate between a variety of important image attributes. By learning to distinguish between an object’s shape, pose, texture/colour and background, the system can employ fine-tuned control to generate new images with any desired combination of attributes.

mixnmatch.png
MixNMatch disentangles and encodes four factors from real images — object pose, shape, texture and background — and combines them to generate new images. Each image in the row of images is a combination of the attributes taken from the four images above it.

Lee continues to build on such work. This year he and his current and former students (Yang Xue, Yuheng Li, and Krishna Kumar Singh) unveiled GIRAFFE HD, a high-resolution generative model that is 3D aware.

This means it can, among other things, coherently rotate, move and scale foreground objects in a scene while independently generating the appropriate background. It is a design tool of enormous power with a near human-like grasp of how an image can be realistically, and seamlessly, transformed.

“As a user, you can tune different ‘knobs’ to change the generated image in highly controllable ways, such as the pose of objects and even the [virtual] camera elevation,” says Lee.

The depth of visual understanding required by such models is too big to depend on supervised learning, he adds.

Mitigating bias
Eliminating the need for annotation makes bias testing much more practical.

“If we want to create systems that can truly absorb all of the visual information that, say, a human will absorb in their lifetime, it's just not going to be feasible for us to curate that kind of dataset,” says Lee.

Nor is it feasible to develop such technology without significant computational resources, which is why Lee’s Amazon award included credits for Amazon Web Services.

“What was particularly beneficial to our lab was Amazon’s EC2 [Elastic Compute Cloud]. At crunch times, when we needed to run lots of different experiments, we could do that in parallel. The scalability and availability of machines on EC2 has been tremendously helpful for our research.”

While Lee is clearly energized by many aspects of vision research, he sees one looming downside: the massive influx of AI-generated art being published online.

“The state of the art now is to learn directly from internet data,” he says. “If that data becomes populated with lots of ML outputs, you’re not actually learning from so-called true knowledge, but instead learning from ‘fake’ information. It isn’t clear how this will affect the training of future models.”

But he remains optimistic about the rate of progress. The semantic understanding already being demonstrated by image-generation systems is surprising, he says.

“Take Dalle-2’s horse-rising astronaut. This kind of semantic concept doesn't really exist in the real world, right, but these systems can construct plausible images of exactly that.”

The takeaway lesson from this is that the power of data is hard to deny, says Lee. Even if the data is ‘noisy’, having enormous amounts of it allows ML models to develop a very deep understanding of the visual world, resulting in creative combinations of semantic concepts.

“Even for somebody working in this field, I still find it fascinating.”

What advice does Lee have for students looking to branch into his dynamic field?

“There is so much activity in this machine learning space, what's really important is to find the topics you're really passionate about, and get some hands-on experience,” says Lee. “Don't just read a paper and then presume you know what you need to know. The best way to learn is to download some cutting-edge open-source code and really play around with it. Have some fun!”

Research areas

Related content

US, CA, Santa Clara
About Amazon Health Amazon Health’s mission is to make it dramatically easier for customers to access the healthcare products and services they need to get and stay healthy. Towards this mission, we (Health Storefront and Shared Tech) are building the technology, products and services, that help customers find, buy, and engage with the healthcare solutions they need. Job summary We are seeking an exceptional Applied Scientist to join a team of experts in the field of machine learning, and work together to break new ground in the world of healthcare to make personalized and empathetic care accessible, convenient, and cost-effective. We leverage and train state-of-the-art large-language-models (LLMs) and develop entirely new experiences to help customers find the right products and services to address their health needs. We work on machine learning problems for intent detection, dialogue systems, and information retrieval. You will work in a highly collaborative environment where you can pursue both near-term productization opportunities to make immediate, meaningful customer impacts while pursuing ambitious, long-term research. You will work on hard science problems that have not been solved before, conduct rapid prototyping to validate your hypothesis, and deploy your algorithmic ideas at scale. You will get the opportunity to pursue work that makes people's lives better and pushes the envelop of science. Key job responsibilities - Translate product and CX requirements into science metrics and rigorous testing methodologies. - Invent and develop scalable methodologies to evaluate LLM outputs against metrics and guardrails. - Design and implement the best-in-class semantic retrieval system by creating high-quality knowledge base and optimizing embedding models and similarity measures. - Conduct tuning, training, and optimization of LLMs to achieve a compelling CX while reducing operational cost to be scalable. A day in the life In a fast-paced innovation environment, you work closely with product, UX, and business teams to understand customer's challenges. You translate product and business requirements into science problems. You dive deep into challenging science problems, enabling entirely new ML and LLM-driven customer experiences. You identify hypothesis and conduct rapid prototyping to learn quickly. You develop and deploy models at scale to pursue productizations. You mentor junior science team members and help influence our org in scientific best practices. About the team We are the ML Science and Engineering team, with a strong focus on Generative AI. The team consists of top-notch ML Scientists with diverse background in healthcare, robotics, customer analytics, and communication. We are committed to building and deploying the most advanced scientific capabilities and solutions for the products and services at Amazon Health. We are open to hiring candidates to work out of one of the following locations: Santa Clara, CA, USA
US, WA, Seattle
We are designing the future. If you are in quest of an iterative fast-paced environment, where you can drive innovation through scientific inquiry, and provide tangible benefit to hundreds of thousands of our associates worldwide, this is your opportunity. Come work on the Amazon Worldwide Fulfillment Design & Engineering Team! We are looking for an experienced and senior Research Scientist with background in Ergonomics and Industrial Human Factors, someone that is excited to work on complex real-world challenges for which a comprehensive scientific approach is necessary to drive solutions. Your investigations will define human factor / ergonomic thresholds resulting in design and implementation of safe and efficient workspaces and processes for our associates. Your role will entail assessment and design of manual material handling tasks throughout the entire Amazon network. You will identify fundamental questions pertaining to the human capabilities and tolerances in a myriad of work environments, and will initiate and lead studies that will drive decision making on an extreme scale. .You will provide definitive human factors/ ergonomics input and participate in design with every single design group in our network, including Amazon Robotics, Engineering R&D, and Operations Engineering. You will work closely with our Worldwide Health and Safety organization to gain feedback on designs and work tenaciously to continuously improve our associate’s experience. Key job responsibilities - Collaborating and designing work processes and workspaces that adhere to human factors / ergonomics standards worldwide. - Producing comprehensive and assessments of workstations and processes covering biomechanical, physiological, and psychophysical demands. - Effectively communicate your design rationale to multiple engineering and operations entities. - Identifying gaps in current human factors standards and guidelines, and lead comprehensive studies to redefine “industry best practices” based on solid scientific foundations. - Continuously strive to gain in-depth knowledge of your profession, as well as branch out to learn about intersecting fields, such as robotics and mechatronics. - Travelling to our various sites to perform thorough assessments and gain in-depth operational feedback, approximately 25%-50% of the time. We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
GB, London
Amazon Advertising is looking for a Data Scientist to join its brand new initiative that powers Amazon’s contextual advertising products. Advertising at Amazon is a fast-growing multi-billion dollar business that spans across desktop, mobile and connected devices; encompasses ads on Amazon and a vast network of hundreds of thousands of third party publishers; and extends across US, EU and an increasing number of international geographies. The Supply Quality organization has the charter to solve optimization problems for ad-programs in Amazon and ensure high-quality ad-impressions. We develop advanced algorithms and infrastructure systems to optimize performance for our advertisers and publishers. We are focused on solving a wide variety of problems in computational advertising like traffic quality prediction (robot and fraud detection), Security forensics and research, Viewability prediction, Brand Safety, Contextual data processing and classification. Our team includes experts in the areas of distributed computing, machine learning, statistics, optimization, text mining, information theory and big data systems. We are looking for a dynamic, innovative and accomplished Data Scientist to work on data science initiatives for contextual data processing and classification that power our contextual advertising solutions. Are you an experienced user of sophisticated analytical techniques that can be applied to answer business questions and chart a sustainable vision? Are you exited by the prospect of communicating insights and recommendations to audiences of varying levels of technical sophistication? Above all, are you an innovator at heart and have a track record of resolving ambiguity to deliver result? As a data scientist, you help our data science team build cutting edge models and measurement solutions to power our contextual classification technology. As this is a new initiative, you will get an opportunity to act as a thought leader, work backwards from the customer needs, dive deep into data to understand the issues, define metrics, conceptualize and build algorithms and collaborate with multiple cross-functional teams. Key job responsibilities * Define a long-term science vision for contextual-classification tech, driven fundamentally from the needs of our advertisers and publishers, translating that direction into specific plans for the science team. Interpret complex and interrelated data points and anecdotes to build and communicate this vision. * Collaborate with software engineering teams to Identify and implement elegant statistical and machine learning solutions * Oversee the design, development, and implementation of production level code that handles billions of ad requests. Own the full development cycle: idea, design, prototype, impact assessment, A/B testing (including interpretation of results) and production deployment. * Promote the culture of experimentation and applied science at Amazon. * Demonstrated ability to meet deadlines while managing multiple projects. * Excellent communication and presentation skills working with multiple peer groups and different levels of management * Influence and continuously improve a sustainable team culture that exemplifies Amazon’s leadership principles. We are open to hiring candidates to work out of one of the following locations: London, GBR
JP, 13, Tokyo
We are seeking a Principal Economist to be the science leader in Amazon's customer growth and engagement. The wide remit covers Prime, delivery experiences, loyalty program (Amazon Points), and marketing. We look forward to partnering with you to advance our innovation on customers’ behalf. Amazon has a trailblazing track record of working with Ph.D. economists in the tech industry and offers a unique environment for economists to thrive. As an economist at Amazon, you will apply the frontier of econometric and economic methods to Amazon’s terabytes of data and intriguing customer problems. Your expertise in building reduced-form or structural causal inference models is exemplary in Amazon. Your strategic thinking in designing mechanisms and products influences how Amazon evolves. In this role, you will build ground-breaking, state-of-the-art econometric models to guide multi-billion-dollar investment decisions around the global Amazon marketplaces. You will own, execute, and expand a research roadmap that connects science, business, and engineering and contributes to Amazon's long term success. As one of the first economists outside North America/EU, you will make an outsized impact to our international marketplaces and pioneer in expanding Amazon’s economist community in Asia. The ideal candidate will be an experienced economist in empirical industrial organization, labour economics, or related structural/reduced-form causal inference fields. You are a self-starter who enjoys ambiguity in a fast-paced and ever-changing environment. You think big on the next game-changing opportunity but also dive deep into every detail that matters. You insist on the highest standards and are consistent in delivering results. Key job responsibilities - Work with Product, Finance, Data Science, and Data Engineering teams across the globe to deliver data-driven insights and products for regional and world-wide launches. - Innovate on how Amazon can leverage data analytics to better serve our customers through selection and pricing. - Contribute to building a strong data science community in Amazon Asia. We are open to hiring candidates to work out of one of the following locations: Tokyo, 13, JPN
DE, BE, Berlin
Ops Integration: Concessions team is looking for a motivated, creative and customer obsessed Snr. Applied Scientist with a strong machine learning background, to develop advanced analytics models (Computer Vision, LLMs, etc.) that improve customer experiences We are the voice of the customer in Amazon’s operations, and we take that role very seriously. If you join this team, you will be a key contributor to delivering the Factory of the Future: leveraging Internet of Things (IoT) and advanced analytics to drive tangible, operational change on the ground. You will collaborate with a wide range of stakeholders (You will partner with Research and Applied Scientists, SDEs, Technical Program Managers, Product Managers and Business Leaders) across the business to develop and refine new ways of assessing challenges within Amazon operations. This role will combine Amazon’s oldest Leadership Principle, with the latest analytical innovations, to deliver business change at scale and efficiently The ideal candidate will have deep and broad experience with theoretical approaches and practical implementations of vision techniques for task automation. They will be a motivated self-starter who can thrive in a fast-paced environment. They will be passionate about staying current with sensing technologies and algorithms in the broader machine vision industry. They will enjoy working in a multi-disciplinary team of engineers, scientists and business leaders. They will seek to understand processes behind data so their recommendations are grounded. Key job responsibilities Your solutions will drive new system capabilities with global impact. You will design highly scalable, large enterprise software solutions involving computer vision. You will develop complex perception algorithms integrating across multiple sensing devices. You will develop metrics to quantify the benefits of a solution and influence project resources. You will validate system performance and use insights from your live models to drive the next generation of model development. Common tasks include: • Research, design, implement and evaluate complex perception and decision making algorithms integrating across multiple disciplines • Work closely with software engineering teams to drive scalable, real-time implementations • Collaborate closely with team members on developing systems from prototyping to production level • Collaborate with teams spread all over the world • Track general business activity and provide clear, compelling management reports on a regular basis We are open to hiring candidates to work out of one of the following locations: Berlin, BE, DEU | Berlin, DEU
DE, BY, Munich
Ops Integration: Concessions team is looking for a motivated, creative and customer obsessed Snr. Applied Scientist with a strong machine learning background, to develop advanced analytics models (Computer Vision, LLMs, etc.) that improve customer experiences We are the voice of the customer in Amazon’s operations, and we take that role very seriously. If you join this team, you will be a key contributor to delivering the Factory of the Future: leveraging Internet of Things (IoT) and advanced analytics to drive tangible, operational change on the ground. You will collaborate with a wide range of stakeholders (You will partner with Research and Applied Scientists, SDEs, Technical Program Managers, Product Managers and Business Leaders) across the business to develop and refine new ways of assessing challenges within Amazon operations. This role will combine Amazon’s oldest Leadership Principle, with the latest analytical innovations, to deliver business change at scale and efficiently The ideal candidate will have deep and broad experience with theoretical approaches and practical implementations of vision techniques for task automation. They will be a motivated self-starter who can thrive in a fast-paced environment. They will be passionate about staying current with sensing technologies and algorithms in the broader machine vision industry. They will enjoy working in a multi-disciplinary team of engineers, scientists and business leaders. They will seek to understand processes behind data so their recommendations are grounded. Key job responsibilities Your solutions will drive new system capabilities with global impact. You will design highly scalable, large enterprise software solutions involving computer vision. You will develop complex perception algorithms integrating across multiple sensing devices. You will develop metrics to quantify the benefits of a solution and influence project resources. You will validate system performance and use insights from your live models to drive the next generation of model development. Common tasks include: • Research, design, implement and evaluate complex perception and decision making algorithms integrating across multiple disciplines • Work closely with software engineering teams to drive scalable, real-time implementations • Collaborate closely with team members on developing systems from prototyping to production level • Collaborate with teams spread all over the world • Track general business activity and provide clear, compelling management reports on a regular basis We are open to hiring candidates to work out of one of the following locations: Munich, BE, DEU | Munich, BY, DEU | Munich, DEU
IT, MI, Milan
Ops Integration: Concessions team is looking for a motivated, creative and customer obsessed Snr. Applied Scientist with a strong machine learning background, to develop advanced analytics models (Computer Vision, LLMs, etc.) that improve customer experiences We are the voice of the customer in Amazon’s operations, and we take that role very seriously. If you join this team, you will be a key contributor to delivering the Factory of the Future: leveraging Internet of Things (IoT) and advanced analytics to drive tangible, operational change on the ground. You will collaborate with a wide range of stakeholders (You will partner with Research and Applied Scientists, SDEs, Technical Program Managers, Product Managers and Business Leaders) across the business to develop and refine new ways of assessing challenges within Amazon operations. This role will combine Amazon’s oldest Leadership Principle, with the latest analytical innovations, to deliver business change at scale and efficiently The ideal candidate will have deep and broad experience with theoretical approaches and practical implementations of vision techniques for task automation. They will be a motivated self-starter who can thrive in a fast-paced environment. They will be passionate about staying current with sensing technologies and algorithms in the broader machine vision industry. They will enjoy working in a multi-disciplinary team of engineers, scientists and business leaders. They will seek to understand processes behind data so their recommendations are grounded. Key job responsibilities Your solutions will drive new system capabilities with global impact. You will design highly scalable, large enterprise software solutions involving computer vision. You will develop complex perception algorithms integrating across multiple sensing devices. You will develop metrics to quantify the benefits of a solution and influence project resources. You will validate system performance and use insights from your live models to drive the next generation of model development. Common tasks include: • Research, design, implement and evaluate complex perception and decision making algorithms integrating across multiple disciplines • Work closely with software engineering teams to drive scalable, real-time implementations • Collaborate closely with team members on developing systems from prototyping to production level • Collaborate with teams spread all over the world • Track general business activity and provide clear, compelling management reports on a regular basis We are open to hiring candidates to work out of one of the following locations: Milan, MI, ITA
ES, M, Madrid
Ops Integration: Concessions team is looking for a motivated, creative and customer obsessed Snr. Applied Scientist with a strong machine learning background, to develop advanced analytics models (Computer Vision, LLMs, etc.) that improve customer experiences We are the voice of the customer in Amazon’s operations, and we take that role very seriously. If you join this team, you will be a key contributor to delivering the Factory of the Future: leveraging Internet of Things (IoT) and advanced analytics to drive tangible, operational change on the ground. You will collaborate with a wide range of stakeholders (You will partner with Research and Applied Scientists, SDEs, Technical Program Managers, Product Managers and Business Leaders) across the business to develop and refine new ways of assessing challenges within Amazon operations. This role will combine Amazon’s oldest Leadership Principle, with the latest analytical innovations, to deliver business change at scale and efficiently The ideal candidate will have deep and broad experience with theoretical approaches and practical implementations of vision techniques for task automation. They will be a motivated self-starter who can thrive in a fast-paced environment. They will be passionate about staying current with sensing technologies and algorithms in the broader machine vision industry. They will enjoy working in a multi-disciplinary team of engineers, scientists and business leaders. They will seek to understand processes behind data so their recommendations are grounded. Key job responsibilities Your solutions will drive new system capabilities with global impact. You will design highly scalable, large enterprise software solutions involving computer vision. You will develop complex perception algorithms integrating across multiple sensing devices. You will develop metrics to quantify the benefits of a solution and influence project resources. You will validate system performance and use insights from your live models to drive the next generation of model development. Common tasks include: • Research, design, implement and evaluate complex perception and decision making algorithms integrating across multiple disciplines • Work closely with software engineering teams to drive scalable, real-time implementations • Collaborate closely with team members on developing systems from prototyping to production level • Collaborate with teams spread all over the world • Track general business activity and provide clear, compelling management reports on a regular basis We are open to hiring candidates to work out of one of the following locations: Madrid, ESP | Madrid, M, ESP
US, CA, San Diego
The Private Brands team is looking for an Applied Scientist to join the team in building science solutions at scale. Our team applies Optimization, Machine Learning, Statistics, Causal Inference, and Econometrics/Economics to derive actionable insights. We are an interdisciplinary team of Scientists, Engineers, and Economists and primary focus on building optimization and machine learning solutions in supply chain domain with specific focus on Amazon private brand products. Key job responsibilities You will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable optimization solutions and ML models. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale problems, enable measurable actions on the consumer economy, and work closely with scientists and economists. As a scientist, you bring business and industry context to science and technology decisions. You set the standard for scientific excellence and make decisions that affect the way we build and integrate algorithms. Your solutions are exemplary in terms of algorithm design, clarity, model structure, efficiency, and extensibility. You tackle intrinsically hard problems, acquiring expertise as needed. You decompose complex problems into straightforward solutions. We are particularly interested in candidates with experience in predictive and machine learning models and working with distributed systems. Academic and/or practical background in Machine Learning are particularly relevant for this position. Familiarity and experience in applying Operations Research techniques to supply chain problems is a plus. To know more about Amazon science, Please visit https://www.amazon.science We are open to hiring candidates to work out of one of the following locations: San Diego, CA, USA | Seattle, WA, USA
LU, Luxembourg
Ops Integration: Concessions team is looking for a motivated, creative and customer obsessed Snr. Applied Scientist with a strong machine learning background, to develop advanced analytics models (Computer Vision, LLMs, etc.) that improve customer experiences We are the voice of the customer in Amazon’s operations, and we take that role very seriously. If you join this team, you will be a key contributor to delivering the Factory of the Future: leveraging Internet of Things (IoT) and advanced analytics to drive tangible, operational change on the ground. You will collaborate with a wide range of stakeholders (You will partner with Research and Applied Scientists, SDEs, Technical Program Managers, Product Managers and Business Leaders) across the business to develop and refine new ways of assessing challenges within Amazon operations. This role will combine Amazon’s oldest Leadership Principle, with the latest analytical innovations, to deliver business change at scale and efficiently The ideal candidate will have deep and broad experience with theoretical approaches and practical implementations of vision techniques for task automation. They will be a motivated self-starter who can thrive in a fast-paced environment. They will be passionate about staying current with sensing technologies and algorithms in the broader machine vision industry. They will enjoy working in a multi-disciplinary team of engineers, scientists and business leaders. They will seek to understand processes behind data so their recommendations are grounded. Key job responsibilities Your solutions will drive new system capabilities with global impact. You will design highly scalable, large enterprise software solutions involving computer vision. You will develop complex perception algorithms integrating across multiple sensing devices. You will develop metrics to quantify the benefits of a solution and influence project resources. You will validate system performance and use insights from your live models to drive the next generation of model development. Common tasks include: • Research, design, implement and evaluate complex perception and decision making algorithms integrating across multiple disciplines • Work closely with software engineering teams to drive scalable, real-time implementations • Collaborate closely with team members on developing systems from prototyping to production level • Collaborate with teams spread all over the world • Track general business activity and provide clear, compelling management reports on a regular basis We are open to hiring candidates to work out of one of the following locations: Luxembourg, LUX