How to do fast, accurate multi-category classification

Many of today’s most useful AI systems are multilabel classifiers: they map input data into multiple categories at once. An object recognizer, for instance, might classify a given image as containing sky, sea, and boats but not desert or clouds.

Earlier this month, at the International Conference on Machine Learning, my colleagues and I presented a new approach to doing computationally efficient multilabel classification. In tests, we compared our approach to four leading alternatives using three different data sets. On five different performance measures, our system demonstrated improvements across the board.

The need for multilabel classification arises in many different contexts. Originally, it was investigated as a means of doing text classification: a single news article, for instance, might touch on multiple topics. Since then, it’s been used for everything from predicting protein function from raw sequence data to classifying audio files by genre.

The challenge of multilabel classification is to capture dependencies between different labels. If an image features sea, for instance, it’s much more likely to also feature boats than an image that features desert. In principle, the way to capture such dependencies is to learn a joint probability, which represents the likelihood of any combination of probabilities for all labels.

In practice, calculating an accurate joint probability for more than a handful of labels requires an impractically large amount of data. Two years ago, at NeurIPS, my colleagues and I demonstrated that using recurrent neural networks to chain single-label classifiers in sequence was a more computationally efficient way to capture label dependencies. Recurrent neural networks, or RNNs, process sequenced inputs in order, so that the output corresponding to any given input factors in the inputs and outputs that preceded it. RNNs thus automatically consider dependencies.

Our earlier paper, however, assumed that the order in which the classifiers are applied is fixed, regardless of context. Order matters because errors by classifiers early in the chain propagate through subsequent classifications. If the classifier chain erroneously labels an image as containing sea, for instance, it becomes more likely to mislabel it as containing boats.

The way to combat this problem is to move more error-prone classifiers later in the chain. But a given classifier’s propensity for error is relative to its inputs: some classifiers are very reliable for certain types of data but unreliable for others.

Our new approach is to train a system to dynamically vary the order in which the chained classifiers process the inputs, according to features of the input data. This ensures that the classifiers that are most error prone relative to a particular input move to the back of the chain.

In our experiments, we explored two different techniques for doing this. Our first technique used an RNN to generate a sequence of labels for a particular input. Then we excised the erroneous labels while preserving the order of the correct ones. If the resulting sequence omitted any correct labels, we appended them at the end in a random order. This new sequence then became a target output, which we used to re-train the RNN on the same input data.

Multilabel-classification_targets.png._CB442727461_.png
In this example, the numbers 1–5 represent the correct labels for a single input to a recurrent neural network (RNN). The RNN outputs a sequence of labels, some of which are correct (blue) but most of which are wrong (red). We excise the erroneous labels, preserving the order of the correct ones, and append all the missing labels (green) in a random order. This new sequence serves as a target output for re-training the RNN on the same data.

By preserving the order of the correct labels, we ensure that classifiers later in the chain learn to take advantage of classifications earlier in the chain. Initially, the output of the RNN is entirely random, but it eventually learns to tailor its label sequences to the input data. Note that while the training data is annotated to indicate the true labels, it is not annotated to indicate the ideal sequence of classifications. Hence the need for us to generate new training targets on the fly.

Reinforcement learning is the second technique we used to train an RNN to do dynamic classifier chaining. In reinforcement learning, a machine learning system learns a “policy,” which consists of a set of actions to take under different circumstances. It learns the policy through trial and error, gauging its success according to a “reward function”, which measures how far a selected action takes it toward some predefined goal.

In our case, the actions were simply the applications of labels to the input data, and the reward function measured the accuracy of the classification that resulted from the chain of classifiers.

In tests, we compared our systems to four different baselines. Each adopted a different strategy for determining classifier order. Two chained classifiers according to how common their labels were, the third used an arbitrary but fixed ordering, and the fourth generated a separate arbitrary ordering for each input.

We used five different metrics to evaluate system performance. One considered only the accuracy of the single most probable label, two the accuracies of the three most probable labels, and two the accuracies of the five most probable labels. For the three-label and five-label cases, we used two different evaluation strategies. One measured overall accuracy across labels, while the other assigned greater value to accurate assessments of the first few labels.

Our best-performing system combined the outputs of our two dynamic-chaining algorithms to produce a composite classification. On each of the five metrics, and on three different data sets, that combination outperformed whichever of the four baselines offered the best performance, usually by about 2% to 3% and in one instance by nearly 5%.

Research areas
About the Author
Jinseok Nam is an applied scientist in the Alexa AI group.

Related content

View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Sign up for our monthly newsletter

Work with us

See more jobs
GB, London
Come build the future of entertainment with us. Are you interested in shaping the future of movies and television? Do you want to define the next generation of how and what Amazon customers are watching?Prime Video is a premium streaming service that offers customers a vast collection of TV shows and movies - all with the ease of finding what they love to watch in one place. We offer customers thousands of popular movies and TV shows from Originals and Exclusive content to exciting live sports events. We also offer our members the opportunity to subscribe to add-on channels which they can cancel at anytime and to rent or buy new release movies and TV box sets on the Prime Video Store. Prime Video is a fast-paced, growth business - available in over 240 countries and territories worldwide. The team works in a dynamic environment where innovating on behalf of our customers is at the heart of everything we do. If this sounds exciting to you, please read on.The Research & Insight team operates in a dynamic and entrepreneurial environment across Europe and the world to understand our customers holistically. We work with senior stakeholders across a wide range of functions to plan, execute, and deliver continuous knowledge and insight to marketing and business teams to deliver customer centric growth.We are looking for a Sr. Data Scientist, Market Research to build our data strategy from the ground up. In this role, you will define quantitative research data standards and structures for end-to-end research processes, ensuring fit for purpose databases that are widely accessible and in usable formats for a variety of stakeholders. You will architect tools such as dashboards and automated reporting to help democratize our market research data. You’ll leverage predictive models, advanced machine learning/AI capabilities and exploratory data analysis (EDA) to solve problems and deliver insight.Responsibilities· Collecting and structuring data from a variety of 3P agency partners and APIs.· Collaborate closely with research, marketing, finance and content partner teams to ensure your data roadmap is fit for purpose across stakeholder groups.· Define and develop data management platforms, analysis tools and user friendly dashboards and reports.· Conduct ad-hoc analysis and predictive modelling to extract value from data to uncover opportunities and recommend actions.· Visualize data sets to inspire stakeholders (word clouds, infographics).· Perform regular data and system audits and feedback to ensure complete, accurate and usable data.· Optimize self-serve platform for data reporting and analysis to accelerate information-to-action at scale.· Build capability training across data and tools.· An SME for research data platform tools and analysis capabilities and delivery.
US, WA, Seattle
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses, responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products.Worldwide Ad Success team (WASE) is at the forefront of our amazing growth machine enabling our teams to deliver at scale. Our goal is to scale account management multifold by investing in strategic self-service applications that improve productivity of external advertising customers and internal account management executives. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.As part of our team evolution we are investing in improving our understanding of the advertisers on Amazon through advanced ML modeling and building an ML service that delivers recommendations to advertisers and solves the prioritization and selection of most optimum recommendations and measure impact with explain-ability.We are moving fast and have the ability to shape our tech infrastructure that will combine science and scalable engineering at a rapid pace. We are looking for a senior Applied Scientist to join the team to drive key science methods and delivery of the project, working closely with product and engineering leads, as well as our leadership. This is a relatively new team, with a focused initiative. We’re a fast-growing team with high visibility from the leadership team and lots of new opportunities.As an Applied Scientist on this team you will:· Solve business problems using state of the art machine learning methods.· Work on full life-cycle projects - from researching the optimal machine learning models to use to deploying your models into production.· Drive processes, tools, and statistical methods that support rational decision-making.· Be technically fearless: You aren't satisfied by performing 'as expected' and push the tech teams past conventional boundaries. Your dial goes to '11'.· Help grow recruit other scientists to achieve outstanding results.· Foster a creative atmosphere to let engineers and other PMs innovate, while holding them accountable for making smart decisions and delivering results.· Explore new problem spaces with unique constraints and thus non-obvious solutions; identify any gaps in the solutions and/or approach.Why you love this opportunityAmazon is investing heavily in building a world-class advertising business. We are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long-term growth. We deliver billions of ad impressions and millions of clicks daily and break fresh ground to create world-class products. We are highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit. With a broad mandate to experiment and innovate.Impact and Career GrowthYou will invent new shopper and advertiser experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon. Define a long-term science vision for our advertising business, driven fundamentally from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.Team video https://youtu.be/zD_6Lzw8raEAmazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us
US, NY, New York
Sponsored Products (SP) is Amazon's largest and fastest growing ad business. SP ads are shown prominently throughout search and product detail pages and allow shoppers to seamlessly discover products sold on Amazon. These are native ads that appear visually similar to other content on the page, which presents a huge opportunity for growth and impact, but also a significant responsibility to protect shopper experience.Job Responsibilities:· Design, develop, and deploy machine learning solutions.· Conduct hands-on data analysis, build large-scale machine-learning models and pipelines.· Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production.· Run regular A/B experiments, gather data, perform statistical analysis, and communicate the impact to senior leaders.· Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving.· Provide technical leadership, research new machine learning approaches to drive continued scientific innovation.· Be a member of the Amazon-wide Machine Learning Community, participating in internal and external MeetUps, Hackathons and Conferences.Impact and Career Growth:· Opportunity to grow and broaden your machine learning skills a make impact – the work you deliver directly impacts customers and revenue!· Work in an environment that thrives on creativity, experimentation, and product innovation.· Drive real-time algorithms to allocate billions of ads per day in advertising auctions.· Have the ability to experiment autonomously with meaningful projects.· Mentor others.Why you love this opportunity:Amazon is investing heavily in building a world class advertising business and we are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.
US, WA, Seattle
Interested in using AI to improve the shopping experience of millions of customers? Amazon Search has the perfect job for you.Amazon Search Customer Experience is looking for an experienced scientist to lead the innovation in Search Whole Page Optimization (WPO). Your research spans deep learning, reinforcement learning, and personalized recommendations. You will work with a team of scientists and engineers to make Amazon’s search experience intelligent, intuitive, and enjoyable.A successful candidate has strong customer obsession, highly-cited publications in relevant areas, and a track record of deploying research outcomes in production. You will bring deep technical expertise and strong business acumen. Amazon leaders are visionaries who are not afraid of rolling up their sleeves and getting their hands dirty. You will help shape the future of Amazon’s search customer experience by painting a compelling vision and leading the journey to get there. You must have the desire to make industry-wide impact and the ability to work within a fast moving environment to rapidly deliver innovations.As a senior leader, you will be responsible for the holistic optimization of Amazon search pages. From page layout to content ranking, from the navigation experience to product display optimization, you will rethink the assumptions behind traditional e-commerce experience and leverage AI to make the shopping journey of each customer a delightful one. You will be part of the Search technical leadership community that forms the backbone of the company. You will play a critical role in business planning, work closely with senior executives, and influence our long-term technical and business strategy.If you like the challenges and opportunities in this exciting space, come join us to work hard, have fun, and make history.
US, WA, Seattle
Amazon is looking for a creative Senior Research Scientist to tackle some of the most interesting problems on the leading edge of natural language processing (NLP), machine learning (ML), search and related areas with our Alexa AI team. Alexa AI aims to reinvent search and information retrieval for a voice-forward, multi-modal future. It enables customers to interact with unstructured and semi-structured content via a broad range of technologies including question answering, summarization, search, and multi-turn dialogues.If you are looking for an opportunity to solve deep technical problems and build innovative solutions in a fast-paced environment working within a smart and passionate team, this might be the role for you. You will develop and implement novel scalable algorithms and modeling techniques to advance the state-of-the-art in technology areas at the intersection of ML, NLP, search, and deep learning. You will innovate, help move the needle for research in these exciting areas and build cutting-edge technologies that enable delightful experiences for hundreds of millions of people.In this role you will:· Work collaboratively with other scientists and developers to design and implement scalable models for accessing and presenting information;· · Drive scalable solutions from the business to prototyping, production testing and through engineering directly to production;· · Drive best practices on the team, deal with ambiguity and competing objectives, and mentor and guide junior members to achieve their career growth potential.
US, WA, Seattle
Where will AWS’s growth come from in the next year? What about over the next five? What are the drivers and factors of such growth? Are we investing enough in our infrastructure, or too much? How do our customers react to changes in prices, product selection, or delivery times? These are among the most important questions at AWS today.A day in the lifeThe central AWS econ team is dedicated to answering these questions using quantitative and statistical methods. We develop cutting edge data pipelines, build accurate predictive models, and deploy automated software solutions to provide forecasting insights to business leaders at the most senior levels throughout the company. We are looking for a talented, driven, and analytical researcher to help us answer these (and many more) questions.About the hiring groupThe central AWS Econ team leads the way in developing innovative models, algorithms and strategies that will help us gain insights into how our business will grow and what will the drivers of such growth. These predictive models and insights will be based along products and product categories, customer segments, regions and locations, etc.Job responsibilitiesThis Research Scientist role will explore and develop innovative quantitative approaches and models, generate features, test hypotheses, design experiments, build predictive models, and work with very large complex data sets in order to make predictions and forecasts. These forecasts and insights will provide a foundation of the highest level of visibility and importance for financial and operational planning. The successful candidate will be a problem solver who enjoys diving into data, is excited by difficult modeling challenges, and possesses strong communication skills to effectively interface between technical and business teams, working together with Economists, Data Scientists, Software Engineers, Product Managers, and Business Analysts.Key Responsibilities:· Research, develop and build predictive models and forecasting systems· Analyze and research features and engineer features that help support predictive models and forecasting systems· Provide insights by analyzing historical data· Create experiments and prototype implementations of new learning algorithms and prediction techniques.· Collaborate with engineering teams to design and implement software solutions for science problems.
GB, Cambridge
Do you want to work as part of a team developing architectures and algorithms for Amazon Quantum Computing Centre?Quantum Computing is rapidly emerging from the realms of science-fiction, and our customers can the see the potential it has to address their challenges. One of our missions at AWS is to give customers access to the most innovative technology available and help them continuously reinvent their business. Quantum Computing is a technology that holds promise to be transformational in many industries, and we are adding quantum computing resources to the toolkits of every researcher and developer.Amazon Quantum Computing is growing and looking for Research Scientists to join an exceptional team of researchers and engineers. As a Research Scientist on the quantum algorithms team, you will:· Develop architectures, algorithms and applications for quantum computing, including resource estimation for practical problems· Play a key role in shaping the development roadmap of the service, and evangelizing new features and capabilities· Have the opportunity to create narrative papers, write blogs, build demos and generate other reusable collateral that can be used by our customers· Establish thought leadership of AWS in the field by publishing research in journals and speaking at international scientific conferences· Most importantly, work closely with our quantum computing hardware teams, as well as industry and academic partnersThis role is based in the AWS Cambridge office but will involve collaboration internationally across the entire Amazon Quantum Computing organization.Our team is dedicated to supporting new team members. Our team has a broad mix of experience levels and Amazon tenures, and we’re building an environment that celebrates knowledge sharing and mentorship.Our team is intentional about attracting, developing, and retaining amazing talent from diverse backgrounds. We’re looking for a new teammate who is enthusiastic, empathetic, curious, motivated, reliable, and able to work effectively with a diverse team of peers; someone who will help us amplify the positive & inclusive team culture we’ve been building.
SG, Singapore
The Amazon Prime Team is hiring Interns in Economics. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to apply economic and econometric theories to large-scale business problems and big data sets.These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will work in a team of economists, data scientists, and engineers and in collaboration with product and finance managers. These experiences will translate well into writing applied chapters in your dissertation and prepare you with placement in academia or private sector.Roughly 50% of research assistants from previous cohorts have converted to full time data science or economics employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com.
US, WA, Seattle
Have you ever ordered a product on Amazon and when that box with the smile arrives you wonder how it got to you so fast? Wondered where it came from and how much it would have cost Amazon? If so, Amazon’s Supply Chain Optimization Technologies (SCOT) team is for you. We build systems to peer into the future and estimate the distribution of tens of millions of products every week to Amazon’s warehouses in the most cost-effective way. When customers place orders, our systems use real time, large scale optimization techniques to optimally choose where to ship from and how to consolidate multiple orders so that customers get their shipments on time or faster with the lowest possible transportation costs. This team is focused on saving hundreds of millions of dollars using cutting edge science, machine learning, and scalable distributed software on the Cloud that automates and optimizes inventory and shipments to customers under the uncertainty of demand, pricing and supply.To help describe some of our challenges, we created a short video about Supply Chain Optimization at Amazon - http://bit.ly/amazon-scotA day in the lifeAs a Sr. Data Scientist, you will solve real world problems by analyzing large amounts of business data, defining new metrics and business cases, designing simulations and experiments, creating ML models, and collaborating with teammates in business, software, and research. The successful candidate will have a strong quantitative background and can thrive in an environment that leverages statistics, machine learning, operations research, econometrics, and business analysis.About the hiring groupFulfillment-by-Amazon (FBA) Inventory Technology (FIT) at Amazon’s Supply Chain Optimization Technologies (SCOT) focuses on driving long term free cash flow by automating and optimizing our third-party supply chain. The team’s efforts will address the key challenges facing the worldwide FBA Seller business, including 1) improving FBA Seller inventory efficiency, 2) efficiently balancing the supply and demand of FBA Seller capacity, 3) closing worldwide selection gap by enabling global selling profitability, and 4) driving out costs across the FBA supply chain to spin the flywheel. This is truly a unique problem space – optimizing for inventory in Amazon’s pipeline when you don’t control the process or own the inventory.Job responsibilities· Working with product managers, software engineers, data engineers, other data scientists, applied scientists to design, develop, and evaluate highly innovative statistics and ML models to drive FBA growth and efficiency through inventory optimization and the design of new policies and incentives.· Guide and establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation· Proactively seek to identify business opportunities and insights and provide solutions to automate and optimize key business processes and policies based on a broad and deep knowledge of Amazon data, industry best-practices, and work done by other teams.· Collaborating with our dedicated software team to create production implementations for large-scale data analysis and/or ML models.· Developing and owning key business metrics / KPIs and providing clear, compelling analysis that shapes the direction of our businessAmazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
US, NY, New York
Alexa is Amazon’s voice-first AI present in over 100 million Alexa-enabled devices. As devices continue to integrate more deeply into our lives, Alexa Human Effects (HFX) seeks to raise the bar of what is considered a compelling customer experience. We are a paradigm-changing, generative practice that investigates intrinsic anatomical, biomechanical, neurological, perceptual, and cultural parameters, and codifies this intelligence into Alexa hardware, software, and wetware.As the first Applied Scientist within the team, you are a versatile master. You are eccentric and experimental, but also have a love for discipline and rigor. You are comfortable within the realm of research, but are always hungry to force-multiply knowledge gained through the development of applied software / middleware utilities.In this role, you always start with common sense, and then establish appropriate field research methodologies to ensure that hard data surfaced from research can be reliably integrated into elastic design tools. You set technical direction, and bridge the study design with tool development. The elastic tools you build will strike a fine balance between high-precision physics simulations and a real-time game environment. The tool will distill highly-complex knowledge into aesthetically-intuitive, animated interfaces, for use by product / industrial designers.As far as tech-specs: you leverage relevant mathematical modeling and statistical analyses techniques to estimate, simulate, and test the validity of design decisions. You are able to capture data from physical systems (e.g. sensors, biometrics), develop prototypes in high level modeling languages, call optimization libraries, develop/leverage simulations, employ statistical/ML libraries + tools, ingest/process data using scripting languages, and visualize/animate results for intuitive understanding and usage. You are on your way to becoming a world-leading pioneer on how to use generative neural networks (e.g. GANs / VAE) to scale & accelerate data generation / harvesting / application / interaction.Additional locations for this role include: [Sunnyvale, CA], [Irvine, CA], [New York City, NY]
US, NY, New York
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. Amazon's advertising portfolio helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth.Our Forecasting Products team builds end-to-end solutions for publishers, advertisers and Amazon DSP. This includes data pipelines, machine learning models, large scale data structures and indexes, advertiser recommendations (bids, products) and data visualizations. We match supply (human eyeballs) and demand (advertisers interests) in thousands of audience targeting dimensions, and recommend optimal prices.As a Senior Applied Scientist, you will:· Drive applied science (machine learning) projects end-to-end ~ from ideation, analysis, prototyping, development, metrics, and monitoring.· Conduct deep analyses on massive Ad user and contextual data sets· Propose viable modelling ideas to advance optimization or efficiency, with supporting argument, data, or, preferably, preliminary results.· Invent ways to overcome technical limitations and enable new forms of analyses to drive key technical and business decisions.· Design, develop, and maintain scalable, Machine Learning models with automated training, validation, monitoring and reporting.· Stay familiar with the field and apply state-of-the-art Machine Learning techniques to our domain problems, around forecasting, bidding, allocation, and optimization.· Produce peer-reviewed scientific paper in top journals and conferences.· Present results, reports, and data insights to both technical and business leadership.· Work across teams and lead a group of talented scientists and engineers to solve problems in the domains of forecasting, auction theory, bid optimization, and user clustering.Team video - Amazon Advertising: https://youtu.be/zD_6Lzw8raEWhy you love this opportunityAmazon is investing heavily in building a world-class advertising business. We are responsible for defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long-term growth. We deliver billions of ad impressions and millions of clicks daily and break fresh ground to create world-class products. We are highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit. With a broad mandate to experiment and innovate.Impact and Career GrowthYou will invent new shopper and advertiser experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon. Define a long-term science vision for our advertising business, driven fundamentally from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us
US, WA, Seattle
Do you get excited about working on scientific problems that will have a large, significant and lasting impact for Amazon customers? Are you interested in data compression and machine learning? Are you thrilled about working in a startup-like environment? Search no more! We have an opening for an Applied Scientist in Amazon S3.As an Applied Scientist, you will be driving prototypes, projects and solutions in the fields of data compression and information coding. Our work sits right at the intersection of research and engineering and we are delivering products with real world impact for AWS customers. Your ideas will have a fast path to production at scale and help us reshape the way we think about data storage and content delivery at Amazon.Research areas our team is excited about include but are not limited to:· General purpose and domain-specific data compression· Information coding and data redundancy· Performance-preserving inline compression methodsOur team works together to provide work/life balance for all team members. We recognize that the circumstances of our team members vary, and we balance work across the team so that we’re all able to maintain high standards on behalf of our customers, while at the same time allowing for rich and happy personal lives.Upon joining S3, every employee is paired with a peer buddy who will help you to quickly come up to speed in understanding the technology we’re building, the tools we use and the business problems we’re trying to solve. We hire exceptional people and we recognize the importance of coaching others. We have a formal mentor search application that lets you find a mentor that works best for you based on location, job family, and job level. On our internal “Broadcast” video streaming website, you’ll find countless presentations by some of the brightest minds in computing discussing solutions to many of the exciting problems we’re solving in AWS today with new live-streaming videos added every week.We have a diverse team and drive towards an inclusive culture and work environment. Our team is intentional about attracting, developing, and retaining amazing talent from diverse backgrounds. Our team members are active in Amazon’s 10+ affinity groups, sometimes known as employee resource groups, which bring employees together across businesses and locations around the world. These range from groups such as the Black Employee Network (BEN), Amazon Women in Engineering (AWE) and Glamazon (our LGBTQ+ community).US EEO: Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, visit US Disability Accommodations.
US, WA, Seattle
The Selection team decides what is being sold by Amazon.com. As one component of the famous Amazon Flywheel, Selection is strategically important to our topline growth. We are responsible for thought leadership and innovation to improve the existing and develop new solutions for optimizing Amazon’s selection, better serving our customers. We build the models to make these selection decisions, driven by signals such as historical search and sales data, understanding of item economics, and product substitutability and complementarity.You will partner with scientists, economists and engineers to propose, prototype, analyze, implement and deliver to production a variety of data science initiatives. You will develop a comprehensive understanding of Amazon’s business and supply chain processes and make use of rich data to make inferences about customer behavior. You will propose, analyze and communicate the results of experiments that measure causal effects of selection changes with great autonomy. You will define, create and beautifully present metrics that deepen our understanding of how customers interact with our selection. You will creatively translate our collected data into initiatives with measurable impacts on customer satisfaction and business growth. You will define team data science processes and mentor your colleagues in the art of data science.In Amazon, everything operates at world scale. You will gain hands-on experience with big data technologies that handle the volume, development processes that supply the robustness, and recent modeling techniques that cut through the complexity. You will be inspired by and participate in the rich scientific community centered around Amazon’s unparalleled universe of econometric questions.
SG, Singapore
We are seeking an experienced Economist to accelerate the growth of Amazon Prime through econometrics and data analytics. Our vision is for Prime to be the Earth’s largest and most loved membership program. We look forward to partnering with you to advance our innovation on customers’ behalf.Amazon has a proven history of working with Ph.D. economists and offers a unique environment for economists to thrive. As an economist at Prime, you will apply the frontier of econometric and economic methods to Amazon’s terabytes of data and intriguing customer problems. Your expertise in building reduced-form or structural causal inference models is exemplary in Amazon. Your strategic thinking in designing mechanisms and products influences how Amazon evolves.In this role, you will build ground-breaking, state-of-the-art causal inference models to guide multi-million-dollar investment decisions. You will execute and expand a research roadmap that connects science, business, and engineering and contributes to Amazon Prime’s long term success. As one of the first economists outside North America/EU, you will make an outsized impact to our international marketplaces and pioneer in expanding Amazon’s economist community in Asia.The ideal candidate will be an economist in empirical industrial organization, labour economics, or related structural/reduced-form causal inference fields. You are a self-starter who enjoys ambiguity in a fast-paced and ever-changing environment. You think big on the next game-changing opportunity but also dive deep into every detail that matters. You insist on the highest standards and are consistent in delivering results.Primary Responsibilities:- Work with Product, Finance, Data Science, and Data Engineering teams across the globe to deliver data-driven insights and products for regional and world-wide launches.- Innovate on how Prime can leverage data analytics to grow faster.- Contribute to building a strong data science community in Amazon Asia.
SG, Singapore
We are seeking a seasoned Senior Data Scientist to accelerate the growth of Amazon Prime through data analytics. Our vision is for Prime to be the Earth’s largest and most loved membership program. We look forward to partnering with you to advance our innovation on customers’ behalf.As a senior science thought leader, you will build ground-breaking, state-of-the-art causal inference models to guide multi-million-dollar investment decisions. You will architect and execute a research roadmap that connects science, business, and engineering and contributes to Amazon Prime’s long term success. As an independent contributor, you mentor and nurture junior scientists on the team to set them up for success.The ideal candidate will be an established expert in data science, especially in the area of causal inference. You are comfortable leading through ambiguity in a fast-paced and ever-changing environment. You think big on the next game-changing opportunity but also dive deep into every detail that matters. You insist on the highest standards and are consistent in delivering results.Primary Responsibilities:- Own a multi-year research roadmap that guides Prime’s investment decisions.- Work with Product, Finance, Economist, and Data Engineering teams across the globe to deliver data-driven insights and products for regional and world-wide launches.- Innovate on how Prime can leverage data analytics to grow faster.- Contribute to building a strong data science community in Amazon Asia.
CA, BC, Vancouver
Are you excited about making multi-million dollar decisions more Science and Data Driven? Are you interested in supporting Consumer Hardware device concepts from innovative idea inception to launch? Do you want to work on a Science Product team focused on scaling Statistics and Econometrics via internal to Amazon tools? Then this could be the role for you!Amazon.com strives to be Earth's most customer-centric company where people can find and discover anything they want to buy online. We hire the world's brightest minds, offering them a fast paced, technologically sophisticated and friendly work environment. Amazon Devices and Services team is the area of Amazon focused on inventing platforms that delight customers by eliminating friction they have in supplying, entertaining, and managing the home and beyond.The Device Economics team owns demand estimates and pricing recommendations of concept devices before customers know they exist. We support over 100 device-specific analyses a year on hardware and services ranging from Echo Frames to Kindle Paperwhite to Blink Video Camera subscriptions to the Amazon Smart Plug…all prior to launch.. We are a cross-functional Product team working to scale Econometrics through Amazon and beyond by incorporating Science into internal facing tools and making it easier for others to do so as well.In this role, you will support up to SVP*-level decision meetings around approving confidential funding requests (PRFAQs) for brand new devices and services, build decisions around how many hardware devices to manufacture prior to receiving any customer signal, and pricing decisions around how to price and promote products and services. You will leverage Science and Tools produced by the Device Economics team such as conjoint demand models to produce these recommendations. As part of the stakeholder-facing arm of the team, you will own relationships with decision makers to help improve the end-customer experience by making the decisions that impact those end-customers more data and Science-driven. In parallel, you will work with Scientists, Economists, Product Managers, and Software Developers to provide meaningful feedback about stakeholder problems to inform business solutions and increase the velocity, quality, and scope behind our recommendations. You will own projects to make progress on Decision Science itself. Through this all, we will invest in your development to pursue your career goals.*The SVP of Amazon Devices and Services is Dave Limp. He reports to Jeff Bezos.
US, WA, Seattle
Amazon Web Services’ Marketing Science team is looking for a Data Scientist with excellent technical skills in machine learning and forecasting to drive methodology improvements and tool inventions for marketing planning, goal tracking, targeting and segmentation, while developing our long term roadmap.The successful candidate will be a hands-on self-starter, comfortable with ambiguity, humble to seek feedback and learn from peers (scientists, economists, marketing practitioners), bold to think big, has solid attention to detail and believes in challenging themselves to raise the bar continuously. The scientist will develop production models using geographical, macro-economic and customer-level panel data on demand forecasting and account-based marketing. They will work closely with media and portfolio teams to understand business needs and with data engineering team to productionize solutions.This role requires an individual with excellent quantitative modeling skills and the ability to apply statistical, machine learning, econometric, and coding skills.The candidate should have excellent communication skills to work closely with stakeholders to translate business needs into methodology and data-driven findings into actionable insights. The successful candidate will be a self-starter comfortable with ambiguity, with excellent attention to detail, and ability to work in a fast-paced and ever-changing environment.Location: Position may be located in Seattle, or near any Amazon/AWS U.S. Corp office only. Relocation offered from within the US only to these locations.Responsibilities:· · Simplify and drive automation of the forecasting process by building new tools and onboarding existing ones from Amazon Retail or AWS · Design and build scalable analytic solutions using forecasting and machine learning models to measure the financial impact of cross channel marketing spend · Design and implement forecasting and machine learning models in partnership with other scientists on the team · Work with the data acquisition team on data requirements · Work closely with both business units and engineering teams to formulate forecasting and targeting problems and associated technical solution strategies · Work with peer science and marketing teams in Amazon Retail to identify technical solutions that could be used or adopted in AWS Marketing forecasting and targeting framework · Develop a library of forecasting and machine learning algorithms to enable data-driven decision making · Support engineering teams to build tools and applications on our unique big data platform to efficiently generate and deploy insights into decision-making systems at AWS · Raise the bar on applications of machine learning and deep learning for forecasting and targeting
US, CA, Sunnyvale
Amazon Lab126 is an inventive research and development company that designs and engineers high-profile consumer electronics. Lab126 began in 2004 as a subsidiary of Amazon.com, Inc., originally creating the best-selling Kindle family of products. Since then, we have produced groundbreaking devices like Fire tablets, Fire TV and Amazon Echo.What will you help us create?The Role:We are looking for a passionate, talented and inventive Senior Applied Scientist - Audio to join our team. As part of the larger technology team working on new consumer technology, your work will have a large impact to hardware, internal software developers, ecosystem, and ultimately the lives of Amazon customers. You must love high quality signal processing, enjoy adjusting codecs, optimizing audio frameworks, and have a feel for what a good consumer experience should sound like.In this role, you will:· Engage with an experienced cross-disciplinary staff to conceive and design innovative consumer products· Work closely with an internal inter-disciplinary team, and outside partners to drive key aspects of product definition, execution and test· Development of new audio and speech processing algorithms· Optimization and port to different platforms of audio and speech processing algorithms with focus in Voice Communication and Speech Recognition· Integrate vendor hardware and software stacks· Be able, and willing, to multi-task and learn new technologies quickly· Be responsive, flexible and able to succeed within an open collaborative peer environment
US, WA, Seattle
The Networking Science team is looking for an exceptional Applied Scientist who is passionate about leveraging big data solutions for devising the next generation of network monitoring systems. Our mission is to develop models and tools to measure performance of AWS network. We deliver actionable insights across the Networking organization through descriptive analysis, data science, data engineering and ML/anomaly detection techniques.The ideal candidate will share our excitement about the incredible opportunity cloud computing and Big Data analytics represent, and will be passionate about delivering high quality services. You will have good knowledge of distributed systems with design and implementation experience, as well as the ability to lead and mentor other engineers. Experience with databases, data warehousing, business intelligence, and machine learning are particularly valued, as is experience delivering large-scale big data services. You will be customer centric and enjoy working in a fast-paced environment that requires excellent technical and communication skills.Networking Science team is based in Dublin, Ireland and Seattle, WA. You will join a tenured team of Research Scientists, Data Engineers and Software development engineers to lead the future of network availability and performance.
US, MA, Boston
Machine learning (ML) has been strategic to Amazon from the early years. We are pioneers in areas such as recommendation engines, product search, eCommerce fraud detection, and large-scale optimization of fulfillment center operations.The Amazon ML Solutions Lab team helps AWS customers accelerate the use of machine learning to solve business and operational challenges and promote innovation in their organization. We are looking for a passionate, talented, and inventive Applied Scientist with a strong machine learning background to help develop solutions by pushing the envelope in Time Series, Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), Machine Learning (ML) and Computer Vision (CV).As a ML Solutions Lab Applied Scientist, you are proficient in designing and developing advanced ML models to solve diverse challenges and opportunities. You will be working with terabytes of text, images, and other types of data and develop novel models to solve real-world problems. You'll design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. You will apply classical ML algorithms and cutting-edge deep learning (DL) and reinforcement learning approaches to areas such as drug discovery, customer segmentation, fraud prevention, capacity planning, predictive maintenance, pricing optimization, call center analytics, player pose estimation, event detection, and virtual assistant among others.The primary responsibilities of this role are to:· Design, develop, and evaluate innovative ML/DL models to solve diverse challenges and opportunities across industries· Interact with customer directly to understand their business problems, and help them with defining and implementing scalable ML/DL solutions to solve them· Work closely with account teams, research scientist teams, and product engineering teams to drive model implementations and new algorithmsThis position requires travel of up to 25%.About UsInclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.This position involves on-call responsibilities, typically for one week every two months. We don’t like getting paged in the middle of the night or on the weekend, so we work to ensure that our systems are fault tolerant. When we do get paged, we work together to resolve the root cause so that we don’t get paged for the same issue twice.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.