RecSys 2022: “Recommenders are ubiquitous”

Adapting natural-language-processing techniques to recommendation systems and algorithmic fairness are two central topics at this year’s conference.

The ACM Conference on Recommender Systems (RecSys), the leading conference in the field of recommendation systems, takes place this week, and two Amazon scientists — Max Harper, a senior applied scientist, and Vanessa Murdock, a senior applied-science manager, both in the Alexa Shopping organization — are among the conference’s three general chairs, along with Jennifer Golbeck of the University of Maryland. Harper and Murdock spoke to Amazon Science about the conference program and what it indicates about the state of research on recommender systems.

Amazon Science: Can you tell us a little bit about RecSys?

Max Harper: RecSys has been around for a long time — since the ’90s — and it's a community that's interested in both algorithms and applications of machine learning techniques that model the behavior of users. In particular, RecSys focuses on domains where the definition of the best thing for the model to return depends on which person you ask. So it's personalized.

RecSys portrait.png
Senior applied scientist Max Harper (left) and senior applied-science manager Vanessa Murdock, both of the Alexa Shopping organization, are two of the three general chairs at this year's RecSys.

The classical applications include movies, music, and books, which are obviously taste-driven domains. But these days, it's expanded into tons of areas, including travel, fashion, and job finding.

In addition to algorithms and applications, I'd say about 20% of the field is interested in people, how people perceive recommendations, how to design user interfaces that work well and how to shape the user experience in a variety of ways.

There's also a whole host of machine learning issues that comes along with it, including how to measure performance, how to scale the algorithms, how to preserve users’ privacy. And finally, an increasingly important issue is the societal impacts of these algorithms.

Related content
In 2017, when the journal IEEE Internet Computing was celebrating its 20th anniversary, its editorial board decided to identify the single paper from its publication history that had best withstood the “test of time”. The honor went to a 2003 paper called “Amazon.com Recommendations: Item-to-Item Collaborative Filtering”, by then Amazon researchers Greg Linden, Brent Smith, and Jeremy York.

Vanessa Murdock: I sit between the fields of search and recommendation, and they're somewhat different in that recommendations can be made even if the user isn't asking for them, whereas search is usually in response to a request.

Recommenders are ubiquitous — they’re in many of the apps and tools we use every day. For example, if you're looking for a coffee in Seattle, and you look at a map, the resolution of the map that you see on the first view will show you some points of interest, and then, if you zoom in, you'll see more. You can view those first points of interest as recommendations, but it's not what you usually think of as a recommender.

Your Instagram feed and Tik Tok are all recommendations. Your Twitter feed is a set of recommended tweets. It's central to our experience with the digital world in everything that we do.
Vanessa Murdock

All of this research on deciding what people would like to engage with has had significant influence on online commerce and ads and sponsored placements. Your Instagram feed and Tik Tok are all recommendations. Your Twitter feed is a set of recommended tweets. It's central to our experience with the digital world in everything that we do.

AS: In 2017, when IEEE Internet Computing celebrated its 20th anniversary, it gave its test-of-time award to Amazon’s 2003 paper on item-to-item collaborative filtering. How has the field evolved since that paper?

MH: The concept of collaborative filtering is still very, very relevant. These days, matrix factorization techniques are much more common; you use them to complete an item-customer matrix. But it's essentially the same class of techniques.

There's a paper at this year's RecSys, “Revisiting the performance of iALS on item recommendation benchmarks”, and it's part of the RecSys replicability track, which is kind of a unique thing at RecSys. This paper has to do with matrix factorization, which the field thinks of as an old-fashioned technique. And the point that authors make in this paper is that a well-tuned matrix factorization algorithm can hold its own against a whole range of more modern deep-learning algorithms.

Related content
Learn how the Amazon Music Conversations team is using pioneering machine learning to make Alexa's discernment better than ever.

VM: The reproducibility track at RecSys is especially good because a lot of reported research is incremental gains over many years. In every paper, the numbers always go up, and the results are always significant, but the improvements don’t always add up over time. Having a reproducibility track really sets RecSys apart. It means that as we are making gains in some area, we can look back and say, “Is this really true?”

In my own work, I've found that when I've tried to reproduce work from other people, the results depend on the collection or the queries or the system parameters. And that's not what a scientific advance really should be. So I think that that's a very important track, and more conferences should add it.

Sequential recommendation

AS: What are some of the newer ideas in the field that you find most intriguing?

MH: If I were to pick the number one thing that seems to have taken over the conference, it would be the application of techniques from natural-language processing to the field of recommender systems. In particular, Transformers and large language models like BERT have been adapted to the context of recommendations in an interesting way.

Related content
Two-day RecSys workshop that extends the popular REVEAL to include CONSEQUENCES features Amazon organizers, speakers.

Essentially, these language models learn the semantics of sentences by modeling which words go with which other words, and you can take an analogous approach in the field of recommendations by looking at, not sentences of words, but sequences of items — for example, products at Amazon or movies at Netflix that users engage with. And by using similar training techniques to what they use in NLP, they can solve problems like next-item prediction: given that the user has looked at these three products most recently, what's the product that they're most likely to look at next?

Language models learn the semantics of sentences by modeling which words go with which other words, and you can take an analogous approach in the field of recommendations by looking at, not sentences of words, but sequences of items.
Max Harper

That concept is called sequential recommendation, and it is everywhere at RecSys this year.

AS: Does sequential recommendation use the same kind of masked training that language models do?

MH: Yeah, it does. You take a sequence of user behavior, and you hide one of the items that they actually interacted with and try to predict that that's part of the sequence.

AS: How is that approach adapted to the new setting?

MH: Two examples I can think of: One is that there's aren’t necessarily natural boundaries in a sequence of user interactions, so you might be tempted to look at the entire sequence of interactions in order to predict the next one. Researchers are looking at the degree to which recency is important in next-item recommendation.

Another one is that sentences are more predictable: if you're missing a word in a sentence, it's more likely that a human could guess what that word is. With a sequence of item clicks or ratings or purchases, there might be a lot of noise with certain items.

Related content
The scientist's work is driving practical outcomes within an exploding machine learning research field.

Yusan Lin, who joined Amazon Fashion this year as an applied scientist, is a coauthor on a RecSys paper called “Denoising self-attentive sequential recommendation”, and it's about that concept: how do you find those items that are potentially harmful to the performance of the system and essentially hide them from the training so that the system learns more of a clean language, if you will, of what people are interested in?

VM: Sometimes the sequence of interactions is way too predictable. In e-commerce, if you think about reordering, where, say, you order the same brand of coffee absolutely every week, there's not really a benefit to recommending that coffee to you, even though it's very accurate. So there's some subtlety in there when we're talking about predicting the next recommendation — the next good recommendation — from a sequence of user interactions.

Fairness

AS: Vanessa, are there any other recent research trends that you find particularly interesting?

VM: In the last, say, 10 years, the attention that researchers have been paying to bias and fairness is tremendously important. As we get better at predicting what people need, and as we become more embedded in everyday life, the effort to make sure that we're not introducing unintended biases is very, very important. It's a hard problem, and I'm very happy to see attention to that.

AS: What kind of approaches do people take to that problem?

VM: The first thing is that the researcher actually has to be aware of the problem. A lot of times the data is very large, and the items you are trying to predict are a very small subset. Suppose that you have a group of people who have blue hair, and they're very interested in products for blue hair. You can imagine they are a tiny, tiny proportion of your data. If your recommender is based on what most people like, you're never going to offer them anything for their blue hair.

It's a class of problems called unknown unknowns, where there’s a small positive class, but you don't know how big it is, and you don't have a way to find that in your data. You know there are some people with blue hair because they've interacted with blue-hair things, but you don't know how many of your customers actually have blue hair.

Related content
Research investigates how to construct recommendation algorithms when the search space is massive and how to perform natural-language searches on the COVID-19 literature.

Some approaches for that are to sample in a clever way or to create synthetic data or to do domain adaptation, where you have a large amount of known data from some other domain that you can adapt to this new area. For instance, you have a lot of data about people who have green hair, and you can adapt that to people with blue hair.

Another is to look at whether the data itself has a skew in the features. Maybe the features are accidentally correlated, or maybe something is not represented well, because the feature space for the blue haired items is too small. Those are all things to look at.

MH: I totally agree that fairness, along with privacy and explainability, are big topics at this year's RecSys. There definitely is research into news recommendation, which is a big, important topic to the world. There's this idea of filter bubbles, which is a long-hypothesized problem, but one that we're seeing in practice, in which personalization technology makes the range of opinions that we see online shallower and shallower. So for instance, we'll see news that confirms our own beliefs rather than seeing a diversity of viewpoints.

There's some work on those topics at this year's RecSys. One paper in particular I thought was quite interesting because they took a principled approach to looking at what it means for a news article to be diverse. There's a shallow, algorithmic definition of diversity that most prior research has used that may or may not line up with what humans perceive as diversity in news articles.

So they took this more principled approach to measuring diversity using natural-language techniques. They provided a mathematical foundation for measuring the diversity of a set of articles and looked at how different algorithms actually behave on a news dataset. I think that work on fairness is really important and will be very influential in years to come.

Related content

US, CA, Santa Clara
Job summaryAmazon is looking for a passionate, talented, and inventive Applied Scientist with a strong machine learning background to help build industry-leading language technology.Our mission is to provide a delightful experience to Amazon’s customers by pushing the envelope in Natural Language Processing (NLP), Natural Language Understanding (NLU), Dialog management, conversational AI and Machine Learning (ML).As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services, as well as contributing to the wider research community. You will gain hands on experience with Amazon’s heterogeneous text and structured data sources, and large-scale computing resources to accelerate advances in language understanding.We are hiring primarily in Conversational AI / Dialog System Development areas: NLP, NLU, Dialog Management, NLG.This role can be based in NYC, Seattle or Palo Alto.Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.
US, NY, New York
Job summaryAmazon is looking for a passionate, talented, and inventive Applied Scientist with a strong machine learning background to help build industry-leading language technology.Our mission is to provide a delightful experience to Amazon’s customers by pushing the envelope in Natural Language Processing (NLP), Natural Language Understanding (NLU), Dialog management, conversational AI and Machine Learning (ML).As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services, as well as contributing to the wider research community. You will gain hands on experience with Amazon’s heterogeneous text and structured data sources, and large-scale computing resources to accelerate advances in language understanding.We are hiring primarily in Conversational AI / Dialog System Development areas: NLP, NLU, Dialog Management, NLG.This role can be based in NYC, Seattle or Palo Alto.Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.
US, CA, Santa Clara
Job summaryAWS AI/ML is looking for world class scientists and engineers to join its AI Research and Education group working on building automated ML solutions for planetary-scale sustainability and geospatial applications. Our team's mission is to develop ready-to-use and automated solutions that solve important sustainability and geospatial problems. We live in a time wherein geospatial data, such as climate, agricultural crop yield, weather, landcover, etc., has become ubiquitous. Cloud computing has made it easy to gather and process the data that describes the earth system and are generated by satellites, mobile devices, and IoT devices. Our vision is to bring the best ML/AI algorithms to solve practical environmental and sustainability-related R&D problems at scale. Building these solutions require a solid foundation in machine learning infrastructure and deep learning technologies. The team specializes in developing popular open source software libraries like AutoGluon, GluonCV, GluonNLP, DGL, Apache/MXNet (incubating). Our strategy is to bring the best of ML based automation to the geospatial and sustainability area.We are seeking an experienced Applied Scientist for the team. This is a role that combines science knowledge (around machine learning, computer vision, earth science), technical strength, and product focus. It will be your job to develop ML system and solutions and work closely with the engineering team to ship them to our customers. You will interact closely with our customers and with the academic and research communities. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. You are also expected to work closely with other applied scientists and demonstrate Amazon Leadership Principles (https://www.amazon.jobs/en/principles). Strong technical skills and experience with machine learning and computer vision are required. Experience working with earth science, mapping, and geospatial data is a plus. Our customers are extremely technical and the solutions we build for them are strongly coupled to technical feasibility.About the teamInclusive Team CultureAt AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded scientist and enable them to take on more complex tasks in the future.Interested in this role? Reach out to the recruiting team with questions or apply directly via amazon.jobs.
US, CA, Santa Clara
Job summaryAWS AI/ML is looking for world class scientists and engineers to join its AI Research and Education group working on building automated ML solutions for planetary-scale sustainability and geospatial applications. Our team's mission is to develop ready-to-use and automated solutions that solve important sustainability and geospatial problems. We live in a time wherein geospatial data, such as climate, agricultural crop yield, weather, landcover, etc., has become ubiquitous. Cloud computing has made it easy to gather and process the data that describes the earth system and are generated by satellites, mobile devices, and IoT devices. Our vision is to bring the best ML/AI algorithms to solve practical environmental and sustainability-related R&D problems at scale. Building these solutions require a solid foundation in machine learning infrastructure and deep learning technologies. The team specializes in developing popular open source software libraries like AutoGluon, GluonCV, GluonNLP, DGL, Apache/MXNet (incubating). Our strategy is to bring the best of ML based automation to the geospatial and sustainability area.We are seeking an experienced Applied Scientist for the team. This is a role that combines science knowledge (around machine learning, computer vision, earth science), technical strength, and product focus. It will be your job to develop ML system and solutions and work closely with the engineering team to ship them to our customers. You will interact closely with our customers and with the academic and research communities. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. You are also expected to work closely with other applied scientists and demonstrate Amazon Leadership Principles (https://www.amazon.jobs/en/principles). Strong technical skills and experience with machine learning and computer vision are required. Experience working with earth science, mapping, and geospatial data is a plus. Our customers are extremely technical and the solutions we build for them are strongly coupled to technical feasibility.About the teamInclusive Team CultureAt AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded scientist and enable them to take on more complex tasks in the future.Interested in this role? Reach out to the recruiting team with questions or apply directly via amazon.jobs.
US, WA, Seattle
Job summaryHow can we create a rich, data-driven shopping experience on Amazon? How do we build data models that helps us innovate different ways to enhance customer experience? How do we combine the world's greatest online shopping dataset with Amazon's computing power to create models that deeply understand our customers? Recommendations at Amazon is a way to help customers discover products. Our team's stated mission is to "grow each customer’s relationship with Amazon by leveraging our deep understanding of them to provide relevant and timely product, program, and content recommendations". We strive to better understand how customers shop on Amazon (and elsewhere) and build recommendations models to streamline customers' shopping experience by showing the right products at the right time. Understanding the complexities of customers' shopping needs and helping them explore the depth and breadth of Amazon's catalog is a challenge we take on every day. Using Amazon’s large-scale computing resources you will ask research questions about customer behavior, build models to generate recommendations, and run these models directly on the retail website. You will participate in the Amazon ML community and mentor Applied Scientists and software development engineers with a strong interest in and knowledge of ML. Your work will directly benefit customers and the retail business and you will measure the impact using scientific tools. We are looking for passionate, hard-working, and talented Applied scientist who have experience building mission critical, high volume applications that customers love. You will have an enormous opportunity to make a large impact on the design, architecture, and implementation of cutting edge products used every day, by people you know.Key job responsibilitiesScaling state of the art techniques to Amazon-scaleWorking independently and collaborating with SDEs to deploy models to productionDeveloping long-term roadmaps for the team's scientific agendaDesigning experiments to measure business impact of the team's effortsMentoring scientists in the departmentContributing back to the machine learning science community
US, NY, New York
Job summaryAmazon Web Services is looking for world class scientists to join the Security Analytics and AI Research team within AWS Security Services. This group is entrusted with researching and developing core data mining and machine learning algorithms for various AWS security services like GuardDuty (https://aws.amazon.com/guardduty/) and Macie (https://aws.amazon.com/macie/). In this group, you will invent and implement innovative solutions for never-before-solved problems. If you have passion for security and experience with large scale machine learning problems, this will be an exciting opportunity.The AWS Security Services team builds technologies that help customers strengthen their security posture and better meet security requirements in the AWS Cloud. The team interacts with security researchers to codify our own learnings and best practices and make them available for customers. We are building massively scalable and globally distributed security systems to power next generation services.Inclusive Team Culture Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life Balance Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. Mentorship & Career Growth Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. We care about your career growth and strive to assign projects based on what will help each team member develop and enable them to take on more complex tasks in the future.A day in the lifeAbout the hiring groupJob responsibilities* Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, making use of both quantitative and business judgment.* Collaborate with software engineering teams to integrate successful experiments into large scale, highly complex production services.* Report results in a scientifically rigorous way.* Interact with security engineers, product managers and related domain experts to dive deep into the types of challenges that we need innovative solutions for.
US, WA, Seattle
Job summaryPrime Video is changing the way millions of customers interact with video content. Our team delivers high-quality video to Amazon customers through subscriptions (Amazon Prime) as well as purchases and rentals. Amazon believes so deeply in the mission of Video that we've launched our own studio to create original and exclusive content.Every day we face the challenges of a fast-paced market and expanding technology set. You will have the freedom and encouragement to explore your own ideas and the reward of seeing your contributions benefit millions of Amazon.com customers worldwide.We’re looking for a Senior Data Scientist to join the Prime Video Global Video Supply Chain team. Data Scientist help us build the future of advanced analytics across the Prime Video Supply Chain.You will partner with Business Intelligence Engineers, Data Engineers, and Software Engineering teams to drive experimentation, forward-looking insights, predictive modeling, and machine learning algorithms. You will build best-in-class forecasting solutions at scale.If you have extensive experience with algorithmic demand forecasting, defect detection and algorithm development, think strategically and analytically about business, product, and technical challenges, if you can contribute to the development of current and future technical roadmaps and can build and communicate compelling value propositions, you will be successful in this role.The position can be based out of markets with a Prime Video office:· Seattle, Washington· Culver City, California· Arlington, Virginia· Austin, Texas· New York, New YorkKey job responsibilitiesImplement code (Python, R, Scala, etc.) for analyzing data and building statistical models to solve specific business problems.Work with software engineering teams to productionize algorithms where appropriate.Lead the development of the scientific roadmap, guide and develop junior engineers in designing and implementing scientific solutions.Directly contribute to the design and development of scaled forecasting solutions via machine learning based demand forecasting techniques.Translate analytic insights into concrete, actionable recommendations for business or product improvement.Develop and present these as papers to senior stakeholders with ranging levels of technical knowledge.Create, enhance, and maintain technical documentation, and present to other scientists, engineers and business leaders.Demonstrate thorough technical knowledge on feature engineering of massive datasets, effective exploratory data analysis, and model building to deliver accurate and effective business insights.Innovate by researching, learning, and adapting new modeling techniques and procedures to existing business problems.Manage and execute entire project from start to finish including problem solving, data gathering and manipulation, predictive modeling, and stakeholder engagement.About the teamGlobal Video Supply Chain (GVSC). Prime Video’s Global Video Supply Chain group owns the largest, richest catalog of video content in the world, available to customers as early as possible. We enable Prime Video customers all around the world to discover, purchase, and stream any title in any language anywhere.
CA, BC, Vancouver
Job summary Amazon Brand Protection organization focuses on building trust with all brands by accurately representing and completely protecting their brands on Amazon. We strive to be the most trusted thought leader in the space and ensure that public perception mirrors the trustworthy experience we deliver. The Brand Protection machine learning (ML) team is responsible to provide data driven long term strategies and solutions. The team is responsible to develop the state of art ML algorithms to ensure each product is brand authentic and to ensure no abuse or infringements on any brands. The ML team faces the challenges to work with huge amount of structured and unstructured data including images and product descriptions and to develop ML solutions that can scale to protect millions of brands and billions of products worldwide. The team also faces the challenge to fast update our ML systems to stay ahead of bad actors who constantly circumvent our algorithms. If you are excited at these responsibilities and challenges and if you love data and machine learning, we have a position for you. We are looking for a strong manager to manage the ML science team in Vancouver. As the manager, you will hire and develop ML talents. You will design long terms plans and define SMART goals. You will build roadmaps to achieve team’s vision and goals. You will lead the ML directions. You will lead roadmap and plan executions. You will be able to deep dive and guide your team both in directions and in details. You understand ML cycles and advocate ML best practices. You will keep abreast with new ML technologies. Major responsibilities:Work with business/tech teams to identify opportunities, design solution, implement and monitor ML models.Understand business challenges by analyzing data and customer feedbackGuide team members on model building strategies and model experiment, implementation, measurement and continuous improvementBuild and manage team roadmapsCreate long term plans to address complicated business problems at scale using MLDeep dive to provide business insightsCreate business and analytics reports and present to the senior management teamsLead research and implement novel machine learning and statistical approaches
US, WA, Bellevue
Job summaryAre you passionate about leveraging your data science and machine learning skills to make an impact at scale? Do you enjoy developing innovative algorithms, optimization and predictive models to generate recommendations that will be used by automated systems to drive hundreds of millions of impact on Amazon Retail's cash flow? If these questions get you excited, we definitely want to hear from you. Strategic Sourcing team, as part of Amazon Supply Chain Optimization and Technology organization, is seeking an experienced and motivated Data Science leader. Strategic Sourcing team owns systems that are designed to: 1) reduce end to end costs from inbound supply chain and (2) improve vendor performance. Some of the key decisions that these systems make: when and if we should source a product (e.g. is the product obsolete or temporarily unavailable); from which vendor and at what cost we should source an ASIN; what is the ideal supply chain setup (e.g. Pallet, Truckload, Vendor Initiated PO, etc.) for an ASIN/vendor; when should vendor ship/deliver inventory to Amazon FCs; which inbound lanes – vendor warehouse to Amazon FC – should have pre-allocated transportation with how many shipments; when should we penalize vendors for defects/infractions through chargebacks and by how much. Together these set of decisions and systems work together to ensure Amazon’s inventory needs are met on time and in the most efficient way. We develop sophisticated algorithms that involve learning from large amounts of data from diverse sources such Vendors, Transportation carriers, Amazon warehouses etc. Key job responsibilitiesAs the Data Science Senior Manager on this team, you will: • Lead of team of scientists on solving science problems with a high degree of complexity and ambiguity • Develop science roadmaps, run annual planning, and foster cross-team collaboration to execute complex projects • Perform hands-on data analysis, build machine-learning models, run regular A/B tests, and communicate the impact to senior management • Hire and develop top talent, provide technical and career development guidance to scientists and engineers in the organization • Analyze historical data to identify trends and support optimal decision making • Apply statistical and machine learning knowledge to specific business problems and data • Formalize assumptions about how our systems should work, create statistical definitions of outliers, and develop methods to systematically identify outliers. Work out why such examples are outliers and define if any actions needed
CA, ON, Toronto
Job summaryThe Customer Behavior Analytics (CBA) organization owns Amazon’s insights pipeline from data collection to deep analytics. We aspire to be the place where Amazon teams come for answers, a trusted source for data and insights that empower our systems and business leaders to make better decisions. Our outputs shape Amazons marketing teams’ decisions and thus how Amazon customers see, use, and value their experience.CMO (Campaign measurement and Optimization) team within CBA org's mission is to make Amazon’s marketing the most measurably effective in the world. Our long-term objective is to measure the incremental impact of all Amazon’s marketing investments on consumer perceptions, actions, and sales. This requires measuring Amazon’s marketing comparably and consistently across channels, business teams and countries using a comprehensive approach that integrates all Paid, Owned and Earned marketing activity. As the experts on marketing performance, we will lead the Amazon worldwide marketing community by providing critical global insights that can power marketing best practices and tenets globally.Are you passionate about Deep Learning, Causal Inference, and Big Data Systems? Interested in building new state-of-the-art measurement products at petabyte scale? Be part of a team of industry leading experts that operates one of the largest big data and machine learning stacks at Amazon. Amazon is leveraging its highly unique data and applying the latest machine learning and big data technologies to change the way marketers optimize their advertising spend. Our campaign measurement and reporting systems apply these technologies on many billions of events in near real time.You'll be one of the lead scientists tackling some of the hardest problems in advertising; measuring ads incrementality, providing estimated counterfactuals and predicting the success of advertising strategies for omni-channel campaign measurement. Working with a cross-functional team of product managers, program managers, economists and engineers you will develop state of the art causal learning, deep learning, and predictive techniques to help marketers understand the performance of their omni-channel campaigns and optimize their spends.Some things you'll do in this role:Lead full life-cycle Data Science solutions from beginning to end.Deliver with independence on challenging large-scale problems with complexity and ambiguity.Write code (Python, R, Scala, SQL, etc.) to obtain, manipulate, and analyze data.Build Machine Learning and statistical models to solve specific business problems.Retrieve, synthesize, and present critical data in a format that is immediately useful to answering specific questions or improving system performance.Analyze historical data to identify trends and support optimal decision making.Apply statistical and machine learning knowledge to specific business problems and data.Formalize assumptions about how our systems should work, create statistical definitions of outliers, and develop methods to systematically identify outliers. Work out why such examples are outliers and define if any actions needed.Given anecdotes about anomalies or generate automatic scripts to define anomalies, deep dive to explain why they happen, and identify fixes.Build decision-making models and propose effective solutions for the business problems you define.Conduct written and verbal presentations to share insights to audiences of varying levels of technical sophistication.Impact and Career Growth: You will invent solutions that can make billion dollar impact for Amazon as an advertiser. Define a long-term science vision for our business, driven from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding.This position is based in Irvine, San Francisco, Sunnyvale, San Jose or Seattle. Key job responsibilitiesDive deep into petabyte-scale data to drive insights, identify machine-learning modeling gaps and business opportunitiesEstablish scalable, efficient, automated processes for large-scale data analysisRun regular A/B experiments, gather data, and perform statistical analysisWork with scientists, engineers and product partners to develop new machine learning approaches, and monetization strategiesConduct written and verbal presentation to share insights and recommendations to audiences of varying levels of technical sophistication