Ren Zhang, director of research science for personalization strategic initiatives, is seen sitting on a couch, posing for a photo
Ren Zhang, director of research science for personalization strategic initiatives, and her team use science to present customers with the most relevant offerings possible for things like Amazon Fresh, Amazon Business, Pets, and Beauty.

Why Amazon's homepage feels like your own

Ren Zhang and her team tackle the interesting science challenges behind surfacing the most relevant offerings.

Amazon's massive catalogue makes it possible to personalize a customer's shopping experience in countless ways. It's not surprising when the Amazon Store anticipates you've come back to buy the treats your dog loves, and you see the option up front on the homepage.

But there may also be similar brands worth considering for your pup. Also, you might be due for more face wash, an item in your wishlist is on sale, and several options match the jeans you were looking for the other day.

Forecasting at Amazon
The story of a decade-plus long journey toward a unified forecasting model.

Ren Zhang's team uses science to present customers with the most relevant offerings possible. As director of research science for personalization strategic initiatives including Buy Again, Amazon Fresh, Amazon Business, Pets, and Beauty, Zhang applies deep learning for tailored, real-time recommendations to customers.

"Our goal is to understand the customer's intention for that moment and provide relevant experiences and products," she says. "The complexity we're solving, and the impact we have, make our team a haven for scientists."

Anticipating the customer's needs

As every shopper knows, what you intend to do when you get to a store and what you actually do are often two different things. Did you buy dog treats because you set out to do that or because seeing them reminded you to?

Understanding customers’ preferences can be challenging in a noisy environment where customers may have a number of potential factors motivating a decision. For example, if a product's placement influences a decision, that's a case of bias caused by position. Some items are more likely to be selected by customers simply because they are placed at more visible locations. This is just one of many challenges Zhang's team is trying to solve.

The science of product recommendation
Dual embeddings of each node, as both source and target, and a novel loss function enable 30% to 160% improvements over predecessors.

"There are all kinds of interesting science problems in a very noisy environment," Zhang says. "How do you design an experiment with so much going on and truly learn the right thing?"

Zhang and her team have published research on how to refine recommendations in a field of likely choices — e.g., should Toy Story be recommended because it’s relevant to you or just because it’s a wildly popular movie? In the paper, “Don’t recommend the obvious: Estimate probability ratios,” they demonstrate how popularity-sampled metrics are a better fit for personalized recommendations because they directly measure a system’s ability to identify items special to a user over generally probable items.

Another challenge is to bring transparency to the recommender. While deep-learning algorithms outperform traditional methods, they don’t, by default, provide any rationale for the conclusions they reach.

"Typically, customers are more likely to engage with a recommendation if they know why they're getting one. But we're talking about millions of records to come up with a response. How do you decipher that for customers?" she says. "The complexity of the methodology, the massive data, and the expected response time latency makes the problem extremely challenging."

Finding the right product
Augmenting query-product graphs with hypergraphs describing product-product relationships improves recall score by more than 48%.

Zhang and her team collaborate with Amazon Scholars like Iain Murray on solutions to such problems. A professor of machine learning and inference at the University of Edinburgh, Murray contributes to the team's research and brings ideas of his own.

"There are a lot of brainstorming sessions," Zhang says of the relationship with Murray. "It's not only helping us advance, but it's a great talent magnet as well. Scientists are attracted to a community that fosters innovation and research in science."

Zhang ensures that 20 to 30% of her team's time goes to exploratory research that they believe is the future — work that in the near term may not have a direct business impact.

"I have a much higher tolerance for failure when it comes to this kind of research," she says. "Scientists need that kind of freedom to explore, to feel excited. That kind of energy is what you need to have a high-performing team and have the right culture."

Focused on business impact

Zhang joined Amazon in July 2021 after a series of leading data science roles in the finance world at companies including Prudential and American Express. A love of math led her to study probability and statistics at Peking University in China, where she earned her bachelor's degree, and to eventually earn her doctorate in statistics from the University of Pennsylvania's Wharton School.

I love to drive business impact, and I could see Amazon as an innovation powerhouse that was going to be a fabulous place for me to do that. I've seen extremely talented people in my career, but the average bar at Amazon is so high.
Ren Zhang

Her advisor at Wharton, the late Lawrence Brown, was a scholar through and through, and her studies were geared toward academia. But after some corporate internships and conversations with former department mates, Zhang realized she wanted to be out solving practical problems.

"It took me quite some courage to go to him and say, 'Hi, Larry, I don't think I want to be a professor, I need to go into industry,’" she recalls.

Her first job was at American Express, where she says she "grew up, career-wise." Over nearly 13 years there, she took on roles in a variety of divisions, from business loans to membership rewards to risk and information management. When she was the chief data scientist for BMO Financial, an Amazon recruiter reached out. She recognized the opportunity to move to the tech sector, which had long fascinated her.

"I love to drive business impact, and I could see Amazon as an innovation powerhouse that was going to be a fabulous place for me to do that," she says, adding that she has been impressed with Amazon's talent. "I've seen extremely talented people in my career, but the average bar at Amazon is so high."

WiDS 2022 - Ren Zhang

Zhang joined as director of data science for worldwide selling partner development, helping Amazon sellers extract insights such as how customers were finding their products and ways to build customer loyalty. She switched to her current role in personalization in spring 2022, fascinated by the ability to experiment and ultimately see direct benefits for customers.

Any scientist who wants to work in the industry has to have a practical mindset, according to Zhang. "You need to be able to use data science to solve a business problem," she says. "The toughest talent to find, typically, is a talent who can bridge fields."

Data science at Amazon
Matt Taddy, the chief economist for Amazon’s North America Consumer organization, talks about his recent book, and explains why economists should consider pursuing a career at the company.

Zhang herself works hard to make sure she knows the business and engineering side of any scientific topic at Amazon. She points to Ash Gupta, with whom she worked early in her career when he was chief risk officer at American Express, as a role model. Even as a senior executive, Gupta would often delve into details and challenge his staff on various areas like systems, processes, and policies. He had a deep knowledge of the risk management space while driving the vision at the same time.

It's a level of discipline Zhang considers important for anyone who wants to foster good science but also deliver results.

"No matter what level of leader you are," she says, "you should never be disconnected from what's going on in the field beyond your own area. Science will not be able to deliver impact without an appealing customer experience design and robust engineering system."

Related content

US, CA, Sunnyvale
The Amazon Artificial General Intelligence (AGI) Personalization team is looking for a passionate, highly skilled and inventive Applied Scientist with strong machine learning background to build state-of-the-art ML systems for personalizing large-scale, high-quality conversational assistant systems. As a Applied Scientist, you will play a critical role in driving the development of personalization techniques enabling conversational systems, in particular those based on large language models, information retrieval, recommender systems and knowledge graph, to be tailored to customer needs. You will handle Amazon-scale use cases with significant impact on our customers' experiences. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized assistant systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for contextual knowledge extraction and information retrieval, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, compute, latency and quality - Research in advanced customer understanding and behavior modeling techniques - Collaborate with cross-functional teams of scientists, engineers, and product managers to identify and solve complex problems in personal knowledge aggregation, processing, modeling, and verification - Design and execute experiments to evaluate the performance of state-of-the-art algorithms and models, and iterate quickly to improve results - Think Big on conversational assistant system personalization over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports About the team The AGI Personalization org uses various contextual signals to personalize Large Language Model output for our customers while maintaining privacy and security of customer data. We work across multiple Amazon products, including Alexa, to enhance the user experience by bringing more personal context and relevance to customer interactions.
US, NY, New York
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. As a core product offering within our advertising portfolio, Sponsored Products (SP) helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The SP team's primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! Why you love this opportunity Amazon is investing heavily in building a world-class advertising business. This team is responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven fundamentally from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Key job responsibilities Key job responsibilities As an Applied Scientist III on this team you will: * Lead complex and ambiguous projects to deliver bidding recommendation products to advertisers. * Build machine learning models and utilize data analysis to deliver scalable solutions to business problems. * Perform hands-on analysis and modeling with very large data sets to develop insights that increase traffic monetization and merchandise sales without compromising shopper experience. * Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production. * Design and run A/B experiments that affect hundreds of millions of customers, evaluate the impact of your optimizations and communicate your results to various business stakeholders. * Work with scientists and economists to model the interaction between organic sales and sponsored content and to further evolve Amazon's marketplace. * Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. * Research new predictive learning approaches for the sponsored products business. * Write production code to bring models into production. * Mentor junior scientists and engineer in the team.
CA, ON, Toronto
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. As a core product offering within our advertising portfolio, Sponsored Products (SP) helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The SP team's primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! Why you love this opportunity Amazon is investing heavily in building a world-class advertising business. This team is responsible for defining and delivering a collection of advertising products that drive discovery and sales. Our solutions generate billions in revenue and drive long-term growth for Amazon’s Retail and Marketplace businesses. We deliver billions of ad impressions, millions of clicks daily, and break fresh ground to create world-class products. We are highly motivated, collaborative, and fun-loving team with an entrepreneurial spirit - with a broad mandate to experiment and innovate. Impact and Career Growth You will invent new experiences and influence customer-facing shopping experiences to help suppliers grow their retail business and the auction dynamics that leverage native advertising; this is your opportunity to work within the fastest-growing businesses across all of Amazon! Define a long-term science vision for our advertising business, driven fundamentally from our customers' needs, translating that direction into specific plans for research and applied scientists, as well as engineering and product teams. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding. Key job responsibilities As an Applied Scientist on this team you will: * Build machine learning models and utilize data analysis to deliver scalable solutions to business problems. * Perform hands-on analysis and modeling with very large data sets to develop insights that increase traffic monetization and merchandise sales without compromising shopper experience. * Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production. * Design and run A/B experiments that affect hundreds of millions of customers, evaluate the impact of your optimizations and communicate your results to various business stakeholders. * Work with scientists and economists to model the interaction between organic sales and sponsored content and to further evolve Amazon's marketplace. * Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. * Research new predictive learning approaches for the sponsored products business. * Write production code to bring models into production.
US, WA, Seattle
We are seeking a Senior Applied Scientist to join our AI Security team, which builds security tooling and paved path solutions to ensure Generative AI (GenAI) based experiences developed by Amazon uphold our high security standards to protect customer data and trust. As a Senior Applied Scientist, you’ll be responsible for researching, modeling, designing, and implementing state-of-the-art solutions, to secure GenAI products and services at Amazon scale. You will collaborate with applied scientists, security engineers, software engineers, as well as internal partners and external researchers to develop innovative technologies to solve some of our hardest security problems, and build paved path solutions that support builder teams across Amazon throughout their software development journey, enabling Amazon businesses to accelerate the use of GenAI to enhance our user experiences and delight our customers. Key job responsibilities • Research and develop accurate and scalable methods to solve our hardest AI security problems. • Lead and partner with applied scientists and engineers to drive modeling and technical design for complex problems in GenAI security. • Build security tooling and paved path solutions that support builder teams throughout their software development journey. About the team About the team Diverse Experiences Amazon Security values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon Security At Amazon, security is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for security across all of Amazon’s products and services. We offer talented security professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve. Inclusive Team Culture In Amazon Security, it’s in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest security challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional.
US, CA, Sunnyvale
The Amazon Artificial General Intelligence (AGI) Personalization team is looking for a passionate, highly skilled and inventive Senior Applied Scientist with strong machine learning background to lead the development and implementation of state-of-the-art ML systems for personalizing large-scale, high-quality conversational assistant systems. As a Senior Applied Scientist, you will play a critical role in driving the development of personalization techniques enabling conversational systems, in particular those based on large language models, information retrieval, recommender systems and knowledge graph, to be tailored to customer needs. You will handle Amazon-scale use cases with significant impact on our customers' experiences. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized assistant systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for contextual knowledge extraction and information retrieval, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, compute, latency and quality - Research in advanced customer understanding and behavior modeling techniques - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in personal knowledge aggregation, processing, modeling, and verification - Design and execute experiments to evaluate the performance of state-of-the-art algorithms and models, and iterate quickly to improve results - Think Big about the arc of development of conversational assistant system personalization over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports - Mentor and guide junior scientists and engineers, and contribute to the overall growth and development of the team About the team The AGI Personalization org uses various contextual signals to personalize Large Language Model output for our customers while maintaining privacy and security of customer data. We work across multiple Amazon products, including Alexa, to enhance the user experience by bringing more personal context and relevance to customer interactions.
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the extreme. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best.
US, NY, New York
Are you passionate about innovating and delivering shopping experiences that empower every Amazon customer based on their unique shopping intent? Are you looking for new challenges and solving hard problems while using top-notch LLM and GenAI techniques? Join us and you'll help millions of customers to make informed purchase decisions from a multi-billion product selection! Key job responsibilities You’ll be utilizing your Generative AI, time series and predictive modeling skills, and creative problem-solving skills to drive new projects from ideation to implementation. You will lead efforts in foundational models to develop new approaches to Personalization, and provide opportunities for scientists and engineers to invent and implement scalable ML recommendations supporting new Customer Experiences. Your science expertise will be leveraged to research and deliver often novel solutions to existing problems, explore emerging problems spaces, and create or organize knowledge around them. You’ll publish papers and file patents. And you’ll work closely with engineers to put your ideas into production. You will participate in the Amazon ML community by authoring scientific papers and submitting them to ML conferences. About the team The Discover Innovation team is dedicated to assisting customers in fulfilling their shopping objectives and empowering other Amazon teams by leveraging our deep understanding of customer intent. Our team’s mission is to delight customers by helping them discover the right content at the right time, tailoring the shopping experience to the customer’s intent and unique shopping objectives. We are a team of dedicated Scientists, Engineers, and Designers working together to deliver new, personalized shopping experiences. In tailoring the shopping experience to each customer’s intent, we envision operating seamlessly across Amazon as a talented personal shopping assistant — a partner that is trusted, knowledgeable, and understands your unique preferences.
US, CA, San Francisco
If you are interested in this position, please apply on Twitch's Career site https://www.twitch.tv/jobs/en/ About Us: Twitch is the world’s biggest live streaming service, with global communities built around gaming, entertainment, music, sports, cooking, and more. It is where thousands of communities come together for whatever, every day. We’re about community, inside and out. You’ll find coworkers who are eager to team up, collaborate, and smash (or elegantly solve) problems together. We’re on a quest to empower live communities, so if this sounds good to you, see what we’re up to on LinkedIn and Twitter, and discover the projects we’re solving on our Blog. Be sure to explore our Interviewing Guide to learn how to ace our interview process. About the Role: We are looking for an Applied Scientist to solve challenging and open-ended problems in the domain of recommendations, search, ranking and information retrieval. As an Applied Scientist on Twitch's Community team, you will use ML to help viewers find streamers and communities they’ll love. You will collaborate with a team of passionate scientists and engineers to develop these models and put them into production, where they can help Twitch's creators and viewers succeed and build communities. You will report to the Applied Science Manager on the Community Discovery Team. This position is located in San Francisco, CA. You Will: - Develop and Productionize ML algorithms for recommendations, ranking and search problems that can improve discovery on Twitch. - Collaborate with our Product and Engineering teams to work backwards from customer discovery problems, to determine the ML solution (algorithm and pipeline) to have the biggest impact on our user base in the real world. - Participate in the scientific community at Twitch, Amazon, and the broader ML and risk community. Perks - Medical, Dental, Vision & Disability Insurance - 401(k) - Maternity & Parental Leave - Flexible PTO - Amazon Employee Discount
US, CA, Sunnyvale
The Amazon Artificial General Intelligence (AGI) Personalization team is looking for a passionate, highly skilled and inventive Senior Applied Scientist with strong machine learning background to lead the development and implementation of state-of-the-art ML systems for personalizing large-scale, high-quality conversational assistant systems. As a Senior Applied Scientist, you will play a critical role in driving the development of personalization techniques enabling conversational systems, in particular those based on large language models, information retrieval, recommender systems and knowledge graph, to be tailored to customer needs. You will handle Amazon-scale use cases with significant impact on our customers' experiences. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized assistant systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for contextual knowledge extraction and information retrieval, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, compute, latency and quality - Research in advanced customer understanding and behavior modeling techniques - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in personal knowledge aggregation, processing, modeling, and verification - Design and execute experiments to evaluate the performance of state-of-the-art algorithms and models, and iterate quickly to improve results - Think Big about the arc of development of conversational assistant system personalization over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports - Mentor and guide junior scientists and engineers, and contribute to the overall growth and development of the team About the team The AGI Personalization org uses various contextual signals to personalize Large Language Model output for our customers while maintaining privacy and security of customer data. We work across multiple Amazon products, including Alexa, to enhance the user experience by bringing more personal context and relevance to customer interactions.
US, MA, North Reading
We are looking for experienced scientists and engineers to explore new ideas, invent new approaches, and develop new solutions in the areas of Controls, Dynamic modeling and System identification. Are you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart team of doers that work passionately to apply cutting edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even imagine yet. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling and fun. Key job responsibilities Applied Scientists take on big unanswered questions and guide development team to state-of-the-art solutions. We want to hear from you if you have deep industry experience in the Mechatronics domain and : * the ability to think big and conceive of new ideas and novel solutions; * the insight to correctly identify those worth exploring; * the hands-on skills to quickly develop proofs-of-concept; * the rigor to conduct careful experimental evaluations; * the discipline to fast-fail when data refutes theory; * and the fortitude to continue exploring until your solution is found