Social
On the left is the logo for the Alexa Prize Socialbot Grand Challenge 4. On the right is a group photo of Team Alquist from Czech Technical University which won the Alexa Prize SocialBot Grand Challenge 4 competition.
Team Alquist from Czech Technical University has won the Alexa Prize SocialBot Grand Challenge 4 competition. The team, which was awarded the $500,000 first prize for earning the top score in the finals competition, already is looking forward to the next challenge says its faculty advisor, Jan Sedivy (far right).
Credit: Glynis Condon
Conversational AI

Czech Technical University team wins Alexa Prize SocialBot Grand Challenge 4

Team Alquist awarded $500,000 prize for top score in finals competition; teams from Stanford University and the University of Buffalo place second and third.

By Alexa Prize team
August 16, 2021
4 min read
Share

The Czech Republic captured four gold medals at the 2021 summer Olympics in Japan, quite a feat for the Central European country with a population of just over 10 million people.

The country now has another gold-medal-winning team — Alquist from Czech Technical University (CTU) in Prague, which today learned it is the winner of the 2021 Alexa Prize SocialBot Grand Challenge.

“We are incredibly excited to learn that we have won this year’s competition,”said Jakub Konrád, a CTU PhD student and Alquist’s team leader. “I am delighted and proud of our entire team for building a bot that managed to reach the finals for the fourth consecutive year. This year we strove to create a system capable of flexible conversation by synthesizing generative approaches with prepared scenarios that could adjust to users’ needs.”

Faculty advisors provide their perspectives

Faculty advisors to each of the finalists provided their perspectives on the Alexa Prize Socialbot Grand Challenge 4 competition. Learn more about their insights, and read the teams' research papers.

The team’s faculty advisor, Jan Sedivy, added that this year’s team had great fun designing “catchy and attractive dialogues”, and that the team already is looking forward to joining the next challenge.

The Alexa Prize SocialBot Grand Challenge, launched in 2016, is a competition for university students dedicated to advancing the field of conversational AI. Teams are challenged to design socialbots that Alexa customers can interact with via Alexa-enabled devices. The ultimate goal: meet the Grand Challenge by earning a composite score of 4.0 or higher (out of 5) from competition judges. Additionally, the finals’ judges must determine that at least two-thirds of their interactions with the socialbot were coherent and engaging for a minimum of 20 minutes. The first team to meet the Grand Challenge will win a $1 million research grant for its university.

Although none of this year’s teams met the Grand Challenge, each finalist demonstrated impressive progress toward the goal. Alquist, the socialbot from CTU, earned first place with a 3.28 average rating, and an average finals’ competition interaction duration of 14 minutes and 14 seconds. For the second consecutive year, Stanford University’s Chirpy Cardinal socialbot earned second-place honors and a $100,000 prize by achieving a 3.25 average rating, and an average of 13 minutes and 25 seconds of interaction duration. PROTO, the socialbot from the University of Buffalo team, earned third-place honors with an average rating of 3.16, and an average of 14 minutes and 45 seconds of interaction duration.

Stanford University Alexa Prize team
Stanford University's Chirpy Cardinal team earned second place and a $100,000 prize.
Credit: Stanford

“Team Chirpy Cardinal really enjoyed participating in the Alexa Prize for the second time,” said Ethan Chi, a research assistant within Stanford’s NLP Group, and team leader. “Throughout this experience, we've learned so much about real-world dialogue. We almost completely rebuilt our system from the ground up, allowing us to handle a greatly expanded variety of user comments and interjections, and we developed new neural techniques to blend factual knowledge fluidly into our conversations. We even integrated news from The Guardian into our system, allowing us to build common ground with our users over recent events. Compared to our socialbot's previous incarnation, we more than doubled our average finals conversation length, bringing us closer to our shared goal of fluent conversational AI.”

“From an innovation perspective, our aim was to create an agent that wouldn’t restrict interaction to a defined set of topics,” said Sougata Saha, a PhD student at the University of Buffalo, and PROTO’s team lead. “Our use of an ensemble of factual and chit-chat neural generators, coupled with a robust dialogue manager, helped us achieve our third-place finish.”

University of Buffalo Alexa Prize team
The PROTO team from the University of Buffalo earned third place in the competition, and a $50,000 prize.
Credit: University of Buffalo

Last November, nine teams were selected to participate in the competition, and in July five finalists were selected to compete in the finals competition, which took place July 27-29. The finals competition also included teams from Emory University, and The University of California, Santa Cruz.

“Building open-domain conversational systems that allow customers to engage on topics ranging from sports and entertainment, to politics and technology is an incredibly challenging task,” said Prem Natarajan, Alexa AI vice president of Natural Understanding. “Creating socialbots that can conduct these kinds of multi-turn, open-domain interactions is still far from a solved problem. The fact that the top teams participating in this year’s finals competition more than doubled the average duration of interactions over the previous challenge demonstrates that we continue to make impressive progress toward that goal.”

Each of the nine teams participating in Alexa Prize Grand Challenge 4 has published a research paper outlining their approaches to this year’s competition. The papers are now available on the Alexa Prize website.

Since 2017, Alexa customers have engaged with Alexa Prize socialbots for more than 900,000 hours. Alexa customers can continue to engage with the winning teams’ socialbots simply by saying, “Alexa, let’s chat.” 

Previous challenge winners include teams from the University of Washington, the University of California, Davis, and Emory University.  

Alexa Prize SocialBot Grand Challenge 4

Alexa Prize program expands

The Alexa Prize program has expanded to include another competition, as well. Earlier this year, Amazon launched the Alexa Prize TaskBot Challenge, in which 10 participating teams are competing to develop agents that assist customers in completing tasks that require multiple steps and decisions. It is the first conversational AI challenge to incorporate multimodal (voice and vision) and interactive customer experiences. The year-long competition concludes in May 2022, with winners announced the following month.

More information about the challenge is available on the competition’s frequently asked questions page. Alexa customers will have the opportunity to interact with the taskbots beginning in October 2021.

In the coming months, Amazon Science will have details on the forthcoming Alexa Prize Socialbot Grand Challenge 5.

Research areas
Tags
About the Author
Alexa Prize team

Latest news

The latest updates, stories, and more about Alexa Prize.
Load more

Work with us

See more jobs See more jobs
Wireless Science Manager, Device Connectivity
US, CA, Sunnyvale
Come join the Device connectivity team in building the next generation of innovative wireless solution that create a magical experience on our products and services. We actively engage in strategic initiatives, foster partnerships with industry and academia, leverage foundational artificial intelligence and large language models to stay at the forefront of the technological advancements. We are seeking an experienced Applied Science Manager to lead and grow a team of applied scientists who are pushing the boundaries of AI/ML in wireless connectivity and sensing. In this role, you will combine deep technical expertise with strong people leadership to drive scientific innovation that directly impacts millions of customers worldwide. Key job responsibilities As a Applied Science Manager in the team, you will: Build, mentor, and develop a high-performing team of applied scientists, setting the technical bar through code reviews, design reviews, and hands-on contributions while fostering a culture of scientific excellence, innovation, and operational rigor. Define and drive the AI/ML science roadmap for wireless solutions by developing a deep understanding of Amazon's Devices and Services offerings, translating complex business problems into well-defined scientific challenges, identifying high-risk and high-impact technical directions, and guiding your team to deliver them from conception through production. Collaborate cross-functionally with engineering, product, and business partners to drive ML development from research through optimization and onto production devices, aligning science investments with product goals while meeting on-device performance, latency, and resource constraints. Balance exploratory research with production delivery timelines, ensuring the team maintains scientific rigor while meeting business commitments. Represent the team's AI innovations to both internal leadership and the external scientific community through leadership reviews, publications, patents, and conference presentations, providing clear articulation of science strategy, progress, and impact. About the team About the team Device Connectivity team is empowering possibilities through wireless innovation on our devices and through services, our vision is to design and develop transformative products and services that consistently exceed our customers' expectations.
Data Scientist, Demand Forecasting
US, WA, Bellevue
What does it take to build a foundation model that can forecast demand for hundreds of millions of products — including ones that have never been sold before? At Amazon, our Demand Forecasting team is tackling one of the most ambitious challenges in applied time series research: building large-scale foundation models that generalize across an enormous and diverse catalog of products, geographies, and business contexts. This is not incremental modeling work. We are redefining what's possible in demand forecasting. Our team operates at a scale that is unmatched in industry. We run experiments across millions of products simultaneously, pushing the boundaries of what foundation models can learn from vast, heterogeneous time series data. We are also exploring novel data generation techniques that augment our already unprecedented dataset — opening new frontiers in model generalization and forecasting for products with limited or no sales history. The models you build here will ship to production and directly influence hundreds of millions of dollars in automated inventory decisions every week, labor plans for tens of thousands of employees, and Amazon's financial outlook. Beyond operational impact, this team contributes to the broader scientific community and advances the state of the art in time series foundation models. If you are a scientist who wants to work at the frontier of time series research, at a scale no academic lab or startup can match, and see your work deployed to real-world impact — this is the team for you. Key job responsibilities - Design and run rigorous experiments at scale to evaluate and improve foundation model performance across hundreds of millions of products, geographies, and business verticals - Lead the end-to-end lifecycle of forecasting models — from research and experimentation through production launch — including defining success metrics, obtaining stakeholder sign-off, and managing rollout - Conduct online and offline labs to measure the real-world impact of forecast improvements beyond accuracy, including downstream supply chain, inventory, and financial outcomes - Develop and deploy production-grade deep learning and statistical models using Python, Scala, SQL, and related tools - Perform large-scale exploratory data analysis to uncover patterns, identify opportunities, and inform model development - Translate complex research findings into clear insights and recommendations for technical and non-technical stakeholders at all levels - Contribute to Amazon's scientific community and the broader research field through collaboration and publication in top-tier venues A day in the life No two days look the same, but most will involve some combination of deep technical work, cross-functional collaboration, and scientific thinking at a scale you won't find anywhere else. You might start the morning reviewing the results of an experiment running across hundreds of millions of products — analyzing whether a new foundation model variant is improving generalization on cold-start items, or whether a novel data generation approach is meaningfully shifting forecast quality. You'll dig into the numbers, form a hypothesis, and design the next iteration. Later in the day, you could be in a stakeholder review, walking business and engineering partners through a set of launch metrics — explaining not just forecast accuracy, but the downstream supply chain and financial impact your model is driving. Getting a model to production at Amazon requires rigor: you'll define success criteria, run online and offline labs to validate real-world impact, and build the case for sign-off across technical and business stakeholders. You'll write code — Python, Scala, SQL — to process and analyze data at a scale most scientists never encounter. You'll collaborate closely with scientists, engineers, and business teams, and contribute to research that has a real chance of being published and advancing the field. The work is hard, the problems are unsolved, and the impact is immediate. If you want to do research that ships — this is where you do it. About the team The Demand Forecasting team sits at the heart of Amazon's supply chain, building the science that determines what products are available, when, and at what cost — for hundreds of millions of customers around the world. Our mission is to push the frontier of what's possible in large-scale time series forecasting, and to deploy that science where it creates real, measurable impact. We are a team of scientists who care deeply about both research rigor and real-world outcomes. We don't just publish — we ship. And we don't just ship — we measure, iterate, and raise the bar. Our work spans the full lifecycle: from foundational research and large-scale experimentation to production deployment and downstream impact measurement across supply chain, inventory, and financial planning.
Applied Scientist II, Alexa Ads
IN, KA, Bengaluru
Alexa+ is the world’s best Generative AI powered personal assistant / agent for consumers, and is becoming the conversational AI interface for Amazon services with the launch of Alexa for Shopping on Amazon.com and Amazon mobile app. At Alexa Ads, we are creating industry's first and most advanced Agentic Advertising products to drive Agentic Commerce. We are seeking an Applied Scientist to join our newly expanding team in India focused on Alexa Agentic/Conversational Ads and Personalization. In this role, you will build machine learning models that seamlessly and naturally integrate relevant advertising into the Alexa experience while deeply personalizing user interactions. You will work closely with other scientists, engineers, and product managers to take models from conception to production. Key job responsibilities - Design, develop, and evaluate innovative machine learning and deep learning models for natural language processing (NLP), recommendation systems, and personalization. - Conduct hands-on data analysis and build scalable ML pipelines. - Design and run A/B experiments to measure the impact of new models on customer experience and ad performance. - Collaborate with software development engineers to deploy models into high-scale, real-time production environments. About the team We are building a new science team in Bangalore to solve some of the most impactful problems in computational advertising. This isn't about tweaking existing models as we are rethinking how ads are ranked, priced, and personalized across voice-first and screen-first surfaces. These are problems that don't have textbook solutions. Key points to note about the team: 🧪 Greenfield team - you are not joining a mature org with rigid processes. You will shape the science roadmap, pick the problems, and define the culture from day one. 📈 Direct business impact — your models directly drive revenue. No yearly cycles to see if your work matters. 🌏 Global scope, local autonomy — collaborate with scientists and engineers across Seattle, Sunnyvale, and Bangalore, but own your problem space end-to-end. 🎓 Ship AND Publish: We encourage top-tier publications (NeurIPS, ACL, EMNLP, KDD, ICML, WWW) while ensuring your research hits production.
Manager, Research Science, Special Projects
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the extreme. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best.
Applied Scientist, Fauna
US, CA, Sunnyvale
We are seeking an Applied Scientist to focus on Robotics Spatial Intelligence and Semantic Understanding. In this role, you'll research and build advanced semantic and world understanding algorithms that enable robots to observe, understand, and reason about complex and dynamic home environments. You'll work across a broad spectrum of 3D perception, contextual understanding, and world modeling approaches to build robust solutions that support autonomous decision making, task planning, navigation, and manipulation. Key job responsibilities - Develop and implement robust World Understanding and Modeling algorithms for a domestic robot. - Build simulation-based and on-robot evaluation frameworks with comprehensive benchmarks and metrics for systematic evaluation of Our Spatial Intelligence stack. - Conduct sim-to-real transfer experiments, analyzing performance gaps and developing techniques to ensure reliable real-world performance. - Collaborate with navigation, manipulation, and other teams to ensure seamless integration of World Understanding capabilities. - Stay current with the latest advances in World Modeling, Spatial Reasoning, and related fields and apply relevant findings to improve system performance About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
Airspace Management Simulation Applied Scientist, Prime Air Flight Sciences High Fidelity Methods
US, WA, Seattle
Here at Amazon, we embrace our differences. We are committed to furthering our culture of diversity and inclusion of our teams within the organization. How do you get items to customers quickly, cost-effectively, and—most importantly—safely, in less than an hour? And how do you do it in a way that can scale? Our teams of hundreds of scientists, engineers, aerospace professionals, and futurists have been working hard to do just that! We are delivering to customers, and are excited for what’s to come. Check out more information about Prime Air on the About Amazon blog (https://www.aboutamazon.com/news/transportation/amazon-prime-air-delivery-drone-reveal-photos). If you are seeking an iterative environment where you can drive innovation, apply state-of-the-art technologies to solve real world delivery challenges, and provide benefits to customers, Prime Air is the place for you. Come work on the Amazon Prime Air Team! We're looking for a Research Scientist with a background in developing simulations for traffic management algorithms, including expert knowledge in strategic deconfliction, tactical deconfliction, or detect-and-avoid systems. Managing a large number of concurrent autonomous drone flights that share airspace with other autonomous or manned aircraft is a challenging problem. Be part of the team building simulation tools and algorithms to solve this at scale. This role will contribute to a portfolio of simulation tools managing concurrent airspace traffic for aviation systems. This will include developing new methodologies in the areas of conflict detection and resolution, as well as developing related software systems that will be used in operation to enable package delivery at scale. The ideal candidate is comfortable with risk-taking and ambiguity and able to build consensus on critical, controversial technical decisions. If you enjoy the process of solving real-world problems that haven’t been solved at scale anywhere before, Prime Air is right for you. Along the way, we guarantee you’ll get opportunities to be a disruptor, prolific innovator, and a reputed problem solver and directly impact Amazon’s customers worldwide. Key job responsibilities The primary focus of this role will be on modeling traffic management frameworks that use a layered conflict detection and resolution strategy to ensure safe and efficient flight operations. This will include developing fundamental simulation infrastructure code, including discrete event simulation tooling. In addition, it will involve developing expert knowledge of the layers of mitigation and conducting in-depth scientific research on alternative solutions for conflict resolution. The candidate will contribute to significant and impactful systems that will provide value for Amazon customers and will drive these projects from the concept stage through development. This role will include substantial software development in prototyping and production environments.
Applied Scientist, AWS Automated Reasoning
US, WA, Seattle
Applied Scientists in AWS Automated Reasoning are dedicated to making AWS the best computing service in the world for customers who require advanced and rigorous solutions for automated reasoning, privacy, and sovereignty. Key job responsibilities The successful candidate will: - Solve large or significantly complex problems that require deep knowledge and understanding of your domain and scientific innovation. - Own strategic problem solving, and take the lead on the design, implementation, and delivery for solutions that have a long-term quantifiable impact. - Provide cross-organizational technical influence, increasing productivity and effectiveness by sharing your deep knowledge and experience. - Develop strategic plans to identify fundamentally new solutions for business problems. - Assist in the career development of others, actively mentoring individuals and the community on advanced technical issues. A day in the life This is a unique and rare opportunity to get in early on a fast-growing segment of AWS and help shape the technology, product and the business. You will have a chance to utilize your deep technical experience within a fast moving, start-up environment and make a large business and customer impact. About the team Diverse Experiences Amazon Automated Reasoning values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying. Why Amazon Automated Reasoning? At Amazon, automated reasoning is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for automated reasoning across all of Amazon's products and services. We offer talented automated reasoning professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Automated Reasoning, it's in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest automated reasoning challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve.
Sr. Applied Scientist, AWS Automated Reasoning
US, WA, Seattle
Applied Scientists in AWS Automated Reasoning are dedicated to making AWS the best computing service in the world for customers who require advanced and rigorous solutions for automated reasoning, privacy, and sovereignty. Key job responsibilities The successful candidate will: - Solve large or significantly complex problems that require deep knowledge and understanding of your domain and scientific innovation. - Own strategic problem solving, and take the lead on the design, implementation, and delivery for solutions that have a long-term quantifiable impact. - Provide cross-organizational technical influence, increasing productivity and effectiveness by sharing your deep knowledge and experience. - Develop strategic plans to identify fundamentally new solutions for business problems. - Assist in the career development of others, actively mentoring individuals and the community on advanced technical issues. A day in the life This is a unique and rare opportunity to get in early on a fast-growing segment of AWS and help shape the technology, product and the business. You will have a chance to utilize your deep technical experience within a fast moving, start-up environment and make a large business and customer impact. About the team Diverse Experiences Amazon Automated Reasoning values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying. Why Amazon Automated Reasoning? At Amazon, automated reasoning is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for automated reasoning across all of Amazon's products and services. We offer talented automated reasoning professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Automated Reasoning, it's in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest automated reasoning challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve.
Applied scientist, Agentic AI, AWS Agentic AI
IL, Tel Aviv
Come join the AWS Agentic AI science team in building the next generation models for intelligent automation. AWS, the world-leading provider of cloud services, has fostered the creation and growth of countless new businesses, and is a positive force for good. Our customers bring problems that will give Applied Scientists like you endless opportunities to see your research have a positive and immediate impact in the world. You will have the opportunity to partner with technology and business teams to solve real-world problems, have access to virtually endless data and computational resources, and to world-class engineers and developers that can help bring your ideas into the world. As part of the team, we expect that you will develop innovative solutions to hard problems, and publish your findings at peer reviewed conferences and workshops. We are looking for world class researchers with experience in one or more of the following areas - autonomous agents, API orchestration, Planning, large multimodal models (especially vision-language models), reinforcement learning (RL) and sequential decision making.
Sr Applied Scientist, Support Agent Intelligence and Evaluation
US, WA, Seattle
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. Amazon's advertising portfolio helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Millions of advertisers rely on Amazon's self-service support experience to resolve issues, unblock campaigns, and grow their business. Our Support Agents team is building the science behind intelligent, conversational support — systems that understand advertiser intent, retrieve the right knowledge, generate accurate answers, and know when to escalate. We serve ~2M monthly active advertisers across dozens of locales and languages, and every percentage point of improvement in resolution quality translates directly into advertiser success and retention. We are seeking an Applied Scientist who is passionate about building evaluation science, NLP systems, and quality measurement at scale. You will define how we measure "good" — designing LLM-as-a-judge evaluation pipelines, developing our next-generation Issue Resolution Rate (IRR) metrics, and closing the quality gap between English and non-English markets. Your work will directly shape what ships to advertisers and what leadership uses to assess the health of our support experience. Key job responsibilities 1. Enhance support agent capabilities across the broad suite of Amazon Advertising products — expanding coverage, depth of resolution, and advertiser task completion across Sponsored Products, Sponsored Brands, DSP, AMC, and more 2. Design and own the evaluation framework for agent quality — including automated LLM-based scoring of answer correctness, confidence calibration, and conversation-level resolution signals 3. Develop novel metrics that capture whether advertisers actually got the help they needed (beyond surface-level deflection rates) 4. Build and improve retrieval and generation models that power real-time advertiser interactions under strict latency SLAs 5. Drive multilingual science — improve non-English resolution rates through cross-lingual retrieval, translation quality modeling, and locale-aware evaluation 6. Partner with product, engineering, and business teams to productize research and inform roadmap decisions with data A day in the life You might start the morning reviewing overnight evaluation results from your LLM-as-a-judge pipeline, then jump into a whiteboard session designing a new resolution metric that captures whether advertisers actually unblocked their campaign. After lunch you're running offline experiments on a cross-lingual retrieval model to close the quality gap for non-English markets, and by end of day you're syncing with engineering on latency trade-offs for next week's A/B test. The constant: your science directly changes the experience millions of advertisers have when they need help. About the team This role sits within Amazon Advertising's broader Agentic Intelligence organization — a community of multiple Applied Science and Engineering teams building the next generation of AI-powered experiences for advertisers. You'll have access to Principal Engineers and Principal Applied Scientists to pressure-test ideas and elevate your work. What makes this team unique is the balance: you'll drive product-facing science through customer support agents that touch millions of advertisers, while also influencing and collaborating with a core AI infrastructure team within Amazon. The team is cross-functional — scientists and engineers work shoulder-to-shoulder, from problem framing through production deployment.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social