publications-lead-image-2.png

Publications

Amazon is a great place to practice science and have real business impact, but that's only one part of the story. Our scientists continue to publish, teach, and engage with the worldwide research community, sharing insights across diverse disciplines from machine learning to operations research. Through these contributions, we're advancing scientific knowledge while developing innovations that address complex challenges for customers and society. 
4,376 results found
  • Czech Technical University in Prague
    Amazon Nova AI Challenge Proceedings
    2025
    We introduce AlquistCoder, a code-generating system that effectively minimizes the risk of producing malicious content or vulnerable code while maintaining excellent Python coding and question answering standards across a wide range of tasks. The architecture of AlquistCoder employs a sophisticated input guardrail classifier that analyzes whether the user’s intention is benign, potentially harmful, or falls
  • University of Texas at Dallas
    Amazon Nova AI Challenge Proceedings
    2025
    WARNING: Contains harmful content that can be offensive in nature Large language models (LLMs) for code generation have enhanced developer productivity while introducing new misuse vectors, as these models can generate potentially harmful code. Existing evaluation methods fail to assess such misuse scenarios, and structured red teaming pipelines for code generation remain under-developed. This paper presents
  • University of California, Davis
    Amazon Nova AI Challenge Proceedings
    2025
    Large Language Models (LLMs) for code generation (i.e., Code LLMs) have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to gen-erating vulnerable or even malicious code under adversarial settings. Existing red-teaming approaches rely on extensive human effort, limiting their scalability and practicality,
  • Virginia Tech
    Amazon Nova AI Challenge Proceedings
    2025
    While large language models (LLMs) achieve strong performance in code generation, persistent security vulnerabilities hinder their safe deployment. Starting from a pretrained CodeGen model without inherent safety mechanisms, we develop a systematic synthetic instruction-tuning workflow to progressively enhance model security. Our pipeline begins with taxonomy-guided synthetic data, capturing diverse attack
  • Columbia University
    Amazon Nova AI Challenge Proceedings
    2025
    We present SecureLion, a trustworthy AI assistant designed to securely han-dle cybersecurity queries and generate vulnerability-free code. Compared to the base model, our system achieves a 76.6% relative reduction on insecure messages in an adversarial competition setting at the cost of negligible utility loss. The success of SecureLion stems primarily from: (1) pervasive security-focused reasoning integrated
  • Purdue University
    Amazon Nova AI Challenge Proceedings
    2025
    AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain—especially in high-stakes domains like cybersecurity. Current red-teaming tools often rely on fixed bench-marks or unrealistic prompts, missing many real-world vulnerabilities. We present ASTRA, an automated agent system designed to systematically uncover safety flaws in AI-driven
  • University of Illinois at Urbana-Champaign
    Amazon Nova AI Challenge Proceedings
    2025
    We introduce PurpCode, a novel post-training method that aligns coding assistants to perform safety reasoning to defend against malicious cyber activities and provide secure and functional code. Our approach trains a reasoning model in two stages:(i) Rule learning, which explicitly teaches the model to reference cyber safety rules to avoid facilitating malicious cyber events and to generate vulnerability-free
  • Carnegie Mellon University
    Amazon Nova AI Challenge Proceedings
    2025
    Warning: This report contains partially redacted content that may be offensive to the reader Large language models (LLMs) may assist users with malicious cybersecurity at-tacks or inadvertently generate code with critical security flaws. These failures stem from their broader inability to reliably identify safe data or generate safe outputs, despite advances in alignment research. We identify three potential
  • NOVA University Lisbon
    Amazon Nova AI Challenge Proceedings
    2025
    This paper presents the vision, scientific contributions, and technical details of RedTWIZ: an adaptive and diverse multi-turn red teaming framework, to audit the robustness of Large Language Models (LLMs) in AI-assisted software development. Our work is driven by three major research streams: (1) robust and systematic assessment of LLM conversational jailbreaks; (2) a diverse generative multi-turn attack
  • University of Wisconsin-Madison
    Amazon Nova AI Challenge Proceedings
    2025
    In this technical report, we present our automated red-teaming framework designed to induce jailbreaks in targeted code-generating Large Language Models (LLMs), prompting them to generate malicious and vulnerable code. As of May 13, according to the latest competition leaderboard, our solution has achieved top performance in the second tournament. Our solution consists of three primary modules. First, we
  • Haris Riaz, Sourav Bhabesh, Vinayak Arannil, Miguel Ballesteros, Graham Horwood
    2025
    Recent smaller language models such Phi-3.5 and Phi-4 rely on synthetic data generated using larger Language models. Questions remain about leveraging synthetic data for other use cases, such as adapting LLMs to specific domains. A key limitation of synthetic data is low diversity, which negatively impacts its downstream applicability for improving other models. To address this, we propose MetaSynth, a
  • KDD 2025 Workshop on Evaluation and Trustworthiness of Agentic and Generative AI Models
    2025
    The emergence of AI-driven web automation through Large Language Models (LLMs) offers unprecedented opportunities for optimizing digital workflows. However, deploying such systems within industry's real-world environments presents four core challenges: (1) ensuring consistent execution, (2) accurately identifying critical HTML elements, (3) meeting human-like accuracy in order to automate operations at
  • Sanghun Jung, Jingjing Zheng, Ke Zhang, Nan Qiao, Albert Chen, Lu Xia, Chi Liu, Yuyin Sun, Xiao Zeng, Hsiang-Wei Huang, Byron Boots, Min Sun, Cheng-Hao Kuo
    2025
    Unlike closed-vocabulary 3D instance segmentation that is often trained end-to-end, open-vocabulary 3D instance segmentation (OV-3DIS) often leverages vision-language models (VLMs) to generate 3D instance proposals and classify them. While various concepts have been proposed from existing research, we observe that these individual concepts are not mutually exclusive but complementary. In this paper, we
  • Denis Mazzucato, Abdal Mohamed, June Lee, Clark Barrett, Jim Grundy, John Harrison, Corina Pasareanu
    CAV 2025
    2025
    Many security- and performance-critical domains, such as cryptography, rely on low-level verification to minimize the trusted computing surface and allow code to be written directly in assembly. However, verifying assembly code against a realistic machine model is a challenging task. Furthermore, certain security properties—such as constant-time behavior—require relational reasoning that goes beyond traditional
  • Jihyeok Kim, Sungjin Lee, Seung-won Hwang, Yang Liu
    ACL Findings 2025
    2025
    Targeting long-form question-answering, chain-of-query (CoQ) has been studied, integrating chain-of-thought (CoT) with retrieval-augmented generation. CoQ breaks down complex questions into simpler subquestions (SQs), allowing relevant information to be retrieved step by step. By doing so, CoQ aims to improve the answer comprehensiveness and verifiability, at the expense of latency. Our first contribution

Latest news

US, CA, Sunnyvale
Come join the Device connectivity team in building the next generation of innovative wireless solution that create a magical experience on our products and services. We actively engage in strategic initiatives, foster partnerships with industry and academia, leverage foundational artificial intelligence and large language models to stay at the forefront of the technological advancements. We are seeking an experienced Applied Science Manager to lead and grow a team of applied scientists who are pushing the boundaries of AI/ML in wireless connectivity and sensing. In this role, you will combine deep technical expertise with strong people leadership to drive scientific innovation that directly impacts millions of customers worldwide. Key job responsibilities As a Applied Science Manager in the team, you will: Build, mentor, and develop a high-performing team of applied scientists, setting the technical bar through code reviews, design reviews, and hands-on contributions while fostering a culture of scientific excellence, innovation, and operational rigor. Define and drive the AI/ML science roadmap for wireless solutions by developing a deep understanding of Amazon's Devices and Services offerings, translating complex business problems into well-defined scientific challenges, identifying high-risk and high-impact technical directions, and guiding your team to deliver them from conception through production. Collaborate cross-functionally with engineering, product, and business partners to drive ML development from research through optimization and onto production devices, aligning science investments with product goals while meeting on-device performance, latency, and resource constraints. Balance exploratory research with production delivery timelines, ensuring the team maintains scientific rigor while meeting business commitments. Represent the team's AI innovations to both internal leadership and the external scientific community through leadership reviews, publications, patents, and conference presentations, providing clear articulation of science strategy, progress, and impact. About the team About the team Device Connectivity team is empowering possibilities through wireless innovation on our devices and through services, our vision is to design and develop transformative products and services that consistently exceed our customers' expectations.
US, WA, Bellevue
What does it take to build a foundation model that can forecast demand for hundreds of millions of products — including ones that have never been sold before? At Amazon, our Demand Forecasting team is tackling one of the most ambitious challenges in applied time series research: building large-scale foundation models that generalize across an enormous and diverse catalog of products, geographies, and business contexts. This is not incremental modeling work. We are redefining what's possible in demand forecasting. Our team operates at a scale that is unmatched in industry. We run experiments across millions of products simultaneously, pushing the boundaries of what foundation models can learn from vast, heterogeneous time series data. We are also exploring novel data generation techniques that augment our already unprecedented dataset — opening new frontiers in model generalization and forecasting for products with limited or no sales history. The models you build here will ship to production and directly influence hundreds of millions of dollars in automated inventory decisions every week, labor plans for tens of thousands of employees, and Amazon's financial outlook. Beyond operational impact, this team contributes to the broader scientific community and advances the state of the art in time series foundation models. If you are a scientist who wants to work at the frontier of time series research, at a scale no academic lab or startup can match, and see your work deployed to real-world impact — this is the team for you. Key job responsibilities - Design and run rigorous experiments at scale to evaluate and improve foundation model performance across hundreds of millions of products, geographies, and business verticals - Lead the end-to-end lifecycle of forecasting models — from research and experimentation through production launch — including defining success metrics, obtaining stakeholder sign-off, and managing rollout - Conduct online and offline labs to measure the real-world impact of forecast improvements beyond accuracy, including downstream supply chain, inventory, and financial outcomes - Develop and deploy production-grade deep learning and statistical models using Python, Scala, SQL, and related tools - Perform large-scale exploratory data analysis to uncover patterns, identify opportunities, and inform model development - Translate complex research findings into clear insights and recommendations for technical and non-technical stakeholders at all levels - Contribute to Amazon's scientific community and the broader research field through collaboration and publication in top-tier venues A day in the life No two days look the same, but most will involve some combination of deep technical work, cross-functional collaboration, and scientific thinking at a scale you won't find anywhere else. You might start the morning reviewing the results of an experiment running across hundreds of millions of products — analyzing whether a new foundation model variant is improving generalization on cold-start items, or whether a novel data generation approach is meaningfully shifting forecast quality. You'll dig into the numbers, form a hypothesis, and design the next iteration. Later in the day, you could be in a stakeholder review, walking business and engineering partners through a set of launch metrics — explaining not just forecast accuracy, but the downstream supply chain and financial impact your model is driving. Getting a model to production at Amazon requires rigor: you'll define success criteria, run online and offline labs to validate real-world impact, and build the case for sign-off across technical and business stakeholders. You'll write code — Python, Scala, SQL — to process and analyze data at a scale most scientists never encounter. You'll collaborate closely with scientists, engineers, and business teams, and contribute to research that has a real chance of being published and advancing the field. The work is hard, the problems are unsolved, and the impact is immediate. If you want to do research that ships — this is where you do it. About the team The Demand Forecasting team sits at the heart of Amazon's supply chain, building the science that determines what products are available, when, and at what cost — for hundreds of millions of customers around the world. Our mission is to push the frontier of what's possible in large-scale time series forecasting, and to deploy that science where it creates real, measurable impact. We are a team of scientists who care deeply about both research rigor and real-world outcomes. We don't just publish — we ship. And we don't just ship — we measure, iterate, and raise the bar. Our work spans the full lifecycle: from foundational research and large-scale experimentation to production deployment and downstream impact measurement across supply chain, inventory, and financial planning.
IN, KA, Bengaluru
Alexa+ is the world’s best Generative AI powered personal assistant / agent for consumers, and is becoming the conversational AI interface for Amazon services with the launch of Alexa for Shopping on Amazon.com and Amazon mobile app. At Alexa Ads, we are creating industry's first and most advanced Agentic Advertising products to drive Agentic Commerce. We are seeking an Applied Scientist to join our newly expanding team in India focused on Alexa Agentic/Conversational Ads and Personalization. In this role, you will build machine learning models that seamlessly and naturally integrate relevant advertising into the Alexa experience while deeply personalizing user interactions. You will work closely with other scientists, engineers, and product managers to take models from conception to production. Key job responsibilities - Design, develop, and evaluate innovative machine learning and deep learning models for natural language processing (NLP), recommendation systems, and personalization. - Conduct hands-on data analysis and build scalable ML pipelines. - Design and run A/B experiments to measure the impact of new models on customer experience and ad performance. - Collaborate with software development engineers to deploy models into high-scale, real-time production environments. About the team We are building a new science team in Bangalore to solve some of the most impactful problems in computational advertising. This isn't about tweaking existing models as we are rethinking how ads are ranked, priced, and personalized across voice-first and screen-first surfaces. These are problems that don't have textbook solutions. Key points to note about the team: 🧪 Greenfield team - you are not joining a mature org with rigid processes. You will shape the science roadmap, pick the problems, and define the culture from day one. 📈 Direct business impact — your models directly drive revenue. No yearly cycles to see if your work matters. 🌏 Global scope, local autonomy — collaborate with scientists and engineers across Seattle, Sunnyvale, and Bangalore, but own your problem space end-to-end. 🎓 Ship AND Publish: We encourage top-tier publications (NeurIPS, ACL, EMNLP, KDD, ICML, WWW) while ensuring your research hits production.
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the extreme. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best.
US, CA, Sunnyvale
We are seeking an Applied Scientist to focus on Robotics Spatial Intelligence and Semantic Understanding. In this role, you'll research and build advanced semantic and world understanding algorithms that enable robots to observe, understand, and reason about complex and dynamic home environments. You'll work across a broad spectrum of 3D perception, contextual understanding, and world modeling approaches to build robust solutions that support autonomous decision making, task planning, navigation, and manipulation. Key job responsibilities - Develop and implement robust World Understanding and Modeling algorithms for a domestic robot. - Build simulation-based and on-robot evaluation frameworks with comprehensive benchmarks and metrics for systematic evaluation of Our Spatial Intelligence stack. - Conduct sim-to-real transfer experiments, analyzing performance gaps and developing techniques to ensure reliable real-world performance. - Collaborate with navigation, manipulation, and other teams to ensure seamless integration of World Understanding capabilities. - Stay current with the latest advances in World Modeling, Spatial Reasoning, and related fields and apply relevant findings to improve system performance About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
US, WA, Seattle
Here at Amazon, we embrace our differences. We are committed to furthering our culture of diversity and inclusion of our teams within the organization. How do you get items to customers quickly, cost-effectively, and—most importantly—safely, in less than an hour? And how do you do it in a way that can scale? Our teams of hundreds of scientists, engineers, aerospace professionals, and futurists have been working hard to do just that! We are delivering to customers, and are excited for what’s to come. Check out more information about Prime Air on the About Amazon blog (https://www.aboutamazon.com/news/transportation/amazon-prime-air-delivery-drone-reveal-photos). If you are seeking an iterative environment where you can drive innovation, apply state-of-the-art technologies to solve real world delivery challenges, and provide benefits to customers, Prime Air is the place for you. Come work on the Amazon Prime Air Team! We're looking for a Research Scientist with a background in developing simulations for traffic management algorithms, including expert knowledge in strategic deconfliction, tactical deconfliction, or detect-and-avoid systems. Managing a large number of concurrent autonomous drone flights that share airspace with other autonomous or manned aircraft is a challenging problem. Be part of the team building simulation tools and algorithms to solve this at scale. This role will contribute to a portfolio of simulation tools managing concurrent airspace traffic for aviation systems. This will include developing new methodologies in the areas of conflict detection and resolution, as well as developing related software systems that will be used in operation to enable package delivery at scale. The ideal candidate is comfortable with risk-taking and ambiguity and able to build consensus on critical, controversial technical decisions. If you enjoy the process of solving real-world problems that haven’t been solved at scale anywhere before, Prime Air is right for you. Along the way, we guarantee you’ll get opportunities to be a disruptor, prolific innovator, and a reputed problem solver and directly impact Amazon’s customers worldwide. Key job responsibilities The primary focus of this role will be on modeling traffic management frameworks that use a layered conflict detection and resolution strategy to ensure safe and efficient flight operations. This will include developing fundamental simulation infrastructure code, including discrete event simulation tooling. In addition, it will involve developing expert knowledge of the layers of mitigation and conducting in-depth scientific research on alternative solutions for conflict resolution. The candidate will contribute to significant and impactful systems that will provide value for Amazon customers and will drive these projects from the concept stage through development. This role will include substantial software development in prototyping and production environments.
US, WA, Seattle
Applied Scientists in AWS Automated Reasoning are dedicated to making AWS the best computing service in the world for customers who require advanced and rigorous solutions for automated reasoning, privacy, and sovereignty. Key job responsibilities The successful candidate will: - Solve large or significantly complex problems that require deep knowledge and understanding of your domain and scientific innovation. - Own strategic problem solving, and take the lead on the design, implementation, and delivery for solutions that have a long-term quantifiable impact. - Provide cross-organizational technical influence, increasing productivity and effectiveness by sharing your deep knowledge and experience. - Develop strategic plans to identify fundamentally new solutions for business problems. - Assist in the career development of others, actively mentoring individuals and the community on advanced technical issues. A day in the life This is a unique and rare opportunity to get in early on a fast-growing segment of AWS and help shape the technology, product and the business. You will have a chance to utilize your deep technical experience within a fast moving, start-up environment and make a large business and customer impact. About the team Diverse Experiences Amazon Automated Reasoning values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying. Why Amazon Automated Reasoning? At Amazon, automated reasoning is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for automated reasoning across all of Amazon's products and services. We offer talented automated reasoning professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Automated Reasoning, it's in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest automated reasoning challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve.
US, WA, Seattle
Applied Scientists in AWS Automated Reasoning are dedicated to making AWS the best computing service in the world for customers who require advanced and rigorous solutions for automated reasoning, privacy, and sovereignty. Key job responsibilities The successful candidate will: - Solve large or significantly complex problems that require deep knowledge and understanding of your domain and scientific innovation. - Own strategic problem solving, and take the lead on the design, implementation, and delivery for solutions that have a long-term quantifiable impact. - Provide cross-organizational technical influence, increasing productivity and effectiveness by sharing your deep knowledge and experience. - Develop strategic plans to identify fundamentally new solutions for business problems. - Assist in the career development of others, actively mentoring individuals and the community on advanced technical issues. A day in the life This is a unique and rare opportunity to get in early on a fast-growing segment of AWS and help shape the technology, product and the business. You will have a chance to utilize your deep technical experience within a fast moving, start-up environment and make a large business and customer impact. About the team Diverse Experiences Amazon Automated Reasoning values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying. Why Amazon Automated Reasoning? At Amazon, automated reasoning is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for automated reasoning across all of Amazon's products and services. We offer talented automated reasoning professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Automated Reasoning, it's in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest automated reasoning challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve.
IL, Tel Aviv
Come join the AWS Agentic AI science team in building the next generation models for intelligent automation. AWS, the world-leading provider of cloud services, has fostered the creation and growth of countless new businesses, and is a positive force for good. Our customers bring problems that will give Applied Scientists like you endless opportunities to see your research have a positive and immediate impact in the world. You will have the opportunity to partner with technology and business teams to solve real-world problems, have access to virtually endless data and computational resources, and to world-class engineers and developers that can help bring your ideas into the world. As part of the team, we expect that you will develop innovative solutions to hard problems, and publish your findings at peer reviewed conferences and workshops. We are looking for world class researchers with experience in one or more of the following areas - autonomous agents, API orchestration, Planning, large multimodal models (especially vision-language models), reinforcement learning (RL) and sequential decision making.
US, WA, Seattle
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. Amazon's advertising portfolio helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Millions of advertisers rely on Amazon's self-service support experience to resolve issues, unblock campaigns, and grow their business. Our Support Agents team is building the science behind intelligent, conversational support — systems that understand advertiser intent, retrieve the right knowledge, generate accurate answers, and know when to escalate. We serve ~2M monthly active advertisers across dozens of locales and languages, and every percentage point of improvement in resolution quality translates directly into advertiser success and retention. We are seeking an Applied Scientist who is passionate about building evaluation science, NLP systems, and quality measurement at scale. You will define how we measure "good" — designing LLM-as-a-judge evaluation pipelines, developing our next-generation Issue Resolution Rate (IRR) metrics, and closing the quality gap between English and non-English markets. Your work will directly shape what ships to advertisers and what leadership uses to assess the health of our support experience. Key job responsibilities 1. Enhance support agent capabilities across the broad suite of Amazon Advertising products — expanding coverage, depth of resolution, and advertiser task completion across Sponsored Products, Sponsored Brands, DSP, AMC, and more 2. Design and own the evaluation framework for agent quality — including automated LLM-based scoring of answer correctness, confidence calibration, and conversation-level resolution signals 3. Develop novel metrics that capture whether advertisers actually got the help they needed (beyond surface-level deflection rates) 4. Build and improve retrieval and generation models that power real-time advertiser interactions under strict latency SLAs 5. Drive multilingual science — improve non-English resolution rates through cross-lingual retrieval, translation quality modeling, and locale-aware evaluation 6. Partner with product, engineering, and business teams to productize research and inform roadmap decisions with data A day in the life You might start the morning reviewing overnight evaluation results from your LLM-as-a-judge pipeline, then jump into a whiteboard session designing a new resolution metric that captures whether advertisers actually unblocked their campaign. After lunch you're running offline experiments on a cross-lingual retrieval model to close the quality gap for non-English markets, and by end of day you're syncing with engineering on latency trade-offs for next week's A/B test. The constant: your science directly changes the experience millions of advertisers have when they need help. About the team This role sits within Amazon Advertising's broader Agentic Intelligence organization — a community of multiple Applied Science and Engineering teams building the next generation of AI-powered experiences for advertisers. You'll have access to Principal Engineers and Principal Applied Scientists to pressure-test ideas and elevate your work. What makes this team unique is the balance: you'll drive product-facing science through customer support agents that touch millions of advertisers, while also influencing and collaborating with a core AI infrastructure team within Amazon. The team is cross-functional — scientists and engineers work shoulder-to-shoulder, from problem framing through production deployment.