105 Amazon Research Awards recipients announced

Awardees, who represent 51 universities in 15 countries, have access to Amazon public datasets, along with AWS AI/ML services and tools.

Amazon Research Awards (ARA) provides unrestricted funds and AWS Promotional Credits to academic researchers investigating various research topics in multiple disciplines. This cycle, ARA received many excellent research proposals from across the world and today is publicly announcing 105 award recipients who represent 51 universities in 15 countries.

This announcement includes awards funded under six call for proposals during the fall 2023 cycle: AI for Information Security, Automated Reasoning, AWS AI, AWS Cryptography and Privacy, AWS Database Services, and Sustainability. Proposals were reviewed for the quality of their scientific content and their potential to impact both the research community and society.

Additionally, Amazon encourages the publication of research results, presentations of research at Amazon offices worldwide, and the release of related code under open-source licenses.

Recipients have access to more than 300 Amazon public datasets and can utilize AWS AI/ML services and tools through their AWS Promotional Credits. Recipients also are assigned an Amazon research contact who offers consultation and advice, along with opportunities to participate in Amazon events and training sessions.

Recommended reads
Using time to last byte — rather than time to first byte — to assess the effects of data-heavy TLS 1.3 on real-world connections yields more encouraging results.

“We received a fantastic response to the cryptography and privacy engineering’s call for proposals. This was the first time we offered ARAs for cryptography and privacy, and the response far exceeded our expectations, in terms of both the number and quality of the proposals,” said Rod Chapman, senior principal applied scientist with AWS Cryptography. “Advanced cryptography plays a crucial role in building trust with our customers and regulators, especially in emerging domains such as cryptographic computing, generative AI, and privacy-preserving applications. We look forward to working with the new principal investigators to bring ever more impactful cryptographic technologies to fruition.”

Recommended reads
Generative AI raises new challenges in defining, measuring, and mitigating concerns about fairness, toxicity, and intellectual property, among other things. But work has started on the solutions.

“Given that data is central to Amazon’s core businesses, I am excited by this opportunity to collaborate with universities on cutting-edge technologies for modern database systems,” said Doug Terry, vice president and distinguished scientist in AWS Database and AI Leadership. “These Amazon Research Awards allow us to support projects that have the potential for substantial advancement in important areas from correctness testing of SQL queries to new data models for generative AI applications.”

ARA funds proposals throughout the year in a variety of research areas. Applicants are encouraged to visit the ARA call for proposals page for more information or send an email to be notified of future open calls.

The tables below list, in alphabetical order by last name, fall 2023 cycle call-for-proposal recipients, sorted by research area.

AI for Information Security

Photo grid shows the recipients of the 2023 fall AI for information security Amazon Research Awards

Recipient

University

Research title

Murat Kocaoglu

Purdue University

Causal Anomaly Detection from Non-stationary Time-series in the Cloud

Hui Liu

Michigan State University

Harnessing the Power of Weakly-Supervised Graph Representation Learning for Cybersecurity

Xiaorui Liu

North Carolina State University

Harnessing the Power of Weakly-Supervised Graph Representation Learning for Cybersecurity

Thomas Pasquier

University of British Columbia

Building Robust Provenance-based Intrusion Detection

Michalis Polychronakis

Stony Brook University

SafeTrans: AI-assisted Transcompilation to Memory-safe Languages

Automated Reasoning

Photo grid shows the recipients of the 2023 fall automated reasoning Amazon Research Awards

Recipient

University

Research title

Armin Biere

University of Freiburg

From Mavericks to Teamplayers: Fostering Solver Cooperation in Distributed SAT Solving

Victor Braberman

Universidad de Buenos Aires

Abstractions for Validating Distributed Protocol Reference Implementations

Varun Chandrasekaran

University of Illinois Urbana-Champaign

Automating Privacy Compliance

Maria Christakis

TU Wien

Testing Dafny for Unsoundness and Brittleness Bugs

Werner Dietl

University of Waterloo

Optional Type Systems for Model-Implementation Consistency

Alastair Donaldson

Imperial College London

Validating Compilers for the Dafny Verified Programming Language

Azadeh Farzan

University of Toronto

Better Predictability in Dynamic Data Race Detection

Sicun Gao

University Of California, San Diego

Proof Optimization and Generalization in dReal

Tobias Grosser

University Of Cambridge

Correct and High-Performance Domain-Specific Compilation with Lean and MLIR

Andrew Head

University Of Pennsylvania

TYCHE: An IDE for Property-Based Testing

Kihong Heo

Korea Advanced Institute Of Science and Technology - KAIST

Generative Translation Validation for JIT Compiler in the V8 JavaScript Engine

Frans Kaashoek

Massachusetts Institute of Technology

Flotilla: Compositional Formal Verification of Liveness of Distributed Systems Implementations

Baris Kasikci

University of Washington - Seattle

Privacy-Conscious Failure Reproduction for Root Cause Diagnosis in Large-Scale Distributed Systems

Laura Kovacs

TU Wien

QuAT: Quantifiers with Arithmetic Theories are Friends with Benefits

Shriram Krishnamurthi

Brown University

Paralegal: Scalable Tooling to Find Privacy Bugs in Application Code

Corina Pasareanu

Carnegie Mellon University

Proving the Absence of Timing Side Channels in Cryptographic Applications

Jean Pichon-Pharabod

Aarhus University

Validating Isolation of Virtual Machines in the Cloud

Benjamin Pierce

University Of Pennsylvania

TYCHE: An IDE for Property-Based Testing

Ruzica Piskac

Yale University

Democratizing the Law - Using LLMs and Automated Reasoning for Legal Reasoning

Malte Schwarzkopf

Brown University

Paralegal: Scalable Tooling to Find Privacy Bugs in Application Code

Peter Sewell

University Of Cambridge

The Foundations of Cloud Virtual-machine Isolation

Scott Shapiro

Yale University

Democratizing the Law - Using LLMs and Automated Reasoning for Legal Reasoning

Geoffrey Sutcliffe

University Of Miami

Automated Theorem Proving Community Infrastructure in the AWS Cloud

Joseph Tassarotti

New York University

Asynchronous Couplings for Probabilistic Relational Reasoning in Dafny

Sebastian Uchitel

Universidad de Buenos Aires

Abstractions for Validating Distributed Protocol Reference Implementations

Josef Urban

Czech Technical University

Learning Based Synthesis Meets Learning Guided Reasoning

Thomas Wies

New York University

Automating Privacy Compliance

Nickolai Zeldovich

Massachusetts Institute of Technology

Flotilla: Compositional Formal Verification of Liveness of Distributed Systems Implementations

AWS AI

Photo grid shows the recipients of the 2023 fall AWS AI Amazon Research Awards

Recipient

University

Research title

Pulkit Agrawal

Massachusetts Institute Of Technology

Adapting Foundation Models without Finetuning

Niranjan Balasubramanian

Stony Brook University

An API Sandbox for Complex Tasks on Common Applications

Osbert Bastani

University Of Pennsylvania

Uncertainty Quantification for Trustworthy Language Generation

Matei Ciocarlie

Columbia University

Do You Speak EMG? Generative Pre-training on Electromyographic Signals for Controlling a Rehabilitation Robot after Stroke

Caiwen Ding

University of Connecticut

Graph of Thought: Boosting Logical Reasoning in Large Language Models

Yufei Ding

University Of California, San Diego

A Hollistic Compiler and Runtime System for Efficient and Scalable LLM Serving

Xinya Du

University Of Texas At Dallas

Process-guided Fine-tuning for Answering Complex Questions

Luciana Ferrer

University of Buenos Aires - CONICET

Efficient Adaptation of Generative Language Models through Unsupervised Calibration

Jakob Foerster

University Of Oxford

Compute-only Scaling of Large Language Models

Nikhil Garg

Cornell University

Recommendation systems in high-stakes settings

Georgia Gkioxari

California Institute Of Technology

Towards a 3D Foundation Model: Recognize and Reconstruct Anything

Tom Goldstein

University of Maryland

Building Safer Diffusion Models

Aditya Grover

University of California, Los Angeles

Personalizing Multimodal Generative Models via In-Context Preference Modeling

Albert Gu

Carnegie Mellon University

Scaling the Next Generation of Foundation Model Architectures

Mahdi S. Hosseini

Concordia University

Toward Auto-Populating Synoptic Reports in Diagnostic Pathology

Maliheh Izadi

Delft University Of Technology

Understanding and Regulating Memorization in Large Language Models for Code

Vijay Janapa Reddi

Harvard University

Benchmarking the Safety of Generative AI Models with Data-centric AI Challenges

Adel Javanmard

University of Southern California

Reliable AI for Generation of Medical Reports from MRI Scans

Jianbo Jiao

University Of Birmingham

PCo3D: Physically Plausible Controllable 3D Generative Models

Subbarao Kambhampati

Arizona State University

Understanding and Leveraging Planning, Reasoning & Self-Critiquing Capabilities of Large Language Models

Kangwook Lee

University Of Wisconsin–Madison

Information and Coding Theory-Based Framework for Prompt Engineering

Ales Leonardis

University Of Birmingham

PCo3D: Physically Plausible Controllable 3D Generative Models

Anqi Liu

Johns Hopkins University

(Multi-)Calibrated Active Learning under Subpopulation Shift

Lydia Liu

Princeton University

From Predictions to Positive Impact: Foundations of Responsible AI in Social Systems

Song Mei

University Of California, Berkeley

Mathematical Foundations and Physical Principles of Foundation Models and Generative AI

Pablo Piantanida

National Centre for Scientific Research (CNRS)

Efficient Adaptation of Generative Language Models through Unsupervised Calibration

Chara Podimata

Massachusetts Institute Of Technology

Responsible AI through User Incentive-Awareness

Bhiksha Raj

Carnegie Mellon University

Text and Speech Large Language Models

Christian Rupprecht

University Of Oxford

Viewset Diffusion for Probabilistic 3D Reconstruction

Olga Russakovsky

Princeton University

Diffusion models: Generative models beyond data generation

Vatsal Sharan

University Of Southern California

Debiasing ML-based Decision Making using Multicalibration

Abhinav Shrivastava

University Of Maryland

Audio-conditioned Diffusion Models for Generating Lip-synchronized Videos

Rachee Singh

Cornell University

Accelerating collective communication for distributed ML

Vincent Sitzmann

Massachusetts Institute Of Technology

2D and 3D Animation via Image-Conditional Generative Flow Models

Justin Solomon

Massachusetts Institute Of Technology

Lightweight Algorithms for Generative AI

Mahdi Soltanolkotabi

University of Southern California

Reliable AI for Generation of Medical Reports from MRI Scans

Qian Tao

Delft University of Technology

Φ-Generative Medical Imaging by Physics and AI (PhAI)

Yapeng Tian

University Of Texas At Dallas

Integrating Visual Alignment and Text Interaction for Multi-modal Audio Content Generation

Sherry Tongshuang Wu

Carnegie Mellon University

Generating Deployable Models from Natural Language Instructions through Adaptive Data Curation

Florian Tramer

Eth Zurich

Can Technology Protect us from Generative AI?

Arie van Deursen

Delft University Of Technology

Understanding and Regulating Memorization in Large Language Models for Code

Andrea Vedaldi

University Of Oxford

Viewset Diffusion for Probabilistic 3D Reconstruction

Carl Vondrick

Columbia University

Viper: Visual Inference via Python Execution for Reasoning

Xiaolong Wang

University of California, San Diego

Generating Compositional 3D Scenes and Embodied Tasks with Large Language Models

Eric Wong

University Of Pennsylvania

Adversarial Manipulation of Prompting Interfaces

Saining Xie

New York University

Image Sculpting: Precise Image Generation and Editing with Interactive Geometry Control

Rex Ying

Yale University

Diff-H: Hyperbolic Text-to-Image Diffusion Generative Model

Minlan Yu

Harvard University

Troubleshooting Distributed Training Systems

Zhiru Zhang

Cornell University

A Unified Approach to Tensor Graph Optimization

AWS Cryptography and Privacy

Photo grid shows the recipients of the 2023 fall AWS Cryptography and Privacy Amazon Research Awards

Recipient

University

Research title

Christopher Brzuska

Aalto University

Secure Messaging: Updates Efficiency & Verification

Tevfik Bultan

University of California, Santa Barbara

Detecting and Quantifying Information Leakages in Crypto Libraries

Muhammed Esgin

Monash University

Practical Post-Quantum Oblivious Pseudorandom Functions Supporting Verifiability

Nadia Heninger

University of California, San Diego

Bringing Modern Security Guarantees to End-to-End Encrypted Cloud Storage

Tal Malkin

Columbia University

Cryptographic Techniques for Machine Learning

Peihan Miao

Brown University

Advancing Private Set Intersection for Wider Industrial Adoption

Virginia Smith

Carnegie Mellon University

Rethinking Watermark Embedding and Detection for LLMs

Ron Steinfeld

Monash University

Practical Post-Quantum Oblivious Pseudorandom Functions Supporting Verifiability

AWS Database Services

Photo grid shows the recipients of the fall 2023 AWS Database Services Amazon Research Awards

Recipient

University

Research title

Lei Cao

University Of Arizona

SEED: Simple, Efficient, and Effective Data Management via Large Language Models

Natacha Crooks

University Of California, Berkeley

Mammoths Are Slow: The Overlooked Transactions of Graph Data

Samuel Madden

Massachusetts Institute Of Technology

SEED: Simple, Efficient, and Effective Data Management via Large Language Models

Manuel Rigger

National University Of Singapore

Democratizing Database Fuzzing

Kexin Rong

Georgia Institute Of Technology

Dynamic Data Layout Optimization with Worst-case Guarantees

Sustainability

Photo grid shows the recipients of the fall 2023 sustainability Amazon Research Awards

Recipient

University

Research title

Kate Armstrong

New York Botanical Garden

VERDEX: remote sensing of plant biodiversity

Praveen Bollini

University Of Houston

Data-driven design and optimization of selective nanoporous catalysts for biofuel conversion

Brandon Bukowski

Johns Hopkins University

Data-driven design and optimization of selective nanoporous catalysts for biofuel conversion

Alan Edelman

Massachusetts Institute of Technology

Scientific Machine Learning with Application to Probabilistic Climate Forecasting and Sustainability

Kosa Goucher-Lambert

University of California, Berkeley

LCAssist: An Interactive System for Life-Cycle-Informed Sustainable Design Decision-Making

Vikram Iyer

University of Washington - Seattle

Data-Driven Sustainable Polymer Design for Circuits, Packaging, and Actuators

Can Li

Purdue University

Design and Analysis of Sustainable Supply Chains Using Optimization and Large Language Models

Damon Little

New York Botanical Garden

VERDEX: remote sensing of plant biodiversity

Aniruddh Vashisth

University of Washington - Seattle

Data-Driven Sustainable Polymer Design for Circuits, Packaging, and Actuators

Ming Xu

Tsinghua University

Advancing Sustainable Practices in the AI Era: Integrating Large Language Models for Automated Life Cycle Assessment Modeling

Related content

US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
GB, London
As a STRUC Economist Intern, you'll specialize in structural econometric analysis to estimate fundamental preferences and strategic effects in complex business environments. Your responsibilities include: Analyze large-scale datasets using structural econometric techniques to solve complex business challenges Applying discrete choice models and methods, including logistic regression family models (such as BLP, nested logit) and models with alternative distributional assumptions Utilizing advanced structural methods including dynamic models of customer or firm decisions over time, applied game theory (entry and exit of firms), auction models, and labor market models Building datasets and performing data analysis at scale Collaborating with economists, scientists, and business leaders to develop data-driven insights and strategic recommendations Tackling diverse challenges including pricing analysis, competition modeling, strategic behavior estimation, contract design, and marketing strategy optimization Helping business partners formalize and estimate business objectives to drive optimal decision-making and customer value Build and refine comprehensive datasets for in-depth structural economic analysis Present complex analytical findings to business leaders and stakeholders
US, WA, Seattle
At Amazon Selection and Catalog Systems (ASCS), our mission is to power the online buying experience for customers worldwide so they can find, discover, and buy any product they want. We innovate on behalf of our customers to ensure uniqueness and consistency of product identity and to infer relationships between products in Amazon Catalog to drive the selection gateway for the search and browse experiences on the website. We're solving a fundamental AI challenge: establishing product identity and relationships at unprecedented scale. Using Generative AI, Visual Language Models (VLMs), and multimodal reasoning, we determine what makes each product unique and how products relate to one another across Amazon's catalog. The scale is staggering: billions of products, petabytes of multimodal data, millions of sellers, dozens of languages, and infinite product diversity—from electronics to groceries to digital content. The research challenges are immense. GenAI and VLMs hold transformative promise for catalog understanding, but we operate where traditional methods fail: ambiguous problem spaces, incomplete and noisy data, inherent uncertainty, reasoning across both images and textual data, and explaining decisions at scale. Establishing product identities and groupings requires sophisticated models that reason across text, images, and structured data—while maintaining accuracy and trust for high-stakes business decisions affecting millions of customers daily. Amazon's Item and Relationship Platform group is looking for an innovative and customer-focused applied scientist to help us make the world's best product catalog even better. In this role, you will partner with technology and business leaders to build new state-of-the-art algorithms, models, and services to infer product-to-product relationships that matter to our customers. You will pioneer advanced GenAI solutions that power next-generation agentic shopping experiences, working in a collaborative environment where you can experiment with massive data from the world's largest product catalog, tackle problems at the frontier of AI research, rapidly implement and deploy your algorithmic ideas at scale, across millions of customers. Key job responsibilities Key job responsibilities include: * Formulate novel research problems at the intersection of GenAI, multimodal learning, and large-scale information retrieval—translating ambiguous business challenges into tractable scientific frameworks * Design and implement leading models leveraging VLMs, foundation models, and agentic architectures to solve product identity, relationship inference, and catalog understanding at billion-product scale * Pioneer explainable AI methodologies that balance model performance with scalability requirements for production systems impacting millions of daily customer decisions * Design and execute model distillation strategies—distilling large frontier LLMs and VLMs into compact, production-grade models—that preserve multimodal reasoning capability while dramatically reducing serving latency, cost, and infrastructure footprint at billion-product catalog scale * Own end-to-end ML pipelines from research ideation to production deployment—processing petabytes of multimodal data with rigorous evaluation frameworks * Define research roadmaps aligned with business priorities, balancing foundational research with incremental product improvements * Mentor peer scientists and engineers on advanced ML techniques, experimental design, and scientific rigor—building organizational capability in GenAI and multimodal AI * Represent the team in the broader science community—publishing findings, delivering tech talks, and staying at the forefront of GenAI, VLM, and agentic system research