79 Amazon Research Awards recipients announced

Awardees, who represent 54 universities in 14 countries, have access to Amazon public datasets, along with AWS AI/ML services and tools.

Amazon Research Awards (ARA) provides unrestricted funds and AWS Promotional Credits to academic researchers investigating various research topics in multiple disciplines. This cycle, ARA received many excellent research proposals from across the world and today is publicly announcing 79 award recipients who represent 54 universities in 14 countries.

This announcement includes awards funded under four call for proposals during the fall 2022 cycle: AWS AI, Automated Reasoning, Prime Video, and Sustainability. Proposals were reviewed for the quality of their scientific content and their potential to impact both the research community and society.

Recipients have access to more than 300 Amazon public datasets and can utilize AWS AI/ML services and tools through their AWS Promotional Credits. Recipients also are assigned an Amazon research contact who offers consultation and advice, along with opportunities to participate in Amazon events and training sessions.

Additionally, Amazon encourages the publication of research results, presentations of research at Amazon offices worldwide, and the release of related code under open-source licenses.

"Complexities of AI/ML challenges at scale often intersect more than one discipline and require creative and diverse approaches to tackle these issues," said Arash Nourian, AWS general manager, Machine Learning Engines. "I was amazed by the diversity of disciplines and the scientific content of Awardee’s submissions that collectively could represent significant potential impact on both the AI/ML research community and society."

“The incredible response to Prime Video’s fall 2022 Call for Proposals is a testament to the exciting work the ARAs have inspired across four cutting-edge research categories,” said BA Winston, VP of Technology at Prime Video. “I am delighted by the winning proposals and look forward to the ongoing research across several areas in Prime Video that is helping us create even more impactful customer-obsessed technology.”

ARA funds proposals throughout the year in a variety of research areas. Applicants are encouraged to visit the ARA call for proposals page for more information or send an email to be notified of future open calls.

The tables below list, in alphabetical order, fall 2022 cycle call-for-proposal recipients, sorted by research area.

AWS AI

AWS AI - ARA fall 2022.png

Recipient

University

Research title

Jonathan Afilalo

McGill University

Coreslicer: deep learning of CT images for frailty assessment in clinical care

Saman Amarasinghe

Massachusetts Institute of Technology

Reimagining the compiler in the cloud

Akshay Chaudhari

Stanford University

Large-scale self-supervised learning for medical imaging

Soheil Feizi

University Of Maryland, College Park

Towards mitigating spurious correlations in deep learning

Aikaterini Fragkiadaki

Carnegie Mellon University

Analogical networks for continual memory-modulated visual learning and language understanding

Mark Gerstein

Yale University

Privacy-preserving storage, sharing, and analysis for genomics data

Joseph Gonzalez

University Of California, Berkeley

A unified platform for training and serving large models

Michael Gubanov

Florida State University

An interactive polygraph for robust access to scientific knowledge

Yan Huang

Carnegie Mellon University

Combating algorithmic bias inherited from human decision making: a human-AI perspective

CV Jawahar

The International Institute of Information Technology - Hyderabad

Deeper understanding of multilingual handwritten documents: from recognition to dialogues

Zhihao Jia

Carnegie Mellon University

Combining ML and systems optimizations for sustainable and affordable ML

Daniel Khashabi

Johns Hopkins University

Crowdsourcing with machine backbone

Rahul Krishnan

University Of Toronto

Towards a learning healthcare system

Anastasios Kyrillidis

Rice University

Efficient and affordable transformers for distributed platforms

Kevin Leach

Vanderbilt University

Documentnet: iterative data collection for building a robust document understanding dataset

Lei Li

University Of California, Santa Barbara

Real-time robust simultaneous interpretation with few samples

Xiaoyi Lu

University Of California, Merced

Scaling collective communication for distributed deep learning training

Yunan Luo

Georgia Institute of Technology

Calibrated and interpretable geometric deep learning for robust drug screening

Graham Neubig

Carnegie Mellon University

Towards more reliable and interpretable code language models

Qing Qu

University of Michigan, Ann Arbor

Principles of deep representation learning via neural collapse

Mirco Ravanelli

Concordia University

Toward empathetic conversational AI

Amit Roy-Chowdhury

University of California, Riverside

Exploring privacy in deep metric learning: applications in computer vision

Chirag Shah

University of Washington

Fairness as a service: operationalizing fairness in search and recommendation applications through a novel multi-objective optimization framework

Kristina Simonyan

Massachusetts Eye and Ear/Harvard Medical School

Machine learning for automated speech processing for real-time speech prosthesis in neurological disorders

Berrak Sisman

University of Texas, Dallas

Explainable AI for expressive voice synthesis

Dawn Song

University Of California, Berkeley

FedOps: an abstraction for trustworthy federated learning

Peter Spirtes

Carnegie Mellon University

System-level and long-term fairness through causal learning and reasoning

Ion Stoica

University Of California, Berkeley

A unified platform for training and serving large models

Vasileios Syrgkanis

Stanford University

Automating the causal machine learning pipeline

Carlo Tomasi

Duke University

Deep neural network classifiers with margins in input space

Yatish Turakhia

University Of California, San Diego

Machine learning enabled wastewater-based epidemiology

Xiaolong Wang

University of California, San Diego

Learning implicit neural foundation models

Neeraja Yadwadkar

University Of Texas, Austin

Easy-to-use and cost-efficient distributed inference serving

Hamed Zamani

University Of Massachusetts Amherst

On the optimization of retrieval-enhanced machine learning models

Ce Zhang

ETH Zurich

FedOps: an abstraction for trustworthy federated learning

Tianyi Zhang

Purdue University

Human-in-the-loop deep learning optimization for better usability, transparency, and user trust

Yiying Zhang

University Of California, San Diego

Training deep neural networks with “zero” activations

Jishen Zhao

University Of California, San Diego

Semantic-informed document structure recognition with large language models

Ben Zhao

University Of Chicago

Digital forensics for deep neural networks

Heather Zheng

University of Chicago

Digital forensics for deep neural networks

Jun-Yan Zhu

Carnegie Mellon University

Compositional personalization of large-scale generative models

Jia Zou

Arizona State University

A compilation framework for accelerating machine learning inference queries

Amazon Sustainability

Amazon Sustainability ARA fall 2022.png

Recipient

University

Research title

Vikram Iyer

University of Washington

Computational design and circular fabrication for sustainable electronics

Adriana Schulz

University of Washington

Computational design and circular fabrication for sustainable electronics

Mari Winkler

University of Washington

A novel bioreactor platform for continuous high‐rate bio-production

Automated Reasoning

Automated Reasoning ARA fall 2022.png

Recipient

University

Research title

Maria Paola Bonacina

Università degli Studi di Verona

Advances in conflict-driven SATisfiability modulo theories and assignments

Ahmed Bouajjani

Universite Paris-Cite

Safe composition of distributed off-the-shelf components

Martin Nyx Brain

City, University Of London

Snowshoes: overapproximating code footprints for safe program exploration

Anton Burtsev

University Of Utah

Atmosphere: leveraging language safety and operating system design for verification

Alastair Donaldson

Imperial College London

DafnyDefender: automated testing for the Dafny ecosystem

Francois Dupressoir

University Of Bristol

Formosa cryptography: computer-aided reasoning for high-assurance cryptographic design and engineering

Gidon Ernst

Ludwig Maximilian University of Munich

Security specifications for Dafny

Pascal Fontaine

University of Liège

SMT: modules, formats, and standards

Jeffrey Foster

Tufts University

Automated testing of external methods in Dafny

Sicun Gao

University Of California, San Diego

Monte Carlo tree methods for decision-making in dReal

Philippa Gardner

Imperial College London

Gillian-Rust: unbounded verification for unsafe rust code

Limin Jia

Carnegie Mellon University

Enabling one-line rust verification with program synthesis

Patrick Lam

University Of Waterloo

Statically inferring contracts from assertions & tests

Aravind Machiry

Purdue University

Security verification and hardening of CI workflows

Anders Møller

Aarhus University

Securing node.js programs with static resource analysis

Magnus Myreen

Chalmers University Of Technology

Compiling Dafny to CakeML

ThanhVu Nguyen

George Mason University

Scalable and precise DNN constraint solving with abstraction and conflict clause learning

Burcu Kulahcioglu Ozkan

Delft University of Technology

Coverage-directed randomized testing of distributed systems

Bryan Parno

Carnegie Mellon University

Verus: developing provably correct and reliable rust code

Corina Pasareanu

Carnegie Mellon University

Enabling one-line rust verification with program synthesis

Ruzica Piskac

Yale University

Formalizing FISA: using automated reasoning to formalize legal reasoning

Elizabeth Polgreen

University of Edinburgh

Automated and provably correct code modernization

Fred Schneider

Cornell University

Using non-deterministic executable specification to test properties that relate executions

Scott Shapiro

Yale University

Formalizing FISA: using automated reasoning to formalize legal reasoning

Marc Shapiro

INRIA & Sorbonne Universite Paris

Safe composition of distributed off-the-shelf components

Alexandra Silva

Cornell University

Automated reasoning for correctness and incorrectness

Yakir Vizel

Technion – Israel Institute Of Technology

Lazy and incremental framework for solving CHCs

Florian Zuleger

Technische Universität Wien

Automated cost analysis of data structures

Prime Video

Prime Video ARA fall 2022.png

Recipient

University

Research title

David Bull

University of Bristol

Generic deep video quality assessment in the extended parameter space

Eamonn Keogh

University of California Riverside

A proposal to make any time series anomaly detection algorithm faster, more accurate and more practical

Xiaorui Liu

North Carolina State University

Deep reinforcement learning for the mixed ranking of recommendations and advertisements with page-wise display

Jiliang Tang

Michigan State University

Deep reinforcement learning for the mixed ranking of recommendations and advertisements with page-wise display

Hanghang Tong

University of Illinois Urbana-Champaign

Graph algorithms for personalized recommendation

Fan Zhang

University of Bristol

Generic deep video quality assessment in the extended parameter space

Related content

ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.