Garegin Papoian, the Monroe Martin Professor at the University of Maryland, is seen sitting at a desk with an open laptop in front of him. He has turned around in his seat to face the camera.
Garegin Papoian is the Monroe Martin Professor at the University of Maryland. Within his Papoian Lab, a theoretical physical chemistry group located at the university, his team is working toward developing fundamental molecular models of the whole cell, a concept still in its infancy
Courtesy of Garegin Papoian

Garegin Papoian’s quest to model an elusive class of proteins

With the support of an Amazon Research Award, Papoian’s team is deciphering the dynamics of intrinsically disordered proteins.

How do molecules come together and start to behave like a living system? This is the type of question that drives Garegin Papoian’s research. At the University of Maryland, where he is the Monroe Martin Professor, he has been focusing on computational modeling of biological molecules like proteins and DNA. Within his Papoian Lab, a theoretical physical chemistry group also located at the university, his team is also working toward developing fundamental molecular models of the whole cell, a concept still in its infancy.

Papoian’s path into science was determined early on. Growing up in Armenia, then a part of the Soviet Union, he went to a special school of physics and mathematics, where he was introduced to Science Olympiads. While in high school, he won the first place in the Republic of Armenia in separate Olympiads in chemistry, physics, mathematics and biology. “Science Olympiads were a big reason why I got drawn into science, in particular to chemistry and physics”, he says.

Because of his success in the competitions, he was invited to study at an advanced chemistry college in Moscow established specifically for Olympiad winners.

“I was 16,” he says, “but it was assumed that we already knew all university level chemistry. So, they would start immediately with a very high-level training.” The program included an internship in the United States, at the University of Kansas. From there he eventually enrolled as a graduate student at Cornell University, where he pursued his PhD in quantum chemistry, working under the Nobel Laureate, Roald Hoffmann.

During his postdoc, he turned to classical physics with a particular emphasis on biophysics. “I was interested in bringing concepts of physical chemistry to understand biological phenomena from the molecular perspective,” he says. “And my long-term career goal is to develop concepts both for proteins and cells.”

Predicting a protein’s shape

A protein is a large molecule essential to all living things. The sequence of amino acids that form a protein determines its three-dimensional structure. Each protein has a unique shape that dictates its function. Being able to predict what a protein structure looks like from its amino acid sequence has been a long-standing scientific challenge and one of the research interests of Papoian’s group, for which he received an AWS Machine Learning Research Award in 2018.

This animation shows the structure of a protein called linker histone H1
This animation shows the structure of a protein called linker histone H1, including its disordered tails, predicted by Papoian's team. "We discovered that interactions of those disordered tails with DNA help to structurally position H1 with respect to the nucleosome. In terms of the bigger picture, the H1-nucleosome interactions regulate epigenetic processes, determining for example which particular genes should be turned on or off,” says Papoian.

One of the applications of protein structure prediction is drug design. “When you design a drug, you need to know what the target looks like,” says Papoian. If you know that the target protein has a certain pocket, for example, you can develop a molecule that will fit nicely into that pocket. While identifying genes associated with diseases has become easier, the sequence of a gene doesn’t tell you what the protein expressed by it looks like, and experimental methods to determine the protein shape are lengthy and expensive.

IDPs ... are more like this crazy spaghetti. It's very hard to deal with them both experimentally and computationally.
Garegin Papoian

Even in the wake of DeepMind demonstrating that AlphaFold is capable of predicting protein structures with an unprecedented level of accuracy, challenges still remain.

It turns out that a large proportion of human proteins are not completely structured in neat three-dimensional shapes. These are called the intrinsically disordered proteins (IDPs). “They are much more dynamic and mostly never fall into a single structure,” says Papoian. “They are more like this crazy spaghetti. It's very hard to deal with them both experimentally and computationally because they are so elusive.” He notes that about a third of human proteins are like that, including many important disease-causing proteins.

Papoian’s AWS Machine Learning Research Award enabled his team to advance the development of a system that is better suited to simulating these proteins.

Tackling disordered proteins

For the past few years, Papoian Lab has been working with a protein modeling framework called AWSEM-MD (pronounces “awesome”), which stands for associative memory, water-mediated, structure and energy model — molecular dynamics. It has been developed jointly with Peter Wolynes, Papoian’s former postdoctoral advisor who is currently at Rice University and with whom he continued to collaborate over the years.

Using the AWS Machine Learning Research Award, Papoian and his colleagues developed AWSEM-IDP, an AWSEM branch specifically designed to simulate intrinsically disordered proteins.

This system uses a database of protein fragment structures obtained experimentally, for example, through nuclear magnetic resonance (NMR) spectroscopy — a technique that determines the structure and dynamics of proteins. "These fragments serve as structural memories that guide the IDP to undergo structural transformations that are informed by the experiment,” Papoian explains. “This allows simulating more realistic IDP dynamics.”

The fragment database may also contain structures from atomistic simulations — a type of simulation where every atom of a protein is present. “The reason why we prefer not to do those in general is that they’re very expensive, so we cannot do very big simulations. But we can do atomistic simulations of short fragments to give us good fragment memories, again improving the accuracy of IDP’s structural exploration in AWSEM simulations,” he says.

An IDP will prefer multiple structures, not just one.

“That's the key difference from regular proteins: IDPs are multi-faceted in essence. But they still prefer certain structures over others. And the AWSEM-IDP model allows you to correctly describe those preferences,” Papoian explained. This model was described in a 2018 article published at the Journal of Physical Chemistry B.

In another work published earlier this year that was supported by the AWS Machine Learning Award, Papoian and his colleagues applied AWSEM-IDP to study a protein called linker histone H1, which plays an essential role in regulating many important biological processes. This protein has two intrinsically disordered regions, parts of its structure that are not well folded and resemble two tails. Because they are disordered, it’s much harder to understand what they do and how they interact.

Proteins like linker histone H1 regulate histone complexes, which act like a spool around which the DNA wraps to create structures called nucleosomes. “In this paper, we used AWSEM-IDP to model the nucleosome with linker histone H1, in particular with these disordered tails. And that allowed us to understand how the linker histone and the nucleosome come together and interact, and what's the role of these disordered tails,” says Papoian. Understanding proteins’ interactions with nucleosomes may give important insights on epigenetics, which is one of Papoian Lab’s interests.

Future challenges

Because making sense of IDPs is such a difficult process, Papoian says that AWSEM-IDP is an ongoing program with room for improvement. “What we have currently works better in some classes of proteins, and not so much in others. So next we’ll explore what are the challenges for what we currently have in ASWEM-IDP and try to come up with new advances to overcome them.”

In addition to IDPs, Papoian Lab will also continue to pursue the use of deep learning for structure prediction of well-folded proteins. Although there is some conceptual overlap with AlphaFold, Papoian believes that AWSEM-MD is a powerful tool and has advantages to other approaches when it comes to molecular dynamics.

Proteins are not frozen objects. Some of them are well structured, but many are not structured at all, and they are dynamic and move and shape-shift incessantly.
Garegin Papoian

“Proteins are not frozen objects,” he says. “Some of them are well structured, but many are not structured at all, and they are dynamic and move and shape-shift incessantly. So, to understand how these proteins function, you must model their dynamics and that’s what AWSEM-MD can do best.”

Papoian thinks one exciting area to be explored in coming decades will be combining machine learning and physics to work on protein structure prediction, protein dynamics, multiprotein complexes, and epigenetics.

“There are lots of things that still remain to be understood in our models. And I think that probably neither physics nor machine learning by themselves can tackle them. But a program that brings them together in a productive way can be very powerful,” he said.

Modeling an entire cell

Another ambitious project that Papoian and his colleagues are pursuing is to develop a computational model of an entire cell. “We still don’t have a blueprint of a cell the way we have a blueprint of a car or a Boeing airplane.”

To do that, his group develops their own software from scratch.

Garegin Papoian: How do cells move? Chemistry meets mechanics

“We basically do the science, the physics, and biophysics of what is needed to model our cells. We derive the needed algorithms from scratch based on the laws of physics and chemistry and then we program that into a computer and run simulations on a supercomputer,” he explained. This has to be done at a single molecule resolution, he adds, meaning that they have to track every single molecule within a cell.

To achieve that, the Papoian Lab developed a model called MEDYAN.

“We can already model some number of proteins, the membrane, we can model rich chemistry. We have developed some of the fundamental chemistry and physics components of what needs to be done,” he says. The next step is to scale it. “We usually do simulations with several types of proteins. So instead of several, you will need maybe hundreds or thousands of different types of proteins, so it just brings more complexity.”

When that happens, it will be a huge revolution in biomedicine, he says. “Then lots of things that people laboriously spend years doing in the laboratory could just run on AWS servers. And you could do your experiments and search for treatments computationally, which would be much cheaper and faster.”

Research areas

Related content

ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.