AfroTech logo and headshots of  Dela Agbemabiese, Justin Barry, Nashlie Sephus and Colby Wise.
With AfroTech World occurring this week, we asked some of the company's Black scientists what they consider some of the systemic issues limiting underrepresented minorities from being more involved in the technology industry. We heard from Dela Agbemabiese (lower left), a data scientist, Justin Barry (upper left), applied scientist, Nashlie Sephus (upper right), applied science manager, and Colby Wise (lower right), senior deep learning scientist.
Credit: Glynis Condon

Issues of racial, ethnic and gender diversity are on the agenda at AfroTech World

Amazon scientists provide insights on issues related to lack of involvement of underrepresented minorities in the technology industry.

As CNBC reported earlier this year, six years after initially disclosing diversity reports, major technology companies have made little progress in hiring more minorities, especially Black employees with science and technology skills.

This presents a series of ongoing challenges. According the US Bureau of Labor Statistics (BLS), nearly one-quarter of the country’s total economic output is produced by high-tech industries, and in 2017 BLS projected there would be more than 1 million job openings in computer and information technology over the next 10 years. Moreover, computing occupation salaries are more than twice the median wage for all other occupations, according to BLS.

“When we look at tech and its impact on our economy, and the simultaneous underrepresentation of the Black community, it is a critically important racial and economic justice issue," says Allison Scott, CEO of the Kapor Center. “When the tech workforce and leadership reflects the diverse experiences and backgrounds of our nation, I believe tech can begin to play an integral role in addressing long-standing disparities that exist in this country.”

As of December 31, 2019, Amazon reported that 26.5% of its global workforce identifies as Black/African American, 26.5% Asian, 18.5% Hispanic/Latinx, 1.3% as Native American, and 3.6% as two or more races.  The 26.5% of employees who identify as Black/African American work in both non-technical and technical roles.

This week at AfroTech World, issues related to the lack of adequate racial, ethnic, and gender diversity within the technology industry are on the agenda as leaders in technology and business come together to exchange ideas for creating greater opportunity for Blacks in technology.  Amazon is a Diamond Sponsor of this year’s event, and has a virtual recruiting booth.  

On Nov. 13, the company is hosting a virtual event, “Our Voices, Our Power”, presented by Amazon’s Black Employee Network (BEN) affinity group. Attendees will hear employees share their Amazon journey stories, learn about career opportunities, and enjoy entertainment.

In advance of AfroTech, Amazon Science asked some of the company’s Black scientists what they consider some of the systemic issues limiting underrepresented minorities’ involvement in the technology industry, about some of the issues they have had to overcome in pursuing their science careers, who or what inspired them to pursue their science careers, and what lessons we might take from their individual experiences. 

Dela Agbemabiese is a data scientist within Amazon’s advertising organization. He earned his master’s degree in business administration from Drexel University.

Dela Agbemabiese
Dela Agbemabiese

What do you consider some of the systemic issues limiting underrepresented minorities from greater employment opportunities in the technology industry?

Lack of financial resources to stimulate curiosity in tech, lack of mentors or heroes to look up to due to low representation, and societal prejudice hindering opportunities.

Lack of financial resources to stimulate curiosity in tech. I have been fortunate and blessed my entire life.  All gratitude goes to my parents. I was born in Ghana, West Africa. My mom was a nurse, and my dad an economist. Due to the nature of my dad’s work, I got the opportunity to travel a lot as a kid, got enrolled into a course at eight years old to get a Linux command line certificate, and always had access to tech resources. My parents sacrificed to ensure I attended the best schools, and there is not a single thing I ever asked for that I did not get. This may not be the case for all children, whose parents are possibly working hard doing multiple jobs, and in some cases are single parents. If the financial resources I had were similar to that of many minority children, it would be unlikely for me to be where I am today.

"Students don’t see themselves represented in the [economics] profession"

Four economists from diverse backgrounds explain why diversity is essential—and what needs to happen to achieve it.

Lack of mentors or heroes to look up to due to low representation. While my dad was heavy on econometrics and I learned a thing or two from him, it was my cousin Martey to whom I looked up. He was brilliant academically, and I always wanted to be like him. He tutored me in math and physics, thus giving me an edge over my classmates. Martey was not my only mentor, in fact, I had many, including Yao Obeng, who helped me nurture my creativity and problem-solving skills. Many minority children may not have mentors or heroes within tech to encourage and inspire interest in tech-related careers. If I did not have these mentors to motivate me, it would be unlikely for me to be where I am today.

Societal prejudice hindering opportunities. Growing up in Ghana, prejudice did not exist from a racial standpoint. Once I moved to the United States for my undergraduate degree, this became a reality. My minority friends and I have had to work twice as hard as our peers to prove we are as good as our credentials. We strived to invalidate stereotypes about minorities through the quality of our work and our work ethic. With everything I do, in the back of my mind I am thinking about how my actions or inactions affect the perception towards minorities: am I enabling some of these unfounded prejudices? Or am I, through my work, educating my peers and superiors? For me, this societal prejudice only began when I came to the United States for my undergraduate degree, but imagine the minority children out there who have had to live with this their entire lives. It sure can get demoralizing.

What are some of the obstacles you had to overcome in pursuing your science career?

Societal prejudice hindering opportunities. I have been lucky to have managers and peers that are inclusive and open-minded, that judge me based on the quality of my work. Rachel McKitrick was my first manager in Amazon. I joined Amazon as a business analyst, despite my previous role as a senior data scientist. I just wanted to join Amazon! Rachel knew my business analyst role was not ideal, and gave me projects that were science oriented, which ultimately enabled my transition to scientist. My second manager, Monica Wu, always made herself available to chat and made me feel like my voice and opinion mattered. My current team managed by Dauwe Vercamer and Andrew Petschek welcomed me with open arms, gave me opportunities to shine and lead within the team. They provide direct feedback that has made me a much better scientist today.

I have had the privilege of learning from a lot of people. Societal prejudice may be harder to solve for, but I believe a good place to start will be to find means for minority youth to gain access to some of the brilliant minds within the technology industry, be it through some virtual teaching programs, or through some mentoring programs. The prejudice may exist, the financial resources may be sparse or non-existent, but with heroes and mentors to look up to, a child’s imagination can be sparked for what could be.

Who or what inspired you to pursue your science career, and what lessons can we take from your experience?

My dad due to his econometrics background, and my childhood mentors who encouraged me to put math and science ahead of basketball and soccer. Since then, I have had lots of mentors along the way, especially here at Amazon. Individuals such as Leo Razoumov, Pranjal Mallick, Amy Ruschak, John Lafayette, and Oded Netzer, who have helped shape me into a better scientist.

My advice to Black students interested in a STEM career, or other Black scientists is to find mentors, and get them involved in your work. Meet with them once a week for even 10 minutes, and let them influence your work.

Justin Barry is an applied scientist with Amazon’s Prime Video organization. He earned his master’s degree in computer science from the University of Central Florida.

Justin Barry
Justin Barry

What do you consider some of the systemic issues limiting underrepresented minorities from greater employment opportunities in the technology industry?

This is a massive topic with a myriad of associated socioeconomic issues. One issue that jumps to the forefront for me is the schools where leading companies within the tech industry recruit from. Traditionally, these companies have limited their recruitment to top universities where Blacks and other underrepresented minorities comprise a small percentage of the student population. This is beginning to change, but I believe technology companies need to more aggressively expand their recruitment efforts, especially among historically Black colleges and universities (HBCUs).

What are some of the obstacles you had to overcome in pursuing your science career?

One issue is imposter syndrome — the idea that you're not good enough and you’re only in your position because you’ve been given special treatment. Although imposter syndrome is something everyone experiences, I think it’s particularly acute for Blacks given the clear underrepresentation within the technology industry. Imposter syndrome can touch all aspects of your job if you’re unaware, or if you don’t have the tools to deal with it. Not everyone has the tools to deal with it, and I suspect not everyone has correctly identified the problem.

Who or what inspired you to pursue your science career, and what lessons can we take from your experience?

Video games sparked my interest in computer science, and more specifically artificial intelligence. My undergraduate degree is in computer science and math, and machine learning and AI provide the opportunity to apply my computer science and math skills to real-world applications.

Nashlie Sephus is an applied science manager within Amazon Web Services Ai. She earned her PhD in electrical and computer engineering from Georgia Tech.

Nashlie Sephus
Nashlie Sephus

What do you consider some of the systemic issues limiting underrepresented minorities from greater employment opportunities in the technology industry?

Imposter syndrome is one issue I find common within underrepresented minority groups. It’s a feeling of being convinced that you don’t belong in the industry, or within advanced roles in the industry, regardless of your accolades and accomplishments. It is as if they are not real or didn’t happen. This is often due to not seeing many others who look like you in similar or higher positions. ‘You can’t be what you can’t see’ is a common thought. Also, there are few mentors or support systems for these groups, and as a black/female/engineer/scientist, you sometimes feel like the minority of the minority, which further isolates you.  

What are some of the obstacles you had to overcome in pursuing your science career?

At times, I have had to fight for myself and members of my teams for equal pay and advancement in my career. I also have needed to develop mechanisms to be heard when it was difficult to convey messages to those around me. I’m usually quiet and reserved, but over the years I’ve learned how to gain respect from peers by being more outspoken even, or especially, when I disagreed. This is one reason why I appreciate Amazon’s leadership principle: Have Backbone; Disagree and Commit. 

Who or what inspired you to pursue your science career, and what lessons can we take from your experience?

I grew up in a house full of women where we often did our own chores, like fixing and repairing things around the house. I was also always going to summer math and science camps in elementary and middle school, especially a summer engineering camp for girls after my eighth grade science teacher recommended I attend. This was when I was first introduced to the various areas of engineering, and fell in love with computer science. Being able to control the hardware with software was fascinating to me. I knew then that’s what I wanted to do. This early exposure to science was key to me figuring out one of my passions, in addition to music and sports.

Colby Wise is a senior deep learning scientist and manager within the AWS Machine Learning Solutions Lab. He earned his master’s degree in computer science from the Columbia University Fu Foundation School of Engineering and Applied Sciences.

Colby Wise
Colby Wise

What do you consider some of the systemic issues limiting underrepresented minorities from greater employment opportunities in the technology industry?

Educational opportunity. Science, technology, engineering, and math (STEM) careers in the technology industry are highly competitive. Over the years, we’ve seen advanced tools and technologies like cloud technology, machine learning, and deep learning, that were once reserved only for large companies or prestigious universities being utilized by students as early as junior high school. While this has created and accelerated educational opportunities for millions of students globally, the reality is that not all have been able to benefit. In the United States, public school funding varies significantly by geography, and where you grow up is a major factor in access to educational resources. Schools with advanced STEM courses and other after-school programs are valuable inroads for STEM students to accelerate their learning opportunities and explore careers in science. What’s more, these opportunities compound positively from lower educational levels to higher educational levels. While not the only factor, these programs are important when understanding the pipeline of underrepresented minorities in highly competitive industries like technology. For example, the US Federal Reserve conducted a study highlighting how educational attainment of parents plays an important role in children’s educational pursuits. Studies like this and others indicate that lower parental educational attainment may present a unique challenge for students. One potential consequence of underrepresentation of minorities in advanced degrees is that employment opportunities often arise from one’s social network, employee referrals, for example. This can be summarized as both an employment funnel problem and a network problem. While not always the case, a more diverse workforce can build connections to underrepresented talent pools. 

Financial equality. In a study from 2020, the US Federal Reserve found large and persistent gaps in net wealth and earnings by race and ethnicity. While education is a significant factor in wage gaps, the St. Louis Federal Reserve found net wealth by race was not as positively correlated with educational attainment for minorities. Educational attainment is extremely important. Many highly technical roles require advanced degrees. Financial equality and opportunity as characterized by job salary prospects, current income and net wealth, and access to educational funding sources like loans are all potential factors impacting lower minority employment. In 2016, the Brookings Institution found the median household net wealth for Black and Hispanic families to be 1/8th  that of white households. When you consider the rising cost of college and advanced degrees, this income and net wealth gap may also play a factor in why employment among underrepresented minorities is lower in highly competitive industries like technology. Specifically, minorities whose households cannot readily pay for advanced degrees choose between the implications of high debt burdens and lower comparative earnings, and often must forsake advanced degrees to enter or stay in the workforce.

Leadership representation. Representation of minorities in leadership positions is relatively low. It is unclear how much educational opportunity and financial equality contribute to this, compared to other issues such as equitable pathways to senior leadership positions. In many companies in which I have worked, you notice a similar triangular pattern of minority leadership where representation at junior levels is more in-line with industry trends, while there is a dearth of representation as you reach more senior positions. No doubt there is work to be done to drive greater employment of underrepresented minorities at all levels. But simply increasing the representation at entry levels does not address other attrition and talent-retention hurdles. Overall, companies need to take a more systematic, data-driven approach to move the needle and find solutions to underrepresentation of minorities in the tech industry. For instance, companies should not be afraid to tackle the complex issues at multiple hierarchies, such as creating innovative solutions to drive educational opportunity while objectively measuring current pathways to employment within the tech industry. Furthermore, companies should ensure financial equality by aligning corporate incentives with fair pay distributions, minority leadership representation, and talent development and retention.  

What are some of the obstacles you had to overcome in pursuing your science career?

Educational opportunity. While everyone’s path is different, unfortunately my story is rather common given its similarity to those of many underrepresented minorities. I faced and overcame obstacles in educational and financial opportunity plus roadblocks to leadership roles. I attribute my luck mainly to the many individuals who provided a helping hand, plus a little bit of hard work sprinkled in. I grew up in a single-parent household in an impoverished, high-crime inner-city area. Despite this, my family valued education highly, and one of my parents had an advanced degree which was extremely rare for the area. Given that, I always ranked in the top 1% in my coursework while very young. That said, district educational attainment rates were low, and advanced coursework or programs for gifted students were nonexistent. However, prior to high school an unfortunate family event led to me moving from one of the poorest areas in the country to one of the best school districts nationally. After discovering how far behind I was in math and science, my family and I worked extremely hard over several years to get me back in line with my expected academic grade level. Now fast-forwarding to college: I, like many other minorities, did not have the means to pay for college, nor easy access to loans. After being selected to a number of great schools, my decision was ultimately driven by the amount of money I received in scholarships and grants. During college I followed the same recipe for success: tons of luck, humility to ask for help, and a bit of hard work to land an internship as a sophomore at a prestigious Wall Street investment bank. There I was surrounded by some of the smartest minds in STEM, with many having achieved advanced degrees from top universities around the world. The vast majority of these individuals did not look like me. Desperately wanting to be accepted and succeed among my peers in industry is what drove me to pursue a career in science, and many years later brought me to AWS.

Who or what inspired you to pursue your science career, and what lessons can we take from your experience?

Family and friends. Ultimately, doing what you love and constantly learning while being curious is the greatest inspiration one needs to pursue a career in science. As discussed above, studies have shown a correlation between parental educational attainment and children’s attainment. Thinking forward a bit, I combined my passion for what I love in science — AI/ML — with a selfish goal of wanting to be a living model for a career in science for my children. My greatest inspiration, however, is my wife. She discovered her passion for science at a very young age with plentiful opportunities to explore that passion, ultimately helping her reach the pinnacles of academia, where she received undergraduate and graduate degrees from two of the top universities in the world. Her passion for science, hard work, and humility continue to inspire me on a daily basis.

Related content

ES, B, Barcelona
Are you interested in defining the science strategy that enables Amazon to market to millions of customers based on their lifecycle needs rather than one-size-fits-all campaigns? We are seeking a Applied Scientist to lead the science strategy for our Lifecycle Marketing Experimentation roadmap within the PRIMAS (Prime & Marketing analytics and science) team. The position is open to candidates in Amsterdam and Barcelona. In this role, you will own the end-to-end science approach that enables EU marketing to shift from broad, generic campaigns to targeted, cohort-based marketing that changes customer behavior. This is a high-ambiguity, high-impact role where you will define what problems are worth solving, build the science foundation from scratch, and influence senior business leaders on marketing strategy. You will work directly with Business Directors and channel leaders to solve critical business problems: how do we win back customers lost to competitors, convert Young Adults to Prime, and optimize marketing spend by de-averaging across customer cohorts. Key job responsibilities Science Strategy & Leadership: 1. Own the end-to-end science strategy for lifecycle marketing, defining the roadmap across audience targeting, behavioral modeling, and measurement 2. Navigate high ambiguity in defining customer journey frameworks and behavioral models – our most challenging science problem with no established playbook 3. Lead strategic discussions with business leaders translating business needs into science solutions and building trust across business and tech partners 4. Mentor and guide a team of 2-3 scientists and BIEs on technical execution while contributing hands-on to the hardest problems Advanced Customer Behavior Modeling: 1. Build sophisticated propensity models identifying customer cohorts based on lifecycle stage and complex behavioral patterns (e.g., Bargain hunters, Young adults Prime prospects) 2. Define customer journey frameworks using advanced techniques (Hidden Markov Models, sequential decision-making) to model how customers transition across lifecycle stages 3. Identify which customer behaviors and triggers drive lifecycle progression and what messaging/levers are most effective for each cohort 4. Integrate 1P behavioral data with 2P survey insights to create rich, actionable audience definitions Measurement & Cross-Workstream Integration: 1. Partner with measurement scientist to design experiments (RCTs) that isolate audience targeting effects from creative effects 2. Ensure audience definitions, journey models, and measurement frameworks work coherently across Meta, LiveRamp, and owned channels 3. Establish feedback loops connecting measurement insights back to model improvements About the team The PRIMAS (Prime & Marketing Analytics and Science) is the team that support the science & analytics needs of the EU Prime and Marketing organization, an org that supports the Prime and Marketing programs in European marketplaces and comprises 250-300 employees. The PRIMAS team, is part of a larger tech tech team of 100+ people called WIMSI (WW Integrated Marketing Systems and Intelligence). WIMSI core mission is to accelerate marketing technology capabilities that enable de-averaged customer experiences across the marketing funnel: awareness, consideration, and conversion.
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various business operations - Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation - Mentor other scientists and engineers in the use of ML techniques
ES, M, Madrid
At Amazon, we are committed to being the Earth's most customer-centric company. The European International Technology group (EU INTech) owns the enhancement and delivery of Amazon's engineering to all the varied customers and cultures of the world. We do this through a combination of partnerships with other Amazon technical teams and our own innovative new projects. You will be joining the Tamale team to work on Haul. As part of EU INTech and Haul, Tamale strives to create a discovery-driven shopping experience using challenging machine learning and ranking solutions. You will be exposed to large-scale recommendation systems, multi-objective optimization, and state-of-the-art deep learning architectures, and you'll be part of a key effort to improve our customers' browsing experience by building next-generation ranking models for Amazon Haul's endless scroll experience. We are looking for a passionate, talented, and inventive Scientist with a strong machine learning background to help build industry-leading ranking solutions. We strongly value your hard work and obsession to solve complex problems on behalf of Amazon customers. Key job responsibilities We look for applied scientists who possess a wide variety of skills. As the successful applicant for this role, you will work closely with your business partners to identify opportunities for innovation. You will apply machine learning solutions to optimize multi-objective ranking, improve discovery engagement through contextual signals, and scale ranking systems across multiple marketplaces. You will work with business leaders, scientists, and product managers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed ranking services. You will be part of a team of scientists and engineers working on solving ranking and personalization challenges at scale. You will be able to influence the scientific roadmap of the team, setting the standards for scientific excellence. You will be working with state-of-the-art architectures and real-time feature serving systems. Your work will improve the experience of millions of daily customers using Amazon Haul worldwide. You will have the chance to have great customer impact and continue growing in one of the most innovative companies in the world. You will learn a huge amount - and have a lot of fun - in the process!
IN, HR, Gurugram
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for International Emerging Stores (IES). Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the International Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions across International Emerging Store (India, MENA, Far-East, LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
US, MA, Boston
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As part of the AGI team, an Applied Scientist will collaborate closely with core scientist team developing Amazon Nova models. They will lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. This includes designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. The Applied Scientist will perform expert-level manual audits, conduct meta-audits to evaluate auditor performance, and provide targeted coaching to uplift overall quality capabilities. A critical aspect of this role involves developing and maintaining LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. The Applied Scientist will also set up the configuration of data collection workflows and communicate quality feedback to stakeholders. An Applied Scientist will also have a direct impact on enhancing customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. A day in the life An Applied Scientist with the AGI team will support quality solution design, conduct root cause analysis on data quality issues, research new auditing methodologies, and find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. Besides theoretical analysis and quality framework development, an Applied Scientist will also work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.