Top row, left to right, Ruomeng Cui, Christos Faloutsos, Nicholas Kullman, and Niklas Karlsson; bottom row, left to right, Joan Feigenbaum, Hugo Krawczyk, Aaditya Ramdas, and Aaron Roth.
Top row, left to right, Ruomeng Cui, Christos Faloutsos, Nicholas Kullman, and Niklas Karlsson; bottom row, left to right, Joan Feigenbaum, Hugo Krawczyk, Aaditya Ramdas, and Aaron Roth.

Recent honors and awards for Amazon scientists

Researchers honored for their contributions to the scientific community in 2023.

Ruomeng Cui won Management Science best paper award

Ruomeng Cui, an Amazon Visiting Academic with Amazon’s Supply Chain Optimization Technologies (SCOT) team, won the 2023 Management Science Best Paper Award in Operations Management.

Cui, who is on leave from her role as an associate professor in the department of Information System and Operations Management at the Goizueta Business School, Emory University, won the award along with her co-authors Jun Li and Dennis Zhang for their 2020 paper, “Reducing discrimination with reviews in the sharing economy: Evidence from field experiments on Airbnb.”

Their paper explored ways to reduce “widespread discrimination by hosts against guests of certain races in online marketplaces” by using peer-generated online reviews. Their work has influenced sharing platforms’ strategies to fight discrimination.

The award is given “to the manuscript judged to be most deserving for its contribution to the theory and practice of operations management among all operations papers published in the past 3 years at Management Science.”

Cui earned her PhD in operations management from the Kellogg School of Management, Northwestern University in 2014. In June 2022, she joined Amazon as a Visiting Academic. In that role, she is building and implementing cutting-edge causal inference, machine learning, optimization, and economic models to make supply chain decisions.

Christos Faloutsos won Donald G. Fink Overview Paper Award

Christos Faloutsos, an Amazon Scholar and the Fredkin Professor of Computer Science at Carnegie Mellon University, was part of a team that received the 2023 IEEE Signal Processing Society Donald G. Fink Overview Paper Award by the IEE Signal Processing Society for "Tensor Decomposition for Signal Processing and Machine Learning."

In their 2016 overview paper, Faloutsos and his coauthors — Nicholas D. Sidiropoulos, Lieven De Lathauwer, Xiao Fu, Kejun Huang, and Evangelos E. Papalexakis — noted that while tensors, which are a higher-dimensional analogue of a matrix, already had “a rich history, stretching over almost a century, and touching upon numerous disciplines” their usage had only then “become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining and machine learning." Their overview aimed “to provide a good starting point for researchers and practitioners interested in learning about and working with tensors.”

The IEEE Signal Processing Society Overview Paper Award honors the authors “of a journal article of broad interest that has had substantial impact over several years on a subject related to the Society’s technical scope.”

Faloutsos said he believes the paper’s impact can be attributed to the fact that tensors are powerful tools. “They can handle static graphs, time evolving graphs, knowledge graphs which consist of triplets such as subject, verb, object, e.g., who plays in what team, who lives in, what city, who is friends with whom.”

Faloutsos, who joined Amazon as a Scholar in 2018, researches large-scale data mining with emphasis on graphs and time sequences, anomaly detection, tensors, and fractals.

Nicholas Kullman won 2023 Transportation Science Journal Paper of the Year

Nicholas Kullman, a senior research scientist with Amazon Line Haul, won the 2023 Transportation Science Journal Paper of the Year. Kullman and his coauthors — Martin Cousineau, Justin C. Goodson, and Jorge E. Mendoza — were awarded for their 2021 paper, “Dynamic Ride-Hailing with Electric Vehicles”.

In the paper, the authors “consider the problem of an operator controlling a fleet of electric vehicles for use in a ride-hailing service. The operator, seeking to maximize profit, must assign vehicles to requests as they arise as well as recharge and reposition vehicles in anticipation of future requests.”

“As autonomous vehicles become more common, fleets of taxis may become more centrally coordinated,” Kullman explained. “We wanted to consider this case where there's a central authority that controls whether or not requests are accepted or rejected.

“We wanted to look at good policies for figuring out which vehicles should serve which requests and what do you do with your vehicles when they're not serving requests so that they are better positioned to be able to serve future requests — a sort of dynamic stochastic vehicle routing problem.”

The team utilized deep reinforcement learning to develop new policies. Those policies were compared “against some more classical operations research approaches” and “and against dual bounds on the value of an optimal policy.”

“I think one of the other reasons why the paper was well received was that we had dual bounds,” Kullman explained. “We built out a benchmark where we knew we could not have done better than that standard. Basically, if you're the taxi authority and you know exactly where and when these requests are going to pop up, what would you do?”

The team found its “best policy trained with deep reinforcement learning outperforms the reoptimization approach.” Kullman, who joined Amazon in 2021, earned a PhD in operations research from Université de Tours. At Amazon, he researches optimization of middle-mile linehaul operations.

Niklas Karlsson named IEEE Fellow

Niklas Karlsson, a senior principal research scientist in Amazon Advertising Engineering, was recently named an IEEE Fellow for “technical leadership to vSLAM and online advertising.” The designation took effect on Jan. 1. Karlsson leads a team within Amazon DSP (ADSP) engineering, where he oversees research pertaining to ADSP bidding and optimization.

Karlsson earned a master’s in engineering physics from Lund University and then earned both a master’s in statistics and applied probability and a PhD in control, dynamic systems, and robotics, from UC Santa Barbara. After graduating he joined Evolution Robotics as senior navigation and control engineer. While there, he and his colleagues developed and patented vSLAM (visual simultaneous localization and mapping), an odometry- and vision-based SLAM system.

In 2005, Karlsson transitioned to a role as principal control engineer with Advertising.com. There he leveraged his expertise in feedback control and systems engineering to develop a next generation of scalable and adaptive bidding solutions for ad campaign optimization. By way of acquisitions and mergers, he ended up with Yahoo, where, after 17 years in online advertising, he departed as the chief scientist and vice president of research and development for Yahoo’s Demand Side Platform.

The IEEE Fellow designation is conferred by the IEEE board of directors upon individuals with outstanding records of accomplishment in any of the IEEE fields of interest. The total number selected in any one year cannot exceed 0.1% of the total voting membership. IEEE Fellow is the highest grade of membership and is recognized by the technical community as a prestigious honor and an important career achievement.

Joan Feigenbaum named IEEE Fellow

Joan Feigenbaum, an Amazon Scholar and the Grace Murray Hopper professor of computer science at Yale University, will be elevated to IEEE Fellow grade in 2024. The grade of IEEE Fellow “recognizes exceptional distinction in the engineering profession.”

Feigenbaum, who works in the AWS Cryptographic Algorithms group on privacy-preserving computation, was awarded “for contributions to trust-management systems and Internet algorithmics.”

Hugo Krawczyk named IACR Distinguished speaker

Hugo Krawczyk, senior principal scientist, Amazon Web Services, was selected to present the 2023 IACR Distinguished Lecture.

The International Association for Cryptologic Research (IACR) Distinguished Lectures are awarded “to people who have made important contributions to cryptology research.”

Krawczyk, who is also an IACR Fellow, has made fundamental contributions to the cryptographic design of Internet standards like IPsec, IKE, and TLS. He also co-invented numerous cryptographic algorithms including the HMAC message authentication algorithm.

Prior to joining Amazon in July 2023, he was a principal researcher at the Algorand Foundation and part of its founding team. Prior to that, he was an IBM Fellow and Distinguished Research Staff Member at the IBM T.J. Watson Research Center as a member of the Cryptography Research group from 1992 to 1997, and again from 2004 to 2019. He was an associate professor at the Department of Electrical Engineering at the Technion in Israel from 1997 until 2004.

Aaditya Ramdas won Peter Gavin Hall IMS Early Career Prize

Aaditya Ramdas, an Amazon Visiting Academic who is also an assistant professor of statistics and machine learning at Carnegie Mellon University (CMU), won the Peter Gavin Hall Institute of Mathematical Statistics (IMS) Early Career Prize. Ramdas was recognized “for significant contributions in the areas of reproducibility in science and technology; active, sequential decision-making; and assumption-light uncertainty quantification.”

The prize “recognizes one researcher annually who is within the first eight years of completing their doctoral degree.” Ramdas has a bacehlor’s degree in computer science and engineering from IIT-Bombay and earned both a master’s and a PhD in statistics and machine learning from CMU.

Ramdas researches selective and simultaneous inference, game-theoretic statistics, and black-box predictive inference. His areas of applied interest include neuroscience, genetics and auditing.

Aaron Roth named CyLab's 2023 Distinguished Alumni Award winner

Aaron Roth, an Amazon Scholar who is the Henry Salvatori Professor of Computer and Cognitive Science at the University of Pennsylvania, was named Distinguished Alumni Award winner by CyLab, Carnegie Mellon University's security and privacy research institute. The award recognizes “Roth's excellence in algorithms and machine learning, leadership in the field, and commitment to his students.”

Roth, who joined Amazon as a Scholar in 2020, researches the algorithmic foundations of data privacy, algorithmic fairness, game theory, learning theory, and machine learning.

Related content

US, NY, New York
Fauna Robotics is building capable, safe, and delightful robots for everyday life, and voice is one of the most natural ways people will interact with them. Cloud speech and language models are good and getting better, but they can only work with the audio they receive, and a robot is a hard place to listen. Its microphones sit beside motors, fans, and moving joints. It speaks through its own loudspeaker while people talk over it. It moves, turns, and shares a room with several people at once. We are hiring a Principal Audio Scientist to be Fauna's technical authority on how our robots hear. You will design, prototype, and ship the hardest algorithms in the robot's audio system. You will set the audio architecture that other engineers build on, shape hardware decisions across robot generations, mentor the engineers and scientists working on audio, and be the person teams come to when the robot can't hear. Key job responsibilities - Set the long-range science roadmap and technical architecture for the robot's audio system, and serve as Fauna's primary technical authority on robot hearing - Design and implement suppression of the robot's own noise from motors, fans, and moving joints - Design and implement echo cancellation for the robot's own voice, so people can interrupt it naturally - Develop multi-microphone processing that holds up as the robot and the people around it move, including locating who is speaking so the robot can turn toward them - Make on-robot listening decisions robust to internal and external noise sources: wake word, voice activity, and whether speech is directed at the robot - Drive microphone and speaker placement, enclosure acoustics, and vibration isolation decisions with mechanical, electrical, and industrial design, backed by your own measurements - Design the robot-specific data collection and evaluation methods to validate the performance of our audio design - Present audio science and its tradeoffs to senior leadership and partner teams - Mentor scientists and engineers, raising the scientific bar for audio across the organization through design reviews, code reviews, and hiring
IN, KA, Bengaluru
Interested to build the next generation Financial systems that can handle billions of dollars in transactions? Interested to build highly scalable next generation systems that could utilize Amazon Cloud? Massive data volume + complex business rules in a highly distributed and service oriented architecture, a world class information collection and delivery challenge. Our challenge is to deliver the software systems which accurately capture, process, and report on the huge volume of financial transactions that are generated each day as millions of customers make purchases, as thousands of Vendors and Partners are paid, as inventory moves in and out of warehouses, as commissions are calculated, and as taxes are collected in hundreds of jurisdictions worldwide. Key job responsibilities • Understand the business and discover actionable insights from large volumes of data through application of machine learning, statistics or causal inference. • Analyse and extract relevant information from large amounts of Amazon’s historical transactions data to help automate and optimize key processes • Research, develop and implement novel machine learning and statistical approaches for anomaly, theft, fraud, abusive and wasteful transactions detection. • Use machine learning and analytical techniques to create scalable solutions for business problems. • Identify new areas where machine learning can be applied for solving business problems. • Partner with developers and business teams to put your models in production. • Mentor other scientists and engineers in the use of ML techniques. A day in the life • Understand the business and discover actionable insights from large volumes of data through application of machine learning, statistics or causal inference. • Analyse and extract relevant information from large amounts of Amazon’s historical transactions data to help automate and optimize key processes • Research, develop and implement novel machine learning and statistical approaches for anomaly, theft, fraud, abusive and wasteful transactions detection. • Use machine learning and analytical techniques to create scalable solutions for business problems. • Identify new areas where machine learning can be applied for solving business problems. • Partner with developers and business teams to put your models in production. • Mentor other scientists and engineers in the use of ML techniques. About the team The FinAuto TFAW(theft, fraud, abuse, waste) team is part of FGBS Org and focuses on building applications utilizing machine learning models to identify and prevent theft, fraud, abusive and wasteful(TFAW) financial transactions across Amazon. Our mission is to prevent every single TFAW transaction. As a Machine Learning Scientist in the team, you will be driving the TFAW Sciences roadmap, conduct research to develop state-of-the-art solutions through a combination of data mining, statistical and machine learning techniques, and coordinate with Engineering team to put these models into production. You will need to collaborate effectively with internal stakeholders, cross-functional teams to solve problems, create operational efficiencies, and deliver successfully against high organizational standards.
US, NY, New York
We are seeking a Robotics/AI Motor Control Scientist to develop cutting-edge machine learning algorithms for motor control systems in robots. In this role, you will focus on creating and optimizing intelligent motor control strategies to enable robots to perform complex, whole-body tasks. Your contributions will be essential in advancing robotics by enabling fluid, reliable, and safe interactions between robots and their environments. Key job responsibilities - Develop controllers that leverage reinforcement learning, imitation learning, or other advanced AI techniques to achieve natural, robust, and adaptive motor behaviors - Collaborate with multi-disciplinary teams to integrate motor control systems with robotic hardware, ensuring alignment with real-world constraints such as actuator dynamics and energy efficiency - Use simulation and real-world testing to refine and validate control algorithms - Stay updated on advancements in robotics, AI, and control systems to apply advanced techniques to robotic motion challenges - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers - Bridge research initiatives with practical engineering implementation About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you. an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
US, CA, Sunnyvale
Amazon's Artificial General Intelligence (AGI) organization is seeking an Applied Scientist III to advance the science of Responsible AI evaluation for large language models and generative AI. In this role, you will lead the design and development of rigorous evaluation methods, benchmarks, and metrics that measure the safety, fairness, robustness, and trustworthiness of frontier models. You will work with large-scale datasets, modern deep learning frameworks, and world-class scientists and engineers to turn research into evaluation systems that shape model launch decisions at Amazon scale. Key job responsibilities - Lead the design and implementation of evaluation frameworks, benchmarks, and metrics for responsible AI, including safety, fairness, robustness, and harmful content. - Build scalable automated evaluation pipelines for large language models, including model-based and human-in-the-loop evaluation. - Partner with pretraining, post-training, and product teams to translate evaluation results into model improvements and launch decisions. - Conduct rigorous experimentation and statistical analysis, and publish research at top venues. - Mentor junior scientists and help raise the scientific bar of the team. - Champion responsible AI practices across the model development lifecycle. About the team The AGI Responsible AI (RAI) team builds the science and systems that make Amazon's large language models safe, fair, and trustworthy. We work on problems spanning safety evaluation, content moderation, watermarking, bias mitigation, and alignment. Our team values scientific rigor, customer obsession, and rapid iteration, and we collaborate closely with pretraining, post-training, and product teams across AGI.
CH, Zurich
RIVR, an Amazon company, is building Physical AI by deploying autonomous robots for real-world doorstep delivery. Operating daily in diverse urban environments, RIVR's robots continuously learn from and navigate the millions of scenarios encountered during deliveries. By owning the full stack from software to hardware, RIVR is purpose-built for safety, reliability, and the customer from day one. Reinforcement learning is transforming our robotic intelligence, enabling autonomous behavior without human guidance. We are seeking a Senior AI Engineer with deep expertise in reinforcement learning and deep learning, including supervised and self-supervised learning with a focus on dexterous manipulation. Your role will involve leveraging both simulated and real-world data to address practical challenges in dynamic grasping, contact-rich manipulation, and object interaction. If you are passionate about advancing AI and developing innovative solutions, join us in shaping the future of intelligent robotics. Key job responsibilities Develop cutting-edge reinforcement learning algorithms to enable robust, contact-rich dexterous manipulation, translating vision, depth, tactile, and proprioceptive sensor input into precise end-effector and joint-level motor commands. Design, test, and refine algorithms to solve complex real-world manipulation challenges, such as handling diverse package form factors, dynamic hand-offs, and operating door handles or latches. Collaborate with the foundation model team to innovate methods that leverage both simulated and real-world data.
US, WA, Seattle
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Selling Partner Trust & Store Integrity Science Team. We are looking for a talented scientist who is passionate to build advanced machine learning systems that help manage the safety of millions of transactions every day and scale up our operation with automation. Key job responsibilities Innovate with the latest GenAI/LLM/VLM technology to build highly automated solutions for efficient fraud detection, risk evaluation and automated operations Design, develop and deploy end-to-end advance machine learning solutions with vision anf GenAi technologies in the Amazon production environment to create impactful business value Learn, explore and experiment with the latest machine learning advancements to create the best customer experience A day in the life You will be working within a dynamic, diverse, and supportive group of scientists who share your passion for innovation and excellence. You'll be working closely with business partners and engineering teams to create end-to-end scalable machine learning solutions that address real-world problems. You will build scalable, efficient, and automated processes for large-scale data analyses, model development, model validation, and model implementation. You will also be providing clear and compelling reports for your solutions and contributing to the ongoing innovation and knowledge-sharing that are central to the team's success.
US, NY, New York
We are seeking an Applied Scientist to contribute to research and development of novel security validation and monitoring techniques for AI systems at scale. You will own and contribute to four critical work-streams: 1. Real-Time Agent Monitoring Design and implement scientific approaches for continuous behavioral analysis of AI agents in production—detecting anomalous actions, prompt injection exploitation, and policy violations in real time. 2. Protection & Automated Remediation Invent and deliver novel protection technologies and automated remediation techniques building on research in security, cryptography, privacy, automated reasoning, and others domains, to enable safe and secure agentic AI models and AI applications. 3. AI Application and Capabilities Validation Invent and deliver scalable methodologies for security testing of AI applications and AI capabilities (e.g. MCP, skills), including adversarial robustness evaluation, safety guardrail bypass detection, tool-use authorization boundaries, and trust boundary verification. 4. AI Asset Discovery & Inventory Research and build scalable techniques to automatically discover, identify, and catalog all AI-enabled applications, services, and capabilities across the company—maintaining a comprehensive, continuously updated database of AI assets. Key job responsibilities Invent • Identify and frame new research challenges in AI security where problems are ill-defined and require novel scientific paradigms at the product level. • Contribute to the team's scientific agenda for agent monitoring, protection, remediation, validation research, and AI asset discovery. • Publish research results at peer-reviewed internal and external venues (e.g., USENIX Security, ACM CCS, IEEE S&P, NeurIPS, ICML security workshops, ICSE, PETS) when appropriate. • Articulate key scientific challenges of current and future AI security threats and deliver novel research to address them. • Design, implementation, and successful delivery of scientifically complex security solutions into production—both brand new systems and evolutions of existing ones. • Write significant portions of critical-path code for detection models, validation engines, protection technologies, and asset discovery / classification systems. • Assess and select appropriate technologies (e.g., data protection, private inference, graph-based anomaly detection, NLP-based service classification, code/traffic analysis for AI fingerprinting) for production systems. • Use best practices in scientific methodology and software engineering across the team; provide insightful peer reviews of code, design, and architecture artifacts. • Deliver solutions that are inventive, maintainable, scalable, and extensible. Influence • Autonomously drive discussions with security engineers, software engineers, product managers, and scientist peers across multiple teams. • Build consensus on larger cross-team security initiatives and factor complex efforts into independent workstreams. • Identify and resolve endemic problems, including areas where current security tooling limits innovation of partner teams. • Contribute to the broader internal and external scientific communities as a subject matter expert in AI security. About the team Diverse Experiences Amazon Security values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon Security? At Amazon, security is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for security across all of Amazon’s products and services. We offer talented security professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Security, it’s in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest security challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve.
IN, KA, Bangalore
Amazon’s Last Mile Team is looking for a passionate individual with strong optimization and analytical skills to join its Last Mile Science team in the endeavor of designing and improving the most complex planning of delivery network in the world. Last Mile builds global solutions that enable Amazon to attract an elastic supply of drivers, companies, and assets needed to deliver Amazon's and other shippers' volumes at the lowest cost and with the best customer delivery experience. Last Mile Science team owns the core decision models in the space of jurisdiction planning, delivery channel and modes network design, capacity planning for on the road and at delivery stations, routing inputs estimation and optimization. Our research has direct impact on customer experience, driver and station associate experience, Delivery Service Partner (DSP)’s success and the sustainable growth of Amazon. Optimizing the last mile delivery requires deep understanding of transportation, supply chain management, pricing strategies and forecasting. Only through innovative and strategic thinking, we will make the right capital investments in technology, assets and infrastructures that allows for long-term success. Our team members have an opportunity to be on the forefront of supply chain thought leadership by working on some of the most difficult problems in the industry with some of the best product managers, scientists, and software engineers in the industry. Key job responsibilities Candidates will be responsible for developing solutions to better manage and optimize delivery capacity in the last mile network. The successful candidate should have solid research experience in one or more technical areas of Operations Research or Machine Learning. These positions will focus on identifying and analyzing opportunities to improve existing algorithms and also on optimizing the system policies across the management of external delivery service providers and internal planning strategies. They require superior logical thinkers who are able to quickly approach large ambiguous problems, turn high-level business requirements into mathematical models, identify the right solution approach, and contribute to the software development for production systems. To support their proposals, candidates should be able to independently mine and analyze data, and be able to use any necessary programming and statistical analysis software to do so. Successful candidates must thrive in fast-paced environments, which encourage collaborative and creative problem solving, be able to measure and estimate risks, constructively critique peer research, and align research focuses with the Amazon's strategic needs. As a senior scientist, you will also help coach/mentor junior scientists in the team.
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers