Calibrating human-AI collaboration: impact of risk, ambiguity and transparency on algorithmic bias

2020
Download Copy BibTeX
Copy BibTeX
Transparent Machine Learning (ML) is often argued to increase trust into predictions of algorithms however the growth of new interpretability approaches is not accompanied by a growth in studies investigating how interaction of humans and Artificial Intelligence (AI) systems benefits from transparency. The right level of transparency can increase trust in an AI system, while inappropriate levels of transparency can lead to algorithmic bias. In this study, we demonstrate that depending on certain personality traits, humans exhibit different susceptibilities for algorithmic bias. Our main finding is that susceptibility to algorithmic bias significantly depends on annotators’ affinity to risk. These findings help to shed light on the previously underrepresented role of human personality in human-AI interaction. We believe that taking these aspects into account when building transparent AI systems can help to ensure more responsible usage of AI systems.
Research areas

Latest news

US, WA, Bellevue
Have you ever ordered a product on Amazon and when that box with the smile arrived you wondered how it got to you so fast? Have you wondered where it came from and how much it cost Amazon to deliver it to you? If so, the WW Amazon Logistics, Business Analytics team is for you. We manage the delivery of tens of millions of products every week to Amazon’s customers, achieving on-time delivery in a cost-effective manner. We are looking for an enthusiastic, customer obsessed, Sr. Applied Scientist with good analytical skills to help manage projects and operations, implement scheduling solutions, improve metrics, and develop scalable processes and tools. The primary role of an Operations Research Scientist within Amazon is to address business challenges through building a compelling case, and using data to influence change across the organization. This individual will be given responsibility on their first day to own those business challenges and the autonomy to think strategically and make data driven decisions. Decisions and tools made in this role will have significant impact to the customer experience, as it will have a major impact on how the final phase of delivery is done at Amazon. Ideal candidates will be a high potential, strategic and analytic graduate with a PhD in (Operations Research, Statistics, Engineering, and Supply Chain) ready for challenging opportunities in the core of our world class operations space. Great candidates have a history of operations research, and the ability to use data and research to make changes. This role requires robust program management skills and research science skills in order to act on research outcomes. This individual will need to be able to work with a team, but also be comfortable making decisions independently, in what is often times an ambiguous environment. Responsibilities may include: - Develop input and assumptions based preexisting models to estimate the costs and savings opportunities associated with varying levels of network growth and operations - Creating metrics to measure business performance, identify root causes and trends, and prescribe action plans - Managing multiple projects simultaneously - Working with technology teams and product managers to develop new tools and systems to support the growth of the business - Communicating with and supporting various internal stakeholders and external audiences
US, CA, Sunnyvale
Amazon Industrial Robotics is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. We leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. We are pioneering the development of robotics foundation models that: - Enable unprecedented generalization across diverse tasks - Integrate multi-modal learning capabilities (visual, tactile, linguistic) - Accelerate skill acquisition through demonstration learning - Enhance robotic perception and environmental understanding - Streamline development processes through reusable capabilities The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. As an Applied Scientist, you will develop and improve machine learning systems that help robots perceive, reason, and act in real-world environments. You will leverage state-of-the-art models (open source and internal research), evaluate them on representative tasks, and adapt/optimize them to meet robustness, safety, and performance needs. You will invent new algorithms where gaps exist. You’ll collaborate closely with research, controls, hardware, and product-facing teams, and your outputs will be used by downstream teams to further customize and deploy on specific robot embodiments. Key job responsibilities - Leverage state-of-the-art models for targeted tasks, environments, and robot embodiments through fine-tuning and optimization. - Execute rapid, rigorous experimentation with reproducible results and solid engineering practices, closing the gap between sim and real environments. - Build and run capability evaluations/benchmarks to clearly profile performance, generalization, and failure modes. - Contribute to the data and training workflow: collection/curation, dataset quality/provenance, and repeatable training recipes. - Write clean, maintainable, well commented and documented code, contribute to training infrastructure, create tools for model evaluation and testing, and implement necessary APIs - Stay current with latest developments in foundation models and robotics, assist in literature reviews and research documentation, prepare technical reports and presentations, and contribute to research discussions and brainstorming sessions. - Work closely with senior scientists, engineers, and leaders across multiple teams, participate in knowledge sharing, support integration efforts with robotics hardware teams, and help document best practices and methodologies.
US, NY, New York
Advertising at Amazon is growing incredibly fast and we are responsible for defining and delivering a collection of advertising products that drive discovery and sales. Amazon Business Ads is equally growing fast ($XXXMs to $XBs) and owns engineering and science for the AB WW ad experience. We build business-to-business (“B2B”) specific ad solutions distributed across retail and ad systems for shopper and advertiser experiences. Some include new ad placements or widgets, creatives, sourcing techniques, ad campaign management capabilities and much more! We consider unique AB qualities which are differentiated from the consumer experience such as varying shopper role types, purchasing complexities based on business size and industry (eg education vs healthcare), AB specific features (eg business discounts, buying policies to restrict and prefer products), and AB buyer behaviors (eg buying in bulk). We are seeking a scientific leader who can drive innovation in complex problem areas and new business initiatives. The ideal candidate will: Technical & Research Requirements: * Demonstrate fluency in Python, R, Matlab or other statistical languages and familiarity with deep learning frameworks like PyTorch, TensorFlow * Lead end-to-end solution development from research to prototyping and experimentation * Write and deploy significant parts of scientifically novel software solutions into production Leadership & Influence: * Drive team's scientific agenda by proposing new initiatives and securing management buy-in including PM, SDM * Mentor colleagues and contribute to their professional development * Build consensus on large projects and influence decisions across different teams in Ads Key Leadership Principles: * Dive Deep: Uncover non-obvious insights in data * Deliver Results: Create solutions aligned with customer and product needs * Learn and Be Curious: Demonstrate self-driven desire to explore new research areas * Earn Trust: Build relationships with stakeholders through understanding business needs
JP, 13, Tokyo
Are you a Graduate Student interested in machine learning, natural language processing, computer vision, automated reasoning, robotics? We are looking for skilled scientists capable of putting theory into practice through experimentation and invention, leveraging science techniques and implementing systems to work on massive datasets in an effort to tackle never-before-solved problems. A successful candidate will be a self-starter comfortable with ambiguity, strong attention to detail, and the ability to work in a fast-paced, ever-changing environment. As an Applied Scientist, you will own the design and development of end-to-end systems. You’ll have the opportunity to create technical roadmaps, and drive production level projects that will support Amazon Science. You will work closely with Amazon scientists, and other science interns to develop solutions and deploy them into production. The ideal scientist must have the ability to work with diverse groups of people and cross-functional teams to solve complex business problems. Key job responsibilities Amazon Science gives insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Amazon Scientist use our working backwards method to enrich the way we live and work. A day in the life Come teach us a few things, and we’ll teach you a few things as we navigate the most customer-centric company on Earth.
US, NY, New York
At Amazon Selection and Catalog Systems (ASCS), our mission is to power the online buying experience for customers worldwide so they can find, discover, and buy any product they want. We innovate on behalf of our customers to ensure uniqueness and consistency of product identity and to infer relationships between products in Amazon Catalog to drive the selection gateway for the search and browse experiences on the website. We're solving a fundamental AI challenge: establishing product identity and relationships at unprecedented scale. Using Generative AI, Visual Language Models (VLMs), and multimodal reasoning, we determine what makes each product unique and how products relate to one another across Amazon's catalog. The scale is staggering: billions of products, petabytes of multimodal data, millions of sellers, dozens of languages, and infinite product diversity—from electronics to groceries to digital content. The research challenges are immense. GenAI and VLMs hold transformative promise for catalog understanding, but we operate where traditional methods fail: ambiguous problem spaces, incomplete and noisy data, inherent uncertainty, reasoning across both images and textual data, and explaining decisions at scale. Establishing product identities and groupings requires sophisticated models that reason across text, images, and structured data—while maintaining accuracy and trust for high-stakes business decisions affecting millions of customers daily. Amazon's Item and Relationship Platform group is looking for an innovative and customer-focused applied scientist to help us make the world's best product catalog even better. In this role, you will partner with technology and business leaders to build new state-of-the-art algorithms, models, and services to infer product-to-product relationships that matter to our customers. You will pioneer advanced GenAI solutions that power next-generation agentic shopping experiences, working in a collaborative environment where you can experiment with massive data from the world's largest product catalog, tackle problems at the frontier of AI research, rapidly implement and deploy your algorithmic ideas at scale, across millions of customers. Key job responsibilities * Formulate novel research problems at the intersection of GenAI, multimodal learning, and large-scale information retrieval—translating ambiguous business challenges into tractable scientific frameworks * Design and implement leading models leveraging VLMs, foundation models, and agentic architectures to solve product identity, relationship inference, and catalog understanding at billion-product scale * Pioneer explainable AI methodologies that balance model performance with scalability requirements for production systems impacting millions of daily customer decisions * Own end-to-end ML pipelines from research ideation to production deployment—processing petabytes of multimodal data with rigorous evaluation frameworks * Define research roadmaps aligned with business priorities, balancing foundational research with incremental product improvements * Mentor peer scientists and engineers on advanced ML techniques, experimental design, and scientific rigor—building organizational capability in GenAI and multimodal AI * Represent the team in the broader science community—publishing findings, delivering tech talks, and staying at the forefront of GenAI, VLM, and agentic system research
US, WA, Bellevue
The Amazon Fulfillment Technology (AFT) Science team is seeking an exceptional Applied Scientist with strong operations research and optimization expertise to develop production solutions for one of the most complex systems in the world: Amazon's Fulfillment Network. At AFT Science, we design, build, and deploy optimization, statistics, machine learning, and GenAI/LLM solutions that power production systems running across Amazon Fulfillment Centers worldwide. We solve a wide range of challenges encountered throughout the network, including labor planning and staffing, pick scheduling, stow guidance, and capacity risk management. We are tasked with developing innovative, scalable, and reliable science-driven production solutions that exceed the published state of the art, enabling systems to run frequently (ranging from every few minutes to every few hours per use case) and continuously across our large-scale network. Key job responsibilities As an Applied Scientist, you will collaborate with other scientists, software engineers, product managers, and operations leaders to develop optimization-driven solutions using a variety of tools and observe direct impact on process efficiency and associate experience in the fulfillment network. Key responsibilities include: - Develop understanding and domain knowledge of operational processes, system architecture and functions, and business requirements - Deep dive into data and code to identify opportunities for continuous improvement and/or disruptive new approaches - Develop scalable mathematical models for production systems to derive optimal or near-optimal solutions for existing and new challenges - Create prototypes and simulations for agile experimentation of devised solutions - Advocate for technical solutions with business stakeholders, engineering teams, and senior leadership - Partner with engineers to integrate prototypes into production systems - Design experiments to test new or incremental solutions launched in production and build metrics to track performance A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team Amazon Fulfillment Technology (AFT) designs, develops, and operates end-to-end fulfillment technology solutions for all Amazon Fulfillment Centers (FCs). We harmonize the physical and virtual worlds so Amazon customers can get what they want, when they want it. The AFT Science team has expertise in operations research, optimization, statistics, machine learning, and GenAI/LLM. We also possess deep domain expertise in operational processes within FCs and their challenges. We prioritize advancements that support AFT tech teams and focus areas rather than specific fields of research or individual business partners. We influence each stage of innovation from inception to deployment, which includes both developing novel solutions and improving existing approaches. Resulting production systems rely on a diverse set of technologies; our teams therefore invest in multiple specialties as the needs of each focus area evolve.
US, MA, N.reading
Amazon Industrial Robotics is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at an unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic manipulation, locomotion, and human-robot interaction. As an Applied Scientist in Sensing, you will develop innovative and complex sensing systems for our emerging robotic solutions and improve existing on-robot sensing to optimize performance and enhance customer experience. The ideal candidate has demonstrated experience designing and troubleshooting custom sensor systems from the ground up. They enjoy analytical problem solving and possess practical knowledge of robotic design, fabrication, assembly, and rapid prototyping. They thrive in an interdisciplinary environment and have led the development of complex sensing systems. Key job responsibilities - Design and adapt holistic on-robot sensing solutions for ambiguous problems with fluid requirements - Mentor and develop junior scientists and engineers - Work with an interdisciplinary team to execute product designs from concept to production including specification, design, prototyping, validation and testing - Have responsibility for the designs and performance of a sensing system design - Work with the Operations, Manufacturing, Supply Chain and Quality organizations as well as vendors to ensure a fast development and delivery of the sensing concepts to the team - Develop overall safety concept of the sensing platform - Exhibit role model behaviors of applied science best practices, thorough and predictive analysis and cradle to grave ownership
US, CA, San Francisco
Amazon has launched a new research lab in San Francisco to develop foundational capabilities for useful AI agents. We’re enabling practical AI to make our customers more productive, empowered, and fulfilled. In particular, our work combines large language models (LLMs) with reinforcement learning (RL) to solve reasoning, planning, and world modeling in both virtual and physical environments. Our research builds on that of Amazon’s broader AGI organization, which recently introduced Amazon Nova, a new generation of state-of-the-art foundation models (FMs). Our lab is a small, talent-dense team with the resources and scale of Amazon. Each team in the lab has the autonomy to move fast and the long-term commitment to pursue high-risk, high-payoff research. We’re entering an exciting new era where agents can redefine what AI makes possible. We’d love for you to join our lab and build it from the ground up! Key job responsibilities You will be responsible for maintaining our task management system which supports many internal and external stakeholders and ensures we are able to continue adding orders of magnitude more data and reliability.
US, WA, Seattle
The AWS Marketplace & Partner Services (AMPS) Science team is at the forefront of driving AWS's AI-powered growth through developing and evaluating the next generation search, recommendations, and agentic systems. As a Principal Applied Scientist on our team, you'll own the technical strategy and execution for our most ambitious AI-driven initiatives that directly accelerate AWS revenue growth through both customer discovery experiences and partner co-selling capabilities. Your work will establish Marketplace as customers' primary solution discovery and transaction platform while simultaneously transforming how AWS partners scale their business through insights and agentic workflow automation. You will be a trusted technical leader who tackles intrinsically complex scientific problems by leveraging deep expertise in information retrieval, recommendation systems, LLMs, and agentic systems. You'll innovate and set the standard for scientific excellence, making decisions that affect how products are built and integrated across AMPS. You'll align teams toward coherent strategies, guide peers in adopting the latest scientific trends, and force-multiply by decomposing hard problems and executing through collaboration. Key job responsibilities - Define direction for next-generation recommendation systems, including deep personalization, agentic architecture, and primitives that integrate customer usage patterns, infrastructure requirements, and business objectives to deliver personalized, outcome-oriented guidance - Lead information retrieval innovations, multi-objective ranking, and reasoning over heterogeneous solution types - Architect, implement, and design agentic AI systems that orchestrate complex workflows - Bridge theoretical innovations with practical solutions, making critical judgments to select the best technical approaches for both short and long-term objectives - Mentor and guide applied scientists, holding the team to high standards of technical rigor and scientific excellence - Contribute to the broader scientific community through patents, publications at conferences, and engagement with academic partners About the team The AWS Marketplace & Partner Services Science team is at the forefront of developing and deploying AI/ML systems that serve multiple critical stakeholders: - AWS Customers: Through the AWS Marketplace, we support discovery experiences that streamline cloud adoption and innovation. - AWS Partners: Via Partner Central, we offer advanced tools and insights to enhance collaboration and drive mutual growth. - Internal AWS Sellers: We equip our sales teams with data-driven recommendations to better serve our customers and partners.
US, MA, North Reading
At Amazon Robotics, we design advanced robotic systems capable of intelligent perception, learning, and action alongside humans, all on a large scale. Our goal is to develop robots that increase productivity and efficiency at the Amazon fulfillment centers while ensuring the safety of workers. We are seeking an Applied Scientist to develop innovative, scalable solutions in feedback control and state estimation for robotic systems, with a focus on contact-rich manipulation tasks. In this role, you will formulate physics-based models of robotic systems, perform analytical and numerical studies, and design control and estimation algorithms that integrate fundamental principles with data-driven techniques. You will collaborate with a world-class team of experts in perception, machine learning, motion planning, and feedback controls to innovate and develop solutions for complex real-world problems. As part of your work, you will investigate applicable academic and industry research to develop, implement, and test solutions that support product features. You will also design and validate production designs. To succeed in this role, you should demonstrate a strong working knowledge of physical systems, a desire to learn from new challenges, and the problem-solving and communication skills to work within a highly interactive and experienced team. Candidates must show a hands-on passion for their work and the ability to communicate their ideas and concepts both verbally and visually. Key job responsibilities - Research, design, implement, and evaluate feedback control, estimation, and motion-planning algorithms, ensuring effective integration with perception, manipulation, and system-level components. - Develop experiments, simulations, and hardware prototypes to validate control algorithms, and optimization techniques in contact-rich manipulation and other challenging scenarios. - Collaborate with software engineering teams to enable scalable, real-time, and maintainable implementations of algorithms in production systems. - Partner with cross-functional teams across hardware, systems engineering, science, and operations to transition algorithms from early prototyping to robust, production-ready solutions. - Engage with stakeholders at all levels to iterate on system design, define requirements, and drive integration of control and estimation capabilities into Amazon Robotics platforms. A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply!