Social
ConversationalAI.svg
Research Area

Conversational AI

Building software and systems that help people communicate with computers naturally, as if communicating with family and friends.
Explore more

Publications

View all View all
  • Pyramid dynamic inference: Encouraging faster inference via early exit boosting
    Ershad Banijamali, Pegah Kharazmi, Sepehr Eghbali, Jixuan Wang, Clement Chung, Samridhi Choudhary
    NeurIPS 2022 Workshop on Efficient Natural Language and Speech Processing (ENLSP), ICASSP 2023
    2022
    Transformer-based models demonstrate state of the art results on several natural language understanding tasks. However, their deployment comes at the cost of increased footprint and inference latency, limiting their adoption to real-time applications. Early exit strategies are designed to speed-up the inference by routing out a subset of samples at the earlier layers of the model. Exiting early causes losing
    Conversational AI
  • GEMv2: Multilingual NLG benchmarking in a single line of code
    Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina McMillan-Major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez-Beltrachini, Leonardo Ribeiro, Lewis Tunstall, Li Zhang, Mahima Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanch, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
    EMNLP 2022
    2022
    Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. The compatibility, often facilitated through leaderboards, thus leads to outdated but standardized evaluation practices. We pose that the standardization is taking place in the wrong spot. Evaluation infrastructure should enable researchers to use the latest methods
    Conversational AI
  • Fact checking machine generated text with dependency trees
    Alex Estes, Nikhita Vedula, Marcus Collins, Matthew Cecil, Oleg Rokhlenko
    EMNLP 2022
    2022
    Factual and logical errors made by Natural Language Generation (NLG) systems limit their applicability in many settings. We study this problem in a conversational search and recommendation setting, and observe that we can often make two simplifying assumptions in this domain: (i) there exists a body of structured knowledge we can use for verifying factuality of generated text; and (ii) the text to be factually
    Conversational AI
  • Weakly supervised data augmentation through prompting for dialogue understanding
    Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Andy Rosenbaum, Seokhwan Kim, Yang Liu, Zhou Yu, Dilek Hakkani-Tür
    NeurIPS 2022 Workshop on SyntheticData4ML
    2022
    Dialogue understanding tasks often necessitate abundant annotated data to achieve good performance and that presents challenges in low-resource settings. To alleviate this barrier, we explore few-shot data augmentation for dialogue understanding by prompting large pre-trained language models and present a novel approach that iterates on augmentation quality by applying weakly-supervised filters. We evaluate
    Conversational AI
  • Class incremental learning for intent classification with limited or no old data
    Debjit Paul, Daniil Sorokin, Judith Gaspers
    EMNLP 2022 Workshop on Ever Evolving NLP
    2022
    In this paper, we explore class-incremental learning for intent classification (IC) in a setting with limited old data available. IC is the task of mapping user utterances to their corresponding intents. Even though class incremental learning without storing the old data yields high potential of reducing human and computational resources in industry NLP model releases, to the best of our knowledge, it hasn
    Conversational AI
Load more

Related content

Load more

Work with us

See more jobs See more jobs
Applied Scientist II, Machine Learning Accelerator - Annapurna Labs
US, TX, Austin
Our team is involved with pre-silicon design verification for custom IP. A critical requirement of the verification flow is the requirement of legal and realistic stimulus of a custom Machine Learning Accelerator Chip. Content creation is built using formal methods that model legal behavior of the design and then solving the problem to create the specific assembly tests. The entire frame work for creating these custom tests is developed using a SMT solver and custom software code to guide the solution space into templated scenarios. This highly visible and innovative role requires the design of this solving framework and collaborating with design verification engineers, hardware architects and designers to ensure that interesting content can be created for the projects needs. Key job responsibilities Develop an understanding for a custom machine learning instruction set architecture. Model correctness of instruction streams using first order logic. Create custom API's to allow control over scheduling and randomness. Deploy algorithms to ensure concurrent code is safely constructed. Create coverage metrics to ensure solution space coverage. Use novel methods like machine learning to automate content creation.
Applied Scientist, Personalization, Personalization
IL, Tel Aviv
We are seeking an Applied Scientist to help build Amazon’s next-generation customer memory and personalization systems. Are you interested in building systems that move beyond reacting to customer behavior, to actually understanding and remembering it over time? Our team is building Amazon’s customer memory layer – a system that extracts, curates, and reasons over customer knowledge to power next-generation personalization. This includes transforming noisy, unstructured signals into durable, high-quality representations of customer preferences, intents, and life events, and using them in real time to improve customer experiences. We are part of Amazon’s Personalization organization, a high-performing group that leverages large-scale machine learning, generative AI, and distributed systems to deliver highly relevant customer experiences. We tackle challenging problems at the intersection of information extraction, knowledge representation, LLM reasoning, and recommendation systems. Our systems operate under real-world constraints of scale, latency, and quality, requiring careful tradeoffs between precision, recall, and responsiveness. This team plays a central role in defining how Amazon understands its customers, and how that understanding is applied across the shopping experience. As an Applied Scientist, you will design and build ML and LLM-powered solutions for Amazon's customer memory and personalization systems. You will work on how customer knowledge is extracted, validated, and applied in production systems. You will own the end-to-end delivery of ML solutions, from problem formulation and modeling to offline and online experimentation, and production deployment at scale. You will deliver high-quality, scalable systems that power customer-facing experiences. You will drive work across areas such as fact extraction, memory quality and lifecycle, temporal reasoning, and grounded personalization, while navigating tradeoffs between quality, latency, and coverage. You will collaborate closely with engineering and product teams to translate research into measurable customer impact. Please visit https://www.amazon.science for more information.
Economist, Ads Measurement Science
US, NY, New York
The Ads Measurement Science team in the Measurement, Ad Tech, and Data Science (MADS) team of Amazon Ads serves a centralized role developing solutions for a multitude of performance measurement products. We create solutions which measure the comprehensive impact of their ad spend, including sales impacts both online and offline and across timescales, and provide actionable insights that enable our advertisers to optimize their media portfolios. We leverage a host of scientific technologies to accomplish this mission, including Generative AI, classical ML, Causal Inference, Natural Language Processing, and Computer Vision. We are hiring an Economist on the team to develop the next generation of incrementality measurement products, capturing the effect of advertising in driving sales as well as the effects of measurement tools on advertiser engagement with Amazon. As an Economist on the team, you will lead the design, implementation, and validation of large-scale causal inference methodologies to capture these properties. You will communicate your results with science and business leaders, and partner with other scientists and engineers to carry solutions into production. Key job responsibilities Leverage deep expertise in causal inference to develop robust, causally grounded ads measurement solutions Disambiguate problems to propose clear evaluation frameworks and success criteria Work autonomously and write high quality technical documents Partner closely with other scientists to deliver large, multi-faceted technical projects Share and publish works with the broader scientific community through meetings and conferences Communicate clearly to both technical and non-technical audiences and leaders Contribute new ideas that shape the direction of the team's work Mentor more junior scientists and participate in the hiring process
Applied Scientist, Ads Measurement Science
US, NY, New York
The Ads Measurement Science team in the Measurement, Ad Tech, and Data Science (MADS) team of Amazon Ads serves a centralized role developing solutions for a multitude of performance measurement products. We create solutions which measure the comprehensive impact of advertiser's ad spend, including sales impacts both online and offline and across timescales, and provide actionable insights that enable our advertisers to optimize their media portfolios. We also own the science solutions for AI tools that unlock new insights and automate high-effort customer workflows, such as custom query and report generation based on natural language user requests. We leverage a host of scientific technologies to accomplish this mission, including Generative AI, classical ML, Causal Inference, Natural Language Processing, and Computer Vision. As an Applied Scientist on the team, you will lead measurement solutions end-to-end from inception to production. You will propose, design, analyze, and productionize models to provide novel measurement insights to our customers. Key job responsibilities Leverage deep expertise in one or more scientific disciplines to invent solutions to ambiguous ads measurement problems Disambiguate problems to propose clear evaluation frameworks and success criteria Work autonomously and write high quality technical documents Implement a significant portion of critical-path code, and partner with engineers to directly carry solutions into production Partner closely with other scientists to deliver large, multi-faceted technical projects Share and publish works with the broader scientific community through meetings and conferences Communicate clearly to both technical and non-technical audiences Contribute new ideas that shape the direction of the team's work Mentor more junior scientists and participate in the hiring process About the team We are a team of scientists across Applied, Research, Data Science and Economist disciplines. You will work with colleagues with deep expertise in ML, NLP, CV, Gen AI, and Causal Inference with a diverse range of backgrounds. We partner closely with top-notch engineers, product managers, sales leaders, and other scientists with expertise in the ads industry and on building scalable modeling and software solutions.
Fabrication R&D Scientist, Amazon Center for Quantum Computing
US, CA, Pasadena
The Amazon Center for Quantum Computing in Pasadena, CA, is looking to hire a Fabrication R&D Scientist with experience in semiconductor process development who will aid in Amazon’s effort to bring cloud quantum computing services to its worldwide customer base. You will join a multi-disciplinary team of scientists, and hardware and software engineers working at the forefront of quantum computing. Through your work inside and outside of the cleanroom environment in the fabrication research and development group, you will solve problems related to developing next-generation quantum processors. Candidates must have a demonstrated background in sound scientific and engineering principles, and must have excellent data analysis, bias for action, problem solving, and communication skills, and be highly motivated and curious to research and learn new technical topics as needed. As a Fab R&D scientist you will be expected to work on new ideas and stay abreast of novel approaches in fabricating and packaging superconducting quantum processors. Working effectively within a team environment is critical. Diverse Experiences Amazon values diverse experiences. Even if you do not meet all the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Work/Life Balance Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Export Control Requirement Due to applicable export control laws and regulations, candidates must be either a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum, or be able to obtain a US export license. If you are unsure if you meet these requirements, please apply and Amazon will review your application for eligibility. Key job responsibilities Responsibilities include developing and optimizing processes to fabricate high-coherence superconducting qubits; developing advanced 3DI interconnect and routing technologies for integrating superconducting quantum technologies; analyzing inline metrology and electrical test data; developing and maintaining integration documentation, design rules, and standard operating procedures; interacting with project leads to provide feedback that continuously improves different processes; staying updated with the latest advancements and industry trends in process integration and apply knowledge to improve processes and drive innovation providing technical guidance and support to junior colleagues, fostering a collaborative and knowledge-sharing work environment. A day in the life The candidate will develop novel technologies using micro-/nano-fabrication techniques inside the cleanroom (independently or in collaboration with other scientists, engineers, and technicians) for next-generation quantum computing. Outside the cleanroom, the candidate will plan experiments, analyze data, and conceive future innovations.
Applied Scientist, Air Science & Tech
US, WA, Bellevue
The Amazon Middle Mile Science team is seeking an Applied Scientist to be part of a team solving complex airline operations problems to reduce cost and improve performance. You will work closely with product, research science and technical leaders throughout Amazon Air, Amazon Delivery Technology and and will be responsible for influencing funding decisions in areas of investment that you identify as critical future product offerings. You will partner with software developers and data scientists to build end-to-end data pipelines and production code, and you will have exposure to senior leadership as we communicate results and provide scientific guidance to the business. You will analyze large amounts of business data, build the or models that will enable us to continually delight our customers worldwide. The ideal candidate will have extensive experience in Science work, business analytics and have the aptitude to incorporate new approaches and methodologies while dealing with ambiguities. Excellent business and communication skills are a must to develop and define key business questions and build models that answer those questions. You should have a demonstrated ability to think strategically and analytically about business, product, and technical challenges. Further, you must have the ability to build and communicate compelling value propositions, and work across the organization to achieve consensus. This role requires a strong passion for customers, a high level of comfort navigating ambiguity, and a keen sense of ownership and drive to deliver results. Key job responsibilities - Partnership with the engineering and operations to drive modeling and design for complex business problems. - Drive full life-cycle projects. - Design and prototype decision support tools (product) to automate standardized processes and optimize trade-offs across the full decision space. - Execute complex modeling analyses to aid management in making key business decisions and set new policies.
Member of Technical Staff, PM-T, Frontier AI & Robotics (FAR)
US, CA, San Francisco
We are seeking a Product Manager, Data Strategy & Physical AI to define and execute the long-term product vision for FAR's AI-powered robotics platform. The intersection of foundation models and physical intelligence is creating a once-in-a-generation opportunity to reimagine how intelligent systems perceive, reason, and act in the real world. We need a visionary product leader who can treat data as our primary competitive moat and translate research frontiers into scalable, production-grade capabilities. In this role, you will champion our core data strategy for foundation model creation, building a partner and tool ecosystem to systematically acquire, label, and iteratively improve physical AI datasets. You will architect a continuous data collection flywheel across deployed robot fleets, transforming real-world kinematics, video, and force-torque telemetry from edge operations back into high-fidelity training tokens. Recognizing the limitations of real-world environments, you will also lead the strategy to create high-fidelity synthesized datasets, utilizing advanced physics engines and simulation to generate diverse training tokens at massive scale. Key job responsibilities Data Acquisition & Labeling Ecosystem: Establish the partnerships, tools, and vendor pipelines necessary to acquire, curate, and continuously label multi-modal datasets for training large-scale models. Fleet Data Flywheel Infrastructure: Architect the framework for a continuous data flywheel that securely streams high-frequency kinematics, egocentric video, and force-torque telemetry from real-world robot fleets back into the training loop. Synthetic Data & Simulation Strategy: Define the strategy for generating high-fidelity, physics-aligned synthesized datasets using advanced simulation environments to scale training tokens for edge-case scenarios and long-horizon tasks. Data Compliance & Governance: Partner with operations, privacy, legal, and security teams to build enterprise-grade data management pipelines that programmatically enforce data minimization, anonymization, and CCPA/GDPR compliance. Data Quality & Token Curation: Implement automated telemetry filtering and dataset pruning strategies to identify high-value operational logs, eliminate redundant fleet data, and optimize training compute costs. Cross-Functional Physical AI Delivery: Act as the strategic bridge between machine learning research scientists, simulation developers, robotics engineers, and hardware teams to deliver data-ready platform features that improve physical reliability. About the team At Frontier AI & Robotics, we're not just advancing robotics - we're reimagining it from the ground up. Our team is building the future of intelligent robotics through frontier foundation models and end-to-end learned systems. We tackle some of the most challenging problems in AI and robotics, from developing sophisticated perception systems to creating adaptive manipulation strategies that work in complex, real-world scenarios. What sets us apart is our unique combination of ambitious research vision and practical impact. We leverage Amazon's computational infrastructure and rich real-world datasets to train and deploy state-of-the-art foundation models. Our work spans the full spectrum of robotics intelligence - from multimodal perception using images, videos, and sensor data, to sophisticated manipulation strategies that can handle diverse real-world scenarios. We're building systems that don't just work in the lab, but scale to meet the demands of Amazon's global operations. Join us if you're excited about pushing the boundaries of what's possible in robotics, working with world-class researchers, and seeing your innovations deployed at unprecedented scale.
Applied Scientist II, Amazon Search
US, WA, Seattle
Amazon Search is reinventing how customers find products through natural-language and semantic understanding. We are looking for an Applied Scientist II to push the science behind Natural Language Search that interprets complex, constraint-rich shopping queries, retrieves and ranks the most relevant products. You will build and ship large-scale relevance and ranking models that measurably reduce the rate at which customers see irrelevant results, working on problems that span query understanding, semantic matching, and contextual ranking at Amazon scale. Key job responsibilities - Design, train, and ship deep-learning ranking and semantic-matching models that improve search relevance and reduce how often customers see irrelevant results, across hard query types. - Build the training data and evaluation methods that make these models work: synthetic and historical labels, hard-negative mining, and targeted sampling at the cases where search fails. - Develop signals that match product attributes to what the customer actually asked for. - Run offline and online A/B experiments, analyze precision/recall tradeoffs, and iterate to launch. - Work with engineers and scientists across teams to take models from prototype to production at Amazon scale. A day in the life You work alongside scientists and engineers on some of the hardest open problems in search relevance, teaching models to understand what customers really mean when they ask for something specific and nuanced. A typical day blends model development and data curation with sharp experiment analysis: diagnosing where search breaks down for a query segment, designing the fix, and proving the gains through offline metrics and live A/B tests that reach real Amazon customers. The work spans the full range, from surgical fixes that resolve stubborn failure pattern to broad modeling changes that move relevance for millions of queries at once. You'll see your ideas go from whiteboard to production fast, present results regularly to wider team, and help shape the team's relevance roadmap worldwide. About the team We are the science team behind Amazon's semantic search relevance and ranking. We own the models that understand nuanced, multi-constraint shopping queries and show products customers actually want. We operate close to production, measure ourselves on real customer-impact metrics, and run a culture of fast, rigorous experimentation. Every model decision is grounded in data.
Applied Scientist, End User Messaging, AWS Applied AI Solutions Core Services
US, WA, Seattle
As part of the AWS Applied AI Solutions organization, we're advancing the frontier of trust and safety systems for cloud-based communication services. Our vision is to be the trusted foundation for transforming every business with Amazon AI teammates. Our mission is to deliver turnkey, enterprise-grade foundational AI capabilities that create delightful AI powered solutions. We're building sophisticated AI systems that protect infrastructure from evolving threats while enabling legitimate high-volume users to operate without friction, with messaging services at scale as a key application area. Key job responsibilities - Develop advanced machine learning approaches and agentic systems that autonomously adapt to evolving threat patterns across cloud communication services - Create behavioral detection models that quickly identify malicious patterns after onboarding rather than creating friction during signup - Design intelligent resource allocation algorithms that optimize service delivery based on real-time feedback - Develop frameworks operating at scale across diverse usage patterns, analyzing hundreds of thousands of daily active customers - Research novel approaches combining AI agents with trust and safety systems to solve complex security problems - Collaborate with engineering teams to integrate science components into production systems - Conduct rigorous experimentation and establish evaluation frameworks to measure solution performance A day in the life As an Applied Scientist, you'll develop fraud detection algorithms and AI-powered security systems while maintaining a clear path to customer impact. You'll investigate novel approaches to behavioral analysis, develop methods for real-time reputation assessment, and validate ideas through rigorous experimentation. You'll collaborate with other scientists and engineers to transform research insights into scalable solutions, work directly with enterprise customers to understand requirements, and help shape the future of cloud security technology. About the team Our team is a central science organization supporting multiple product teams across AWS Core Services. We tackle fundamental challenges in AI and machine learning that require novel approaches beyond off-the-shelf solutions. Working at the intersection of machine learning, large language models, and domain-specific applications, we develop practical techniques that advance the state-of-the-art while maintaining a clear path to customer impact. Our team builds deep domain expertise across geospatial intelligence, trust and safety systems, autonomous operations, and other critical areas, collaborating closely with engineering teams to transform research insights into scalable production solutions.
Applied Scientist II, Alexa International
IN, KA, Bengaluru
Alexa International is looking for passionate, talented, and inventive Senior Applied Scientists to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. Senior applied scientists will drive cross-team scientific strategy, influence partner teams, and deliver solutions that have broad impact across Alexa's international products and services. Key job responsibilities As a Applied Scientist II with the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs, particularly delivering industry-leading scientific research and applied AI for multi-lingual applications — a challenging area for the industry globally. Your work will directly impact our global customers in the form of products and services that support Alexa+. You will leverage Amazon's heterogeneous data sources and large-scale computing resources to accelerate advances in text, speech, and vision domains. The ideal candidate possesses a solid understanding of machine learning, speech and/or natural language processing, modern LLM architectures, LLM evaluation & tooling, and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environment, like to tackle complex challenges, excel at swiftly delivering impactful solutions while iterating based on user feedback, and are able to influence and align multiple teams around a shared scientific vision. A day in the life * Analyze, understand, and model customer behavior and the customer experience based on large-scale data. * Build novel online & offline evaluation metrics and methodologies for multimodal personal digital assistants. * Fine-tune/post-train LLMs using advanced and innovative techniques like SFT, DPO, Reinforcement Learning (RLHF and RLAIF) for supporting model performance specific to a customer’s location and language. * Quickly experiment and set up experimentation framework for agile model and data analysis or A/B testing. * Contribute through industry-first research to drive innovation forward. * Drive cross-team scientific strategy and influence partner teams on LLM evaluation frameworks, post-training methodologies, and best practices for international speech and language systems. * Lead end-to-end delivery of scientifically complex solutions from research to production, including reusable science components and services that resolve architecture deficiencies across teams. * Serve as a scientific thought leader, communicating solutions clearly to partners, stakeholders, and senior leadership. * Actively mentor junior scientists and contribute to the broader internal and external scientific community through publications and community engagement.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social