Social
Conversational AI

A quick guide to Amazon’s 50-plus ICASSP papers 2022

Topics range from the predictable, such as speech recognition and signal processing, to time series forecasting and personalization.

By Staff writer
May 10, 2022
7 min read
Share

Amazon researchers have more than 50 papers at this year’s International Conference on Acoustics, Speech, and Signal Processing (ICASSP). A plurality of them are on automatic speech recognition and related topics, such as keyword spotting and speaker identification. But others range farther afield, to topics such as computer vision and federated learning.

ICASSP-2022-Header.png
This year's ICASSP includes a virtual component, from May 7 to 13, and an in-person component in Singapore, May 22 to 27.

Acoustic-event detection

Federated self-supervised learning for acoustic event classification
Meng Feng, Chieh-Chi Kao, Qingming Tang, Ming Sun, Viktor Rozgic, Spyros Matsoukas, Chao Wang

Improved representation learning for acoustic event classification using tree-structured ontology
Arman Zharmagambetov, Qingming Tang, Chieh-Chi Kao, Qin Zhang, Ming Sun, Viktor Rozgic, Jasha Droppo, Chao Wang

WikiTAG: Wikipedia-based knowledge embeddings towards improved acoustic event classification
Qin Zhang, Qingming Tang, Chieh-Chi Kao, Ming Sun, Yang Liu, Chao Wang

Automatic speech recognition

A likelihood ratio-based domain adaptation method for end-to-end models
Chhavi Choudhury, Ankur Gandhe, Xiaohan Ding, Ivan Bulyko

Being greedy does not hurt: Sampling strategies for end-to-end speech recognition
Jahn Heymann, Egor Lakomkin, Leif RādellJahn Heymann, Egor Lakomkin, Leif RādelJahn Heymann, Egor Lakomkin, Leif RādelJahn Heymann, Egor Lakomkin, Leif Rādel

Caching networks: Capitalizing on common speech for ASR
Anastasios Alexandridis, Grant P. Strimel, Ariya Rastrow, Pavel Kveton, Jon Webb, Maurizio Omologo, Siegfried Kunzmann, Athanasios Mouchtaris

Lattice attention.png
In "LATTENTION: Lattice attention in ASR rescoring", Amazon researchers show that applying an attention mechanism (colored grid) to a lattice encoding multiple automatic-speech-recognition (ASR) hypotheses improves ASR performance.

Contextual adapters for personalized speech recognition in neural transducers
Kanthashree Mysore Sathyendra, Thejaswi Muniyappa, Feng-Ju Chang, Jing Liu, Jinru Su, Grant P. Strimel, Athanasios Mouchtaris, Siegfried Kunzmann

LATTENTION: Lattice attention in ASR rescoring
Prabhat Pandey, Sergio Duarte Torres, Ali Orkan Bayer, Ankur Gandhe, Volker Leutnant

Listen, know and spell: Knowledge-infused subword modeling for improving ASR performance of out-of-vocabulary (OOV) named entities
Nilaksh Das, Monica Sunkara, Dhanush Bekal, Duen Horng Chau, Sravan Bodapati, Katrin Kirchhoff

KG ASR rescoring.png
In "Listen, know and spell: Knowledge-infused subword modeling for improving ASR performance of OOV named entities", Amazon researchers show how to improve automatic speech recognition by incorporating information from knowledge graphs into the processing pipeline.

Mitigating closed-model adversarial examples with Bayesian neural modeling for enhanced end-to-end speech recognition
Chao-Han Huck Yang, Zeeshan Ahmed, Yile Gu, Joseph Szurley, Roger Ren, Linda Liu, Andreas Stolcke, Ivan Bulyko

Multi-modal pre-training for automated speech recognition
David M. Chan, Shalini Ghosh, Debmalya Chakrabarty, Björn Hoffmeister

Multiturn encoder.png
The model used in "Multi-turn RNN-T for streaming recognition of multi-party speech" to disentangle overlapping speech in multi-party automatic speech recognition.

Multi-turn RNN-T for streaming recognition of multi-party speech
Ilya Sklyar, Anna Piunova, Xianrui Zheng, Yulan Liu

RescoreBERT: Discriminative speech recognition rescoring with BERT
Liyan Xu, Yile Gu, Jari Kolehmainen, Haidar Khan, Ankur Gandhe, Ariya Rastrow, Andreas Stolcke, Ivan Bulyko

USTED: Improving ASR with a unified speech and text encoder-decoder
Bolaji Yusuf, Ankur Gandhe, Alex Sokolov

VADOI: Voice-activity-detection overlapping inference for end-to-end long-form speech recognition
Jinhan Wang, Xiaosu Tong, Jinxi Guo, Di He, Roland Maas

Computer vision

ASD-transformer: Efficient active speaker detection using self and multimodal transformers
Gourav Datta, Tyler Etchart, Vivek Yadav, Varsha Hedau, Pradeep Natarajan, Shih-Fu Chang

Dynamically pruning SegFormer for efficient semantic segmentation
Haoli Bai, Hongda Mao, Dinesh Nair

Enhancing contrastive learning with temporal cognizance for audio-visual representation generation
Chandrashekhar Lavania, Shiva Sundaram, Sundararajan Srinivasan, Katrin Kirchhoff

Few-shot gaze estimation with model offset predictors
Jiawei Ma, Xu Zhang, Yue Wu, Varsha Hedau, Shih-Fu Chang

Visual representation learning with self-supervised attention for low-label high-data regime
Prarthana Bhattacharyya, Chenge Li, Xiaonan Zhao, István Fehérvári, Jason Sun

Federated learning

Federated learning challenges and opportunities: An outlook
Jie Ding, Eric Tramel, Anit Kumar Sahu, Shuang Wu, Salman Avestimehr, Tao Zhang

FL framework.png
The federated-learning scenario considered in "Federated learning challenges and opportunities: An outlook".

Learnings from federated learning in the real world
Christophe Dupuy, Tanya G. Roosta, Leo Long, Clement Chung, Rahul Gupta, Salman Avestimehr

Information retrieval

Contrastive knowledge graph attention network for request-based recipe recommendation
Xiyao Ma, Zheng Gao, Qian Hu, Mohamed Abdelhady

Keyword spotting

Unified speculation, detection, and verification keyword spotting
Geng-shen Fu, Thibaud Senechal, Aaron Challenner, Tao Zhang

Machine translation

Isometric MT: Neural machine translation for automatic dubbing
Surafel Melaku Lakew, Yogesh Virkar, Prashant Mathur, Marcello Federico

Natural-language understanding

ADVIN: Automatically discovering novel domains and intents from user text utterances
Nikhita Vedula, Rahul Gupta, Aman Alok, Mukund Sridhar, Shankar Ananthakrishnan

An efficient DP-SGD mechanism for large scale NLU models
Christophe Dupuy, Radhika Arava, Rahul Gupta, Anna Rumshisky

Paralinguistics

Confidence estimation for speech emotion recognition based on the relationship between emotion categories and primitives
Yang Li, Constantinos Papayiannis, Viktor Rozgic, Elizabeth Shriberg, Chao Wang

Multi-lingual multi-task speech emotion recognition using wav2vec 2.0
Mayank Sharma

Representation learning through cross-modal conditional teacher-student training for speech emotion recognition
Sundararajan Srinivasan, Zhaocheng Huang, Katrin Kirchhoff

Sentiment-aware automatic speech recognition pre-training for enhanced speech emotion recognition
Ayoub Ghriss, Bo Yang, Viktor Rozgic, Elizabeth Shriberg, Chao Wang

Personalization

Incremental user embedding modeling for personalized text classification
Ruixue Lian, Che-Wei Huang, Yuqing Tang, Qilong Gu, Chengyuan Ma, Chenlei (Edward) Guo

Signal processing

Deep adaptive AEC: Hybrid of deep learning and adaptive acoustic echo cancellation
Hao Zhang, Srivatsan Kandadai, Harsha Rao, Minje Kim, Tarun Pruthi, Trausti Kristjansson

Improved singing voice separation with chromagram-based pitch-aware remixing
Siyuan Yuan, Zhepei Wang, Umut Isik, Ritwik Giri, Jean-Marc Valin, Michael M. Goodwin, Arvindh Krishnaswamy

Sparse recovery of acoustic waves
Mohamed Mansour

Upmixing via style transfer: A variational autoencoder for disentangling spatial images and musical content
Haici Yang, Sanna Wager, Spencer Russell, Mike Luo, Minje Kim, Wontak Kim

Sound source localization

End-to-end Alexa device arbitration
Jarred Barber, Yifeng Fan, Tao Zhang

Speaker diarization/identification/verification

ASR-aware end-to-end neural diarization
Aparna Khare, Eunjung Han, Yuguang Yang, Andreas Stolcke

Improving fairness in speaker verification via group-adapted fusion network
Hua Shen, Yuguang Yang, Guoli Sun, Ryan Langman, Eunjung Han, Jasha Droppo, Andreas Stolcke

OpenFEAT: Improving speaker identification by open-set few-shot embedding adaptation with Transformer
Kishan K C, Zhenning Tan, Long Chen, Minho Jin, Eunjung Han, Andreas Stolcke, Chul Lee

Self-supervised speaker recognition training using human-machine dialogues
Metehan Cekic, Ruirui Li, Zeya Chen, Yuguang Yang, Andreas Stolcke, Upamanyu Madhow

Self-supervised speaker verification with simple Siamese network and self-supervised regularization
Mufan Sang, Haoqi Li, Fang Liu, Andrew O. Arnold, Li Wan

Spoken-language understanding

A neural prosody encoder for end-to-end dialogue act classification
Kai Wei, Dillon Knox, Martin Radfar, Thanh Tran, Markus Mueller, Grant P. Strimel, Nathan Susanj, Athanasios Mouchtaris, Maurizio Omologo

Multi-task RNN-T with semantic decoder for streamable spoken language understanding
Xuandi Fu, Feng-Ju Chang, Martin Radfar, Kai Wei, Jing Liu, Grant P. Strimel, Kanthashree Mysore Sathyendra

Tie your embeddings down: Cross-modal latent spaces for end-to-end spoken language understanding
Bhuvan Agrawal, Markus Mueller, Samridhi Choudhary, Martin Radfar, Athanasios Mouchtaris, Ross McGowan, Nathan Susanj, Siegfried Kunzmann

TINYS2I: A small-footprint utterance classification model with contextual support for on-device SLU
Anastasios Alexandridis, Kanthashree Mysore Sathyendra, Grant P. Strimel, Pavel Kveton, Jon Webb, Athanasios Mouchtaris

Text-to-speech

Cross-speaker style transfer for text-to-speech using data augmentation
Manuel Sam Ribeiro, Julian Roth, Giulia Comini, Goeric Huybrechts, Adam Gabrys, Jaime Lorenzo-Trueba

Distribution augmentation for low-resource expressive text-to-speech
Mateusz Lajszczak, Animesh Prasad, Arent van Korlaar, Bajibabu Bollepalli, Antonio Bonafonte, Arnaud Joly, Marco Nicolis, Alexis Moinet, Thomas Drugman, Trevor Wood, Elena Sokolova

Duration modeling of neural TTS for automatic dubbing
Johanes Effendi, Yogesh Virkar, Roberto Barra-Chicote, Marcello Federico

Neural speech synthesis on a shoestring: Improving the efficiency of LPCNET
Jean-Marc Valin, Umut Isik, Paris Smaragdis, Arvindh Krishnaswamy

Text-free non-parallel many-to-many voice conversion using normalising flows
Thomas Merritt, Abdelhamid Ezzerg, Piotr Biliński, Magdalena Proszewska, Kamil Pokora, Roberto Barra-Chicote, Daniel Korzekwa

VoiceFilter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing module
Adam Gabrys, Goeric Huybrechts, Manuel Sam Ribeiro, Chung-Ming Chien, Julian Roth, Giulia Comini, Roberto Barra-Chicote, Bartek Perz, Jaime Lorenzo-Trueba

Time series forecasting

Robust nonparametric distribution forecast with backtest-based bootstrap and adaptive residual selection
Longshaokan Marshall Wang, Lingda Wang, Mina Georgieva, Paulo Machado, Abinaya Ulagappa, Safwan Ahmed, Yan Lu, Arjun Bakshi, Farhad Ghassemi

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Research Scientist, Intelligent Talent Acquisition, Global Hiring Science
US, VA, Arlington
Do you want a role with deep meaning and the ability to have a global impact? Hiring top talent is not only critical to Amazon’s success – it can literally change the world. It took a lot of great hires to deliver innovations like AWS, Prime, and Alexa, which make life better for millions of customers around the world. As part of the Intelligent Talent Acquisition (ITA) team, you'll have the opportunity to reinvent Amazon’s hiring process with unprecedented scale, sophistication, and accuracy. ITA is an industry-leading people science and technology organization made up of scientists, engineers, analysts, product professionals, and more. Our shared goal is to fairly and precisely connect the right people to the right jobs. Last year, we delivered over 6 million online candidate assessments, driving a merit-based hiring approach that gives candidates the opportunity to showcase their true skills. Each year we also help Amazon deliver billions of packages around the world by making it possible to hire hundreds of thousands of associates in the right quantity, at the right location, at exactly the right time. You’ll work on state-of-the-art research with advanced software tools, new AI systems, and machine learning algorithms to solve complex hiring challenges. Join ITA in using cutting-edge technologies to transform the hiring landscape and make a meaningful difference in people's lives. Together, we can solve the world's toughest hiring problems. Within ITA, the Global Hiring Science (GHS) team designs and implements innovative hiring solutions at scale. We work in a fast-paced, global environment where we use research to solve complex problems and build scalable hiring products that deliver measurable impact to our customers. We are seeking selection researchers with a strong foundation in hiring assessment development, legally-defensible validation approaches, research and experimental design, and data analysis. Preferred candidates will have experience across the full hiring assessment lifecycle, from solution design to content development and validation to impact analysis. We are looking for equal parts researcher and consultant, who is able to influence customers with insights derived from science and data. You will work closely with cross-functional teams to design new hiring solutions and experiment with measurement methods intended to precisely define exactly what job success looks like and how best to predict it. Key job responsibilities What you’ll do as a GHS Research Scientist: • Design large-scale personnel selection research that shapes Amazon’s global talent assessment practices across a variety of topics (e.g., assessment validation, measuring post-hire impact) • Partner with key stakeholders to create innovative solutions that blend scientific rigor with real-world business impact while navigating complex legal and professional standards • Apply advanced statistical techniques to analyze massive, diverse datasets to uncover insights that optimize our candidate evaluation processes and drive hiring excellence • Explore emerging technologies and innovative methodologies to enhance talent measurement while maintaining Amazon's commitment to scientific integrity • Translate complex research findings into compelling, actionable strategies that influence senior leader/business decisions and shape Amazon's talent acquisition roadmap • Write impactful documents that distill intricate scientific concepts into clear, persuasive communications for diverse audiences, from data scientists to business leaders • Ensure effective teamwork, communication, collaboration, and commitment across multiple teams with competing priorities A day in the life Imagine diving into challenges that impact millions of employees across Amazon's global operations. As a GHS Research Scientist, you'll tackle questions about hiring and organizational effectiveness on a global scale. Your day might begin with analyzing datasets to inform how we attract and select world-class talent. Throughout the day, you'll collaborate with peers in our research community, discussing different research methodologies and sharing innovative approaches to solving unique personnel challenges. This role offers a blend of focused analytical time and interacting with stakeholders across the globe.
Applied Scientist, AGI Information
US, WA, Seattle
We are looking for a researcher in state-of-the-art LLM technologies for applications across Alexa, AWS, and other Amazon businesses. In this role, you will innovate in the fastest-moving fields of current AI research, in particular in how to integrate a broad range of structured and unstructured information into AI systems (e.g. with RAG techniques), and get to immediately apply your results in highly visible Amazon products. If you are deeply familiar with LLMs, natural language processing, computer vision, and machine learning and thrive in a fast-paced environment, this may be the right opportunity for you. Our fast-paced environment requires a high degree of autonomy to deliver ambitious science innovations all the way to production. You will work with other science and engineering teams as well as business stakeholders to maximize velocity and impact of your deliverables. It's an exciting time to be a leader in AI research. In Amazon's AGI Information team, you can make your mark by improving information-driven experience of Amazon customers worldwide!
Applied Scientist III, Sponsored Products
US, NY, New York
The Sponsored Products and Brands team at Amazon Ads is re-imagining the advertising landscape through cutting-edge generative AI technologies, revolutionizing how millions of customers discover products and engage with brands across Amazon.com and beyond. We are at the forefront of re-inventing advertising experiences, bridging human creativity with artificial intelligence to transform every aspect of the advertising lifecycle from ad creation and optimization to performance analysis and customer insights. We are a passionate group of innovators dedicated to developing responsible and intelligent AI technologies that balance the needs of advertisers, enhance the shopping experience, and strengthen the marketplace. If you're energized by solving complex challenges and pushing the boundaries of what's possible with AI, join us in shaping the future of advertising. Key job responsibilities Participate in the Science hiring process as well as mentor other scientists - improving their skills, their knowledge of your solutions, and their ability to get things done. Identify and devise new video related solutions following a customer-obsessed scientific approach to address customer or business problems when the problem is ill-defined, needs to be framed, and new methodologies or paradigms need to be invented at the product level. Articulate potential scientific challenges of ongoing or future customers’ needs or business problems, and present interventions to address them. Independently assess alternative video related technologies, driving evaluation and adoption of those that fit best A day in the life As an Applied Scientist on the Sponsored Products Video team, you will work with a team of talented and experienced engineers, scientists, and designers to help bring new products to market and ensure that our customers are delighted by what we create. The Sponsored Products Video team is responsible for the design, development, and implementation of Sponsored Products Video experiences worldwide. About the team The Sponsored Products Video team within Sponsored Products and Brands creates relevant and engaging video experiences, connecting advertisers and shoppers. We are on a mission to make Amazon the best in class destination for shoppers to discover, engage and build affinity with brands, making shopping delightful, & personal.
Applied Scientist II, Amazon Business 3P
IN, TS, Hyderabad
We're seeking an Applied Scientist to lead and innovate in applying advanced AI technologies that will reshape how businesses sell on Amazon. Our team is passionate about leveraging Machine Learning, GenAI, and Agentic AI to help B2B sellers optimize their operations and drive growth. Join Amazon Business 3P (Third Party - Sellers) - a rapidly growing global organization where we innovate at the intersection of AI technology and B2B commerce. We're reimagining how sellers reach and serve business customers, creating intelligent solutions that help them grow their B2B business on Amazon. From AI-powered Seller Central tools to smart business certifications, dynamic pricing capabilities, and advanced analytics, we're transforming how B2B selling happens. As an Applied Scientist II on our AB 3P Tech team, you'll drive the development and implementation of state-of-the-art algorithms and models for supervised fine-tuning and reinforcement learning. You'll work with highly technical, entrepreneurial teams to: - Design and implement AI models that power the B2B selling experience - Lead the development of GenAI products that can handle Amazon-scale use cases - Drive research and implementation of advanced algorithms for human feedback and complex reasoning - Make strategic AI technology decisions and mentor technical talent - Own critical AI systems spanning from Seller Central to Amazon Business detail pages Join us in shaping the future of B2B selling - we're building applied AI solutions that businesses love and trust for their day-to-day success. If you are scrappy and bias for action is your favorite Leadership Principle, you'll fit right in as we innovate across the seller experience to create significant impact in this fast-growing business. Key job responsibilities Key job responsibilities: - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in Gen AI - Design and execute experiments to evaluate the performance of different algorithms and models, and iterate quickly to improve results - Think big about the arc of development of Gen AI over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences About the team At Amazon Business Third Party (AB3P) Tech, we're revolutionizing B2B e-commerce by empowering sellers in the business marketplace. Our scope spans the complete B2B selling journey, from Seller Central to Amazon Business detail pages, cart, and checkout for merchant-fulfilled offers. Our entrepreneurial culture and global reach define us. We develop features across seller experience, delivery, certifications, fees, registration, and analytics, collaborating with worldwide teams and leveraging advanced AI technologies to continuously innovate. Working in true Day 1 spirit, we build next-generation solutions that shape the future of B2B commerce. Join us in building next-generation solutions that shape the future of B2B commerce.
Data Scientist, Prime Video
GB, London
Come build the future of entertainment with us. Are you interested in shaping the future of movies and television? Prime Video is a premium streaming service that offers customers a vast collection of TV shows and movies - all with the ease of finding what they love to watch in one place. We offer customers thousands of popular movies and TV shows including Amazon Originals and exclusive licensed content to exciting live sports events. Prime Video is a fast-paced, growth business - available in over 200 countries and territories worldwide. The Video Content Research team works in a dynamic environment where innovating on behalf of our customers is at the heart of everything we do. We are seeking a Data Scientist to develop scalable models that uncover key insights into how, why and when customers engage with Prime Video marketing. Key job responsibilities In this role you will work closely with business stakeholders and technical peers (data scientists, economists and engineers) to develop causal marketing measurement models, analyze experiments and investigate customer, marketing and content related factors that drive engagement with Prime Video. You will create mechanisms and infrastructure to deploy complex models and generate insights at scale. You will have the opportunity to work with large datasets, work with AWS to build and deploy machine learning models that impact Prime Video's marketing decisions. About the team The Video Content Research team uses machine learning, econometrics, and data science to optimize Amazon's marketing and content investments. We generate insights for Amazon's digital video strategy, partnering with finance, marketing, and content teams. We analyze customer behavior on Prime Video (marketing impressions, clicks on owned channels) to identify optimization opportunities.
Research Scientist, Artificial General Intelligence - Data Services
US, MA, Boston
AI is the most transformational technology of our time, capable of tackling some of humanity’s most challenging problems. That is why Amazon is investing in generative AI (GenAI) and the responsible development and deployment of large language models (LLMs) across all of our businesses. Come build the future of human-technology interaction with us. We are looking for a Research Scientist with strong technical skills which includes coding and natural language processing experience in dataset construction, training and evaluating models, and automatic processing of large datasets. You will play a critical role in driving innovation and advancing the state-of-the-art in natural language processing and machine learning. You will work closely with cross-functional teams, including product managers, language engineers, and other scientists. Key job responsibilities Specifically, the Research Scientist will: • Ensure quality of speech/language/other data throughout all stages of acquisition and processing, including data sourcing/collection, ground truth generation, normalization, transformation, cross-lingual alignment/mapping, etc. • Clean, analyze and select speech/language/other data to achieve goals • Build and test models that elevate the customer experience • Collaborate with colleagues from science, engineering and business backgrounds • Present proposals and results in a clear manner backed by data and coupled with actionable conclusions • Work with engineers to develop efficient data querying infrastructure for both offline and online use cases
Economist, People Experience and Technology Central Science
US, VA, Arlington
The People eXperience and Technology Central Science (PXTCS) team uses economics, behavioral science, statistics, and machine learning to proactively identify mechanisms and process improvements which simultaneously improve Amazon and the lives, wellbeing, and the value of work to Amazonians. PXTCS is an interdisciplinary team that combines the talents of science and engineering to develop and deliver solutions that measurably achieve this goal. PXTCS is looking for an economist who can apply economic methods to address business problems. The ideal candidate will work with engineers and computer scientists to estimate models and algorithms on large scale data, design pilots and measure impact, and transform successful prototypes into improved policies and programs at scale. PXTCS is looking for creative thinkers who can combine a strong technical economic toolbox with a desire to learn from other disciplines, and who know how to execute and deliver on big ideas as part of an interdisciplinary technical team. Ideal candidates will work in a team setting with individuals from diverse disciplines and backgrounds. They will work with teammates to develop scientific models and conduct the data analysis, modeling, and experimentation that is necessary for estimating and validating models. They will work closely with engineering teams to develop scalable data resources to support rapid insights, and take successful models and findings into production as new products and services. They will be customer-centric and will communicate scientific approaches and findings to business leaders, listening to and incorporate their feedback, and delivering successful scientific solutions. A day in the life The Economist will work with teammates to apply economic methods to business problems. This might include identifying the appropriate research questions, writing code to implement a DID analysis or estimate a structural model, or writing and presenting a document with findings to business leaders. Our economists also collaborate with partner teams throughout the process, from understanding their challenges, to developing a research agenda that will address those challenges, to help them implement solutions. About the team PXTCS is a multidisciplinary science team that develops innovative solutions to make Amazon Earth's Best Employer
Applied Science Manager, Amazon Industrial Robotics
US, MA, N.reading
Amazon Industrial Robotics is seeking exceptional talent to help develop the next generation of advanced robotics systems that will transform automation at Amazon's scale. We're building revolutionary robotic systems that combine cutting-edge AI, sophisticated control systems, and advanced mechanical design to create adaptable automation solutions capable of working safely alongside humans in dynamic environments. This is a unique opportunity to shape the future of robotics and automation at unprecedented scale, working with world-class teams pushing the boundaries of what's possible in robotic manipulation, locomotion, and human-robot interaction. This role presents an opportunity to shape the future of robotics through innovative applications of deep learning and large language models. At Amazon Industrial Robotics we leverage advanced robotics, machine learning, and artificial intelligence to solve complex operational challenges at unprecedented scale. Our fleet of robots operates across hundreds of facilities worldwide, working in sophisticated coordination to fulfill our mission of customer excellence. We are pioneering the development of robotics foundation models that: - Enable unprecedented generalization across diverse tasks - Enable unprecedented robustness and reliability, industry-ready - Integrate multi-modal learning capabilities (visual, tactile, linguistic) - Accelerate skill acquisition through demonstration learning - Enhance robotic perception and environmental understanding - Streamline development processes through reusable capabilities The ideal candidate will contribute to research that bridges the gap between theoretical advancement and practical implementation in robotics. You will be part of a team that's revolutionizing how robots learn, adapt, and interact with their environment. Join us in building the next generation of intelligent robotics systems that will transform the future of automation and human-robot collaboration. Key job responsibilities As an Applied Science Manager in the Foundations Model team, you will: - Build and lead a team of scientists and developers responsible for foundation model development - Define the right ‘FM recipe’ to reach industry ready solutions - Define the right strategy to ensure fast and efficient development, combining state of the art methods, research and engineering. - Lead Model Development and Training: Designing and implementing the model architectures, training and fine tuning the foundation models using various datasets, and optimize the model performance through iterative experiments - Lead Data Management: Process and prepare training data, including data governance, provenance tracking, data quality checks and creating reusable data pipelines. - Lead Experimentation and Validation: Design and execute experiments to test model capabilities on the simulator and on the embodiment, validate performance across different scenarios, create a baseline and iteratively improve model performance. - Lead Code Development: Write clean, maintainable, well commented and documented code, contribute to training infrastructure, create tools for model evaluation and testing, and implement necessary APIs - Research: Stay current with latest developments in foundation models and robotics, assist in literature reviews and research documentation, prepare technical reports and presentations, and contribute to research discussions and brainstorming sessions. - Collaboration: Work closely with senior scientists, engineers, and leaders across multiple teams, participate in knowledge sharing, support integration efforts with robotics hardware teams, and help document best practices and methodologies.
Sr. Applied Scientist, FAR (Frontier AI & Robotics)
CA, QC, Montreal
Join the next revolution in robotics at Amazon's Frontier AI & Robotics team, where you'll work alongside world-renowned AI pioneers to push the boundaries of what's possible in robotic intelligence. As an Applied Scientist, you'll be at the forefront of developing breakthrough foundation models that enable robots to perceive, understand, and interact with the world in unprecedented ways. You'll drive independent research initiatives in areas such as perception, manipulation, scene understanding, sim2real transfer, multi-modal foundation models, and multi-task learning, designing novel algorithms that bridge the gap between state-of-the-art research and real-world deployment at Amazon scale. In this role, you'll balance innovative technical exploration with practical implementation, collaborating with platform teams to ensure your models and algorithms perform robustly in dynamic real-world environments. You'll have access to Amazon's vast computational resources, enabling you to tackle ambitious problems in areas like very large multi-modal robotic foundation models and efficient, promptable model architectures that can scale across diverse robotic applications. Key job responsibilities - Design and implement novel deep learning architectures that push the boundaries of what robots can understand and accomplish - Drive independent research initiatives in robotics foundation models, focusing on breakthrough approaches in perception, and manipulation, for example open-vocabulary panoptic scene understanding, scaling up multi-modal LLMs, sim2real/real2sim techniques, end-to-end vision-language-action models, efficient model inference, video tokenization - Lead technical projects from conceptualization through deployment, ensuring robust performance in production environments - Collaborate with platform teams to optimize and scale models for real-world applications - Contribute to the team's technical strategy and help shape our approach to next-generation robotics challenges A day in the life - Design and implement novel foundation model architectures, leveraging our extensive compute infrastructure to train and evaluate at scale - Collaborate with our world-class research team to solve complex technical challenges - Lead technical initiatives from conception to deployment, working closely with robotics engineers to integrate your solutions into production systems - Participate in technical discussions and brainstorming sessions with team leaders and fellow scientists - Leverage our massive compute cluster and extensive robotics infrastructure to rapidly prototype and validate new ideas - Transform theoretical insights into practical solutions that can handle the complexities of real-world robotics applications About the team At Frontier AI & Robotics, we're not just advancing robotics – we're reimagining it from the ground up. Our team is building the future of intelligent robotics through ground breaking foundation models and end-to-end learned systems. We tackle some of the most challenging problems in AI and robotics, from developing sophisticated perception systems to creating adaptive manipulation strategies that work in complex, real-world scenarios. What sets us apart is our unique combination of ambitious research vision and practical impact. We leverage Amazon's massive computational infrastructure and rich real-world datasets to train and deploy state-of-the-art foundation models. Our work spans the full spectrum of robotics intelligence – from multimodal perception using images, videos, and sensor data, to sophisticated manipulation strategies that can handle diverse real-world scenarios. We're building systems that don't just work in the lab, but scale to meet the demands of Amazon's global operations. Join us if you're excited about pushing the boundaries of what's possible in robotics, working with world-class researchers, and seeing your innovations deployed at unprecedented scale.
Principal Applied Scientist, AWS AI Fundamental Research
US, WA, Bellevue
Amazon is looking for a Principal Applied Scientist world class scientists to join its AWS Fundamental Research Team working within a variety of machine learning disciplines. This group is entrusted with developing core machine learning solutions for AWS services. At the AWS Fundamental Research Team you will invent, implement, and deploy state of the art machine learning algorithms and systems. You will build prototypes and explore conceptually large scale ML solutions across different domains and computation platforms. You will interact closely with our customers and with the academic community. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. About the team About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our monthly newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2025 Amazon.com, Inc. or its affiliates

Social