Social
Machine learning

A quick guide to Amazon’s papers at NeurIPS 2022

Topics range from specific applications, such as computer vision, to more general problems, such as continual learning, to popular AI methods, such as variational autoencoders.

By Staff writer
November 21, 2022
3 min read
Share

The Conference on Neural Information Processing Systems (NeurIPS) remains the highest-profile conference in AI, and as such, it draws paper submissions from across Amazon’s business lines. Some of those papers concern specific application areas, like computer vision and recommender systems, but many of them address more general problems, such as continual learning, federated learning, and privacy. And some of them investigate ways to improve popular machine learning methods, such as contrastive learning or variational autoencoders.

Below is a quick guide to the main-conference papers from Amazon researchers at this year’s NeurIPS.

Algorithmic fairness

Are two heads the same as one? Identifying disparate treatment in fair neural networks
Michael Lohaus, Matthaus Kleindessner, Krishnaram Kenthapadi, Francesco Locatello, Chris Russell

Computer vision

An in-depth study of stochastic backpropagation
Jun Fang, Mingze Xu, Hao Chen, Bing Shuai, Zhuowen Tu, Joseph Tighe

Self supervised amodal video object segmentation
Jian Yao, Yuxin Hong, Chiyu Wang, Tianjun Xiao, Tong He, Francesco Locatello, David Wipf, Yanwei Fu, Zheng Zhang

Self-supervised pretraining for large-scale point clouds
Zaiwei Zhang, Min Bai, Erran Li

Point clouds.png
The method described in "Self-supervised pretraining for large-scale point clouds" splits a large-scale 3-D point cloud into M occupied volumes, then subjects it to random rotations and scaling to produce two augmented views. The augmented views are then sampled to produce global and local crops.

Semi-supervised vision transformers at scale
Zhaowei Cai, Avinash Ravichandran, Paolo Favaro, Manchen Wang, Davide Modolo, Rahul Bhotika, Zhuowen Tu, Stefano Soatto

Continual learning

Measuring and reducing model update regression in structured prediction for NLP
Deng Cai, Elman Mansimov, Yi-An Lai, Yixuan Su, Lei Shu, Yi Zhang

Memory efficient continual learning with transformers
Beyza Ermis, Giovanni Zappella, Martin Wistuba, Cédric Archambeau

Distribution shifts

Assaying out-of-distribution generalization in transfer learning
Florian Wenzel, Andrea Dittadi, Peter Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, Bernhard Schölkopf, Francesco Locatello

Neural attentive circuits
Martin Weiss, Nasim Rahaman, Francesco Locatello, Chris Pal, Yoshua Bengio, Nicolas Ballas, Erran Li

Earth system forecasting

Earthformer: Exploring space-time transformers for earth system forecasting
Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang (Bernie) Wang, Mu Li, Dit-Yan Yeung

Earth system forecasting.png
A sequence of images of a weather event captured at 10-minute intervals. Color (on a green-to-violet spectrum) indicates precipitation intensity. From "Earthformer: Exploring space-time transformers for earth system forecasting".

Federated learning

Self-aware personalized federated learning
Huili Chen, Jie Ding, Eric Tramel, Shuang Wu, Anit Kumar Sahu, Salman Avestimehr, Tao Zhang

Machine learning methods

Embrace the gap: VAEs perform independent mechanism analysis
Patrik Reizinger, Luigi Gresele, Jack Brady, Julius von Kuegelgen, Dominik Zietlow, Bernhard Schölkopf, Georg Martius, Wieland Brendel, Michel Besserve

Learning manifold dimensions with conditional variational autoencoders
Yijia Zheng, Tong He, Yixuan Qiu, David Wipf

On the detrimental effect of invariances in the likelihood for variational inference
Richard Kurle, Ralf Herbrich, Tim Januschowski, Yuyang (Bernie) Wang, Jan Gasthaus

Variational inference.png
In Bayesian neural networks, weights and biases are treated as random variables whose posterior distribution is induced by a dataset. The most common way to approximate the posterior is mean-field approximation, which is a product of independent normal distributions. In "On the detrimental effect of invariances in the likelihood for variational inference", the authors prove that, under the right conditions, the mean-field approximation induces the same posterior predictive distribution as an invariance-abiding approximation that explicitly models invariances.

Why do we need large batch sizes in contrastive learning? A gradient-bias perspective
Changyou Chen, Jianyi Zhang, Yi Xu, Liqun Chen, Jiali Duan, Yiran Chen, Son Tran, Belinda Zeng, Trishul Chilimbi

Privacy

Private synthetic data for multitask learning and marginal queries
Giuseppe Vietri, Cédric Archambeau, Sergul Aydore, William Brown, Michael Kearns, Aaron Roth, Ankit Siva, Shuai Tang, Steven Wu

Recommender systems

Toward understanding privileged features distillation in learning-to-rank
Shuo Yang, Sujay Sanghavi, Holakou Rahmanian, Jan Bakus, S. V. N. Vishwanathan

Uplifting bandits
Yu-Guan Hsieh, Shiva Kasiviswanathan, Branislav Kveton

Reinforcement learning

Adaptive interest for emphatic reinforcement learning
Martin Klissarov, Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Taesup Kim, Alex Smola

Faster deep reinforcement learning with slower online network
Kavosh Asadi, Rasool Fakoor, Omer Gottesman, Taesup Kim, Michael L. Littman, Alex Smola

Tabular data

Learning enhanced representations for tabular data via neighborhood propagation
Kounianhua Du, Weinan Zhang, Ruiwen Zhou, Yangkun Wang, Xilong Zhao, Jiarui Jin, Quan Gan, Zheng Zhang, David Paul Wipf

Tabular data.png
Unlike existing approaches to making predictions over tabular data, such as tree models (left) and retrieval methods (center), the method proposed in "Learning enhanced representations for tabular data via neighborhood propagation" models multiple data instances as a hypergraph and captures their correlations with the assistance of labels (right).

Research areas
Tags
About the Author
Staff writer

Related content

Work with us

See more jobs See more jobs
Applied Scientist, Navigation
US, CA, Sunnyvale
Amazon is on a mission to redefine the future of automation — and we're looking for exceptional talent to help lead the way. We are building the next generation of advanced robotic systems that seamlessly blend cutting-edge AI, sophisticated control systems, and novel mechanical design to create adaptable, intelligent automation solutions capable of operating safely alongside humans in dynamic, real-world environments. At Amazon, we leverage the power of machine learning, artificial intelligence, and advanced robotics to solve some of the most complex operational challenges at a scale unlike anywhere else in the world. Our fleet of robots spans hundreds of facilities globally, working in sophisticated coordination to deliver on our promise of customer excellence — and we're just getting started. As a Sr. Scientist in Robot Navigation, you will be at the forefront of this transformation — architecting and delivering navigation systems that are intelligent, safe, and scalable. You will bring deep expertise in learning-based planning and control, a strong understanding of foundation models and their application to embodied agents, and as well as have in-depth understanding of control-theoretic approaches such as model predictive control (MPC)-based trajectory planning. You will develop navigation solutions that seamlessly blend data-driven intelligence with principled control-theoretic guarantees. Our vision is bold: to build navigation systems that allow robots to move fluidly and safely through dynamic environments — understanding context, anticipating change, and adapting in real time. You will lead research that bridges the gap between cutting-edge academic advances and production grade deployment, collaborating with world-class teams pushing the boundaries of robotic autonomy, manipulation, and human-robot interaction. Join us in building the next generation of intelligent navigation systems that will define the future of autonomous robotics at scale. Key job responsibilities - Design, develop, and deploy perception algorithms for robotics systems, including object detection, segmentation, tracking, depth estimation, and scene understanding - Lead research initiatives in computer vision, sensor fusion and 3D perception - Collaborate with cross-functional teams including robotics engineers, software engineers, and product managers to define and deliver perception capabilities - Drive end-to-end ownership of ML models — from data collection and labeling strategy to training, evaluation, and deployment - Mentor junior scientists and engineers; contribute to a culture of technical excellence - Define and track key metrics to measure perception system performance in real-world environments - Publish research findings in top-tier venues (CVPR, ICCV, ECCV, ICRA, NeurIPS, etc.) and contribute to patents A day in the life - Train ML models for deployment in simulation and real-world robots, identify and document their limitations post-deployment - Drive technical discussions within your team and with key stakeholders to develop innovative solutions to address identified limitations - Actively contribute to brainstorming sessions on adjacent topics, bringing fresh perspectives that help peers grow and succeed — and in doing so, build lasting trust across the team - Mentor team members while maintaining significant hands-on contribution to technical solutions About the team Our team is a group is a diverse group of scientists and engineers passionate about building intelligent machines. We value curiosity, rigor, and a bias for action. We believe in learning from failure and iterating quickly toward solutions that matter.
Applied Scientist II, Translation Services
IN, TS, Hyderabad
Have you ever wondered how Amazon launches and maintains a consistent customer experience across hundreds of countries and languages it serves its customers? Are you passionate about data and mathematics, and hope to impact the experience of millions of customers? Are you obsessed with designing simple algorithmic solutions to very challenging problems? If so, we look forward to hearing from you! At Amazon, we strive to be Earth's most customer-centric company, where both internal and external customers can find and discover anything they want in their own language of preference. Our Translations Services (TS) team plays a pivotal role in expanding the reach of our marketplace worldwide and enables thousands of developers and other stakeholders (Product Managers, Program Managers, Linguists) in developing locale specific solutions. Amazon Translations Services (TS) is seeking an Applied Scientist to be based in our Hyderabad office. As a key member of the Science and Engineering team of TS, this person will be responsible for designing algorithmic solutions based on data and mathematics for translating billions of words annually across 130+ and expanding set of locales. The successful applicant will ensure that there is minimal human touch involved in any language translation and accurate translated text is available to our worldwide customers in a streamlined and optimized manner. With access to vast amounts of data, technology, and a diverse community of talented individuals, you will have the opportunity to make a meaningful impact on the way customers and stakeholders engage with Amazon and our platform worldwide. Together, we will drive innovation, solve complex problems, and shape the future of e-commerce. Key job responsibilities * Apply your expertise in LLM models to design, develop, and implement scalable machine learning solutions that address complex language translation-related challenges in the eCommerce space. * Collaborate with cross-functional teams, including software engineers, data scientists, and product managers, to define project requirements, establish success metrics, and deliver high-quality solutions. * Conduct thorough data analysis to gain insights, identify patterns, and drive actionable recommendations that enhance seller performance and customer experiences across various international marketplaces. * Continuously explore and evaluate state-of-the-art modeling techniques and methodologies to improve the accuracy and efficiency of language translation-related systems. * Communicate complex technical concepts effectively to both technical and non-technical stakeholders, providing clear explanations and guidance on proposed solutions and their potential impact. About the team We are a start-up mindset team. As the long-term technical strategy is still taking shape, there is a lot of opportunity for this fresh Science team to innovate by leveraging Gen AI technoligies to build scalable solutions from scratch. Our Vision: Language will not stand in the way of anyone on earth using Amazon products and services. Our Mission: We are the enablers and guardians of translation for Amazon's customers. We do this by offering hands-off-the-wheel service to all Amazon teams, optimizing translation quality and speed at the lowest cost possible.
Applied Scientist (GenAI/LLM), Sandstone
US, CA, San Diego
Do you want to join an innovative team of scientists and engineers who use terabytes of data and create state-of-the-art Generative AI algorithms to push the boundaries of AI creativity? We are building foundational behavioral models for Amazon Stores using Generative AI, LLMs and Large Model training techniques that fuses general world knowledge, customer shopping behavior and Amazon e-commerce domain knowledge. We are looking for scientists who are passionate about technology, innovation, and customer experience, and are ready to make a lasting impact on the industry using intelligent and transformative AI applications. Working closely with cross-functional teams, you will be an essential part of every stage of AI development, from ideation and design to rigorous testing and successful deployment, ensuring our AI projects drive innovation and provide value for our customers. If you’re fired up about being part of a dynamic, driven team, then this is your moment to join us on this exciting journey! Key job responsibilities In this role you will leverage your background and expertise to lead developing foundational behavioral model for Amazon Stores using Generative AI, LLM and Large Model training techniques. On a day-to-day basis, you will: - Research and implement new algorithms and architectures for generative AI applications. - Optimize model performance and scalability for inference and deployment. - Collaborate with other talented applied scientists and engineers to gather and preprocess large datasets and develop an improved training infrastructure that accelerates innovation. - Experiment with SOTA methods to improve generative AI model quality. - Provide technical expertise and guidance to support the integration of generative AI solutions into various products and services.
Applied Scientist II, CMT
IN, KA, Bengaluru
As a member of the CMT team, you'll play a key role in the evolution of our Competitive Monitoring systems to solve significantly complex and interesting technical challenges in machine learning, large language models in production, and recommender systems to name a few. The team's work directly impacts customer experience at a worldwide scale. Key job responsibilities Thought leader on the team and help set team directions Research multiple problem domains, suggest various approaches to try and be as hands-on as needed while providing more junior scientists with critical mentorship Collaborate with engineers to come up with the right LLD and HLD to solve key business problems Strong emphasis on communication via writing, internal and external talks, and being able to align with multiple stakeholders A day in the life As an Applied scientist II, a typical day will involve aligning with key product, engineering and business stakeholders ; advising junior scientists on the work they are doing ; reading current research papers and staying up-to-date on AI research ; diving deep as needed to improve CMT models and addressing stakeholders from the science perspective ; writing python code
Applied Scientist, CMT
IN, KA, Bengaluru
As a member of the CMT team, you'll play a key role in the evolution of our Competitive Monitoring systems to solve significantly complex and interesting technical challenges in machine learning, large language models in production, and recommender systems to name a few. The team's work directly impacts customer experience at a worldwide scale. Key job responsibilities 1. Research the problem domain and come up with various approaches to solve the problem. 2. Be willing to experiment quickly and fail fast. 3. Collaborate with engineers to come up with the right end to end solution to the business problems. 4. Ideate on future roadmap for science in CMT 5. Be willing to roll up your sleeves and learn core topics outside applied science, for example ML engineering A day in the life A typical day might involve (a) working on ideas for improving models around product similarity or price recommendations, (b) working closely with other scientists and our ML engineers to ensure that the best models are in production, (c) writing good maintainable code that can be reused and reproduced, (d) sharing your work across CMT and beyond via technical writings and presentations
Applied Scientist II, Console Science
US, CA, Santa Clara
We are looking for passionate, talented, and inventive Applied Scientists with a strong machine learning background to help build industry-leading Conversational AI Systems. Our mission is to provide a delightful experience to Amazon’s customers by pushing the envelope in Natural Language Understanding (NLU), Dialog Systems including Generative AI with Large Language Models (LLMs) and Applied Machine Learning (ML). You will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services that make use language technology. You will gain hands on experience with Amazon’s heterogeneous text, structured data sources, and large-scale computing resources to accelerate advances in language understanding. We are hiring in all areas of human language technology: NLU, Dialog Management, Conversational AI, LLMs and Generative AI. A day in the life The team uses generative AI and foundation models to reimagine the experience of all customers on AWS. We explore new technologies and find creative solutions. Curiosity and an explorative mindset can find a place here to impact the life of engineers around the world. If you are excited about this space and want to enlighten your peers with new capabilities, this is the team for you. We are open to hiring candidates to work out of one of the following locations: Santa Clara, CA, USA About the team AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (Iot), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices.
Applied Scientist, Neuron ARG, Annapurna ML
US, WA, Seattle
The Automated Reasoning Group in the Amazon Neuron team is looking for an Applied Scientist to work on the intersection of Artificial Intelligence and program analysis to raise the code quality bar in our state-of-the-art deep learning compiler stack. This stack is designed to optimize application models across diverse domains, including Large Language and Vision, originating from leading frameworks such as PyTorch and JAX. Your role will involve working closely with our custom-built Machine Learning accelerator, Trainium, which represents the forefront of innovation for advanced ML capabilities, and is the underpinning of Generative AI. In this role as an Applied Scientist, you'll be instrumental in designing, developing, and deploying analyzers for ML compiler stages and compiler IRs. You will architect and implement business-critical tooling, publish research, and mentor a brilliant team of experienced scientists and engineers. You will need to be technically capable, credible, and curious in your own right as a trusted AWS Neuron engineer, innovating on behalf of our customers. Your responsibilities will involve tackling crucial challenges alongside a talented engineering team, contributing to leading-edge design and research in compiler technology and deep-learning systems software. Strong experience in programming languages, compilers, program analyzers, theorem provers, and program synthesis engines will be a benefit in this role. A background in machine learning and AI accelerators is preferred but not required.
Principal Applied Scientist, CX Science
US, CA, Santa Clara
We are looking for passionate, talented, and inventive Principal Applied Scientist with a strong machine learning background to help build industry-leading Conversational AI Systems. Our mission is to provide a delightful experience to Amazon’s customers by pushing the envelope in Natural Language Understanding (NLU), Dialog Systems including Generative AI with Large Language Models (LLMs) and Applied Machine Learning (ML). As part of our team, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services that make use language technology. You will gain hands on experience with Amazon’s heterogeneous text, structured data sources, and large-scale computing resources to accelerate advances in language understanding. We are hiring in all areas of human language technology: NLU, Dialog Management, Conversational AI, LLMs and Generative AI. A day in the life The team uses generative AI and foundation models to reimagine the experience of all customers on AWS. We explore new technologies and find creative solutions. Curiosity and an explorative mindset can find a place here to impact the life of engineers around the world. If you are excited about this space and want to enlighten your peers with new capabilities, this is the team for you.
Principal Applied Scientist, Sponsored Products and Brands - Demand Utilization
US, CA, Palo Alto
The Demand Utilization team within Amazon Advertising is responsible for determining which ads to serve when hundreds of millions of shoppers search for products on Amazon. We sit at the intersection of customer intent understanding and advertiser value, solving one of the most complex matching problems in the industry, identifying the right ad, for the right shopper, at the right moment, across one of the world's largest product catalogs. Our systems deliver billions of ad impressions and millions of clicks daily under strict relevance and latency constraints. We are looking for a Principal Applied Scientist to set the technical vision and drive the science strategy for our ad retrieval and ranking systems. This is a high-impact leadership role where you will tackle unsolved problems at the frontier of large-scale information retrieval, natural language understanding, and multi-objective optimization, all operating in real time at Amazon scale. You will work on challenges such as: - Modeling shopper intent from sparse, ambiguous, and multi-modal signals - Designing retrieval architectures that balance relevance, advertiser and shopper experience across billions of candidate ads - Advancing personalization and cold-start strategies for new advertisers and emerging product categories This is a role for a scientist who wants to shape the future of performance advertising through rigorous research applied to real-world systems that directly impact Amazon's customers, sellers, and business. Key job responsibilities Key Responsibilities: - Own the science roadmap for ad retrieval and ranking within Demand Utilization, defining multi-year research priorities aligned with business goals - Lead the design and development of novel machine learning models and algorithms for relevance, intent understanding, and ad selection at scale - Drive end-to-end execution from problem formulation and experimentation through production deployment, measuring impact on shopper and advertiser outcomes - Mentor and elevate a team of applied scientists and research engineers, raising the technical bar and fostering a culture of scientific rigor - Collaborate cross-functionally with product, engineering, and business leaders to translate science capabilities into product strategy - Represent Amazon externally through publications at top-tier venues, patents, and participation in the broader ML/IR research community
Principal Applied Scientist, Prime Video Personalization & Discovery , PV Personalization & Discovery, Principal Applied Scientist, PVPD
US, WA, Seattle
As a Principal Applied Scientist at Prime Video, you will be a technical and strategic leader responsible for inventing, developing, and deploying groundbreaking AI solutions that power personalized, relevant, and delightful experiences for millions of global customers. You will help shape the vision and direction of key ML systems that support Prime Video’s mission to deliver AI-powered customer experiences. This role demands a unique blend of deep technical expertise in machine learning and recommendation systems, industry leadership, and strong collaboration skills. You will guide the development of high-impact systems end-to-end - leading innovation from foundational research through production deployment - while mentoring scientists and influencing product and engineering roadmaps. We are looking for a thought leader who brings a strong track record of delivering ML innovations at scale, along with the curiosity and drive to push boundaries. This is a rare opportunity to drive meaningful impact at one of the largest streaming services in the world. Key job responsibilities - Invent, prototype, and productionize large-scale AI solutions across Prime Video’s personalization and discovery ecosystem using deep learning, generative AI, reinforcement learning, and optimization techniques; - Provide technical leadership and influence product vision by collaborating closely with engineers, product managers, and senior stakeholders; - Design and lead high-impact A/B tests and data analyses to validate hypotheses and guide product direction; - Drive technical bar-raising across science and engineering teams through mentorship, design reviews, and collaboration; - Stay ahead of industry trends and emerging research; leverage them to evolve long-term strategy and architecture; - Publish impactful research internally and externally (e.g. top-tier conferences and journals).
See more jobs
View from space of a connected network around planet Earth representing the Internet of Things.
Get more from Amazon Science
Subscribe to our newsletter

Amazon.com | Conditions of Use | Privacy | © 1996-2026 Amazon.com, Inc. or its affiliates

Social