ACL: Computational linguistics in the age of large language models

Amazon’s Yang Liu, general chair of this year’s meeting of the Association for Computational Linguistics, on the road ahead for LLMs.

As they are everywhere, large language models are a major topic of conversation at this year’s meeting of the Association for Computational Linguistics (ACL).

Yang-portrait.png
Yang Liu, a senior principal scientist with Alexa AI and general chair of this year's meeting of the Association for Computational Linguistics.

“We have multiple sessions on large language models, which was not a session at past conferences,” says Yang Liu, a senior principal scientist with Alexa AI and general chair of ACL. “And both keynote talks are related to this topic.”

According to the ACL website, one of the keynote speakers, Geoffrey Hinton, who won a share of the 2018 Turing Award for seminal contributions to deep learning, will address “the contentious issue of whether current multimodal LLMs have subjective experience.” The other keynote speaker, Alison Gopnik, a professor of psychology and philosophy at the University of California, Berkeley, has titled her talk “Large Language Models as Cultural Technologies”.

“We also have a panel on large language models, and there's another session on ethics and NLP [natural-language processing] as these models are becoming more and more powerful,” Liu adds. “These are the issues the entire community is paying attention to. And not just our community: the whole world is looking at the development of these technologies and their relevance to society.”

Hallucination

One of the major problems with large language models (LLMs), of course, is their tendency to “hallucinate”, or generate assertions that sound plausible but are in fact false. Currently, Liu says, NLP researchers are attempting to address this problem in several ways. One is through a post-processing step that attempts to verify the LLMs’ outputs.

Related content
Generative AI raises new challenges in defining, measuring, and mitigating concerns about fairness, toxicity, and intellectual property, among other things. But work has started on the solutions.

“Once you have the system response, you can do knowledge retrieval,” Liu explains. “‘Can you find a source for this?’ When a model says that Joe Biden is the current president, you can easily search and find some credible source for that. And then between that source and your current system response, there are these fact-checking models, similar to entailment models, and you can check the consistency.”

Currently, however, “the error rate is pretty high,” Liu says. “Even if I give you two texts, A and B, and I ask you, ‘Do they mean the same thing?’, it's not a solved problem in NLP.”

Another approach, Liu says, is to more carefully curate the data used to train LLMs. “They are trained with trillions of tokens,” she says — where a “token” is a word, multiword term treated as a unit, or subword unit. “If you want to control the information provided to these models, the first step is to make sure the data is high-quality data.”

Researchers are also trying to modify the inner workings of trained LLMs in order to steer their outputs toward factually accurate assertions. An LLM works by computing the probability of the next token in a sequence of tokens; the LLM’s attention heads — perhaps dozens of them per network layer — determine how heavily the model should weight each past token when computing the probability of the next token.

Related content
The Amazon-sponsored FEVEROUS dataset and shared task challenge researchers to create more advanced fact-checking systems.

“One line of work that aims to improve factual accuracy is by activation editing, which changes such probability distributions,” Liu explains. “These methods do not change the trained models but use different strategies to change inference or prediction results. For example, a recent paper on this topic first identifies a sparse set of attention heads that are highly correlated with truthfulness. Then they perform ‘inference intervention’: they shift activations along these truth-correlated directions. There are also various methods that change model parameters to reduce hallucination.”

“Explicit knowledge grounding can also be used to address hallucination,” she adds. “In these approaches, a knowledge retrieval component is first applied. Then the LLM grounds its response in the relevant documents.”

Training by proxy

One difficulty in preventing hallucination has to do with the way LLMs are trained, Liu explains. LLM training uses input masking, in which words in input sentences are randomly removed, and the LLM has to supply them. The masking is done automatically, and the output error is straightforward to compute. But explicitly training the models for factual accuracy would complicate the picture.

Related content
With an encoder-decoder architecture — rather than decoder only — the Alexa Teacher Model excels other large language models on few-shot tasks such as summarization and machine translation.

“What people have found is that predicting tokens is a good proxy for many downstream use cases,” Liu says. “That builds the basic foundation model, and then on top of it, you can try to improve it to make it follow instructions and perform various tasks. But changing that foundation model, adding some additional training loss objective, that's hard, and computationally expensive.”

“I think it makes sense to continuously improve these models after pretraining — for example, via the reward model with human feedback in the loop,” Liu adds. Reinforcement learning with human feedback is a popular method for improving the performance of LLMs, in which, during training, the model seeks human feedback to distinguish between choices to which it assigns low probabilities.

“If factual error is something you care about, you can have models optimize toward those dimensions,” Liu says. “I think the model performance along these dimensions is improving; it's just that the acceptance criterion is very high. Say, 95% — that seems very accurate from the classification point of view. But in search, if you have one single error and then people say, ‘Oh no, you're giving incorrect answers!’, it’s a problem.”

Related content
Two papers from Amazon Web Services AI present algorithms that alleviate the intensive hyperparameter search and fine-tuning required by privacy-preserving deep learning at very large scales.

One possibility, Liu says, is that as researchers find ways to incrementally improve the factual accuracy of LLMs, the public gets better educated about how to use them.

“Maybe users will change their attitudes, and companies will also change,” she says. “People play with LLMs, they see some errors, and people do their own fact checking. You treat them just like any online news source. This is related to our panel on ethics: the entire society is looking at this new tool. How should we be treating these things? Do we take it as ground truth, or is it a tool that provides you something, and you double-check it? I think people are trying to understand these things and then live with them in harmony.”

Research areas

Related content

US, WA, Seattle
Alexa is the Amazon cloud service that powers Echo, the groundbreaking Amazon device designed around your voice. We believe voice is the most natural user interface for interacting with technology across many domains; we are inventing the future. Alexa Audio is responsible for fulfilling customers requests for all types of audio content (Music, Radio, Podcasts, Books, custom sounds) across all Alexa enabled devices. This covers a broad set of experiences including search, browse, recommendations, playback, and devices grouping and controls. We are seeking a talented, self-directed Applied Scientists who would come up with state of the art semantic search and recommendation techniques that work with both voice and visual interfaces. This is a unique opportunity where you will be working on latest technologies including LLMs, and also see it impact customer's lives in meaningful ways. Responsibilities - Apply advance state-of-the-art artificial intelligence techniques and develop algorithms in areas of personalization, voice based dialogue systems and natural language information retrieval. - Design scientifically sound online experiments and offline simulations to study and improve products. - Work closely with talented engineers to create scalable models and put them to production. - Perform statistical analyses on large data sets, identify problems, and propose solutions. - Work with partner science teams to identify collaboration opportunities. Work hard. Have fun. Make history. We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA | Seattle, WA, USA | Sunnyvale, CA, USA
ES, M, Madrid
Amazon's International Technology org in EU (EU INTech) is creating new ways for Amazon customers discovering Amazon catalog through new and innovative Customer experiences. Our vision is to provide the most relevant content and CX for their shopping mission. We are responsible for building the software and machine learning models to surface high quality and relevant content to the Amazon customers worldwide across the site. The team, mainly located in Madrid Technical Hub, London and Luxembourg, comprises Software Developer and ML Engineers, Applied Scientists, Product Managers, Technical Product Managers and UX Designers who are experts on several areas of ranking, computer vision, recommendations systems, Search as well as CX. Are you interested on how the experiences that fuel Catalog and Search are built to scale to customers WW? Are interesting on how we use state of the art AI to generate and provide the most relevant content? Key job responsibilities We are looking for Applied Scientists who are passionate to solve highly ambiguous and challenging problems at global scale. You will be responsible for major science challenges for our team, including working with text to image and image to text state of the art models to scale to enable new Customer Experiences WW. You will design, develop, deliver and support a variety of models in collaboration with a variety of roles and partner teams around the world. You will influence scientific direction and best practices and maintain quality on team deliverables. We are open to hiring candidates to work out of one of the following locations: Madrid, M, ESP
GB, Cambridge
The Amazon Artificial General Intelligence (AGI) team is looking for a passionate, highly skilled and inventive Senior Applied Scientist with strong machine learning background to lead the development and implementation of state-of-the-art ML systems for building large-scale, high-quality conversational assistant systems. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized assistant systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for contextual knowledge extraction and information representation, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, cpu, latency and quality - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in personal knowledge aggregation, processing and verification - Design and execute experiments to evaluate the performance of different algorithms and models, and iterate quickly to improve results - Think Big about the arc of development of conversational assistant system personalization over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports - Mentor and guide junior scientists and engineers, and contribute to the overall growth and development of the team A day in the life As a Senior Applied Scientist, you will play a critical role in driving the development of personalization techniques enabling conversational systems, in particular those based on large language models, to be tailored to customer needs. You will handle Amazon-scale use cases with significant impact on our customers' experiences. We are open to hiring candidates to work out of one of the following locations: Cambridge, GBR | London, GBR
DE, Berlin
The Amazon Artificial General Intelligence (AGI) team is looking for a passionate, highly skilled and inventive Senior Applied Scientist with strong machine learning background to lead the development and implementation of state-of-the-art ML systems for building large-scale, high-quality conversational assistant systems. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized assistant systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for contextual knowledge extraction and information representation, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, cpu, latency and quality - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in personal knowledge aggregation, processing and verification - Design and execute experiments to evaluate the performance of different algorithms and models, and iterate quickly to improve results - Think Big about the arc of development of conversational assistant system personalization over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports - Mentor and guide junior scientists and engineers, and contribute to the overall growth and development of the team A day in the life As a Senior Applied Scientist, you will play a critical role in driving the development of personalization techniques enabling conversational systems, in particular those based on large language models, to be tailored to customer needs. You will handle Amazon-scale use cases with significant impact on our customers' experiences. We are open to hiring candidates to work out of one of the following locations: Berlin, DEU
US, WA, Bellevue
Conversational AI ModEling and Learning (CAMEL) team is part of Amazon Artificial General Intelligence (AGI) organization where our mission is to create a best-in-class Conversational AI that is intuitive, intelligent, and responsive, by developing superior Large Language Models (LLM) solutions and services which increase the capabilities built into the model and which enable utilizing thousands of APIs and external knowledge sources to provide the best experience for each request across millions of customers and endpoints. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), Natural Language Processing (NLP), Recommender Systems and/or Information Retrieval, to invent and build scalable solutions for a state-of-the-art context-aware conversational AI. A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, enjoy operating in dynamic environments, be self-motivated to take on challenging problems to deliver big customer impact, moving fast to ship solutions and then iterating on user feedback and interactions. Key job responsibilities As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel algorithms and modeling techniques to reduce friction and enable natural and contextual conversations. You will analyze, understand and improve user experiences by leveraging Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in artificial intelligence. You will work on core LLM technologies, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc. Your work will directly impact our customers in the form of novel products and services . We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA | Boston, MA, USA | Seattle, WA, USA | Sunnyvale, CA, USA
US, WA, Seattle
Amazon Web Services (AWS) is building a world-class marketing organization, and we are looking for an experienced Applied Scientist to join the central data and science organization for AWS Marketing. You will lead AWS Measurement, targeting, recommendation, forecasting related AI/ML products and initiatives, and own mechanisms to raise the science and measurement standard. You will work with economists, scientists and engineers within the team, and partner with product and business teams across AWS Marketing to build the next generation marketing measurement, valuation and machine learning capabilities directly leading to improvements in our key performance metrics. A successful candidate has an entrepreneurial spirit and wants to make a big impact on AWS growth. You will develop strong working relationships and thrive in a collaborative team environment. You will work closely with business leaders, scientists, and engineers to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable distributed services. The ideal candidate will have experience with machine learning models and causal inference. Additionally, we are seeking candidates with strong rigor in applied sciences and engineering, creativity, curiosity, and great judgment. You will work on high-impact, high-visibility products, with your work improving the experience of AWS leads and customers. AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success. AWS Support also partners with a global list of customers that are building mission-critical applications on top of AWS services. Key job responsibilities * Lead the design, development, deployment, and innovation of advanced science models in the strategic area of marketing measurement and optimization. * Partner with scientists, economists, engineers, and product leaders to break down complex business problems into science approaches. * Understand and mine the large amount of data, prototype and implement new learning algorithms and prediction techniques to improve long-term causal estimation approaches. * Design, build, and deploy effective and innovative ML solutions to improve components of our ML and causal inference pipelines. * Publish and present your work at internal and external scientific venues in the fields of ML and causal inference. * Influence long-term science initiatives and mentor other scientists across AWS. A day in the life Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices. We are open to hiring candidates to work out of one of the following locations: Arlington, VA, USA | Austin, TX, USA | New York City, NY, USA | Seattle, WA, USA
US, CA, Santa Clara
Amazon is looking for world class scientists and engineers to join its AWS AI Labs working within natural language processing. This group is entrusted with developing core data mining, natural language processing, and machine learning solutions for AWS services. At AWS AI Labs you will invent, implement, and deploy state of the art machine learning algorithms and systems. You will build prototypes and explore conceptually large scale natural language processing solutions. You will interact closely with our customers and with the academic community. You will be at the heart of a growing and exciting focus area for AWS and work with other acclaimed engineers and world famous scientists. AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (Iot), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices. We are open to hiring candidates to work out of one of the following locations: Santa Clara, CA, USA
IN, KA, Bangalore
Alexa is the voice activated digital assistant powering devices like Amazon Echo, Echo Dot, Echo Show, and Fire TV, which are at the forefront of this latest technology wave. To preserve our customers’ experience and trust, the Alexa Sensitive Content Intelligence (ASCI) team creates policies and builds services and tools through Machine Learning techniques to detect and mitigate sensitive content across Alexa. We are looking for an experienced Senior Applied Scientist to build industry-leading technologies in attribute extraction and sensitive content detection across all languages and countries. A Senior Applied Scientist will be a tech lead for a team of exceptional scientists to develop novel algorithms and modeling techniques to advance the state of the art in NLP or CV related tasks. You will work in a hybrid, fast-paced organization where scientists, engineers, and product managers work together to build customer facing experiences. You will collaborate with and mentor other scientists to raise the bar of scientific research in Amazon. Your work will directly impact our customers in the form of products and services that make use of speech, language, and computer vision technologies. We are looking for a leader with strong technical experiences a passion for building scientific driven solutions in a fast-paced environment. You should have good understanding of NLP models (e.g. LSTM, transformer based models) or CV models (e.g. CNN, AlexNet, ResNet) and where to apply them in different business cases. You leverage your exceptional technical expertise, a sound understanding of the fundamentals of Computer Science, and practical experience of building large-scale distributed systems to creating reliable, scalable, and high-performance products. In addition to technical depth, you must possess exceptional communication skills and understand how to influence key stakeholders. You will be joining a select group of people making history producing one of the most highly rated products in Amazon's history, so if you are looking for a challenging and innovative role where you can solve important problems while growing as a leader, this may be the place for you. Key job responsibilities You'll lead the science solution design, run experiments, research new algorithms, and find new ways of optimizing customer experience. You set examples for the team on good science practice and standards. Besides theoretical analysis and innovation, you will work closely with talented engineers and ML scientists to put your algorithms and models into practice. Your work will directly impact the trust customers place in Alexa, globally. You contribute directly to our growth by hiring smart and motivated Scientists to establish teams that can deliver swiftly and predictably, adjusting in an agile fashion to deliver what our customers need. A day in the life You will be working with a group of talented scientists on researching algorithm and running experiments to test scientific proposal/solutions to improve our sensitive contents detection and mitigation. This will involve collaboration with partner teams including engineering, PMs, data annotators, and other scientists to discuss data quality, policy, and model development. You will mentor other scientists, review and guide their work, help develop roadmaps for the team. You work closely with partner teams across Alexa to deliver platform features that require cross-team leadership. About the hiring group About the team The mission of the Alexa Sensitive Content Intelligence (ASCI) team is to (1) minimize negative surprises to customers caused by sensitive content, (2) detect and prevent potential brand-damaging interactions, and (3) build customer trust through appropriate interactions on sensitive topics. The term “sensitive content” includes within its scope a wide range of categories of content such as offensive content (e.g., hate speech, racist speech), profanity, content that is suitable only for certain age groups, politically polarizing content, and religiously polarizing content. The term “content” refers to any material that is exposed to customers by Alexa (including both 1P and 3P experiences) and includes text, speech, audio, and video. We are open to hiring candidates to work out of one of the following locations: Bangalore, KA, IND
IN, KA, Bangalore
Alexa is the voice activated digital assistant powering devices like Amazon Echo, Echo Dot, Echo Show, and Fire TV, which are at the forefront of this latest technology wave. To preserve our customers’ experience and trust, the Alexa Sensitive Content Intelligence (ASCI) team creates policies and builds services and tools through Machine Learning techniques to detect and mitigate sensitive content across Alexa. We are looking for an experienced Applied Scientist to build industry-leading technologies in attribute extraction and sensitive content detection across all languages and countries. An Applied Scientist will be working with a team of exceptional scientists to develop novel algorithms and modeling techniques to advance the state of the art in NLP or CV related tasks. You will work in a hybrid, fast-paced organization where scientists, engineers, and product managers work together to build customer facing experiences. You will collaborate with and mentor other scientists to raise the bar of scientific research in Amazon. Your work will directly impact our customers in the form of products and services that make use of speech, language, and computer vision technologies. We are looking for a leader with strong technical experiences a passion for building scientific driven solutions in a fast-paced environment. You should have good understanding of NLP models (e.g. LSTM, transformer based models) or CV models (e.g. CNN, AlexNet, ResNet) and where to apply them in different business cases. You leverage your exceptional technical expertise, a sound understanding of the fundamentals of Computer Science, and practical experience of building large-scale distributed systems to creating reliable, scalable, and high-performance products. In addition to technical depth, you must possess exceptional communication skills and understand how to influence key stakeholders. You will be joining a select group of people making history producing one of the most highly rated products in Amazon's history, so if you are looking for a challenging and innovative role where you can solve important problems while growing as a leader, this may be the place for you. Key job responsibilities You'll participate the science solution design, run experiments, research new algorithms, and find new ways of optimizing customer experience. You set examples for the team on good science practice and standards. Besides theoretical analysis and innovation, you will work closely with talented engineers and ML scientists to put your algorithms and models into practice. Your work will directly impact the trust customers place in Alexa, globally. You contribute directly to our growth by hiring smart and motivated Scientists to establish teams that can deliver swiftly and predictably, adjusting in an agile fashion to deliver what our customers need. A day in the life You will be working with a group of talented scientists on researching algorithm and running experiments to test scientific proposal/solutions to improve our sensitive contents detection and mitigation. This will involve collaboration with partner teams including engineering, PMs, data annotators, and other scientists to discuss data quality, policy, and model development. You will mentor other scientists, review and guide their work, help develop roadmaps for the team. You work closely with partner teams across Alexa to deliver platform features that require cross-team leadership. About the hiring group About the team The mission of the Alexa Sensitive Content Intelligence (ASCI) team is to (1) minimize negative surprises to customers caused by sensitive content, (2) detect and prevent potential brand-damaging interactions, and (3) build customer trust through appropriate interactions on sensitive topics. The term “sensitive content” includes within its scope a wide range of categories of content such as offensive content (e.g., hate speech, racist speech), profanity, content that is suitable only for certain age groups, politically polarizing content, and religiously polarizing content. The term “content” refers to any material that is exposed to customers by Alexa (including both 1P and 3P experiences) and includes text, speech, audio, and video. We are open to hiring candidates to work out of one of the following locations: Bangalore, KA, IND
US, WA, Seattle
Are you a scientist interested in pushing the state of the art in Generative AI, LLMs, LMMs? Are you interested in working on ground-breaking research projects that will lead to great products and scientific publications? Do you wish you had access to large datasets? Answer yes to any of these questions and you’ll fit right in here at Amazon. We are looking for a hands-on researcher, who wants to derive, implement, and test the next generation of Generative AI algorithms in multiple projects ranging from Computer Vision, ML, and NLP. The research we do is innovative, multidisciplinary, and far-reaching. We aim to define, deploy, and publish cutting edge research. In order to achieve our vision, we think big and tackle technology problems that are cutting edge. Where technology does not exist, we will build it. Where it exists we will need to modify it to make it work at Amazon scale. We need members who are passionate and willing to learn. Key job responsibilities - Derive novel computer vision, machine learning, and NLP algorithms. - Define scalable computer vision, machine learning and NLP models. - Invent the next generation of Generative AI models. - Work with large datasets. - Work with software engineering teams to deploy your - Publish your work at top conferences/journals. - Mentor team members. A day in the life We are a team of seasoned scientists. We work on science problems and publish our results at major scientific conferences. We work with multiple other science teams at Amazon. About the team We are a tight-knit group that shares our experiences and help each other succeed. We believe in team work. We love hard problems and like to move fast in a growing and changing environment. We use data to guide our decisions and we always push the technology and process boundaries of what is feasible on behalf of our customers. If that sounds like an environment you like, join us. We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA