Prem Natarajan, Alexa AI vice president of natural understanding, giving a presentation
Prem Natarajan, Alexa AI vice president of natural understanding
Credit: Micron Technology, Inc.

3 questions: Prem Natarajan on issues of AI fairness and bias

Alexa AI vice president of natural understanding Prem Natarajan discusses the upcoming cycle for the National Science Foundation collaboration on fairness in AI, his participation on the Partnership on AI board, and issues related to bias in natural language processing.

A year ago, Amazon and the National Science Foundation (NSF) announced a $20 million collaboration to fund academic research on fairness in AI over a three-year period. Recently, Erwin Gianchandani, deputy assistant director for Computer and Information Science and Engineering at NSF, discussed the work of the first ten recipients of the program’s grants. Here, Prem Natarajan, Alexa AI vice president of natural understanding, and the Amazon executive who helped launch the collaboration with NSF, discusses the next cycle of upcoming proposals from academic researchers, his work with the Partnership on AI, and what can be done to address bias in natural language processing models.

The 2020 award cycle for the Fairness in AI program in conjunction with the NSF recently launched. Full proposals are due by July 13th. What are you hoping to see in the next round of proposals?

We collaborated with the NSF to launch the Fairness in AI program with the goal of promoting academic research in this important aspect of AI. Our primary objective for engaging with academia on issues related to fairness and transparency in AI is to get many different and diverse perspectives focused on the challenge. The teams selected by NSF in the first round are addressing a variety of topics – from principled frameworks for developing and certifying fair AI, to domain-focused applications such as fair recommender systems for foster care services. To that end, I hope that the second round will build upon the success of the first round by bringing an even greater diversity of perspectives on definitions and perceptions of fairness. Without such diversity the entire field of research into fair AI will become a self-defeating exercise.

Another hope I have for the second round, and indeed for all rounds of this program, is that it will drive the creation of a portfolio of open-source artifacts – such as data sets, metrics, tools, and testing methodologies – which all stakeholders in AI can use to promote the use of fair AI. Such readily available artifacts will make it easier for the community to learn from one another, promote the replication of research results, and, ultimately, advance the state of the art more rapidly. Put differently, we hope that open access to the research under this program will form a rising tide that lifts all boats. It also seems natural that methodologies for fairness will benefit from broad and inclusive discussion across relevant academic and scientific communities.

The deadline for this next round of proposal submissions is July 13th. We hope that the response to this round will be even stronger than for the first. NSF selects the recipients, and I am sure NSF’s reviewers are looking forward to a summer of interesting reading!

You are Amazon’s representative on the Partnership on AI (PAI) board of directors. This unique organization has thematic pillars related to safety-critical AI; fair, transparent and accountable AI; AI labor and the economy; collaborations between AI systems and people; social and societal influences of AI; and AI and social good. It’s an ambitious, broad agenda. You’re fairly new in your role with PAI; what most excites you about the work being done there?

The most exciting aspect of the Partnership on AI is that it is a unique multi-sector forum where I get to listen to and learn from the incredible diversity of perspectives – from industry, academia, non-profits, and social justice groups. PAI today counts amongst its members about 59 non-profits, 24 academic institutions, and 18 industrial organizations. While I joined the board just a few months ago, I have already attended several meetings and participated in discussions with other PAI members as well as PAI staff. While every member has their own unique perspective on AI, it’s been really interesting and encouraging to see that we all share the same values and many of the same concerns. It should be of no surprise that the issue of equity is top of mind with a concomitant focus on fairness considerations.

Alexa & Friends Twitch show features Prem Natarajan

Earlier this month, Alexa evangelist Jeff Blankenburg interviewed Prem Natarajan live on the 'Alexa & Friends' Twitch show. In the video, they discuss recent advances in natural understanding , and how those advancements translate into better experiences for customers, developers and third-party device manufacturers.

From a technical perspective, I am excited by the number and quality of research initiatives underway at PAI. Many of these initiatives are of critical importance to the future development of the field of AI. Let me give you a couple of examples.

One is the area of fairness, accountability and transparency. There are several projects underway in this area, but I will mention one that to me exemplifies the kind of work that an organization like PAI can do. PAI researchers interviewed practitioners at twenty different organizations and performed an in-depth case study of how explainable AI is used today. This kind of research is very important to AI practitioners because it gives them a referential basis to assess their own work and to identify useful areas for future contributions.

Another example is ABOUT ML, which is focused on developing and sharing best practices as well as on advancing public understanding of AI. A couple of years ago some researchers had proposed the development of an AI model scorecard, along the lines of the nutritional information you get on the back of most food items we buy today. The scorecard would describe the attributes of the data used to train the models, the way in which it was tested, etc. The motivation behind the scorecard is to give other developers or model builders a sense of the strengths and limitations of the model, so they can better estimate and address potential weaknesses in the model for their target use cases. ABOUT ML goes well beyond such a scorecard, focusing on documentation, provenance of data and code artifacts, and other critical attributes of the model development process. Ultimately, only multisector organizations like PAI can successfully drive this kind of initiative, bringing together people across organizations and sectors.

Lastly, there’s an education role that PAI serves that I believe is unique, serving as the bridge between AI technologists and other stakeholders within society, making sure AI technologists are appropriately factoring in the perspectives and concerns of the other stakeholders within society. Some examples here include PAI’s collaborative work with First Draft, a PAI Partner, to help technologists and journalists at digital platforms address growing issues around manipulated media. PAI also helps those stakeholders understand more about how AI technology works, its strengths and its limitations.

You oversee Alexa’s natural understanding team. Natural language processing models have drawn criticism for capturing common social biases with respect to gender and race. A large body of work is emerging related to bias in word embedding and classifiers, and there are many proposals for countermeasures. Can you describe the challenge of bias in NLP models, and give us insight into some of the countermeasures you think are, or could be, effective?

A word embedding is a vector of real numbers representing that word; the core idea is that words with similar meanings map to vectors that are “close” to each other. Word embeddings have become a central feature of modern NLP. While embeddings can be computed using a variety of different techniques, deep learning techniques have proven to be tremendously effective at numerically representing the semantics of a word and concepts, etc. Today, deep learning based embeddings are used for all kinds of processing, from named entity recognition, to question answering, and natural language generation. As a result, the semantics that these embeddings encode greatly influence how we interpret text, the accuracy of those interpretations, and the actions we take in response to those interpretations.

Bias can also manifest in other ways because any system that is based on data can exhibit a majoritarian bias to it.
Prem Natarajan, Alexa AI VP of natural understanding

As word embeddings became prevalent, researchers naturally started looking into their fragilities and shortcomings. One of those fragilities is that the embeddings derive and encode meaning from context, which means that the meaning of a word is largely controlled by the different contexts in which that word is observed in the training data. While that seems like a reasonable basis for inferring meaning, it leads to undesirable consequences. My friend Kai-Wei Chang at UCLA is one of the early investigators of bias in NLP and he uses the following example: take the vector for doctor and you subtract the vector for man; when you add the vector for woman, you should in principle get the vector for doctor again, or a female doctor. But instead the resulting vector is close to the vector for ‘nurse.’ What this example shows is that the latent biases in human-generated text get encoded into the embeddings. One example of a system that is affected by these biases is natural language generation. Many studies have shown that such biases can result in the generation of text that exhibits the same biases and prejudices as humans, sometimes in an amplified manner. Left unmitigated, such systems could reinforce human biases and stereotypes.

Bias can also manifest in other ways because any system that is based on data can exhibit a majoritarian bias to it. So, for example, different groups in different parts of the world may speak the same language with different dialects, but the most frequent dialect will likely see the best performance only because it forms the major proportion of the training data. But we don’t want dialect or accent to determine how well the system will work for an individual. We want our systems to work equally well for everyone, regardless of geography, dialect, gender, or any other irrelevant factor.

Methodologically, we counter the impact of bias by using a principled approach to characterize the dimensions of bias and associated impact, and by developing techniques that are robust to these biasing factors. For example, it stands to reason that speech recognition systems should ignore parts of the signal that are not useful for recognizing the words that were spoken. It shouldn’t really matter whether the voice is male or female, only the actual words should. Similarly for natural language understanding, we want to be able to understand the queries of different groups of people regardless of the stylistic or syntactic variations of the language used. Scientists at Amazon and elsewhere are exploring a broad variety of approaches such as de-biasing techniques, adversarial invariance, active learning, and selective sampling. Personally, I find the adversarial approaches to both testing and to generating bias or nuisance invariant representations most appealing because of their scalability, but in the next few years, we will all find out what works best for different problems!


Work with us

See more jobs
US, CA, Irvine
Job summaryAs the Applied Science Manager within Personalization, you will help build a team that leads creation and innovation on the next generation of Recommendation and Personalization technologies on Amazon. Our team owns creating recommendation solutions that understand long-term customer shopping patterns, Spanning Academia, research, and applied machine learning techniques operating at world-class scale, we focus on recommending the right product to the right customer at the right time. You will be surrounded by thought-leaders in the Personalization space who are patent-leaders within Amazon.In Personalization we use state-of-the-art machine learning techniques and A/B testing to run experiments on some of Amazon’s most prominent and valuable pages. We work on a diverse range of products, building real-time, low-latency recommendation and ranking systems as well as building algorithms for understanding customer behavior and generating recommendations content. You will hone your skills in areas such as deep learning and collaborative filtering while building scalable industrial machine learning systems that handle millions of requests a day. You will lead a collaborative team of experienced engineers. You will have a unique opportunity to drive direct, measurable impact to our customers, powering features on Amazon.About the team: Our team has the autonomy to decide where we can have the most impact and get down to experimenting. We love metrics and the fast pace. We analyze data to uncover potential opportunities, generate hypotheses, and test them. We refuse to accept constraints, internal or external, and have a strong bias for action. We imagine, build prototypes, validate ideas, and launch follow-up experiments from the successful ones.About you: You are an entrepreneurial Applied Science Manager with an interest in machine learning, data science, search, or recommendation systems. You love keeping abreast of the latest technology and use it to help you innovate. You have strong leadership qualities, great judgment, clear communication skills, and a track record of delivering great products and hiring/coaching successful teams. You enjoy working hard, having fun, and making history!
IN, KA, Bangalore
Job summaryThe Team: Amazon One is a fast, convenient, contactless way for people to use their palm to make everyday activities like paying at a store, presenting a loyalty card, entering a location like a stadium, or badging into work more effortless. The service is designed to be highly secure and uses custom-built algorithms and hardware to create a person’s unique palm signature. Designed and custom-built by Amazonians, it uses a variety of technologies including computer vision, sensor fusion, and advanced machine learning. Innovation is part of our DNA! Our goal is to be Earths’ most customer centric company and we are just getting started. We need people who want to join an ambitious program that continues to push the state of the art in computer vision, machine learning, distributed systems and hardware design.The Role: Everyone on the team needs to be entrepreneurial, wear many hats and work in a highly collaborative environment that’s more startup than big company. We’ll need to tackle problems that span a variety of domains: computer vision, image recognition, machine learning, real-time and distributed systems.If you have expertise leading Computer Vision research teams and have a Ph.D, or an Masters and 2+ years of industry experience and have: - the ability to recognize and champion new ideas and novel solutions;· the insight to correctly identify paths worth exploring;· the discipline to fast-fail when data refutes theory;· and the fortitude to continue exploring until your solution is foundCome join us invent the future and change the world.
US, CA, Sunnyvale
Job summaryAmong the goals of the Alexa AI team is to make Alexa the most knowledgeable and trusted source of local information to enable customers to find places, products, or services around them; ask detailed questions about points of interests; and to provide personalized travel, commute, and navigation experiences - while in the car, on the go, or at home.We are seeking a Data Scientist to be part of the NLU Science team for Alexa Local Information. This is a strategic role to shape and deliver our technical strategy in developing and deploying natural language understanding and machine learning solutions to our hardest customer-facing problems. This role requires collaborating closely with business, engineering and other scientists within Local Information and across the Alexa organization. You will work with a team of Language Engineers and Scientists to launch new customer facing features and improve the current experiences.Key job responsibilities· Apply machine learning and SLU techniques to improve our conversational interfaces and search algorithms· Collaborate with Scientists and Language Engineers in developing coherent and comprehensive meaning representation frameworks· Perform hands-on data analysis and modeling with large data sets to develop insights that increase device usage and customer experience· Run A/B experiments, evaluate the impact of your optimizations and communicate your results to various business stakeholders· Work closely with product managers and software engineers to design experiments and implement end-to-end solutions· Set up and monitor alarms to detect anomalous data patterns and perform root cause analyses to explain and address them· Handle competing requests from a range of data customers· Be a member of the Amazon-wide machine learning community, participating in internal and external Meet-Ups, Hackathons and Conferences· Help attract and recruit technical talent
US, WA, Seattle
Job summaryAmazon Web Services (AWS) is obsessed with ensuring the success of our customers. We want to nurture many new customers through use of technical support, credits, and training programs. To this end, we need to understand what types of customers respond the most to these different benefits. Can we tell who needs help from the sales force vs. those who are more motivated by sticker price? Do the impacts of credits or sales help vary by customer tenure, or industry? These are the types of questions we'll answer to help drive the growth of AWS.Job Locations:· Seattle, WA· Dallas, TX· Arlington, VA· Boston, MA· Vancouver, CAKey job responsibilitiesBeyond the usual technical skills, ownership, customer obsessions, and deliver results are key for this team. We need people who can understand customer needs, build a prototype, collect feedback from the customer, build some more, lather, rinse and repeat. Our team contains a wide diversity of tech talent and we can get you help when you're blocked. But we do need you to understand your customers' requirements and work backwards.About the teamWe're a team of economists, data and applied scientists, and engineers who work on projects to help make the best use of AWS's thousands of talented sales representatives. We have great data support from our sister teams and have more than tripled our team size by thinking big and delivering. We'd love to chat with you and answer any questions you have!
US, WA, Redmond
Job summaryAs an Applied Scientist on the team you will responsible for building out and maintaining the algorithms and software services behind one of the world’s largest satellite constellations. You will be responsible for developing algorithms and applications that provide mission critical information derived from past and predicted satellite orbits to other systems and organizations rapidly, reliably, and at scale.You will be focused on contributing to the design and analysis of software systems responsible across a broad range of areas required for automated management of the Kuiper constellation. You will apply knowledge of mathematical modeling, optimization algorithms, astrodynamics, state estimation, space systems, and software engineering across a wide variety of problems to enable space operations at an unprecedented scale. You will develop features for systems to interface with internal and external teams, predict and plan communication opportunities, manage satellite orbits determination and prediction systems, develop analysis and infrastructure to monitor and support systems performance. Your work will interface with various subsystems within Project Kuiper and Amazon, as well as with external organizations, to enable engineers to safely and efficiently manage the satellite constellation.The ideal candidate will be detail oriented, strong organizational skills, able to work independently, juggle multiple tasks at once, and maintain professionalism under pressure. You should have proven knowledge of mathematical modeling and optimization along with strong software engineering skills. You should be able to independently understand customer requirements, and use data-driven approaches to identify possible solutions, select the best approach, and deliver high-quality applications.Export Control Requirement:Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum.About the teamThe Constellation Management & Space Safety team maintains build the software services responsible for maintaining situational awareness for Kuiper satellites and coordinating with internal and external organizations to maintain the nominal operational state. We build automated systems that use satellite telemetry and other related data to predict future orbits, plan maneuvers to avoid high risk conjunctions, and keep satellites in the desired locations. Using knowledge of software engineering and space systems, we provide visibility information that is used to predict and establish communication channels for Kuiper satellites.
GB, Cambridge
Job summaryWe are looking for someone who is excited to apply cutting-edge techniques from deep learning, speech or natural language processing (NLP) to the text-to-speech (TTS) technology behind Alexa and our AWS cloud speech service.As a Machine Learning Scientist you will be responsible for leading the development and launch of core product features. You will have significant influence on our overall strategy by helping define new product features and carry out research to demonstrate their impact.You will have the opportunity to solve hard problems with voice – we’ve spent years of invention on this. When we started working on this, the technology didn’t even exist – we had to invent it. Join us if you want to apply your deep learning skills in a dynamic field that is revolutionising the way people interact with devices and services.We believe that voice will fundamentally improve the way people interact with technology. It can make the complex simple—it’s the most natural and convenient user interface. Voice is going to be a big part of our future and we are inventing it here.Key job responsibilities· Use your expertise in deep learning to research and implement novel approaches to make improvements to our text-to-speech technology.· Publish your work at scientific conferences,· Lead and mentor junior engineers and scientists.· Participate in the design, development, evaluation, deployment and updating of data-driven models and analytical solutions for spoken language applications.
US, WA, Bellevue
Job summaryAt Amazon we strive to be the most innovative and customer centric company on the planet. Come work with us while we are developing our Science and Technology to deliver the best solutions for our vendors, warehouses and carriers. Help us solve complex problems impacting billions of dollars per year while partnering with world class scientists and researchers. Our mission is to build the most efficient and customer-obsessed solution on the planet. We aim to leverage cutting edge technologies in and techniques, and operate high volume, high efficiency, low latency, and high availability services.You’ll design, model, develop and implement state of the art models and solutions used by Amazon worldwide and will regularly interact with engineering and business leadership. You will participate in the planning and execution of technology projects and operational excellence initiatives. The focus of this role is to research, develop, and deploy models that will inform and support our business. You will have the opportunity to work with some of the leading scientists in the areas of vehicle routing, meta-heuristics, and market analysis, , and methodologies.Tasks/ Responsibilities:· Contribute to the mid- and long-term strategic planning studies and analysis.· Lead and partner with the engineering and operations to drive modeling and design for complex business problems.· Develop accurate and scalable mathematical programming, heuristic, and models to solve our hardest problems.· Lead complex modeling analyses to aid management in making key business decisions and set new policies.Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation / Age.
US, WA, Seattle
Job summaryAre you excited about influencing the payment experience of millions of customers worldwide ? The moment a customer makes a payment on Amazon is when trust is established – trust that the item is delivered on time, a refund is provided quickly if needed, a digital movie purchased will play immediately, a seller receives their disbursement, and hundreds of other experiences across Amazon when a customer completes a payment. The Payment Acceptance & Experience (PAE) team, within the Consumer Payments organization, has the mission to build the most trusted, intuitive, and accessible payment experience on Earth. Applied Science & Machine Learning Engineering (PAE ASMLE) is the core machine learning team within PAE. The team has a mission to enhance customer payments experience that requires advancing the state of the art in machine learning. We work backwards from the customer to create value for them by leveraging an underlying applied science methodology. We deploy our solutions through Native AWS services that operate at Amazon scale. We strive to publish our solutions and share our findings so that the broader Amazon scientific community can benefit.As an applied scientist on our team, your role is to leverage your strong background in Computer Science and Machine Learning to help build the next generation of our model development and assessment pipeline, harness and explain rich data at Amazon scale, and provide automated insights to improve machine learned solution that impacts Payments experience of millions of customers every day. This role requires a pragmatic technical leader comfortable with ambiguity, capable of summarizing complex data and models through clear visual and written explanations. The ideal candidate will have experience with machine learning models and applying science to various business contexts. We are particularly interested in experience applying predictive modeling, natural language processing, deep learning, and reinforcement learning at scale. Additionally, we are seeking candidates with strong rigor in applied sciences and engineering, creativity, curiosity, and great judgment.Your responsibilities include:. Analyze the data and metrics resulting from traffic into Amazon Consumer Payments experiences.. Design, build, and deploy effective and innovative ML solutions to improve various components of the Consumer Payments experience, using predictive modeling, recommendations, anomaly detection, ranking, and forecasting.. Evaluate the proposed solutions via offline benchmark tests as well as online A/B tests in production.. Publish and present your work at internal and external scientific venues in the fields of ML/NLP/IR/Forecasting.Your benefits include:. Working on a high-impact, high-visibility product, with your work improving the experience of millions of customers.. The opportunity to use (and innovate) state-of-the-art ML methods to solve real-world problems.. Excellent opportunities, and ample support, for career growth, development, and mentorship.. Competitive compensation, including relocation support.The PAE ML team operates primarily out of Amazon's Seattle office. We are a new and expanding team where you will have an opportunity to influence our goals and mission. We collaborate with Software Engineering, Data Engineering, Product Management and Marketing teams within Amazon Consumer Payments to solve and deploy machine learning solutions at scale.Please visit https://www.amazon.science for more information
GB, Cambridge
Job summaryJob summaryWe research and develop question understanding techniques, deployed to Alexa world-wide.Key job responsibilitiesWe have an exciting position for a NLP/ML scientist to join Alexa AI. Our team makes Alexa smarter by delivering an end-to-end natural language question answering (QA) technology. We build advanced QA models based on constructing a high precision large scale knowledge graph from multiple sources (e.g. facts extraction from text at Internet-scale; linking and aligning open and proprietary knowledge-bases); developing natural language understanding models; and generating natural language responses based on query results on our knowledge graph.To achieve our ambition we need to develop methods that lie beyond the cutting edge academic and industrial research of today and as a scientist, you will bring academic and/or industrial practical experience and create novel solutions to complex problems at massive scale. We are particularly interested in problems of fact extraction, entity linking, natural language understanding, semantic parsing, natural language generation, masked language models, cross-lingual NLP models, and weakly supervised methods of learning (self-supervised, transfer learning, semi-supervised, curriculum learning).As a research led team, we have been publishing and contributing to the scientific community and you can find some of our recent work at: Rongali et al, "Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing", https://arxiv.org/abs/2001.11458; Harkous et al, "Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity", https://arxiv.org/abs/2004.06577; Sen et al, "What do Models Learn from Question Answering Datasets?", https://arxiv.org/abs/2004.03490; Thorne et al, FEVER: a large-scale dataset for fact extraction and verification, https://arxiv.org/abs/1803.05355.A day in the lifeResearch and development of models for question answering, fact extraction, and reasoning. Publish your findings and innovations in papers.About the teamWe research and develop end-to-end question answering techniques, deployed to Alexa world-wide.
US, WA, Seattle
Fashion is extremely fast-moving, visual, subjective, and it presents numerous challenges in areas such as search relevance ranking and optimization, product discovery experience including recommendations and personalization. The vision for Amazon Fashion is to make Amazon the number one online shopping destination for Fashion customers by providing large selections, inspiring and accurate recommendations and customer experience. Are you excited by solving Fashion customer and business problems by applying machine learning and big data technologies? Are you passionate about building systems that process massive amounts of data and make real customer and business impact?The main focus of Softlines Discovery Science team is to innovate and build engaging search and browse experience for Amazon Fashion customers including organic search relevance ranking, content recommendation and optimization, and understanding of fashion customers shopping intent. The team is looking for an Applied Scientist who is proficient in Machine Learning and can apply the concepts to solve large-scale customer facing problem. We closely collaborate with the core Amazon Search community on building better search algorithms with profound customer impact. The team integrates knowledge on causal Inference, and Econometric/Economic Methodologies to derive actionable insights that change the search results towards improving the long term engagement of our customers. We also develop Statistical Models and Algorithms to drive strategic business decisions and improve operations. Additionally, we are building deep learning models and system to best utilize Amazon customer and product information, such as customer review, product images, etc, for more personalized Softlines shopping experience. We are an interdisciplinary team of Engineers, and Scientists incubating and building day one solutions using cutting-edge technology, to solve large scale problems at Amazon. This role will challenge you to utilize cutting-edge machine learning techniques in the domain of search ranking, deep recommendation system, and computer vision to deliver significant impact for the business.Major Responsibilities:· Act as the key contributor in Machine Learning and drive full life-cycle Machine Learning projects.· Participate in technical efforts within this team and collaboration with other science teams.· Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production.· Run A/B experiments, gather data, and perform statistical tests.· Establish scalable, efficient, automated processes for large-scale data mining, machine-learning model development, model validation and serving.· Work closely with software engineers and product managers to assist in productionizing your ML models.· Explore new research initiatives to help shape our long-term science vision.
US, WA, Seattle
Job summaryThe Central AWS Econ team is dedicated to bringing the most trustworthy evidence-based analysis to the most strategic decisions for AWS leadership.Our studies impact strategic investments, service business model, resource allocation, product priorities and pricing models, go-to-market motions and more.This economist role partners with AWS business leaders across the organization to define and deliver on economic questions that guide their most strategic decisions. The successful candidate will be a problem solver who enjoys diving into data, is excited by difficult modeling challenges and ambiguous starting points, and possesses strong communication skills to effectively interface and collaborate with product, finance, planning and business teams.Specific questions include developing supporting economics for new business model, evaluating the relationship between short and long term growth, mapping and affecting the customer journey through different AWS products and cloud technologies.The Central AWS Econ team is dedicated to answering these (and many more) questions using quantitative, economic and statistical methods.Key Responsibilities:· Frame and conduct economic studies, from question definition to and communicating practical implications to senior leadership· Develop new repeatable data analysis pipelines to be used by non-economists
US, CA, San Francisco
Job summaryConversational interfaces are maturing and proliferating, but significant advances in conversational AI and related technologies will need to be made before a fully conversational web is possible. Alexa.com is seeking a creative, entrepreneurial, and customer-obsessed Applied Scientist who can apply cutting edge research and state-of-the-art machine learning algorithms as part of a new project that will push the boundaries of conversational AI. This position will require you to investigate and solve complex technical problems, innovate on behalf of customers, invent new technologies, and build cornerstone services to power the next generation of online conversational applications.The ideal candidate has a broad and deep background in machine learning, is passionate about science, is highly driven to learn and deploy new technologies, thrives in a fast-paced environment that requires the development of solutions to ambiguous and challenging problems, and enjoys collaborating with both technical and nontechnical peers. As part of our AI team, you will work as a hands-on practitioner and technical leader in multiple areas such as statistical modeling, NLP and NLU, and deep learning. You will formulate and test hypotheses, evaluate and implement ML techniques, train and test new models, and deliver new production services that will be used by customers around the world.Key job responsibilities· Develop novel modeling techniques for pattern recognition, prediction, classification, and other complex data science problems· Develop prototypes and collaborate with stakeholders to assess the feasibility of selected approaches· Write high quality code and contribute to our codebase of scientific applications using relevant technologies· Contribute to strategic planning and project management for a variety of technical initiatives· Effectively communicate with customers, senior management, and colleagues with diverse roles and technical backgrounds· Document methodologies and increase our institutional knowledge based on experimental results and operationalized solutionsAbout the teamAlexa.com is a wholly owned subsidiary of Amazon.com. At Alexa.com, we solve ML problems in natural language processing and understanding, search relevance and ranking, and digital behavior measurement and prediction. We have been gathering and analyzing data from online sources for more than 20 years, with terabytes of archived web content, a data-contributing panel of millions of users in countries around the world, and decades of experience as a leading provider of competitive analytics and marketing intelligence services. For more information, visit www.alexa.com.
US
Job summary“Amazon Science gives you insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Our scientists continue to publish, teach, and engage with the academic community, in addition to utilizing our working backwards method to enrich the way we live and work.”Please visit https://www.amazon.science for more informationAmazon Exports and Expansion builds new experiences which help our customers across the world access the benefits of shopping on Amazon to find products sourced locally for them or exported from other Amazon marketplaces. Do you want to improve how shoppers around the world discover and purchase products that delight them?We are looking for an Applied Scientist who can apply the latest research, state of the art algorithms and machine learning to build highly scalable systems in the international e-Commerce space. Right candidate will develop algorithms and models to handle unique cross border shopping problems such as improving selection data quality, improved product discovery or offering scalable, highly localized customer experience to name few.Does the challenge of AI solving some of the pain points of millions of international shoppers from 100+ countries, in multiple currencies and excite you? Have you ever wanted to work on machine learning problems that will make a lasting impact, solving key problems that impact the experience of millions of Amazon customers? If you are passionate about solving complex problems, in a challenging environment, we would love to talk with you.As a member of our team you will develop and evaluate machine learning models using large data-sets, cloud services, customer behavior, transactional history to improve our customer’s experience. Working closely with best-in-class engineers you will have the opportunity to apply a variety of machine learning algorithms, including deep learning, and work on one of the world's largest data sets to influence the long term evolution of our technology roadmap. You will need to be entrepreneurial, able to deal with ambiguity and work in a highly collaborative environment.
IN, KA, Bengaluru
The Alexa Automatic Speech Recognition (ASR) science team in India is rapidly growing. The team is responsible for advancing Alexa’s core ASR technology (in collaboration with the EU and US counterparts) and building highly accurate ASR production models in multiple languages, such as English dialects (UK, AU, NZ, IN) and Indic (Hindi etc.) languages.The challenge is ASR models must generalize to various device form factors (smart speaker, multi-media devices, etc.), acoustic conditions (far-field, close-talk, mobile, and noisy), and content (natural conversations etc.). As part of the expansion, the team will be responsible for complex ML technologies such as (i) learning without relying on human transcribed data by leveraging abundant unlabelled data via semi-supervised learning (SSL) and self-supervised learning, and customer feedback signals through weak-supervision (WS) (ii) Online Learning and Lifelong Learning, which requires a major overhaul of the learning infrastructure and ML algorithms as data will not be stored for training (iii) Privacy-preserving Federated-Learning, which enables training where audio does not even leave the devices. (iv) multilingual ASR technology, where a single ASR model will serve multiple languages (e.g., Indic languages). Apart from working on cutting-edge ML technology, you will have the opportunity to work on some exciting still confidential new products and also publish your work in top-tier conferences.We are looking to hire Applied Scientists, Senior Applied Scientists and Applied Science Managers, at all levels. For Applied Science roles, Masters or PhD with solid understanding of Machine Learning (ML), Algorithms and Coding is a minimum requirement. Prior experience in ML projects and publications in top-tier conferences are preferred. If you are interested, please apply below.
CA, ON, Toronto
Job summaryThe Sponsored Display Advertising team has an opening for an outstanding ML scientist who is passionate about applying advanced ML and statistical techniques to solve real-world challenges. Amazon's Sponsored Display Advertising program serves millions of personalized Ads every day. We buy Ad impressions in real-time auctions and algorithmically deliver the most relevant Ad. We delight in data, and are constantly working to enhance and improve our models. We relentlessly optimize to keep delivering the best possible Ads for our customers.You will work in an agile and fast-paced team of scientists and software engineers. The team is building a number of new advertising products, including dynamic video advertising, to improve the range of offerings for our advertisers and provide new Ad experiences for our customers. As a scientist on the team, you can be involved in every aspect of the process - from idea generation, business analysis and scientific research, through to development and deployment of advanced models - giving you a real sense of ownership. The systems that you help to build will operate at massive scale to display ads to customers around the world. From day one, you will be working with experienced scientists, engineers, and designers who love what they do.We are looking for ML scientists who can delight our customers by continually learning and inventing. Our ideal candidate is an experienced ML scientist who has a track-record of performing analysis and applying statistical techniques to solve real business problems, who has great leadership and communication skills, and who is motivated to achieve results in a fast-paced environment. The position offers an exceptional opportunity to grow your technical and non-technical skills and make a real difference to the Amazon Advertising business.Key responsibilities:· Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, making use of both quantitative analysis and business judgment.· Collaborate with software engineering teams to integrate successful experimental results into large-scale, highly complex Amazon production systems.· Report results in a manner which is both statistically rigorous and compellingly relevant, exemplifying good scientific practice in a business environment.· Promote the culture of experimentation and applied science at Amazon.
JP, 13, Meguro
Job summary*Please submit English resume when you apply for the job.Sellers are our customers. Our mission is to help every seller grow and succeed on Amazon. Our team invents, test and launches some of the most innovative services, technology and processes for our sellers.Amazon is looking for a talented and passionate hands-on Data Scientist to build world class statistical and machine learning models to understand our sellers, and deliver the right recommendation to help them grow.They’ll be comfortable with ambiguity and enjoy working in a fast-paced, diverse and dynamic environment. The position also requires collaboration with other scientists, engineers, Product and Marketing ManagersThe Data Scientist is accountable for:(1) Deeply understanding the most challenging business questions and use data / modelling / analysis to articulate possible root cause analysis and solutions.(2) Managing and executing entire projects or components of large projects from start to finish including project management, data gathering and manipulation, synthesis and modeling, problem solving, and communication of insights and recommendations.(3) Developing and scaling end-to-end ML Models and solutions(4) Partnering with Other Data scientists and Economists to design and run experiments, research new algorithms, and prove incrementality and drive growth.(5) Understanding drivers, impacts, and key influences on seller growth dynamics.(6) Automating feedback loops for algorithms in production.(7) Utilizing Amazon systems and tools to effectively work with terabytes of data.[Work Life Harmony]We believe, it is important to spend private time such as spending time with your family or doing anything you like to spur innovation. Amazon promotes a fulfilling and flexible work style according to the work volume and lifestyle of each employee. (Example: Flex Time, Work from Home, Maternity /Parenting /Family Care Leave etc.)
US, WA, Seattle
Job summaryThe Sourcing Guidance team is looking for an Applied Scientist to join our team in building science solutions at scale. Our team applies Machine Learning, Reinforcement Learning, Causal Inference, and Econometrics/Economics to derive actionable insights about the complex economy of Amazon’s retail business. We also develop Statistical Models and Algorithms to drive strategic business decisions and improve operations. We are an interdisciplinary team of Economists, Engineers, and Scientists incubating and building day one solutions using cutting-edge technology, to solve some of the toughest business problems at Amazon.You will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable ML models. You will partner with scientists, economists, and engineers to help invent and implement scalable ML, RL, and econometric models while building tools to help our customers gain and apply insights. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale economic problems, enable measurable actions on the Consumer economy, and work closely with scientists and economists. We are particularly interested in candidates with experience building predictive models and working with distributed systems.As an Applied Scientist, you bring business and industry context to science and technology decisions. You set the standard for scientific excellence and make decisions that affect the way we build and integrate algorithms. Your solutions are exemplary in terms of algorithm design, clarity, model structure, efficiency, and extensibility. You tackle intrinsically hard problems, acquiring expertise as needed. You decompose complex problems into straightforward solutions.To know more about Amazon science, Please visit https://www.amazon.science
US, CA, San Diego
Job summaryThe Sourcing Guidance team is looking for an Applied Scientist to join our team in building science solutions at scale. Our team applies Machine Learning, Reinforcement Learning, Causal Inference, and Econometrics/Economics to derive actionable insights about the complex economy of Amazon’s retail business. We also develop Statistical Models and Algorithms to drive strategic business decisions and improve operations. We are an interdisciplinary team of Economists, Engineers, and Scientists incubating and building day one solutions using cutting-edge technology, to solve some of the toughest business problems at Amazon.You will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable ML models. You will partner with scientists, economists, and engineers to help invent and implement scalable ML, RL, and econometric models while building tools to help our customers gain and apply insights. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale economic problems, enable measurable actions on the Consumer economy, and work closely with scientists and economists. We are particularly interested in candidates with experience building predictive models and working with distributed systems.As an Applied Scientist, you bring business and industry context to science and technology decisions. You set the standard for scientific excellence and make decisions that affect the way we build and integrate algorithms. Your solutions are exemplary in terms of algorithm design, clarity, model structure, efficiency, and extensibility. You tackle intrinsically hard problems, acquiring expertise as needed. You decompose complex problems into straightforward solutions.To know more about Amazon science, Please visit https://www.amazon.science
US
Job summaryPayments Security team is looking for an Applied Scientist to apply formal verification, program analysis, and constraint-solving to prove the correctness of critical systems. In this role, you will work closely with internal security teams to design and build formal verification systems that continuously assess safety and security. You will build on top of existing formal verification tools developed by AWS and develop new methods to apply those tools at scale. You will need to be innovative, entrepreneurial, and adaptable. We like to move fast, experiment, iterate and then scale quickly, thoughtfully balancing speed and quality.
US, WA, Seattle
Job summaryWe are a team of doers working passionately to apply cutting-edge advances in technology to solve real-world problems. As a Research Scientist, you will work with a unique and gifted team developing exciting products for consumers and collaborate with cross-functional teams. Our team rewards intellectual curiosity while maintaining a laser-focus in bringing products to market. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the cutting edge of both academic and applied research in this product area, you have the opportunity to work together with some of the most talented scientists, engineers, and product managers.Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.