3 ways reinforcement learning is changing the world around you

Sahika Genc, senior scientist with Amazon AI, writes about three important ways reinforcement learning is used in the real world, and explains how you can get hands on with reinforcement learning.

Sahika Genc
Sahika Genc, senior scientist, Amazon AI
Credit: Alexandra Tatarzyn

Sahika Genc is a senior scientist with Amazon AI. Her team works on reinforcement learning (RL) algorithms for Amazon Sagemaker, which provides every developer and data scientist with the ability to build, train, and deploy machine learning models quickly. Genc also leads the science team on AWS DeepRacer, which enables developers to have a way to get hands-on with RL, experiment, and learn through autonomous driving.

In this article, Genc discusses three important ways reinforcement learning is used in the real world, and explains how you can get hands on with reinforcement learning.

When the concept of reinforcement learning was first introduced in the 1950s, there were two themes – the first focused on developing learning methods via a trial-and-error process, while the other provided a more theoretical framework to solve optimal control problems. These practical and theoretical methods merged in the 1980s to give birth to reinforcement learning as a more formalized field of study and development.

At the time, luminaries like Richard Sutton and Andrew Barto highlighted theories like optimal control and dynamic programming, and identified key component ideas, such as temporal difference learning, dynamic programming, and function approximation.

Fast forward to the 2000s, where deep learning gave reinforcement learning a massive boost by eliminating the need to manually configure features, and use raw sensor data (such as the pixels of an image rather than a segmented image).

But what exactly is reinforcement learning?

As opposed to supervised learning (which uses labeled training data) or unsupervised learning (where you draw inferences from input data without labeled responses), reinforcement learning involves a system making short-term decisions while optimizing for a longer-term goal through trial and error. Deep learning is used to make mathematical representations of important variables, while the reinforcement learning agent learns the actions needed to maximize rewards over a longer period of time.

Here are three applications of reinforcement learning that are changing our world in profound ways:

1. Recommendation systems

Reinforcement learning has obvious advantages in developing recommendation systems for news feeds, products or videos. In this case, the goal of the system is to personalize product recommendations.

The state of a system changes constantly as users interact with it. This makes supervised learning less than ideal for recommendation systems, as you would constantly need additional infrastructure for deploying recurring model updates. On the other hand, systems that use reinforcement learning can continually update recommendations based on user feedback. Deep learning provides mathematical representations of the product, consumer interest, and consumer satisfaction. The reinforcement learning agent can personalize the content to each individual based on their preferences over a period of time, in a way that maximizes the reward over the long term.

In recent years, there has been an increased uptake in deep reinforcement learning for use cases such as push notifications, faster video loading by pre-fetching content and for delivering product recommendations. Visit the Amazon Sagemaker notebook on recommendation systems to get a deep dive on reinforcement learning in action.

2. Energy smart grids

According to the International Energy Agency (IEA), global energy consumption grew by 2.3% in 2018 – twice as fast as the average over the last ten years. Reinforcement learning has outperformed advanced control systems traditionally used for energy optimization for applications like datacenter cooling and select smart grid applications.

Energy systems interact with the environment in complex and non-linear ways. Traditional formula-based engineering and human intuition cannot adapt to rapidly changing conditions like the weather. It is impossible to come up with rules and heuristics for every operating scenario. A general intelligence framework is needed to understand the data center’s interactions with the environment.

Deep reinforcement learning has been used to extract knowledge from past consumption patterns, production time series and available forecasts to tailor energy distribution for datacenters and buildings. Here, deep learning is used to make mathematical representations of complex thermodynamic equations. By seeking reward maximization, the reinforcement learning agent learns the right actions to take (such as which systems to turn on and off) over the course of entire days, weeks, months and years. See the Amazon Sagemaker notebook for energy use cases to get hands on with practical applications of reinforcement learning.

3. Robotics

Most of the industrial robots used in environments like manufacturing floors are blind. This is because image sensing has not been a commodity until recent times. However, there has been an increase in the use of image data from camera, LIDAR or radar sensors.

Consequently, deep reinforcement learning can be used to train robots to take actions such as picking up or moving objects in warehouses and factories. In this scenario, deep learning is used to interpret images by looking at every pixel, while reinforcement learning agents learn how to make the right decisions over a period of time based on which action was successful. The Amazon Sagemaker notebook is a great place to get started with reinforcement learning and robotics.

There are still many challenges we must work through. These have to do with not only the high volume, but the high dimensionality of data, which can make it challenging to design responsive systems. In addition, be it for recommender systems or energy grids, both the data and relationships between variables can change over time. This can make it incredibly difficult to avoid concept drift.

Finally, the moral of the story of Midas is applicable to machine learning. Be careful what you wish for. There can be a huge gap between the intended reward and stated reward – and you can find your system maximizing for end states that aren’t entirely desirable.

AWS Deep Racer.png
AWS DeepRacer gives you an interesting and fun way to get started with reinforcement learning (RL)
image Credit: Amazon

In many ways, it’s still early days when it comes to Deep Reinforcement Learning. There’s no better time to get on board. With AWS DeepRacer, you now have a way to get hands-on with RL, experiment, and learn through autonomous driving. You can get started with the virtual car and tracks in the cloud-based 3D racing simulator. For a real-world experience, you can deploy your trained models onto AWS DeepRacer and race your friends, or take part in the global AWS DeepRacer League. Visit the AWS DeepRacer page to get started.


Work with us

See More Jobs
US, WA, Seattle
The Human Resources Central Science Team (HRCS) uses economics, behavioral science, statistics, and machine learning to proactively identify mechanisms and process improvements which simultaneously improve Amazon and the lives, wellbeing, and the value of work to Amazonians. We are an interdisciplinary team which combines the talents of science and engineering to develop and deliver solutions that measurably achieve this goal.We are looking for an outstanding applied scientist who is able to provide structure around complex science and business problems, work with economists and engineers to estimate and validate their models on large scale data, and who can help business and econ partners turn the results of their analysis into policies, programs, and actions that have a major impact on Amazon’s business. We are looking for creative thinkers who can combine a strong economic toolbox with a desire to learn from others, and who know how to execute and deliver on big ideas.At the senior level, we expect you be able to own the development of scientific models and to manage, in close collaboration with scientists and engineers, the data analysis, modeling, and experimentation that is necessary for estimating and validating your model. You will need to work with our business partners to communicate the properties of your analysis/modeling and be able to work to incorporate their feedback and requests into your project.We are particularly interested in candidates with research background in nature language understanding, experimental design, reinforcement learning, search/recommendation in working on innovative, strategic problems with significant business impact.
US, CA, Santa Clara
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLC, an Amazon.com CompanyTitle: Data Scientist IIWorksite: Santa Clara, CAPosition Responsibilities:Manage metrics reporting, perform data mining and big data analysis, to provide strategic advice on business and improve processes and systems. Collect business use cases, research and evaluate opportunities, and leverage data to support business functions through advanced mathematical modeling. Independently own and solve business problems. Transform business problems into mathematical models, provide data-driven solutions, and write clear and factually correct documents explaining technical concept to non-technical audience. Gather and use complex data sets across domains and proactively gather data when not readily available. Use specialized modelling software and scripting languages to scale methods. Partner with business, science, and engineering to establish scalable, efficient, automated processes for large scale data analyses, model development, model validation, and model implementation. Building and tuning machine learning models. Formulating and solving predictive modeling, machine learning, forecasting or statistical modeling problems in the NLU and Search domain.Amazon.com is an Equal Opportunity-Affirmative Action Employer - Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.#0000
US, CA, Palo Alto
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLC, an Amazon.com CompanyTitle: Research Scientist IIWorksite: Palo Alto, CAPosition Responsibilities:Interact with various software and business groups to develop an understanding of their business requirements and operational processes. Utilize acquired knowledge and business judgment to build decision-supporting and operational tools to improve the bottom line. Build quantitative mathematical models to represent a wide range of supply chain, transportation and logistics systems. Implement these models and tools through the use of modeling languages and engineering code in software languages such as Java, C++, C# or C. Gather required data for analysis and mathematical model building by writing ad-hoc scripts and database queries. Perform quantitative, economic, and numerical performance analyses of these systems under uncertainty using statistical and optimization tools. Create computer simulations to support operational decision-making. Identify areas with potential for improvement and work with internal teams to generate requirements to realize improvements. Design optimal or near optimal solution methodologies to be used by in-house decision support tools and software. Create software prototypes to verify and validate the devised solutions methodologies. Integrate prototypes into production systems using standard software development tools and methodologies.Amazon.com is an Equal Opportunity-Affirmative Action Employer - Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.#0000
US, CA, Palo Alto
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLC, an Amazon.com CompanyTitle: Applied Scientist IIWorksite: Palo Alto, CAPosition Responsibilities:Participate in the design, development, evaluation, deployment and updating of data-driven models and analytical solutions for machine learning (ML) and/or natural language (NL) applications. Develop and/or apply statistical modeling techniques (e.g. Bayesian models and deep neural networks), optimization methods, and other ML techniques to different applications in business and engineering. Routinely build and deploy ML models on available data. Research and implement novel ML and statistical approaches to add value to the business. Mentor junior engineers and scientists.Amazon.com is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation #0000
US, WA, Seattle
LOCATION: Seattle, WAMULTIPLE POSITIONS AVAILABLE1. Perform analytical tasks on vast amounts of structured and unstructured data to extractactionable business insights2. Develop algorithms using advanced mathematical and statistical techniques includingmachine learning to predict business outcomes and recommend optimal actions tomanagement3. Use advanced algorithms to solve business problems, and transform one-off models intoautomated systems4. Analyze and validate data to ensure high data quality and reliable insights5. Run analytical experiments in a methodical manner to find opportunities for product andprocess optimization6. Communicate analytical output and business insights to management using visualizationtechniques and data storytelling7. Write concise documents communicating results to stakeholders and visualize data to drivedecision making#0000
US, Virtual
In Customer Trust and Partner Support (CTPS), thousands of Amazonians spend their days obsessing over how to continually provide a safe and trustworthy store for our customers, brands, and selling partners. In this rich, demanding environment, it is vital that we invest significant effort to hire and develop the best.How do we do this effectively? That’s where you come in. Help us build a brand new core research team focused on talent growth and retention.We are seeking a behavioral research scientist with deep expertise in quantitative and experimental approaches to social science.You're a good fit for the role if you have experience independently owning research in the realm of quantitative social science end to end in industry settings. This includes ideation of research design and analyses to answer ambiguous questions, leading analysis and data collection, and disseminating findings to peers, stakeholders, and leadership.You will be responsible for developing and testing theoretical frameworks, inventing novel ways to generate insights, and identifying and developing rigorous analyses for a large variety of data. This role requires deep knowledge of statistics, including linear and logistic regressions, hierarchical models, structural equation models, and similar.As a research leader in the organization, you will also be responsible for mentorship of others, including mentoring junior team members, and providing learning opportunities for other members of the organization (think tech talks, workshops, etc).A customer-obsessed, relentless curiosity is a must, as is commitment to the highest standards of methodological rigor that a given study allows. This role provides significant opportunity to influence how CTPS hires and develops the best.If you’re hungry to engage and empower Amazonians with your expertise in mixed methods research, let's talk.Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
LU, Luxembourg
Amazon opened its virtual doors in 1995 and strives to be the world’s most customer-centric company, where customers can find and discover anything they might want to buy online.The Amazon Logistics team is offering positions to creative big thinkers who are passionate about using data to direct decision making and solve complex and large-scale challenges. We are looking for individuals with a strong interest in improving one of the world’s most complex logistic systems. As part of the Amazon Logistics team, we design and build systems that support planning decisions, measure and manage the last mile transportation, real estate investments, and resource planning.As a Data Scientist, your main focus will be to perform analysis, build metrics, synthesize information, identify business opportunities, provide project direction, and communicate business and technical requirements within the team and across stakeholder groups. You are indispensable and own the quantification of business opportunities and identification of project direction prioritizing projects based on greatest return on investment. You consider the needs of day-to-day operations and insist on the standards required to build the network of tomorrow. Your job will straddle day-to-day decisions and strategic vision, you will assist in defining trade-offs and quantifying opportunities for a variety of projects. You will learn current processes, build metrics, educate diverse stakeholder groups, assist science groups in initial solution design, and audit all model implementation. A successful candidate in this position will have a background in communicating across significant differences, prioritizing competing requests, and quantifying decisions made. You will balance the trade-offs between customer experience, network speed, and cost, to measurably improve the network in a 3 to 6-month time-frame.This position requires superior analytic thinkers, who are able to quickly approach large ambiguous problems and apply their technical and statistical knowledge to identify opportunities for further research. The ideal candidate will demonstrate a deep understanding of identifying meaningful key performance indicators and building actionable metrics. This candidate is passionate about providing insights that drive larger initiatives and sees the undeniable value of a good metric. You should be able to independently mine and analyze data, and be able to use any necessary programming and statistical analysis software to do so. Successful candidates must thrive in fast-paced environments which encourage collaborative and creative problem solving and be able to measure and estimate risks.About our rewardsWe’ll expect you to go the extra mile, but we’ll also make sure you’re well rewarded. As well as a competitive salary and stock units, we offer a whole host of other benefits, including an employee discount.There are other, more intangible rewards too. Like our commitment to your development, a refreshing lack of hierarchy, the chance to work with some of the brightest minds in the industry, plenty of team spirit and an informal atmosphere – suits and ties are few and far between.Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. We welcome applications from all members of society irrespective of age, sex, disability, sexual orientation, race, religion or belief.
US, WA, Seattle
Do you want the excitement of experimenting with cutting edge machine learning, natural language processing, computer vision, and active learning models to solve real world problems at scale? Imagine experimenting with Deep Neural Networks as your daily job and imagine using your model outputs to affect the product discovery of the biggest e-tailer in the world. Imagine leading research inside of an Amazon team that is always looking to deploy creative solutions to real world problems in product discovery. Your research findings are directly related to Amazon’s Browse experience and impact millions of customers, ingesting images, text and all the structured and unstructured attributes in the Amazon catalog to drive true understanding of products at scale.The Amazon Browse Classification and Discovery team is seeking an Applied Scientist for developing ML systems that can help classify Amazon products into our catalog and build new experiences for improving customer discovery of products. You will be part of Browse AI team consisting of experienced Applied Scientist working on a new set of initiatives, building models and delivering them into the Amazon production ecosystem. Your efforts will build robust ensemble of ML systems that can drive classification of products with a high precision and recall, and scale to new marketplaces and languages. This problem is challenging due to sheer scale (billions of products in the catalog), diversity (products ranging from electronics to groceries to instant video across multiple languages) and multitude of input sources (millions of sellers contributing product data with different quality).We are looking for an experienced Applied Scientist who can develop best in class solutions. Your primary customers are Amazon shoppers who would thank you for correctly identifying products in our catalogs across countries and languages.The ideal Applied Scientist candidate has deep expertise in one or several of the following fields: Web search, Applied/Theoretical Machine Learning, Deep Neural Networks, Classification Systems, Clustering, Label Propagation, Natural Language Processing, Computer Vision, Active learning, and Artificial Intelligence. S/he has a strong publication record at top relevant academic venues and experience in launching products/features in the industry.
US, WA, Seattle
We are looking for a passionate and talented Applied Scientist who will collaborate with other scientists and engineers to leverage machine learning methods and algorithms for the modeling and analysis of data.You'll design and run experiments, research new algorithms, and find new ways of optimizing customer experience. Besides theoretical analysis and innovation, you will work closely with talented engineers to put your algorithms and models into practice. Your work will directly impact the trust customers place in Amazon.You are the ideal candidate if you are passionate about delivering experiences that delight customers and creating solutions that are robust. You thrive in ambiguous environments that require to find solutions to problems that have not been solved before. You leverage your exceptional technical expertise, a sound understanding of the fundamentals of Computer Science, and practical experience to creating reliable, scalable, and high performance products. Your strong communication skills enable you to work effectively with both business and technical partners. You have hands-on experience making the right decisions about technology, models and methodology choices. You strive for simplicity, and demonstrate significant creativity and high judgment backed by statistical proof.Responsibilities include:· Design, implement, test, deploy and maintain innovative data and machine learning solutions to further the customer experience.· Create experiments and prototype implementations of new learning algorithms and prediction techniques· Collaborate with scientists, engineers, product managers and business stakeholders to design and implement software solutions· Use machine learning best practices to ensure a high standard of quality for all of the team deliverables
US, GA, Virtual Location - Georgia
Are you interested in working on state-of-the-art problems in inventory placement, marketplace optimization, network design and fulfillment?The Inventory Planning and Control (IPC) Science team is responsible for the optimization of Amazon's current supply chain, designing systems for predictive modeling of inventory and in designing future supply-chain networks.We are looking for motivated scientists with strong leadership skills who desire to work at the intersection of machine learning, operations research, engineering development, and advanced large-inventory systems. Applications include adaptive systems for buying and pricing, seller economic modeling, real-time supply-chain optimization, robotics and control and dynamic resource management systems design.The ideal candidate is a motivated individual with strong skills in building robust, stochastic modeling systems for applications suited to their strong machine learning background including classification, synthetic controls, and robotics and marketplace dynamical systems.As a Research Scientist you will solve complex problems and design the next generation of algorithms to drive the optimization of Amazon’s Supply Chain. You will continue to contribute to the research community, by working with scientists across Amazon, as well as collaborating with academic researchers and publishing papers (www.aboutamazon.com/research).What you'll do:· · Designing and implementing new features and machine learned models, including the application of state-of-art models in support of buying, pricing and inventory placement and selection.· · Conducting and coordinating process development leading to improved and streamlined processes for model development.· · Working closely with Product Managers to expand depth of our product insights with data, create a variety of experiments, and determine the highest-impact projects to include in planning roadmaps· · Providing technical and scientific guidance to your team members· · Communicating effectively with senior management as well as with colleagues from science, engineering, and business backgrounds· · Build and validate prototypes to demonstrate the benefits from proposed changes to the decision systems and operations.· To help describe some of our challenges, we created a short video about Supply Chain Optimization at Amazon - http://bit.ly/amazon-scot Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation / Age
US, WA, Seattle
Amazon has the world’s richest collection of e-commerce data: Hundreds of millions of customers and products, billions of transactions, petabytes of data. We need your skills to take advantage of it in order to optimize our operations, with the end goal of optimizing the experience of our customers.Do you want to be involved in learning predictive models from huge volumes of past data such as customer transactions and interactions, characteristics of products and their relations, prices, and promotions? Are you interested in using Deep Learning, Reinforcement Learning and cutting edge ML techniques to optimize the supply chain and transportation? Do you want to develop infinitely-scalable algorithms for machine learning on the cloud that optimize Amazon’s systems using cutting edge quantitative techniques? Do you want to see your work have a huge impact to company’s results?AMXL seeks a Manager, Data Science with strong leadership and communication skills to join our team. You will lead a team of data scientists to help us discover and define problems, and apply quantitative solutions to problems to impact our core business. You will be working with cutting edge technologies that enable big data and parallelizable algorithms. You will play an active role in translating business and functional requirements into concrete deliverables.Responsibilities• Apply business judgement to identify opportunities and develop science strategies• Work collaboratively and effectively with engineering and business partners to build algorithms and models for demand forecasting• Lead a team of talented individuals• Hire and grow junior science leaders• Influence, collaborate, and communicate effectively with other engineering, science, and business leaders• Communicate scientific solution and insights effectively to a non-scientific audience
US, CA, Sunnyvale
Amazon is looking for a passionate, talented, and inventive Senior Scientist to help build industry-leading Speech and Language technology. Our mission is to push the envelope in Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), and Audio Signal Processing, in order to provide the best-possible experience for our customers.As a Speech and Language Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in spoken language understanding. Your work will directly impact our customers in the form of novel products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding. You will mentor junior scientists, create and drive new initiatives.We are hiring in all areas of spoken language understanding: ASR, NLU, text-to-speech (TTS), and Dialog Management.
CN, Beijing
Why this job is awesome?· · This is SUPER high-visibility work: Our mission is to provide consistent, accurate, and relevant delivery information to every single page on every Amazon-owned site.· · MILLIONS of customers will be impacted by your contributions: The changes we make directly impact the customer experience on every Amazon site. This is a great position for someone who likes to leverage Machine learning technologies to solve the real customer problems, and also wants to see and measure their direct impact on customers.· · We are a cross-functional team that owns the ENTIRE delivery experience for customers: From the business requirements to the technical systems that allow us to directly affect the on-site experience from a central service, business and technical team members are integrated so everyone is involved through the entire development process.· - Do you want to join an innovative team of scientists and engineers who use machine learning and statistical techniques to deliver the best delivery experience on every Amazon-owned site?- Are you excited by the prospect of analyzing and modeling terabytes of data on the cloud and create state-of-art algorithms to solve real world problems?- Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company?- Do you like to innovate and simplify?If yes, then you may be a great fit to join the Delivery Experience Machine Learning team.Major responsibilities:· Lead a ML team to research and implement machine learning and statistical techniques to create scalable and effective models in Delivery Experience (DEX) systems· Solve business problems and to identify business opportunities to provide the best delivery experience on all Amazon-owned sites.· Design, development and evaluation of highly innovative machine learning models for big data.· Analyzing and understanding large amounts of Amazon’s historical business data to detect patterns, to analyze trends and to identify correlations and causalities· Working closely with other software engineering teams to drive real-time model implementations and new feature creations· Establishing scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation
US, CA, Sunnyvale
Amazon is looking for a passionate, talented, and inventive Senior Applied Scientist to help build industry-leading Speech and Language technology. Our mission is to push the envelope in Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), and Audio Signal Processing, in order to provide the best-possible experience for our customers.As a Speech and Language Scientist, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in spoken language understanding. Your work will directly impact our customers in the form of novel products and services that make use of speech and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in spoken language understanding. You will mentor junior scientists, create and drive new initiatives.We are hiring in all areas of spoken language understanding: ASR, NLU, text-to-speech (TTS), and Dialog Management.
US, WA, Seattle
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLCPosition: Research Scientist IIILocation: Seattle, WAPosition Responsibilities:Interact with various software and business groups to develop an understanding of their business requirements and operational processes. Utilize acquired knowledge and business judgment to build decision-supporting and operational tools to improve the bottom line. Build quantitative mathematical models to represent a wide range of supply chain, transportation and logistics systems. Implement these models and tools through the use of modeling languages and engineering code in software languages such as Java, C++, C# or C. Gather required data for analysis and mathematical model building by writing ad-hoc scripts and database queries. Perform quantitative, economic, and numerical performance analyses of these systems under uncertainty using statistical and optimization tools. Create computer simulations to support operational decision-making. Identify areas with potential for improvement and work with internal teams to generate requirements to realize improvements. Design optimal or near optimal solution methodologies to be used by in-house decision support tools and software. Create software prototypes to verify and validate the devised solutions methodologies. Integrate prototypes into production systems using standard software development tools and methodologies.Amazon.com is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation #0000
US, WA, Seattle
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLC, an Amazon.com companyTitle: Data Scientist IILocation: Seattle, WAPosition Responsibilities:Manage metrics reporting, perform data mining and big data analysis, to provide strategic advice on business and improve processes and systems. Collect business use cases, research and evaluate opportunities, and leverage data to support business functions through advanced mathematical modeling. Independently own and solve business problems. Transform business problems into mathematical models, provide data-driven solutions, and write clear and factually correct documents explaining technical concept to non-technical audience. Gather and use complex data sets across domains and proactively gather data when not readily available. Use specialized modelling software and scripting languages to scale methods. Partner with business, science, and engineering to establish scalable, efficient, automated processes for large scale data analyses, model development, model validation, and model implementation. Building and tuning machine learning models.Amazon.com is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation #0000
US, CA, Sunnyvale
Amazon Lab126 is an inventive research and development company that designs and engineers high-profile consumer electronics. Lab126 began in 2004 as a subsidiary of Amazon.com, Inc., originally creating the best-selling Kindle family of products. Since then, we have produced groundbreaking devices like Fire tablets, Fire TV, and Amazon Echo. What will you help us create?The Role:We are a smart team of doers that work passionately to apply cutting-edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even imagine yet. Key responsibilities will be to conduct research and development in computer vision and related disciplines, and to collaborate with cross-functional engineering teams, including Amazon Robotics, to put the concepts you develop into production. You will determine where commercially available solution and academic research can be applied to solve Amazon business problems, as well as identify opportunities for innovation. You will use a large amount of data to test algorithms to bring them up to production level qualityIf this describes you, come join our team at Lab126 in the heart of Silicon Valley. The team is using computer vision, sensor fusion, real-time and distributed systems to convert requirements into concrete deliverables. A Researcher on this team will translate business and functional requirements into working code. Comfort with a high degree of ambiguity and ability to solve problems that haven’t been solved to scale before are essential.Key responsibilities will be to conduct research and development in computer vision and related disciplines, and to collaborate with cross-functional engineering teams to put the concepts you develop into production. You will determine where commercially available solution and academic research can be applied to solve Amazon business problems, as well as identify opportunities for innovation. You will use a large amount of data to train and test algorithms to bring them up to production level quality. In this role, you will:· Research, design, implement and evaluate novel algorithms· Work on large-scale datasets, focusing on creating scalable and accurate computer vision systems in versatile application fields· Collaborate closely with team members on developing systems from prototyping to production level· Collaborate with teams spread all over the world· Work closely with software engineering teams to drive scalable, real-time implementations· Track general business activity and provide clear, compelling management reports on a regular basis
US, CA, Culver City
Prime Video is an industry leading, high-growth business and a critical driver of Amazon Prime subscriptions, which contribute to customer loyalty and lifetime value. Prime Video is used daily by a massive audience on Amazon's websites and through a variety of devices including the Kindle Fire, game consoles, smart TVs and Blu-ray players.The Prime Video Marketing Analytics team uses machine learning, econometrics, and data science to optimize Amazon’s media spending strategies on Amazon Originals and third party content, driving customer loyalty as well as enhancing lifetime value of our Prime members. We are looking for an applied data scientist to build innovative models for measuring the impact of marketing spending on customer engagement and Prime member acquisition. Key responsibilities of Prime Video applied scientists include the following:· Playing an integral role in developing a roadmap to expand and enhance marketing analytics of Prime Video· Optimizing media planning on Amazon Originals to grow Amazon Prime user engagement and acquisition· Improving model usability by analyzing customer behavior and by gathering data from business owners and other tech teams· Incorporating new data sources and implementing creative methodology innovations to improve model performance· Creating and tracking accuracy and performance metrics· Helping build production systems that take inputs from multiple models and support decision makingTo summarize, the applied scientist will join our team to generate scientific insights to guide Amazon’s digital-video marketing strategy. We use detailed customer behavioral data (e.g. streaming history) and detailed information about content (e.g. IMDb-sourced characteristics) to discern causality of advertising spending on customer engagement and Prime member acquisition, based on which we derive optimize media planning of multi-million dollar marketing budget.
US, WA, Seattle
Interested in modeling and understanding customer behavior through machine learning, artificial intelligence, and data mining over TB scale data with huge business impact on millions of customers? Join our team of Scientists and Engineers developing models to predict customer behavior and optimize the customer experience with Amazon Prime. This includes identifying who our customers are and providing them with personalized relevant content. As an ML expert, you will partner directly with product owners to intake, build, and directly apply your modeling solutions.There are numerous scientific and technical challenges you will get to tackle in this role, such as global scalability of models, combinatorial optimization, cold start problem, accelerated experimentation, short/long term goals modeling, cohort identification in a semi-supervised setting, and multi-step optimization leading to reinforcement learning of the customer journey. We employ techniques from supervised learning, bandits, optimization, and RL.As the central science team within Prime, our expertise gets routinely called upon to weigh in on a variety of topics. We also emphasize the need and value of scientific research and have developed a strong publication and patent record (internally/externally) which you will be a part of.You will also utilize and be exposed to the latest in ML technologies and infrastructure: AWS technologies (EMR/Spark, Redshift, Sagemaker, DynamoDB, S3, ...), various ML algorithms and techniques (XGBoost, Random Forests, Neural Networks, supervised/unsupervised/semi-supervised/reinforcement learning), and statistical modeling techniques.Major responsibilities· Build and develop machine learning models and supporting infrastructure at TB scale, in coordination with software engineering teams.· Leverage Bandits and Reinforcement Learning for Recommendation Systems.· Develop offline policy estimation tools and integrate with reporting systems.· Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation.· Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes.· Work closely with the business to understand their problem space, identify the opportunities and formulate the problems.· Use machine learning, data mining, statistical techniques and others to create actionable, meaningful, and scalable solutions for the business problems.· Design, develop and evaluate highly innovative models and statistical approaches to understand and predict customer behavior and to solve business problems.
US, WA, Seattle
Amazon Prime Video is changing the way millions of customers enjoy digital content. Prime Video delivers premium content to customers through purchase and rental of movies and TV shows, unlimited on-demand streaming through Amazon Prime subscriptions, add-on channels like Showtime and HBO, and live concerts and sporting events like NFL Thursday Night Football. In total, Prime Video offers nearly 200,000 titles and is available across a wide variety of platforms, including PCs and Macs, Android and iOS mobile devices, Fire Tablets and Fire TV, Smart TVs, game consoles, Blu-ray players, set-top-boxes, and video-enabled Alexa devices. Amazon believes so strongly in the future of video that we've launched our own Amazon Studios to produce original movies and TV shows, many of which have already earned critical acclaim and top awards, including Oscars, Emmys and Golden Globes.The Global Consumer Engagement team within Amazon Prime Video builds product and technology solutions that drive customer activation and engagement across all our supported devices and global footprint. We obsess over finding effective, programmatic and scalable ways to reach customers via a broad portfolio of both in-app and out-of-app experiences.We would love to have you join us to work on the core video optimization systems. You will hone your skills in areas such as deep learning, multi-armed bandits, and reinforcement learning while building scalable industrial systems. As a member of a highly leveraged team of talented engineers and ML scientists, you will have a unique opportunity to help determine what content gets shown to every customer on Amazon.Our team consistently strives to innovate, and holds several novel patents and inventions in the motion picture and television industry. We are highly motivated to extend the state of the art. As a member of our team, you will apply your deep knowledge of Data Science and Machine Learning to concrete problems that have broad cross-organizational, global, and technology impact. You will work on large engineering efforts that solve significantly complex problems facing global customers. You will be trusted to operate with independence and are often assigned to focus on areas with significant impact on audience satisfaction. You must be equally comfortable with digging in to customer requirements as you are drilling into design with development teams and developing production ready learning models. You consistently bring strong, data-driven business and technical judgment to decisions.You will work with internal and external stakeholders, cross-functional partners, and end-users around the world at all levels. Our team makes a big impact because nothing is more important to us than pleasing our customers, continually earning their trust, and thinking long term. You are empowered to bring new technologies and deep learning approaches to your solutions. We embrace the challenges of a fast paced market and evolving technologies, paving the way to universal availability of content. You will be encouraged to see the big picture, be innovative, and positively impact millions of customers. This is a young and evolving business where creativity and drive will have a lasting impact on the way video is enjoyed worldwide.Amazon is an Equal Opportunity Employer – Minority / Women / Disability / Veteran / Gender Identity / Sexual Orientation / Age