DeepRacer: Autonomous racing platform for experimentation with Sim2Real reinforcement learning

2020
Download Copy BibTeX
Copy BibTeX
DeepRacer is a platform for end-to-end experimentation with RL and can be used to systematically investigate the key challenges in developing intelligent control systems. Using the platform, we demonstrate how a 1/18th scale car can learn to drive autonomously using RL with a monocular camera. It is trained in simulation with no additional tuning in the physical world and demonstrates: 1) formulation and solution of a robust reinforcement learning algorithm, 2) narrowing the reality gap through joint perception and dynamics, 3) distributed on-demand compute architecture for training optimal policies, and 4) a robust evaluation method to identify when to stop training. It is the first successful large-scale deployment of deep reinforcement learning on a robotic control agent that uses only raw camera images as observations and a model-free learning method to perform robust path planning. We open source our code and video demo on GitHub.
Research areas

Latest news

US, WA, Seattle
Success in any organization begins with its people and having a comprehensive understanding of our workforce and how we best utilize their unique skills and experience is paramount to our future success. AWS is looking for an exceptional Senior Data Scientist with ML expertise to join the Workforce Planning team. This Senior Data Scientist has proficiency in employing workforce and human behavior insights, operations research and machine learning techniques to build models and algorithms that enable the acceleration of revenue growth, improved operational efficiencies & delivery. Ultimately, results will be delivered through a workforce plan of optimized size and level in service of predictably accelerating product innovation, and enhancing the customer experience. Moreover, this Senior Data Scientist will work in partnership with the core WFP team that requires thought leadership to advance WFP content across AWS. This individual has proficiency in employing workforce and human behavior insights, strong business operations fundamentals and good judgement. In this role, you will be responsible for using operational and human capital data and leveraging machine learning methods to map enterprise strategies into actionable delivery plans, guiding data driven business decisions that results in predicting outcomes, understanding complex data relationships, and developing a quantitative return on investment. You will work closely with the business and technology teams. The ideal Senior Data Scientist has a strong sense of ownership, is self-driven, loves breaking new ground. You will bring a mix of experience including complex program management, cross-functional collaboration, strategic thinking, technical expertise, and process improvement. If you enjoy working in a fast-paced dynamic environment and being challenged by new problems, we’d like to speak with you! Key job responsibilities Strategic Analytics & Consultative Guidance: o Develop the next generation of Workforce ML Predictive Analytics & Forecasting models providing simulations, what-if analytics, and prescriptive analytics functionality o Determine the correct usage of core modelling techniques, and their applicability to the available data and use cases of the Workforce Planning team o Facilitate business case development and prioritization of opportunities based on ROI & feasibility assessment. o Provide business executives and stakeholders with thought-leadership and insights to enable continuous improvement across key financial, performance, and consumer metrics. o Partner with key stakeholders to create accurate financial forecasts and identify key drivers impacting performance versus benchmarks o Directly work alongside Data Science team to scope and develop advanced autonomous Machine Learning systems to further improved existing forecasting and prediction capabilities within WFP o Present critical information in a format that is immediately useful to answer questions about the inputs and outputs of Forecasting systems and improving their performance. Project Management: Manage analytical projects by leveraging agile methodologies to partner across business functions, meet deliverables, and actively champion solution adoption across the business. Requirements Gathering: Conduct interviews with key business stakeholders to translate business objectives into analytical project deliverables to realize business goals. Data Preparation: Write high quality code and organize data structures to efficiently drive scalable workflows and analyze data. Data Governance: Develop, maintain and perform processes to continuously monitor data quality and integrity We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
US, CA, Santa Clara
Machine learning (ML) has been strategic to Amazon from the early years. We are pioneers in areas such as recommendation engines, product search, eCommerce fraud detection, and large-scale optimization of fulfillment center operations. The Generative AI team helps AWS customers accelerate the use of Generative AI to solve business and operational challenges and promote innovation in their organization. As an applied scientist, you are proficient in designing and developing advanced ML models to solve diverse challenges and opportunities. You will be working with terabytes of text, images, and other types of data to solve real-world problems. You'll design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. We’re looking for talented scientists capable of applying ML algorithms and cutting-edge deep learning (DL) and reinforcement learning approaches to areas such as drug discovery, customer segmentation, fraud prevention, capacity planning, predictive maintenance, pricing optimization, call center analytics, player pose estimation, event detection, and virtual assistant among others. Key job responsibilities The primary responsibilities of this role are to: • Design, develop, and evaluate innovative ML models to solve diverse challenges and opportunities across industries • Interact with customer directly to understand their business problems, and help them with defining and implementing scalable Generative AI solutions to solve them • Work closely with account teams, research scientist teams, and product engineering teams to drive model implementations and new solutions A day in the life About Us Inclusive Team Culture Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life Balance Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. About the team Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future. We are open to hiring candidates to work out of one of the following locations: San Francisco, CA, USA | San Jose, CA, USA | Santa Clara, CA, USA
US, MA, Boston
Machine learning (ML) has been strategic to Amazon from the early years. We are pioneers in areas such as recommendation engines, product search, eCommerce fraud detection, and large-scale optimization of fulfillment center operations. The Generative AI team helps AWS customers accelerate the use of Generative AI to solve business and operational challenges and promote innovation in their organization. As an applied scientist, you are proficient in designing and developing advanced ML models to solve diverse challenges and opportunities. You will be working with terabytes of text, images, and other types of data to solve real-world problems. You'll design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. We’re looking for talented scientists capable of applying ML algorithms and cutting-edge deep learning (DL) and reinforcement learning approaches to areas such as drug discovery, customer segmentation, fraud prevention, capacity planning, predictive maintenance, pricing optimization, call center analytics, player pose estimation, event detection, and virtual assistant among others. Key job responsibilities The primary responsibilities of this role are to: • Design, develop, and evaluate innovative ML models to solve diverse challenges and opportunities across industries • Interact with customer directly to understand their business problems, and help them with defining and implementing scalable Generative AI solutions to solve them • Work closely with account teams, research scientist teams, and product engineering teams to drive model implementations and new solutions A day in the life About Us Inclusive Team Culture Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life Balance Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. About the team Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future. We are open to hiring candidates to work out of one of the following locations: Boston, MA, USA | San Francisco, CA, USA | San Jose, CA, USA | Santa Clara, CA, USA
US, VA, Arlington
Device Economics (DEcon) is looking for an economist experienced in causal inference, empirical industrial organization, forecasting, and scaled systems to work on business problems to advance critical resource allocation and pricing decisions in the Amazon Devices org. Output will be included in scaled systems to automate existing processes and to maximize business and customer objectives. Amazon Devices designs and builds Amazon first-party consumer electronics products to delight and engage customers. Amazon Devices represents a highly complex space with 100+ products across several product categories (e-readers [Kindle], tablets [Fire Tablets], smart speakers and audio assistants [Echo], wifi routers [eero], and video doorbells and cameras [Ring and Blink]), for sale both online and in offline retailers in several regions. The space becomes more complex with dynamic product offering with new product launches and new marketplace launches. The Device Economics team leads in analyzing these complex marketplace dynamics to enable science-driven decision making in the Devices org. Device Economics achieves this by combining economic expertise with macroeconomic trends, and including both in scientific applications for use by internal analysts, to provide deep understanding of customer preferences. Our team’s outputs inform product development decisions, investments in future product categories, product pricing and promotion, and bundling across complementary product lines. We have achieved substantial impact on the Devices business, and will achieve more. Device Economics seeks an economist adept in measuring customer preferences and behaviors with proven capacity to innovate, scale measurement, and drive rigor. The candidate will work with Amazon Devices science leadership to refine science roadmaps, models, and priorities for innovation and simplification, and advance adoption of insights to influence important resource allocation and prioritization decisions. Effective communication skills (verbal and written) are required to ensure success of this collaboration. The candidate must be passionate about advancing science for business and customer impact. We are open to hiring candidates to work out of one of the following locations: Arlington, VA, USA | Seattle, WA, USA
US, WA, Seattle
About Amazon Advertising: Amazon Advertising operates at the intersection of eCommerce and advertising, offering a rich array of digital display advertising solutions with the goal of helping our customers find and discover anything they want to buy. We help advertisers of all types to reach Amazon customers on Amazon.com, across our other owned and operated sites, on other high quality sites across the web, and on millions of Kindles, tablets, and mobile devices. We start with the customer and work backwards in everything we do, including advertising. If you’re interested in joining a rapidly growing team working to build a unique, world-class advertising group with a relentless focus on the customer, you’ve come to the right place. About our team: Sponsored Display (SD) is Amazon Advertising’s first self-service display advertising offering. It empowers any advertiser – regardless of budget size, advertising experience, or technical expertise – to set up display campaigns that show ads on key placements across Amazon websites and apps (Amazon.com, Twitch, IMDb), devices (Kindle, Fire Tablet, Fire TV, Echo), third party websites (e.g., nytimes.com) and mobile apps. About the role: As science leader on this team you will be involved in every aspect of the process - from idea generation, business analysis and scientific research, through to development and deployment of advanced models - giving you a real sense of ownership. The systems that you help to build will operate at massive scale to display ads to customers around the world. From day one, you will be working with experienced scientists, engineers, and designers who love what they do. We are looking for Senior Applied Scientist who can help us take our products to the next level who has deep passion for building machine-learning solutions, ability to communicate data insights and scientific vision, and a proven track record of execute complex projects. As an Senior Applied Scientist in Sponsored Display, you will: - Conduct hands-on data analysis, build large-scale machine-learning models and pipelines - Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production - Run regular A/B experiments, gather data, perform statistical analysis, and communicate the impact to senior management - Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving - Provide technical leadership, research new machine learning approaches to drive continued scientific innovation - Be a member of the Amazon-wide Machine Learning Community, participating in internal and external seminars and conferences - Help attract and recruit technical talent We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the extreme. We focus on creating entirely new products and services with a goal of positively impacting the lives of our customers. No industries or subject areas are out of bounds. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. As an Associate Scientist, you will work with a unique and gifted team developing exciting products for consumers and collaborate with cross-functional teams. Our team rewards intellectual curiosity while maintaining a laser-focus in bringing products to market. At the cutting edge of both academic and applied research in this product area, you have the opportunity to work together with some of the most talented scientists, engineers, and product managers. Here at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have thirteen employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We are constantly learning through programs that are local, regional, and global. Amazon’s culture of inclusion is reinforced within our 16 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Our team highly values work-life balance, mentorship and career growth. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We care about your career growth and strive to assign projects and offer training that will challenge you to become your best. We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
US, WA, Seattle
Amazon’s Global Media and Entertainment (GME) organization is creating the future of entertainment where creative content, innovation, and commerce come together. We leverage Amazon’s unique expertise across video, music, gaming, and more to create a truly immersive entertainment experience. Our team, GME Economics, is focused on building strategic science tools to optimize Amazon’s entertainment offerings, so that we can provide a great customer experience while operating as a sustainable and profitable business. This role will build and expand long-term measurement models serving all GME businesses, making strong causal inference skills essential. The GME business context is complex, ambiguous, and can change quickly. You'll need to be adaptable and resourceful, converting complex business problems into estimable econometric problems, and building solutions that anticipate future business needs. Once the model is built, you'll partner hands-on with engineers to productionize it, ensuring the output is high quality and that computation is efficient. Our team values excellent written and verbal communication with partners of all levels, with technical and nontechnical backgrounds. You'll need to translate data and findings into actionable insights for leaders with the same skill that you explain the nuances of your identification assumptions to scientists, or describe the logical flow of your estimations to engineers. Our healthy team culture is supportive and fast-paced, and we prioritize learning, growth, and helping each other to continuously raise the bar. We think big, taking on high-risk, high-reward projects based on novel scientific approaches. As a central science team, we work hard to build trust with partners, who always have fresh problems for us to tackle. Impact and Career Growth: In today’s entertainment landscape, critical decisions are made with data and economic models. You’ll help leaders ask the right questions, and then deliver data-driven answers, creating the future of GME at Amazon. You’ll help define a long-term science vision and translate it into an actionable roadmap. This role combines science leadership, organizational ability, technical strength, product focus, and business understanding – a perfect recipe for career growth as an economist in tech. Key job responsibilities • Design and build econometric models, especially causal models, to measure the value of the business and its many features • Develop science products from concept to prototype to production, incorporating feedback from scientists and business partners • Independently identify and pursue new opportunities to leverage economic insights across GME businesses • Collaborate with PMs to build product roadmaps, ensuring our work meets the needs of our stakeholders and then communicating progress and findings to them • Write business and technical documents communicating business context, methods, and results to business leadership and other scientists • Work with engineers within and outside our org to productionize science models • Serve as a technical reviewer for our team and related teams, including document and code reviews We are open to hiring candidates to work out of one of the following locations: Los Angeles, CA, USA | New York, NY, USA | Seattle, WA, USA
US, IL, Chicago
Do you want to use your expertise in translating innovative science into impactful products to improve the lives and work of over a million people worldwide? If you do, People eXperience Technology Central Science (PXTCS) would love to talk to you about how to make that a reality. PXTCS is an interdisciplinary team that uses economics, behavioral science, statistics, and machine learning to identify products, mechanisms, and process improvements that both improve Amazonian’s wellbeing and their ability to deliver value for Amazon’s customers. We work with HR teams across Amazon to make Amazon PXT the most scientific human resources organization in the world. As an applied scientist on our team, you will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, define the science vision and translate it into specific plans for applied scientists, as well as engineering and product teams. You will partner with scientists, economists, and engineers on the design, development, testing, and deployment of scalable ML and econometric models. This is a unique, high visibility opportunity for someone who wants to have impact, dive deep into large-scale solutions, enable measurable actions on the employee experience, and work closely with scientists and economists. This role combines science leadership, organizational ability, and technical strength. Key job responsibilities As an Applied Scientist, ML Applications, you will: • Design, develop, and evaluate innovative machine learning solutions to solve diverse challenges and opportunities for Amazon customers • Advance the team's engineering craftsmanship and drive continued scientific innovation as a thought leader and practitioner. • Partner with the engineering team to deploy your models in production. • Partner with scientists from across PXTCS to solve complex problems and use your team’s expertise to accelerate their ability get their work into production. • Work directly with Amazonians from across the company to understand their business problems and help define and implement scalable ML solutions to solve them. We are open to hiring candidates to work out of one of the following locations: Austin, TX, USA | Chicago, IL, USA | Seattle, WA, USA
US, IL, Chicago
Do you want to use your expertise in translating innovative science into impactful products to improve the lives and work of over a million people worldwide? If you do, People eXperience Technology Central Science (PXTCS) would love to talk to you about how to make that a reality. PXTCS is an interdisciplinary team that uses economics, behavioral science, statistics, and machine learning to identify products, mechanisms, and process improvements that both improve Amazonian’s wellbeing and their ability to deliver value for Amazon’s customers. We work with HR teams across Amazon to make Amazon PXT the most scientific human resources organization in the world. As an applied scientist on our team, you will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, define the science vision and translate it into specific plans for applied scientists, as well as engineering and product teams. You will partner with scientists, economists, and engineers on the design, development, testing, and deployment of scalable ML and econometric models. This is a unique, high visibility opportunity for someone who wants to have impact, dive deep into large-scale solutions, enable measurable actions on the employee experience, and work closely with scientists and economists. This role combines science leadership, organizational ability, and technical strength. Key job responsibilities As an Applied Scientist, ML Applications, you will: • Lead applied scientists to deliver machine-learning and AI solutions to production. • Design, develop, and evaluate innovative machine learning solutions to solve diverse challenges and opportunities for Amazon customers • Advance the team's engineering craftsmanship and drive continued scientific innovation as a thought leader and practitioner. • Partner with the engineering team to deploy your models in production. • Partner with scientists from across PXTCS to solve complex problems and use your team’s expertise to accelerate their ability get their work into production. • Work directly with Amazonians from across the company to understand their business problems and help define and implement scalable ML solutions to solve them. • Mentor and develop junior scientists and developers. We are open to hiring candidates to work out of one of the following locations: Austin, TX, USA | Chicago, IL, USA | Seattle, WA, USA
DE, BE, Berlin
Have you ever wondered how Amazon delivers timely and reliably hundreds of millions of packages to customer’s doorsteps? Are you passionate about data and mathematics, and hope to impact the experience of millions of customers? Are you obsessed with designing simple algorithmic solutions to very challenging problems? If so, we look forward to hearing from you! Amazon Transportation Services is seeking Applied (or Research) Scientists. As a key member of the central Research Science Team of ATS operations, these persons will be responsible for designing algorithmic solutions based on data and mathematics for optimizing the middle-mile Amazon transportation network. The job is opened in the EU Headquarters in Luxembourg (alternatively: Barcelona, Berlin or London), designed to maximize interaction with the team and stakeholders, but we will consider applicants with remote work requirements as well. Basic qualifications * PhD in Operations Research, Machine Learning, Statistics, Applied Mathematics, Computer Science or other field related to algorithms and data (or equivalent experience). * Excellent written and verbal communication skills. * Experience with some programming language (Java/python/C++) * Research experience in one or more: *Combinatorial optimization problems (e.g., scheduling, vehicle routing, facility location). *Continuous optimization problems (e.g., linear programming, convex programming, non-convex programming). *Predictive analytics (e.g., forecasting, time-series, neural networks) *Prescriptive analytics (e.g., stochastic optimization, bandits, reinforcement learning). Preferred qualifications * Experience from working in a fast-paced applied research environment. * Ability to handle ambiguity. * Top tier publications pertinent to the field of study. Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice (https://www.amazon.jobs/en/privacy_page) to know more about how we collect, use and transfer the personal data of our candidates. Key job responsibilities Solve complex optimization and machine learning problems using scalable algorithmic techniques. Design and develop efficient research prototypes that address real-world problems in the middle-mile operations of Amazon. Lead complex time-bound, long-term as well as ad-hoc analyses to assist decision making. Communicate to leadership results from business analysis, strategies and tactics. A day in the life You will be brainstorming algorithmic approaches with team-mates to solve challenging problems for the middle-mile operations of Amazon. You will be developing and testing prototype solutions with above algorithmic techniques. You will be scavenging information from the sea of Amazon data to improve these solutions. You will be meeting with other scientists, engineers, stakeholders and customers to enhance the solutions and get them adopted. About the team The Science and Tech team of ATS EU is looking for candidates who are looking to impact the world with their mathematical and data-driven skills. ATS stands for Amazon Transportation Service, we are the middle-mile planners: we carry the packages from the warehouses to the cities in a limited amount of time to enable the “Amazon experience”. As the core research team, we grow with ATS business to support decision making in an increasingly complex ecosystem of a data-driven supply chain and e-commerce giant. We schedule more than 1 million trucks with Amazon shipments annually; our algorithms are key to reducing CO2 emissions, protecting sites from being overwhelmed during peak days, and ensuring a smile on Amazon’s customer lips. Our mathematical algorithms provide confidence in leadership to invest in programs of several hundreds millions euros every year. Above all, we are having fun solving real-world problems, in real-world speed, while failing & learning along the way. We use modular algorithmic designs in the domain of combinatorial optimization, solving complicated generalizations of core OR problems with the right level of decomposition, employing parallelization and approximation algorithms. We use deep learning, bandits, and reinforcement learning to put data into the loop of decision making. We like to learn new techniques to surprise business stakeholders by making possible what they cannot anticipate. For this reason, we work closely with Amazon scholars and experts from Academic institutions. We code our prototypes to be production-ready We prefer provably optimal solutions than heuristics, though we settle for heuristics when performance dictates it. Overall, we appreciate the value of correct modeling. We are open to hiring candidates to work out of one of the following locations: Berlin, BE, DEU