NASA's Orion spacecraft shown splashing down in the Pacific Ocean, west of Baja California, at 9:40 a.m. PST Sunday, Dec. 11.
NASA's Orion spacecraft shown splashing down in the Pacific Ocean, west of Baja California, at 9:40 a.m. PST Sunday, Dec. 11.
NASA

The story behind how Amazon integrated Alexa into NASA’s Orion spacecraft

From physical constraints to acoustic challenges, learn how Amazon collaborated with NASA and Lockheed Martin to get Alexa to work in space.

In September 2018, Amazon’s principal solutions architect Philippe Lantin received a call from his manager.

“He said that there was something unique on the horizon, and that their team was being roped into a one-in-a-lifetime opportunity,” says Lantin.

This was no understatement: on the horizon was an opportunity for Amazon to collaborate with Lockheed Martin Space, and integrate Alexa into NASA’s Orion spacecraft. Orion is the first human-rated spacecraft to visit the moon in more than 40 years.

“NASA is trying to engage the public more as we enter this new era of space travel, where we are setting the stage for extra-planetary exploration,” says Lantin. “Given that over 100 million Alexa-enabled devices have already been sold, having Alexa answer questions like 'Alexa, how far to the moon?' and 'Alexa, how fast is Orion going?' is a great way to get people around the world involved in NASA’s missions.”

Setting up an Echo device on Earth is simple: all you need is a Wi-Fi connection and the Alexa app. However, things are far more complicated in space.

“We had several constraints we had to contend with,” says Lantin.
The Alexa team had to operate within a key physical constraint: the shape of the device. The contours of a smart speaker greatly influences it acoustics. To give just one example, the round shape of the Echo Dot offers a full cavity behind the woofer for a better bass response.

Related content
NASA is using unsupervised learning and anomaly detection to explore the extreme conditions associated with solar superstorms.

However, when it came to NASA’s Orion spacecraft, Alexa’s acoustic engineers had to work with what was provided by Lockheed Martin and NASA.

“We were somewhat limited by the form factor, which was a small briefcase-like enclosure that was 1.5 feet by one foot and about five inches in depth.” says Lantin.

There were other physical constraints. Equipment developed for the mission had to be resilient to extreme shocks and vibrations, be at least minimally resistant to radiation emissions in space, and utilize highly specific and custom-built components such as power and data cables.

Limited Internet connectivity

The team also had to deal with issues related to the lack of Internet connectivity. Typically, Echo devices use on-device keyword spotting designed to detect when a customer says the wake word. This on-device buffer exists in temporary memory. After the wake word is detected, the device streams the audio to the cloud for speech recognition and natural language processing.

Orion components

“However, for the Orion mission, our ability to communicate with the Alexa cloud was severely constrained,” says Lantin. “NASA’s spacecraft uses the Deep Space Network to communicate with earth. The bandwidth available to us on the downlink connection is slightly better than dial-up modem speeds with latencies of up to five seconds. To further complicate matters, NASA prioritizes traffic for navigation and telemetry for the first payload — traffic for Alexa was consigned to the secondary payload.”

The team also wanted to demonstrate a fully autonomous experience, one that can be used in future missions where Earth connectivity is no longer a practical option for real-time communications. They used Alexa Local Voice Control to get around the limited internet connectivity. Alexa Local Voice control allows select devices to process voice commands locally, rather than sending information to the cloud.

Lantin says that while the team was motivated by demonstrating technology leadership and scientific innovation in a very challenging environment, the real motivator was making a difference in the lives of millions of customers at home on earth.

“At Amazon, we take pride in delivering customer-focused science,” says Lantin. “That was a huge motivator for us at every step along the way. Consider the innovations we drove to Alexa Local Voice Control. These improvements will allow people on earth to do so much more with Alexa in situations where they have limited or no Internet connectivity. Think about when you are in a car and passing through a tunnel, or driving to a remote camping site. You can do things like tune the radio, turn on the AC and continue to use voice commands, even if you have a feeble signal or no cellular connection.”

Lantin says that the acoustic innovations enabled for Orion will also translate directly into improved listening experiences for people interacting with the mission on earth.

Rohit Prasad, Alexa senior vice president and head scientist, on the initial collaboration with Lockheed Martin

“We are planning to have celebrities, politicians, STEM students and a variety of other personalities interacting with Alexa,” says Lantin. “ And so, we also spent a good deal of time thinking about what people might want to ask Alexa about during the mission.”

The nuances of acoustics aboard Orion

Scott Isabelle is a solutions architect at Amazon. Prior to Amazon, Isabelle was a distinguished member of the technical staff at Motorola, where among other projects, he developed systems for enhancing voice quality in mobile devices, methods for generating adaptive ringtones, and a two-microphone system for noise suppression.

“One of the most important things for a voice AI is being in an environment where it is able to pick up your voice,” says Isabelle.

Related content
Parallel processing of microphone inputs and separate detectors for periodicity and dynamics improve performance.

However, this is easier said than done on Orion, where the conical shape of the space capsule, and its metallic surfaces result in increased reverberation.

“The voice can keep bouncing around losing very little energy. This wouldn’t happen in a typical room where soft material like curtains and sofa cushions can absorb some of the sound. In the capsule, the reverberations off the metal surfaces can play up the wrong frequencies that are critical to automatic speech recognition. This can make it really difficult for Alexa to pick up wake word invocations. ”

Alexa also has to contend with increased noise levels aboard Orion.

NASA | Exploration Mission-1 — pushing farther into deep space

The ideal signal to noise ratio (SNR) for systems involving intelligent voice assistants is in the range 20 to 30 decibels (dB). To place this in context, a SNR of 35 dB is what you would find in a face-to-face conversation between two people standing one meter apart in a typical room (higher SNRs are better). However, the SNR onboard the Orion capsule can be much lower than 20 dB, posing an acoustic challenge.

To enhance the comfort of astronauts during crewed missions, NASA would ordinarily place acoustic blankets to damp down the reverberation in the hard-walled cabin, and some of the noise created by engines and pumps.

“However, because this is an uncrewed mission we have to work within an environment with more reverberation and noise than we would like,” says Isabelle.

re:MARS 2022 — Open space: A revolution in robots for space exploration

There’s another challenge that results from the lack of humans on board. For Orion, commands to Alexa have to be sent from ground control. The low-bandwidth connections utilized for the transmission can make it challenging to transmit voices at the wide range of frequencies essential for differentiating between sounds.

During a typical phone call, our voice is typically transmitted in the narrow band, which ranges from 300 HZ to 3,000 HZ. For Alexa to make out individual words aboard the noisier environment of the space capsule, the voice would have to be transmitted at 8,000 HZ.

“Voice commands from mission control are transmitted to Alexa via a speaker,” says Isabelle. “Flight-qualified speakers are typically designed for narrow-band communications. And so for this mission we were required to use a speaker that could operate in the flight environment.”

Alexa in Space | Alexa Innovators | Build with Alexa

The team relied on what Isabelle calls “brute force” to overcome these acoustic challenges.

Related content
A combination of audio and visual signals guide the device’s movement, so the screen is always in view.

“We designed the speaker playback system to play at extremely loud volumes, which allowed us to increase the SNR to where we wanted it to be.”

The team also took advantage of the physical form factor of Alexa on board to overcome the challenges presented by the noisy environment. The speakers, the light ring and the microphones in the briefcase-like enclosure for Alexa are close to each other, which allows acoustic engineers to overcome some of the obstacles presented by the background noise and reverberation.

Finally, the team deployed two microphones in combination with an array processing algorithm. The latter combined the signals from the two microphones in a way that helps Alexa make sense of the commands being issued from mission control. Because the speakers and microphones are in fixed positions relative to each other — as opposed to a room, where people can be located in any number of locations — the algorithms could be more easily designed to distinguish between speech and the surrounding noise.

Related content
Zoox principal software engineer Olivier Toupet on company’s autonomous robotaxi technology

While the Orion mission will not have any crew members on board, the initial mission will lay the groundwork for Alexa to be integrated into future crewed missions — to the moon, Mars, and beyond. Having Alexa onboard in these future missions would allow crew members to be more efficient in day-to-day tasks, and benefit from the comforts of having Alexa on board such as the ability to play relaxing music and to keep in touch with family and friends back home.

Future crewed missions would have their own unique set of challenges, where Alexa would have to respond to commands from astronauts, who might (literally) be free-floating at multiple points within the capsule. Isabelle and Lantin are already looking forward to overcoming the challenges enabled by crewed missions.

“For someone who grew up watching Star Trek, working on this project has been a dream come true,” says Lantin. “It’s great to be able to build the future. But it’s just as exciting to be able to draw on all of this great work, and be able to enjoy all these new Alexa capabilities during my next vacation, and my day-to-day life right here at home.”

Editor's note

This is a reprint of an article that initially ran on the Alexa Skills Kit Blog. To learn more about the technical innovations that helped get Alexa into space and some inspiring facts about the Artemis I mission, visit the Skills Kit blog.

Related content

US, WA, Seattle
We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Knowledge of econometrics, as well as basic familiarity with Python (or R, Matlab, or equivalent) is necessary, and experience with SQL would be a plus. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. Roughly 85% of previous cohorts have converted to full time scientist employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com.
US, WA, Bellevue
As an applied scientist, you will use your experience to initiate the design, development, execution and implementation of scientific research projects. Working closely with fellow research scientists and product managers, you will use your experience in modeling, statistics, and simulation to design models of new policies, simulate their performance, and evaluate their benefits and impacts to cost, reliability, and speed of our fulfillment network. Our teams are looking for experience in network and combinatorial optimization, algorithms, data structures, statistics, and/or machine learning. This position requires superior analytical thinking, and ability to apply their technical and statistical knowledge to identify opportunities for real world applications. You should be able to mine and analyze large data, and be able to use necessary programming and statistical analysis software/tools to do so. Amazon has positions available for Research Scientists in multiple locations across the US and Canada.
US, WA, Virtual Contact Center-WA
We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Some knowledge of econometrics, as well as basic familiarity with Python is necessary, and experience with SQL and UNIX would be a plus. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. Roughly 85% of previous cohorts have converted to full time scientist employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com. About the team The Selling Partner Fees team owns the end-to-end fees experience for two million active third party sellers. We own the fee strategy, fee seller experience, fee accuracy and integrity, fee science and analytics, and we provide scalable technology to monetize all services available to third-party sellers. Within the Science team, our goal is to understand the impact of changing fees on Seller (supply) and Customers (demand) behavior (e.g. price changes, advertising strategy changes, introducing new selection etc.) as well as using this information to optimize our fee structure and maximizing our long term profitability.
US, WA, Seattle
This is a unique opportunity to build technology and science that millions of people will use every day. Are you excited about working on large scale Natural Language Processing (NLP), Machine Learning (ML), and Deep Learning (DL)? We are embarking on a multi-year journey to improve the shopping experience for customers globally. Amazon Search team creates customer-focused search solutions and technologies that makes shopping delightful and effortless for our customers. Our goal is to understand what customers are looking for in whatever language happens to be their choice at the moment and help them find what they need in Amazon's vast catalog of billions of products. As Amazon expands to new geographies, we are faced with the unique challenge of maintaining the bar on Search Quality due to the diversity in user preferences, multilingual search and data scarcity in new locales. We are looking for an applied researcher to work on improving search on Amazon using NLP, ML, and DL technology. As an Applied Scientist, you will lead our efforts in query understanding, semantic matching (e.g. is a drone the same as quadcopter?), relevance ranking (what is a "funny halloween costume"?), language identification (did the customer just switch to their mother tongue?), machine translation (猫の餌を注文する). This is a highly visible role with a huge impact on Amazon customers and business. As part of this role, you will develop high precision, high recall, and low latency solutions for search. Your solutions should work for all languages that Amazon supports and will be used in all Amazon locales world-wide. You will develop scalable science and engineering solutions that work successfully in production. You will work with leaders to develop a strategic vision and long term plans to improve search globally. We are growing our collaborative group of engineers and applied scientists by expanding into new areas. This is a position on Global Search Quality team in Seattle Washington. We are moving fast to change the way Amazon search works. Together with a multi-disciplinary team you will work on building solutions with NLP/ML/DL at its core. Along the way, you’ll learn a ton, have fun and make a positive impact on millions of people. Come and join us as we invent new ways to delight Amazon customers.
US, WA, Seattle
This is a unique opportunity to build technology and science that millions of people will use every day. Are you excited about working on large scale Natural Language Processing (NLP), Machine Learning (ML), and Deep Learning (DL)? We are embarking on a multi-year journey to improve the shopping experience for customers globally. Amazon Search team creates customer-focused search solutions and technologies that makes shopping delightful and effortless for our customers. Our goal is to understand what customers are looking for in whatever language happens to be their choice at the moment and help them find what they need in Amazon's vast catalog of billions of products. As Amazon expands to new geographies, we are faced with the unique challenge of maintaining the bar on Search Quality due to the diversity in user preferences, multilingual search and data scarcity in new locales. We are looking for an applied researcher to work on improving search on Amazon using NLP, ML, and DL technology. As an Applied Scientist, you will lead our efforts in query understanding, semantic matching (e.g. is a drone the same as quadcopter?), relevance ranking (what is a "funny halloween costume"?), language identification (did the customer just switch to their mother tongue?), machine translation (猫の餌を注文する). This is a highly visible role with a huge impact on Amazon customers and business. As part of this role, you will develop high precision, high recall, and low latency solutions for search. Your solutions should work for all languages that Amazon supports and will be used in all Amazon locales world-wide. You will develop scalable science and engineering solutions that work successfully in production. You will work with leaders to develop a strategic vision and long term plans to improve search globally. We are growing our collaborative group of engineers and applied scientists by expanding into new areas. This is a position on Global Search Quality team in Seattle Washington. We are moving fast to change the way Amazon search works. Together with a multi-disciplinary team you will work on building solutions with NLP/ML/DL at its core. Along the way, you’ll learn a ton, have fun and make a positive impact on millions of people. Come and join us as we invent new ways to delight Amazon customers.
US, WA, Seattle
The retail pricing science and research group is a team of scientists and economists who design and implement the analytics powering pricing for Amazon’s on-line retail business. The team uses world-class analytics to make sure that the prices for all of Amazon’s goods and services are aligned with Amazon’s corporate goals. We are seeking an experienced high-energy Economist to help envision, design and build the next generation of retail pricing capabilities. You will work at the intersection of economic theory, statistical inference, and machine learning to design new methods and pricing strategies to deliver game changing value to our customers. Roughly 85% of previous intern cohorts have converted to full time scientist employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com. Key job responsibilities Amazon’s Pricing Science and Research team is seeking an Economist to help envision, design and build the next generation of pricing capabilities behind Amazon’s on-line retail business. As an economist on our team, you will work at the intersection of economic theory, statistical inference, and machine learning to design new methods and pricing strategies with the potential to deliver game changing value to our customers. This is an opportunity for a high-energy individual to work with our unprecedented retail data to bring cutting edge research into real world applications, and communicate the insights we produce to our leadership. This position is perfect for someone who has a deep and broad analytic background and is passionate about using mathematical modeling and statistical analysis to make a real difference. You should be familiar with modern tools for data science and business analysis. We are particularly interested in candidates with research background in applied microeconomics, econometrics, statistical inference and/or finance. A day in the life Discussions with business partners, as well as product managers and tech leaders to understand the business problem. Brainstorming with other scientists and economists to design the right model for the problem in hand. Present the results and new ideas for existing or forward looking problems to leadership. Deep dive into the data. Modeling and creating working prototypes. Analyze the results and review with partners. Partnering with other scientists for research problems. About the team The retail pricing science and research group is a team of scientists and economists who design and implement the analytics powering pricing for Amazon’s on-line retail business. The team uses world-class analytics to make sure that the prices for all of Amazon’s goods and services are aligned with Amazon’s corporate goals.
US, CA, San Francisco
The retail pricing science and research group is a team of scientists and economists who design and implement the analytics powering pricing for Amazon's on-line retail business. The team uses world-class analytics to make sure that the prices for all of Amazon's goods and services are aligned with Amazon's corporate goals. We are seeking an experienced high-energy Economist to help envision, design and build the next generation of retail pricing capabilities. You will work at the intersection of statistical inference, experimentation design, economic theory and machine learning to design new methods and pricing strategies for assessing pricing innovations. Roughly 85% of previous intern cohorts have converted to full time scientist employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com. Key job responsibilities Amazon's Pricing Science and Research team is seeking an Economist to help envision, design and build the next generation of pricing capabilities behind Amazon's on-line retail business. As an economist on our team, you will will have the opportunity to work with our unprecedented retail data to bring cutting edge research into real world applications, and communicate the insights we produce to our leadership. This position is perfect for someone who has a deep and broad analytic background and is passionate about using mathematical modeling and statistical analysis to make a real difference. You should be familiar with modern tools for data science and business analysis. We are particularly interested in candidates with research background in experimentation design, applied microeconomics, econometrics, statistical inference and/or finance. A day in the life Discussions with business partners, as well as product managers and tech leaders to understand the business problem. Brainstorming with other scientists and economists to design the right model for the problem in hand. Present the results and new ideas for existing or forward looking problems to leadership. Deep dive into the data. Modeling and creating working prototypes. Analyze the results and review with partners. Partnering with other scientists for research problems. About the team The retail pricing science and research group is a team of scientists and economists who design and implement the analytics powering pricing for Amazon's on-line retail business. The team uses world-class analytics to make sure that the prices for all of Amazon's goods and services are aligned with Amazon's corporate goals.
US, WA, Bellevue
We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Some knowledge of econometrics, as well as basic familiarity with Python is necessary, and experience with SQL and UNIX would be a plus. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. Roughly 85% of interns from previous cohorts have converted to full time economics employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com.
US
The Amazon Supply Chain Optimization Technology (SCOT) organization is looking for an Intern in Economics to work on exciting and challenging problems related to Amazon's worldwide inventory planning. SCOT provides unique opportunities to both create and see the direct impact of your work on billions of dollars’ worth of inventory, in one of the world’s most advanced supply chains, and at massive scale. We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. We are looking for a PhD candidate with exposure to Program Evaluation/Causal Inference. Knowledge of econometrics and Stata/R/or Python is necessary, and experience with SQL, Hadoop, and Spark would be a plus. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and perform applied econometric analysis at Internet speed collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. Roughly 85% of previous cohorts have converted to full time scientist employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com.
US, WA, Seattle
The Selling Partner Fees team owns the end-to-end fees experience for two million active third party sellers. We own the fee strategy, fee seller experience, fee accuracy and integrity, fee science and analytics, and we provide scalable technology to monetize all services available to third-party sellers. We are looking for an Intern Economist with excellent coding skills to design and develop rigorous models to assess the causal impact of fees on third party sellers’ behavior and business performance. As a Science Intern, you will have access to large datasets with billions of transactions and will translate ambiguous fee related business problems into rigorous scientific models. You will work on real world problems which will help to inform strategic direction and have the opportunity to make an impact for both Amazon and our Selling Partners.