The science behind Echo Frames

How the team behind Echo Frames delivered longer battery life and improved sound quality inside the slim form factor of a pair of eyeglasses.

When the team behind Amazon’s Echo Frames set out to improve the next generation of their product, they needed to strike a delicate balance. Customer feedback on earlier versions of the smart audio eyeglasses centered on three elements: longer battery life, more style options, and improved sound quality.

A man with a beard is seen wearing a pair of Echo Frames glasses. He is standing outside and is pictured in three-quarters view.
Echo Frames feature custom-built speech processing technology that drastically improves word recognition — key for interacting with Alexa in windy or noisy environments.

Achieving all three of those goals would be a challenge in itself; doing that inside the slim form factor of a pair of Alexa-enabled eyeglasses upped the ante.

“All three of those goals are in tension with one another,” says Adam Slaboski, senior manager of product management and product lead for Echo Frames. The easiest way to improve battery and audio would be to increase the size of the device, but that would conflict with feedback around the importance of design. Amping up bass to improve the audio experience would consume more battery, and so on.

Finding that sweet spot was a huge effort in engineering and customer understanding.
Adam Slaboski

“Finding that sweet spot was a huge effort in engineering and customer understanding," Slaboski says.

With Echo Frames (3rd Gen) and Carrera Smart Glasses with Alexa (designed in collaboration with Safilo, one of the world’s leading eyewear companies), the Smart Eyewear team met the challenge.

The smart glasses feature enhanced audio playback, with custom-built speech-processing technology that dramatically improves word recognition — key for interacting with Alexa in windy or noisy environments. The new range of frame styles come in a variety of sizes, and all come with a significant boost in battery life.

From the outside, Echo Frames still look like a pair of regular eyeglasses. “But we changed everything on the inside,” says Jean Wang, general manager and director of Smart Eyewear. “And we learned new lessons along the way.”

Here’s how Amazon engineers and product designers tackled all three customer demands.

Turning up the volume with open-ear audio

Like previous generations of Echo Frames, the current model uses open-ear audio. In addition to fitting the form factor of a pair of glasses, this allows users to maintain awareness of their surroundings while interacting with Alexa or enjoying audio entertainment.

Related content
Combining psychoacoustics, signal processing, and speaker beamforming enhances stereo audio and delivers an immersive sound experience for customers.

The open-ear audio design has been popular with users who are blind or have low vision, notes Jenai Akina, senior product manager for Echo Frames. “It’s really beneficial that it doesn’t obstruct a critical sense like hearing,” she explains. “That form factor is really helpful for daily interactions — especially when we want to be open to engage with our environment and the people around us. Open ear allows customers to maintain awareness, while providing access to a voice assistant.”

Open-ear audio brings a host of unique challenges to the engineering process. Typical headphones and earbuds block off the ear from the outside world, preventing air from escaping. That funnels more of the sound waves from the speakers into the user’s ears. With an open-ear design, sound has to travel farther, and there is less control over direction. That could lower the audio volume and reduce clarity — and importantly, audio could leak out to people standing nearby. The key is to drive the sound pressure as much as possible toward the user’s ears while minimizing the audio leakage.

By bringing people into the lab, we can simulate real environmental noise conditions like wind, background noise in a crowded restaurant, and the sound of cars on the road.
Scott Choi

In working to improve audio quality, the team continued to hone the directionality of the sound while also working to improve volume and bass. A technique called dipole speaker configuration helps to do both. In addition to a sound porthole located near the ear canal, the frames feature a second porthole that cancels unnecessary sound while amping up bass.

With input from in-house audio experts and instruments to analyze measurements like harmonic distortion, the team came up with a set of potential tuning solutions that met objective targets for audio quality. They then tested those “flavors” of tuning in the lab with several user groups.

“By bringing people into the lab, we can simulate real environmental-noise conditions like wind, background noise in a crowded restaurant, and the sound of cars on the road,” explains senior manager of audio Scott Choi. That allowed his team to understand environmental variables in a controlled setting.

With the feedback from those focus groups, the team then selected a few of the most popular tunings to push out to beta testing, where users could provide feedback on a weekly basis.

“We see how the feedback trends change with each tuning change, which gradually allows it to mature and converge into a certain tuning,” Choi says. The result is audio calibrated to maximize intelligibility and volume without leaking private conversations (or guilty-pleasure playlists).

The Echo Frame team used a rotating arch of microphones to lest leakage. This animation shows the array moving in circles around a mannequin wearing the Gen 3 prototype, creating a 3D sphere plot of audio leakage. Via this testing, the team was able to minimize leakage to the side and back.
The Echo Frame team used a rotating arch of microphones to lest leakage. The array moved in circles around a mannequin wearing the Gen 3 prototype, creating a 3D sphere plot of audio leakage. Via this testing, the team was able to minimize leakage to the side and back.

To test leakage, the audio team rigged up a rotating arch of microphones. The array moved in circles around a mannequin wearing the Gen 3 prototype, creating a 3-D sphere plot of audio leakage. Choi explains that they focused on minimizing leakage to the side and back, and ultimately, the speakers were moved much closer to the ear to help minimize leakage and improve loudness.

Leakage isn’t the only privacy consideration. The Echo Frame team also continues to innovate on protecting users from bad actors who may get hold of their smart glasses.

Related content
Amazon senior principal engineer Luu Tran is helping the Alexa team innovate by collaborating closely with scientist colleagues.

Gen 2 protected users by requiring them to authenticate their sessions using a trusted phone. Without authentication, a user can’t invoke sensitive commands like “navigate me home,” unlocking a smart lock, or making a purchase. But customers didn’t like the added friction.

Now customers who enroll in Alexa Voice ID will be able to use their vocal fingerprints for authentication to receive responses to smart-home utterances.

“We’re the first on-the-go Alexa device to use Voice ID for privacy authentication,” Slaboski says.

Boosting battery life without cramping style

Gen 3 improves continuous music playback time to six hours, versus the four hours offered by the previous generation of Echo Frames. It also bumps battery life to up to 14 hours of moderate usage spread across playback, talk time, notifications, and Alexa interactions.

Delivering the desired loudness, bass, and audio quality while optimizing for battery life was a careful balance.
Ravi Sanapala

The team couldn’t simply slap on a bigger battery without making the Echo Frames look less like normal glasses. And with sound quality high on the priority list as well, the devices were going to need as much juice as ever. The team focused on trimming power use in standby mode, ensuring that the overall battery consumption would go down without weakening the speakers when users needed them.

“Delivering the desired loudness, bass, and audio quality while optimizing for battery life was a careful balance,” says senior product manager Ravi Sanapala. “We need the battery to last throughout as much of the day as possible and for Alexa to be available whenever users need it.”

The architectural changes in speaker placement helped keep power needs low while improving audio. The team also tweaked the placement of the battery itself, distributing its capacity differently than in Gen 2. Sanapala adds that algorithmic changes were key in balancing idle-battery conservation and on-demand device usage.

“We had to collaborate with all of our cross-functional teams to optimize everything,” Sanapala says.

Gen 3 also features an all-new charging stand, which is designed for compatibility with all frame shapes and keeps lenses upright, protecting them from scratches while wirelessly charging.

Making smart eyewear look like eyewear

Making glasses that are suitable for everyday wear has always been a priority. “One of our goals has always been to develop technology that appears when you need it and disappears when you don’t,” says Wang.

Previous models of Echo Frames have come in a single, one-size-fits-all style.

A person is seen wearing Echo Frames sunglasses outside. The person carries a notebook and is looking down at it, and there are some buildings and blue sky in the background.
The Echo Frames team consulted with both internal and external eyewear designers to review common and popular styles of frames, and to survey potential customers about their preferences.

“That was a very intentional move,” Wang explains. “We wanted to start simply and learn from customer feedback.”

Gen 2’s flexible spring hinge and adjustable temple tips ensured that the single size fit many different faces. In fact, Wang says, while the goal was to fit around half of all potential users, they’ve found that 85 percent of the adult population can comfortably wear the Gen 2 design.

But with Gen 3, Wang says, the team needed to go beyond designing glasses that looked typical. Customers wanted glasses that looked stylish, too.

The team consulted with both internal and external eyewear designers to review common and popular styles of frames, as well as “edgier” designs, and to survey potential customers about their preferences. After testing options with beta customers, they settled on a variety of styles in various colors that cover a range of aesthetics. They also switched to an acetate material to match the feel of high-end eyewear.

Related content
How a team of designers, scientists, developers, and engineers worked together to create a truly unique device in Echo Show 10.

While each style will still come in a single size, the range of designs will accommodate even more faces than Gen 2, as the collection spans narrow, medium, and wide fits. Each style features adjustable temple tips constructed out of silicone around a lightweight titanium core for better fit. And despite the boost in battery life, the temples of Gen 3 frames have actually been slimmed down. Wang notes that competitive products often place large batteries behind a user’s ears. But presenting Echo Frames users with something that bulky and uncomfortable was never on the table.

“We were working with really heavy constraints,” Wang says. “So we have been very deliberate in making design choices in the service of our customer. That’s challenged us to be innovative and really push the limits of what’s possible in the architecture of our designs.”

Related content

US, WA, Seattle
The Annapurna ML team is looking for a Senior Applied Scientist to work on the intersection of Artificial Intelligence and program analysis to raise the code quality bar in our state-of-the-art deep learning compiler stack. This stack is designed to optimize application models across diverse domains, including Large Language and Vision, originating from leading frameworks such as PyTorch, TensorFlow, and JAX. Your role will involve working closely with our custom-built Machine Learning accelerators, Inferentia and Trainium, which represent the forefront of Annapurna innovation for advanced ML capabilities, and is the underpinning of Generative AI. As a Senior Applied Scientist, you'll be instrumental in designing, developing, and deploying analyzers for ML compiler stages and compiler IRs. You will architect and implement business-critical tooling, publish research, and mentor a brilliant team of experienced scientists and engineers. You will need to be technically capable, credible, and curious in your own right as a trusted scientist, innovating on behalf of our customers. Your responsibilities will involve tackling crucial challenges alongside a talented engineering team, contributing to leading-edge design and research in compiler technology and deep-learning systems software. Strong experience in programming languages, compilers, program analyzers, and program synthesis engines will be a benefit in this role. A background in machine learning and AI accelerators is preferred but not required. A day in the life Diverse Experiences Amazon values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon? We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at Amazon, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
US, WA, Seattle
Join the AWS Perimeter Protection team as a Senior Applied Scientist, where you will bring your deep ML engineering expertise to design, build, and scale AI-driven security solutions that protect AWS customers worldwide. This role is ideal for someone who has already built and shipped production ML systems at industry scale and is looking to apply that experience to high-impact security challenges. You will own the full ML lifecycle — from research and prototyping to production deployment and optimization — powering services including Web Application Firewall, DDoS Protection, Bot Management, and Infrastructure Protection. With services spanning all AWS regions and handling trillions of requests per week, you will solve complex engineering and science problems where model performance, system reliability, and low-latency inference are critical. Key job responsibilities - Design, build, and deploy production-grade ML models and systems for real-time threat detection, mitigation, and protection against evolving cyber threats at cloud scale. - Own the full ML lifecycle end-to-end — from problem formulation, data engineering, and model development through to production deployment, monitoring, and continuous improvement. - Architect and optimize ML pipelines, training infrastructure, and serving systems to meet strict latency, throughput, and reliability requirements at AWS scale. - Bridge the gap between research and production by translating novel ML approaches into robust, scalable, and maintainable systems that operate in real-time security environments. - Design and implement feature engineering workflows and large-scale data processing pipelines to support rapid experimentation and reliable model iteration. - Collaborate closely with software engineering teams to integrate ML models into distributed, low-latency security services, driving engineering decisions around model serving, infrastructure, and system design. - Analyze large-scale production data to identify patterns, anomalies, and emerging threat vectors, and translate findings into measurable improvements to detection and mitigation capabilities. - Establish and improve best practices for ML system design, model evaluation, A/B testing, and production monitoring across the team. - Mentor junior scientists and engineers, raising the bar on both scientific rigor and engineering quality.
US, WA, Seattle
The Automated Reasoning Group in the Amazon Neuron team is looking for an Applied Scientist to work on the intersection of Artificial Intelligence and program analysis to raise the code quality bar in our state-of-the-art deep learning compiler stack. This stack is designed to optimize application models across diverse domains, including Large Language and Vision, originating from leading frameworks such as PyTorch and JAX. Your role will involve working closely with our custom-built Machine Learning accelerator, Trainium, which represents the forefront of innovation for advanced ML capabilities, and is the underpinning of Generative AI. In this role as an Applied Scientist, you'll be instrumental in designing, developing, and deploying analyzers for ML compiler stages and compiler IRs. You will architect and implement business-critical tooling, publish research, and mentor a brilliant team of experienced scientists and engineers. You will need to be technically capable, credible, and curious in your own right as a trusted AWS Neuron engineer, innovating on behalf of our customers. Your responsibilities will involve tackling crucial challenges alongside a talented engineering team, contributing to leading-edge design and research in compiler technology and deep-learning systems software. Strong experience in programming languages, compilers, program analyzers, theorem provers, and program synthesis engines will be a benefit in this role. A background in machine learning and AI accelerators is preferred but not required.
IN, KA, Bengaluru
Customer addresses, Geospatial information and Road-network play a crucial role in Amazon Logistics' Delivery Planning systems. We own exciting science problems in the areas of Address Normalization, Geocode learning, Maps learning, Time estimations including route-time, delivery-time, transit-time predictions which are key inputs in delivery planning. As part of the Science team within DNT (Last Mile), you will partner closely with other scientists and engineers in a collegial environment to develop enterprise ML solutions with a clear path to business impact. The setting also gives you an opportunity to think about a complex large-scale problem for multiple years and building increasingly sophisticated solutions year over year. In the process there will be opportunity to innovate, explore SOTA and publish the research in internal and external ML conferences. Successful candidates will have deep knowledge of competing machine learning methods for large scale predictive modelling, natural language processing, semi-supervised & graph based learning. We also look for the experience to graduate prototype models to production and the communication skills to explain complex technical approaches to the stakeholders of varied technical expertise. Here is a glimpse of the problem spaces and technologies that we deal with on a regular basis: 1. De-duping and organizing addresses into hierarchy while handling noisy, inconsistent, localized and multi-lingual user inputs. We do this at the scale of millions of customers for established (US, EU) as well as emerging geographies (IN, MX). We make use of technologies like LLMs, Weak supervision, Graph-based clustering & Entity matching. We also use additional modalities such as building outlines in maps, street view images and 3P datasets, gazetteers. 2. Build a generic ML framework which leverages relationship between places to improve delivery experience by learning precise delivery locations and propagating attributes, such as business hours and safe places. 3. (Work done in sister teams) Developing systems to consume inputs from areal imagery and optimize our maps to enable efficient delivery planning. Also building models to estimate travel time, turn costs, optimal route and defect propensities. Key job responsibilities As an Applied Scientist I, your responsibility will be to deliver on a well defined but complex business problem, explore SOTA technologies including GenAI and customize the large models as suitable for the application. Your job will be to work on a end-to-end business problem from design to experimentation and implementation. There is also an opportunity to work on open ended ML directions within the space and publish the work in prestigious ML conferences. About the team Last Mile Address Intelligence (LMAI) team owns WW charter for address and location learning solutions which are crucial for efficient Last Mile delivery planning. The team works out of Hyderabad and Bangalore offices in India. LMAI is a part of the DNT Science team, which also owns problems in the space of maps learning and travel time estimations. The rest of the DNT Science team and senior leadership of Last Mile org works out of Bellevue office.
US, CA, Pasadena
We are seeking a Research Science Manager to join the SAF Lab. In this role, you will lead a team of world-class applied scientists, engineers, post-docs and interns developing the next generation of safe autonomy on highly dynamic robotic systems. You will drive technical vision, research strategy, and ensure your team's innovations translate into production systems that operate at Amazon scale. You will interface with top academic researchers at the forefront of safe autonomy, and interdisciplinary teams across Amazon working on autonomous mobile robots, mobile manipulators, and dynamically stable robots. You will bridge academic research with real-world deployable safety layers that enable robots to safely operate around humans. Key job responsibilities • Manage a team of scientists and engineers developing a universal safety layer for robotic systems, with a focus on the next generation of robots • Work with leadership to design and execute multi-year research roadmaps for the development of safe autonomy that spans all types of current and emerging robotic platforms • Lead Scientists, Engineers, post-docs and interns to realize research and development goals and provide evidence of these results in a variety of formats • Drive integration of control barrier functions with planning, perception and learning while maintaining guarantees of safe high-performance robot behavior • Collaborate with product teams and science leaders to set a science roadmap (with eventual impact on real robots). • Publish research findings at top-tier conferences and contribute to the broader robotics community • Build relationships with academic and industry partners to stay at the forefront of safe autonomy A day in the life Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! About the team Work with the inventor of control barrier functions in the Safe Autonomy Frontiers (SAF) Lab. The first industry research lab in safe autonomy, developing a universal safety layer for the next generation of robotic systems: mobile robots, manipulators, mobile manipulators, and future platforms with dynamic stability. You will push the frontiers of performant safety for highly dynamic robots: CBF theory integrated with perception and learning, evaluated on next-generation robots. Your work will underpin robots operating alongside people at Amazon's unprecedented scale.
US, WA, Bellevue
Transform global logistics by developing innovative technological solutions that optimize complex international shipping networks. Join a dynamic team where your advanced scientific expertise will directly impact millions of packages moving across borders, solving intricate transportation challenges through applied science. Key job responsibilities - Develop advanced deep learning and generative AI applications for sophisticated product classification systems - Lead complex modeling analyses to support strategic business decision-making and policy development - Partner with engineering and operations teams to design transformative machine learning solutions for transportation challenges - Design and document technical strategies and applied science roadmaps - Innovate and provide technological direction in international imports and exports logistics A day in the life Your workday will be an exciting blend of data exploration, innovative problem-solving, and collaborative technology development. You'll dive into complex datasets, design sophisticated machine learning models, engage with cross-functional teams, and create technological solutions that reshape international shipping capabilities. About the team We are a passionate group of scientists and engineers dedicated to building comprehensive transportation technologies. Our mission is to create highly scalable, intelligent systems that enable seamless international package movement. By pushing technological boundaries, we're reimagining global logistics through innovative applied science. Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan Learn more about our benefits here: https://www.amazon.jobs/en/internal/benefits/us-benefits-and-stock
US, WA, Seattle
Innovators wanted! Are you an entrepreneur? A builder? A dreamer? This role is part of an Amazon Special Projects team that takes the company’s Think Big leadership principle to the limits. If you’re interested in innovating at scale to address big challenges in the world, this is the team for you. As an Applied Scientist on our team, you will focus on building state-of-the-art ML models for healthcare. Our team rewards curiosity while maintaining a laser-focus in bringing products to market. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the forefront of both academic and applied research in this product area, you have the opportunity to work together with a diverse and talented team of scientists, engineers, and product managers and collaborate with other teams. This role offers a unique opportunity to work on projects that could fundamentally transform healthcare outcomes. Key job responsibilities In this role, you will: • Design and implement novel AI/ML solutions for complex healthcare challenges • Drive advancements in machine learning and data science • Balance theoretical knowledge with practical implementation • Work closely with customers and partners to understand their requirements • Navigate ambiguity and create clarity in early-stage product development • Collaborate with cross-functional teams while fostering innovation in a collaborative work environment to deliver impactful solutions • Establish best practices for ML experimentation, evaluation, development and deployment • Partner with leadership to define roadmap and strategic initiatives You’ll need a strong background in AI/ML, proven leadership skills, and the ability to translate complex concepts into actionable plans. You’ll also need to effectively translate research findings into practical solutions. A day in the life You will solve real-world problems by getting and analyzing large amounts of data, generate insights and opportunities, design simulations and experiments, and develop statistical and ML models. The team is driven by business needs, which requires collaboration with other Scientists, Engineers, and Product Managers across the Special Projects organization. You will prepare written and verbal presentations to share insights to audiences of varying levels of technical sophistication. About the team We represent Amazon's ambitious vision to solve the world's most pressing challenges. We are exploring new approaches to enhance research practices in the healthcare space, leveraging Amazon's scale and technological expertise. We operate with the agility of a startup while backed by Amazon's resources and operational excellence. We're looking for builders who are excited about working on ambitious, undefined problems and are comfortable with ambiguity.
US, TX, Austin
Amazon Leo is an initiative to launch a constellation of Low Earth Orbit satellites that will provide low-latency, high-speed broadband connectivity to unserved and underserved communities around the world. As a Systems Engineer, this role is primarily responsible for the design, development and integration of communication payload and customer terminal systems. The Role: Be part of the team defining the overall communication system and architecture of Amazon Leo’s broadband wireless network. This is a unique opportunity to innovate and define groundbreaking wireless technology at global scale. The team develops and designs the communication system for project Leo and analyzes its overall system level performance such as for overall throughput, latency, system availability, packet loss etc. This role in particular will be responsible for leading the effort in designing and developing advanced technology and solutions for communication system. This role will also be responsible developing advanced physical layer + protocol stacks systems as proof of concept and reference implementation to improve the performance and reliability of the LEO network. In particular this role will be responsible for using concepts from digital signal processing, information theory, wireless communications to develop novel solutions for achieving ultra-high performance LEO network. This role will also be part of a team and develop simulation tools with particular emphasis on modeling the physical layer aspects such as advanced receiver modeling and abstraction, interference cancellation techniques, FEC abstraction models etc. This role will also play a critical role in the integration and verification of various HW and SW sub-systems as a part of system integration and link bring-up and verification. Export Control Requirement: Due to applicable export control laws and regulations, candidates must be a U.S. citizen or national, U.S. permanent resident (i.e., current Green Card holder), or lawfully admitted into the U.S. as a refugee or granted asylum.
US, CA, Sunnyvale
We are looking for a Senior Applied Scientist to help drive the research and development of real-time multimodal conversational AI. You will contribute across two focus areas: advancing foundation models for speech and audio, and building the post-training systems (reward modeling, reinforcement learning) that shape natural, human-like conversational behavior. You will own a significant research area and contribute across the full model lifecycle — from pre-training and architecture design through post-training alignment and real-time deployment. You will work at the frontier of what’s possible in conversational AI, with the compute, data, and runway to pursue problems that few teams in the world have the resources to tackle. As a Senior Scientist, you will drive the technical execution of your research area, contribute to the team’s roadmap, and work closely with inference engineers to ensure your models are designed for real-time production deployment. Key job responsibilities What You’ll Do Foundation Model Scaling - Help build and train large-scale multimodal foundation models for real-time speech and audio generation, from architecture design through production-scale training - Advance the scaling and efficiency of conversational models, including the relationship between data, model size, and real-time performance - Design model architectures informed by hardware constraints and inference requirements, working with inference engineers to ensure models are servable from inception - Develop training methodologies for multimodal models that jointly process and generate speech, language, and audio in real-time streaming contexts - Contribute to the state of the art on efficient architectures and training methods for conversational AI at scale Post-Training & Reinforcement Learning - Design and build reward models and reward functions for speech systems — capturing naturalness, fluency, conversational quality, and real-time responsiveness - Develop and apply reinforcement learning methods to shape conversational behavior — teaching models natural timing, responsiveness, and fluid interaction - Build parts of the post-training pipeline from SFT through RL alignment, optimized for real-time multimodal outputs rather than text-only generation - Design evaluation frameworks that capture the quality dimensions unique to real-time conversation (latency sensitivity, audio quality, prosody, interaction naturalness) Real-Time Perception & Generation - Advance the team’s capabilities in real-time perception — the ability of the model to process incoming audio/speech while simultaneously generating responses - Develop techniques for natural interactive systems where the model handles concurrent input and output with human-like timing - Work at the intersection of model architecture and production constraints to ensure multimodal capabilities function within hard real-time latency budgets
US, WA, Bellevue
Robots are about to step out of the movies and into our homes. If you're excited about this robotics revolution, come join Amazon's Fauna Robotics. As an Applied Scientist in this org, you'll push the state-of-the-art techniques in Robotics. You'll drive technical excellence in areas such as perception, manipulation, sim2real transfer, reinforcement learning and multi-task learning, designing novel algorithms that bridge the gap between research and real-world deployment. In this role, you will integrate hands-on technical expertise with scientific leadership, ensuring your team delivers robust solutions for dynamic real-world environments. While staying updated with the latest advancements in robotics research, you will leverage your experience and expertise to guide the team towards the most promising direction. Collaborating with a talented group of software engineers and scientists, you will bring your ideas to fruition. Deploying robots in home environments requires you to anticipate challenges not encountered in structured settings like warehouses. Additionally, consumer-grade sensors and actuators necessitate working around their limitations, requiring out-of-the-box innovative ideas. As such, you are someone who truly enjoys thinking about problems using first principles at a holistic and systems level to address the aforementioned challenges. Key job responsibilities - Lead technical initiatives in areas such as robotics foundation models, reinforcement learning and manipulation - - Design experiments to identify the limitations of current state-of-the-art models and developing new models or techniques that can surpass these models - Design and implement novel deep learning architectures that push the boundaries of what robots can understand and accomplish - Mentor fellow scientists while maintaining strong individual technical contributions - Collaborate with engineering teams to optimize and scale models for real-world applications - Influence technical decisions and implementation strategies within your area of focus - Experienced in communicating complex technical work to a non-technical audience - The ability to work with minimal guidance, be proactive and to handle ambiguity and the challenge of quickly evolving goals A day in the life - Train ML models for deployment in simulation and real-world robots, identify and document their limitations post-deployment - Drive technical discussions within your team and with key stakeholders to develop innovative solutions to address identified limitations - Guide fellow scientists in solving complex technical challenges, from sim2real transfer to training RL policies - Mentor team members while maintaining significant hands-on contribution to technical solutions About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.