Amazon pioneered the use of robots in order fulfillment. Every day, fleets of robots in fulfillment centers carry pods full of heavy inventory, assisting employees with physically demanding tasks. These robots have come a long way since Amazon began deploying them over a decade ago, but there is still much to be learned from how these systems interact with humans.
Maya Cakmak, an associate professor at the University of Washington (UW), is helping explore those open questions. Through work supported via the UW + Amazon Science Hub, a research collaboration housed in UW’s Allen School of Computer Science & Engineering, Cakmak is researching a generation of robots that can be trained to not only grasp and handle heavy products but also turn to humans for help when needed.
“I focus on making robots that can interact with people and do everyday tasks,” she explains.
Cakmak is helping tackle one of the most challenging parts of the fulfillment process: picking and stowing. In a UW robotics lab replicating a setup found in some Amazon fulfillment centers, Cakmak and her team have built prototype robots that can extend robotic arms into compact storage areas, manipulate and grasp items, and extract them from the clutter of nearby objects.
The goal is to explore how robots and humans can assist one another with challenging tasks while obtaining the right result, every time.
Human-robot interaction
Cakmak has spent her career at the forefront of human-robot interaction, or HRI — a field focused on designing and building robots that can be “trained” by and interact with humans to assist with specific, sometimes personal, tasks. Her pioneering work started with helping those with disabilities.
“I focus mostly on robots that do physical tasks,” says Cakmak, “whether it’s for the visual or hearing impaired, those with cognitive challenges or physical impairments — anything that limits their access to the physical world.” She points to a popular robot vacuum as a good example, as it not only moves around but performs a task — vacuuming — that can be a chore for anyone, but especially those with disabilities.
One of Cakmak’s most inspirational HRI projects was a robot and software interface she designed for Henry Evans, who is quadriplegic and needs assistance to accomplish everyday tasks in his home. Using just his eye movements and a few finger clicks, Henry can use an interface Cakmak helped design to instruct his in-home robot to help feed him and operate devices.
Over the past three summers, the interdisciplinary team that Cakmak was part of enabled Henry to feed himself independently, contribute to household tasks, and even physically participate in social interactions, such as playing cards with his family or play with his grandchildren. Working with Henry inspired new research projects in Cakmak’s lab, including the development of a toolkit that enables the creation of fully customizable interfaces that meet the specific needs and preferences of each unique user. The toolkit was used in early stages of an Amazon project to rapidly test the feasibility of addressing more-challenging picks with a human operator in the loop.
A passion for robotics
Cakmak, who was born in Belgium and grew up in Turkey, loved math from an early age, competing in math Olympiads while in high school. At the Middle East Technical University (METU), where she earned a bachelor’s in electrical and electronics engineering, she developed a passion for both engineering and the idea of using math to solve real-world problems. Her interest in robotics was inspired by a senior capstone project that involved building a robot that climbed stairs taller than itself.
That interest deepened as she continued her education at METU, completing a master’s in computer engineering and leading several robotics research projects. She moved to the United States in 2007 to pursue a PhD in robotics at Georgia Tech, specializing in HRI.
After completing her PhD, Cakmak connected with roboticists at Willow Garage, a pioneering robotics research lab and tech incubator that developed hardware and open-source software for personal robotics applications. She initially joined as the “most senior intern,” she quips, but the company quickly named her a postdoctoral research fellow, and she continued working on real-world robotics applications alongside some of the top roboticists in the field.
With a strong desire to remain in academia, Cakmak applied for professorial positions during her year at Willow Garage. She landed her dream job as an assistant professor at the Allen School in 2013 and was promoted to associate professor in 2019, continuing her work in HRI and helping inspire the next generation of roboticists.
Cakmak’s early research at UW focused on developing robots that could be taught or programmed by end-users. While the approach avoided the immense challenge of building universally capable robots, it also raised a more human-centered challenge: enabling people who aren’t roboticists or software developers to program robots. To achieve this, Cakmak and her team at the Human-Centered Robotics Lab worked closely with end-users to develop and evaluate new tools. The lab’s mission statement calls for projects that “focus on end-user robot programming, robotic tool use, and assistive robotics.”
Several years ago, the lab's work began to draw notice from robotics researchers at Amazon. Cakmak’s expertise in HRI, as well as her highly diverse team’s entry in an Amazon robotics challenge, caught the attention of Michael Wolf, a principal applied scientist in Amazon Robotics. Cakmak's work on human-to-robot software interfaces, including the one she built for Henry, made a big impression on the team.
“She’s a great full-stack roboticist, and she has a deep background in human-robot interaction,” Wolf says. “These skills made her an excellent person to lead this research. We’re committed to investing in the science community and do this in a variety of ways. Supporting Maya’s work, which has both science value and numerous potential social benefits, is one of these efforts.”
Human-in-the-loop
Deploying robots that can quickly identify a massive variety of items and grasp and remove individual items from densely packed spaces is an enormous challenge, particularly at Amazon’s scale.
“It's just inevitable that there will be corner cases and failures," Cakmak said. "So how can we put humans in the loop to reduce errors? That was my pitch, to have a human component when we started the project.”
While Wolf notes that prototype robots can successfully grasp the majority of Amazon’s items without error, he and Cakmak believe success in production will require greater collaboration between humans and robots, due to the sheer diversity of items in Amazon’s catalogue.
Take books, for example. For efficiency, books are stacked densely together, making it challenging for robots to identify the right book, grasp it, and slide it out from the shelf. But with the help of a human teleoperator, the robot can gently slide a book out from above, making it easier to grasp.
“There are limitations to robots’ capabilities,” says Cakmak. “We’ve demonstrated that human operators can use their finer perception and intelligence to help them complete more complex tasks.” The team is also working on developing new methods to enable robots to learn from human operators over time.
Wide-ranging impact
Wolf notes that the work that Cakmak and the Science Hub team are doing has much broader applications, a fact that aligns with the hub’s mission of addressing hard challenges in science and engineering.
“When starting collaborations at the Science Hub, we think about not just where the tech gaps are but also where there is a super hard, real-world problem that maybe isn’t getting a lot of attention,” observes Wolf. “Preferably, these problem statements resonate broadly with the academic community and excite the field beyond our collaborations."
“We often talk about using robots for what they're good at and asking humans to do what they’re good at,” Wolf continues. “There's now this new intersection that Cakmak is helping define.”
Additionally, Cakmak notes that the practical applications of the Science Hub work benefit academics like her.
“I think in robotics we have a bit of a culture to make up problems that robots can solve," she says. "So you'll see robots playing ping pong and using chopsticks. We like to challenge ourselves and make something cool without really thinking about applications. I really like that the Amazon work is grounded in a real problem that’s going to make a real difference.”
