Emine Yilmaz: An Amazon Scholar advancing the state of the art in voice shopping

Emine Yilmaz is a computer science professor at the University College London (UCL) and a faculty fellow at the Alan Turing Institute. Her research interests include information retrieval and natural language processing. Yilmaz is the recipient of several honors and awards in her career, including a 2018 Bloomberg Data Science Research grant for her work on building task-oriented systems, and a 2015 British Computer Society Information Retrieval Specialist Group Karen Spärck Jones Award for her research contributions in the field of information retrieval.

Emine Yilmaz

Yilmaz is also an Amazon Scholar, a select group of academics who work on large-scale technical challenges for Amazon while continuing to teach and conduct research at their universities. At Amazon, Yilmaz is leading a research team based in London that’s responsible for improving the Alexa voice shopping experiences.

Given the nascency of the field—the first Echo speaker was launched six years ago—customer satisfaction in voice shopping is an open area of research. Yilmaz is uniquely positioned to drive meaningful innovations in the field. She has been involved with advancing research in modeling user behavior and predicting user satisfaction for her entire career. One example: a recent paper that Yilmaz coauthored with Manisha Verma, “Search Costs vs User Satisfaction on Mobile”, in which they studied the impact of user actions, such as inputting search queries, reading snippets, or scrolling through a search engine result page, on customer satisfaction.

Amazon Science spoke to Yilmaz about her career, her work at Amazon, and why she thinks academics will enjoy working at Amazon.

Q. What drew you to your research interests in information retrieval and natural language processing?

My interest in machine learning was sparked during my undergraduate program. As part of an assignment for a computer science class, we had to implement a machine learning algorithm that would learn to put a number of small rectangles into the smallest rectangle shape possible. I found the concept of a computer being trained to perform tasks fascinating, and decided to pursue a master’s degree in machine learning.

When I began my PhD, web search technology was newly emerging. I was intrigued by how search engines were able to retrieve results relevant to a query in a near-instantaneous manner. There were, and there still are, many open problems in the domain, and nearly all of them can be tackled using principles from machine learning. I thus decided to choose as my area of research machine learning applied to information retrieval (the computer science discipline behind search) and natural language processing.

Q. What are you working on at Amazon?

At Amazon, I’m part of the Alexa Shopping Research and Science organization headed by Yoelle Maarek. Customers interact with Alexa for a variety of shopping-related tasks—from product research to actual purchases. My team’s goal is to continually improve Alexa so that she is able to help customers no matter where they are in their shopping journey.

Q. What are some of the research problems you’re tackling at Amazon?

Annotating customer interactions with pertinent data is critical to training Alexa to get better over time. However, with billions of interactions every week, it isn’t feasible to annotate even a small percentage of those interactions manually.

Further complicating matters is the growing number of experiences that Alexa-enabled devices provide. To give just a few examples, Alexa is available on a wide range of smart speakers, tablets, smartphones, and an ever-increasing array of smart home devices. A successful customer interaction on an Echo device (adding an item to one’s shopping list) can be quite different from that on a tablet (clicking and zooming in on an image).

My team applies state-of-the-art natural language processing and machine learning models to predict customer satisfaction across all of these diverse experiences. To do this, our models look at implicit criteria to evaluate whether Alexa helped customers meet their goals. These criteria include search query reformulations, how much time customers spend interacting with search results, or even whether they zoomed in to study a product image in greater detail. By studying patterns in user interactions, we are able to drive improvements to the Alexa voice shopping experience at scale.

Q. How do you see the nascent field of voice shopping evolving?

These are early days for voice shopping. That’s one of the primary reasons this is a fascinating area to be involved with. Similar to mobile phones today, I believe that intelligent voice assistants will become an embedded part of our lives. Shopping using our voice is a much more frictionless experience. Most of us speak faster than we type. With voice agents, you don’t have to take your phone out, unlock it, type out a search term and take a series of steps to complete your request. To give just one example, today you see residents of senior living centers, who would ordinarily struggle using computers, but who are using Alexa to stay connected to friends, family, and the world during COVID-19. Intelligent voice agents are going to be an integral part of our day-to-day lives. I’m really excited to be at Amazon, and have the opportunity to shape the future in how people use voice to conduct research on, and buy products.

Q. How did you come to join the Amazon Scholars program?

I received a call from an Amazon recruiter in 2019, who told me about the Amazon Scholars program. This seemed really intriguing. Indeed, to say that the entire ecosystem around Alexa is cutting edge would be a massive understatement. I was excited at the opportunity to find out more about the kind of problems the team was working on, and to see if I could contribute to their research.

I was also impressed by the investments Amazon has been making in research. At the time, Amazon had recently opened the Cambridge Development Center. They were actively hiring great talent to further innovation in multiple AI disciplines.

Lastly, I was drawn to working with scientists I’ve always held in high regard —be it Michael Jordan, Thorsten Joachims or Eugene Agichtein. Some of the world’s leading researchers are working as Scholars at Amazon. And given my prior work and research area interests, I was particularly interested in the work of Yoelle Maarek’s team.

Q. How do you balance your work between Amazon and University College London?

At Alexa Shopping, I’m constantly encouraged to write and publish papers at the top research conferences, both within Amazon and at my university. It certainly helps that my research areas in academia and at Amazon are distinct yet aligned. To give just one example, as part of my academic work, I recently coauthored a paper, From Stances' Imbalance to Their Hierarchical Representation and Detection , that was presented at The Web Conference in 2019. In the paper, we proposed a new approach to detecting fake news—news that purports to be factual, but which contains misstatements of fact with intention to arouse passions, attract viewership, or simply deceive. On one hand, the paper is sufficiently distinct from shopping that I can differentiate between my work in academia and at Amazon. On the other hand, the research outlined in the paper can help me invent methods towards ensuring that sellers’ descriptions on product listings are accurate.

Q. In your mind, why would academics enjoy working at Amazon?

First, the caliber of talent at Amazon is very high. I attribute this to the hiring process based on a set of Leadership Principles. The hiring process is concrete and structured, and ensures that we are always meeting a high bar when it comes to recruitment. Because the bar for hiring is so high, I’m constantly learning from my managers, from my peers, and from people who report to me.

I also think academics will readily appreciate Amazon’s “customer obsession”, one of our key Leadership Principles. In my mind, this is the primary reason academics should consider working at the company. Throughout my career, when I’ve thought about research, I’ve also thought about the end application. At Amazon, you have the opportunity to have a positive impact on the lives of millions of people. Staying focused on the customer and working a solution backward makes our research a lot more fulfilling. It also keeps you grounded, and prevents you from drifting into irrelevance, both in academia and within the industry.