Amazon hosts internal conferences throughout the year to connect the company’s scientists to each other, and to the academic community at large. For example, the Amazon Machine Learning Conference brings together thousands of scientists and engineers to share research results and raise the scientific bar within the company.
Kevin Small is a senior applied scientist within the Alexa organization, and has been involved in organizing many of Amazon’s internal conferences in his more than five years at Amazon. In this conversation, Small explains how Amazon’s internal conferences facilitate important breakthroughs, forge collaborations between groups, and help advance one’s career.
Q. You were an academic prior to working at Amazon. What attracted you to Amazon?
A. I held a research faculty appointment at Tufts Medical Center. We were developing machine learning methods for reducing the manual effort required by doctors when generating systematic reviews. Our work has been used by several Evidence-based Practice centers, with the resulting reports included in the Cochrane Database of Systematic Reviews.
When I joined Amazon, I had planned on it being a one-year intermission between academic positions to get first-hand insight regarding conducting science in a business setting. At the time, machine learning was having an increasingly greater impact on our industry, and I was curious where the field was headed. Amazon offered an incredible opportunity to leverage the company’s computational resources and collaborate with peers to solve problems that make an impact on the lives of our customers.
One thing I find exciting about Amazon is that even my more incremental work has an opportunity to have an impact at scale, which in turn helps point me toward more important problems. For example, one of my first projects was to automate the understanding of customer reviews on amazon.com, a project that helped millions of customers make better purchasing decisions.
I also liked how people worked as peers at Amazon, which was in contrast to the more hierarchical structure of academia. At Amazon, for everything I have worked on, I have always felt part of a larger team and appreciate that even the most junior team members have ownership and agency, and function as a part of a larger community.
Q. It’s interesting you bring up being drawn to a sense of community. Is this why you’ve been involved in organizing internal conferences?
I’ve always enjoyed being a part of the larger academic community. I review papers for conferences like NeurIPS, ICML, AAAI, and ACL, amongst others throughout the year. I wanted to extend my involvement and grow the sense of community within Amazon as well. This feeling of being part of something larger, along with peer feedback, is absolutely vital to researchers and scientists.
Organizing internal conferences is especially important at a company like ours. As you know, Amazon is different from many other companies when it comes to the way our science and research teams are organized. In general, we are spread across business units, as opposed to being part of a central organization. When I joined Amazon, there were fewer formal mechanisms to connect scientists across the company in an intentional way. Thus, we began to work on conferences like AMLC to address this gap.
Q. When we solve customer problems, there's a need for an interdisciplinary kind of approach. Do you feel these conferences help in fostering interdisciplinary thinking?
Definitely. Customer problems are rarely solved within a single scientific discipline. For example, within the Alexa organization, scientists might be interested in developing the best speech recognition or question answering systems. But they are working on this for customers who are looking to find music, to open their garage doors or turn on the lights in their house. This requires experts in multi-modal UX design, systems engineering, computational considerations, operational excellence, and a number of fields working together. We structure our internal conferences in way that scientists almost have no choice but to think about how their work fits in this ecosystem.
As a specific example, there was a paper presented this year at AMLC on the Amazon Photos face clustering problem -- the task of grouping all the photos of a distinct individual with little to preferably no supervision from the customer. The paper described the end-to-end process, from collecting evaluation data to training of embedding models and associated context modeling techniques. This paper brought together scientists from various business units within Amazon, and highlighted that solving customer problems requires collaboration across multiple business units.
Q. How do you determine the kind of content you want to feature at a conference?
For conferences like AMLC where we bring in researchers from across the company, we first look for papers that feature breakthrough work. These are papers with implications for a broader segment of the scientific community.
Of course, true breakthroughs are rare. Thus, we also feature papers from at least two other families of contributions. First, we showcase work demonstrating notable progress on really important customer problems – for example, improving the engagement with product recommendations served on the site or reducing delivery times to customers. Secondly, we like to highlight exemplary work regarding ML pipelines that might serve as templates for work throughout the company.
Conferences can also help scientists prioritize what they should be working on. At Amazon, scientists frequently measure their contributions by the impact that their innovations have had on the business at large and scientific priorities are often correlated with business needs. However, sometimes, they are not perfectly aligned for more disruptive research directions. These conferences provide an opportunity to set an agenda for our scientists, where we identify areas where they can discuss how to deliver longer-term meaningful innovation to customers.
Q. How do you see conferences at Amazon evolving?
Over the last five years, our internal data suggests that the number of accepted publications from our scientists at external conferences has gone up by an estimated 500%. You’ll continue to see increased participation from Amazon at external conferences as we encourage our interns and employees to publish externally even more.
I also expect the kind of papers we present at internal conferences to evolve and reflect more real-world scenarios. For example, when you talk about areas like personalization or advertising, you find that real-world data behaves very differently from offline data sets. The distribution of real-world data is often non-stationary or even adversarial. In addition, the fact that people begin to use a system changes related feedback loops, which in turn impacts their behavior. For internal conferences, I expect we’ll see papers that focus on these kinds of research problems that might not be as relevant in an academic setting, but which can have a positive impact on the lives of millions of customers.
I also see our conferences evolving in terms of even more mechanisms for scientists to network with each other. The community of scientists and researchers is smaller than you would expect -- getting noticed at an internal conference can do wonders for a scientist’s career in terms of visibility.
Finally, I expect we’ll continue to organize conference like Amazon Research Days, where we focus on networking and building ties with the academic community. This is important because we can’t operate in a vacuum. We benefit from the academic community, and they benefit from our work and resources as well.
