From intern to applied scientist: How Theodore Vasiloudis made the transition

In the early days of purchase data analysis, a study determined that people often bought diapers and beer together. When Theodore Vasiloudis, then a computer science undergrad, heard that from a professor at the Aristotle University of Thessaloniki, he was intrigued by the correlation: “I found it fascinating that, by aggregating the data of multiple users, you could extract weird and unexpected things like this.”

That course inspired Vasiloudis, today an applied scientist with Amazon Web Services (AWS), to direct his education toward machine learning. He left Greece in 2012 to study at the KTH Royal Institute of Technology in Stockholm, Sweden, which at the time had one of the few master’s programs in Europe dedicated to machine learning. After finishing his thesis on context-aware recommendations, he pursued an industrial PhD while employed at the Swedish Institute of Computer Science (industrial PhD students develop their research projects while working at a company to gain industrial experience).

In the final years of his PhD, Vasiloudis completed two summer internships at Amazon. One of those resulted in the publication of an award-winning research paper, Block-distributed Gradient Boosted Trees. In that paper, Vasiloudis and his colleagues Hyunsu Cho and Henrik Boström described the development of a new algorithm that was able to drastically reduce the communication cost to train massive, sparse datasets.

A full-time Amazon scientist since February 2020, Vasiloudis now uses his experiences to help customers make the best of AWS resources and his colleagues make the most of working remotely. He has even introduced to his team the custom of fika, the Swedish habit of pausing for a cup of coffee in the middle of the day. Each Friday, he and his teammates congregate over a remote coffee break at 3 p.m., which has helped sustain the team’s spirit during the pandemic. We asked Vasiloudis about his internship, what it was like to make the transition to full-time employee, and more.

Q. What made you interested in working at Amazon, and how was your experience as an intern?

With Amazon, you have the opportunity to reach hundreds of millions of people with your work. You can make changes that affect the everyday lives of such a large population. Also, because of the number of Amazon users, you are forced to design algorithms that can actually analyze massive amounts of data. So that's a very interesting challenge for me, to be able to create scalable algorithms that work regardless of the size of the data set.

For my first internship, I worked with Alexa Shopping and we looked into ways to generate realistic data sets to improve the customer’s experience. The second internship was with AWS, where my manager was Vineet Khare, then an applied science manager. There, I worked on how to get gradient-boosted trees to work with massive data sets that contain millions and billions of records, but also millions and billions of features. From that work, in close collaboration with my mentor Hyunsu Cho, we wrote the paper that won the best short paper award at SIGIR 2019.

These were both good experiences, because I got to work on interesting problems. And most importantly, I got to work with great colleagues. We had multiple interns within the team, and that meant that you could share the experience of being a science intern with other PhD students, and support each other through the internship. My full-time colleagues were also very helpful and fun to hang out with outside work as well. So I had a good time, and that's the main reason why I chose to return to Amazon for the full-time role.

One of the things that I definitely learned during my internships was the importance of writing high-quality code.  A common problem when you're writing research code is that you kind of go along without ensuring that everything works in a formal way. Whereas when writing code for a company, you need to prove and ensure that your code will always work regardless of the circumstances. And this is one of the Amazon leadership principles: That we have to insist on the highest standards.

Q. What set apart the paper that won at SIGIR 2019?

Gradient-boosted trees are designed to deal with very large data sets and are one of the most popular machine learning algorithms, widely used in both academia and industry. However, whenever we deal with very large data sets, often we have to use multiple computers.

Imagine you're trying to classify, for example, text. Let's say that this text is somebody’s loan application. If every possible word in this text is a feature, that means there can be millions of features because the vocabulary is practically limitless. So, when you try to share the model training among multiple computers — which can be a hundred, a thousand, or even more — you will very often run into problems because they are all competing for a tiny amount of bandwidth compared to the data set.

Previous systems were not efficient at communicating because they were wasting a lot of bandwidth with redundant information. Many real-world data sets are very sparse. In sparse data sets, most of the features are actually zeroes. Previous systems were still sending those over the network, and they were consuming a lot of unnecessary bandwidth. Whereas if you only send the non-zeroes over the network, then you're actually saving communication costs and bandwidth. That’s the main idea.

Q. How did you go about trying to find a solution for those sparse data sets?

We had two issues to solve: One regarding prediction and another regarding training. You can imagine a data set as a matrix. It has a bunch of rows, which are the records — for example, the loan application documents. And then each of those will have a number of features, which are the words in the document. So, you can have millions of documents, and millions of features as well. In previous systems, they would only partition the data set along the record dimension. They would take a few documents and put them in one computer, a few in another and then do the training and sync.

But if you want to really speed up the process, you can actually take part of a document and store it in one computer and another part in another computer. This is called block distribution. Instead of taking multiple rows from the same matrix, and storing them in the same computer, now we start taking a block — a few rows and a few columns — from that matrix and put it in one computer. That means that we have some additional communication to do to make predictions.

We used an existing algorithm for that called Quickscorer, which was designed for a completely different purpose, to speed up the prediction process locally. But that exact same approach can allow you to perform a very quick distributed prediction, and we modified that algorithm to adapt our use case. So that's how we solved that prediction issue. And then for the training, we did something similar, where we would only send for a given block the number of records that are necessary with a number of features, and then we would use an aggregation step in order to complete the training.

I think this work provides a good direction for future production systems. The communication pattern for very large data sets should be more flexible than the one that is currently used.

Q. What are you currently working on?

I'm working on SageMaker JumpStart. We create AWS solutions that allow customers to get started with SageMaker faster, and take their ideas to production more quickly and painlessly.  One part of my team’s responsibilities is to work directly with customers when they have a specific problem. But we also do a lot of innovating on the behalf of our customers.

Q. You started your full-time role right at the beginning of the pandemic. Did that affect your work in any way?

We stopped going to the office and started using a digital form of communication. In trying to keep the team spirit alive, one of the things that I try to have in our team is something that we used to have in Sweden, which is called the fika. It’s like a coffee break where you stop working for half an hour and chat with your colleagues about anything you want. It’s just some social time where everybody can relax and interact with colleagues.

I saw that, with COVID-19, the interaction with colleagues goes down significantly, so it's good to have some time allocated in your calendar when you don’t have to work, just have some coffee and chat. An informal conversation is when a lot of important ideas come up, and it’s good to have that opportunity.

Q. What advice would you give to people considering following your footsteps?

If you have the opportunity to work at a company like Amazon, you should definitely take it, because you can gain a lot of experience that is impossible to gain during your PhD. The way that the industry works is very different from the way academia works. If you have done a couple of these internships, you're much more prepared to join the workforce.

For interns at Amazon hoping to migrate into a full-time job, I would say that the regular check-ins with your hiring manager are very important, because you need to be constantly aware whether you're on track for your full-time offer. Every second week you get to sit with your hiring manager, and you can check with them if you should be doing something more, if you're hitting your targets in terms of the progress of the work itself, and in terms of representing the leadership principles of Amazon in your work. And that gives you a better sense of accomplishment. You need to make sure that you set a few milestones in the meantime and make sure that you hit them as you progress through your internship.

Q. Any final tips on how to make the best out of your internship at Amazon?

Amazon values being independent and self-driven. And it's very good if you have a goal to publish a paper by the end of your internship and chase that publication. For example, we completed the writing of our paper after I had finished my internship, so if I hadn't pushed for that, I wouldn't have published this paper, and my co-authors and I wouldn't have gotten this award.

It's important to be motivated to work with your manager to make sure that you get all the necessary approvals before you finish your internship toward publishing the paper, because it's an important step for a career as a scientist, as well as for a PhD student, to publish high-quality papers. And it's a unique opportunity to do that when you have access to the infrastructure and data sets of Amazon.