Amazon today announced that ten teams from around the globe have been selected to participate in the Alexa Prize TaskBot Challenge year 2, a university challenge focused on developing multimodal (voice and vision) conversational agents that assist customers in completing tasks requiring multiple steps and decisions.
Alexa Prize is a flagship industry-academic collaboration dedicated to accelerating the science of conversational artificial intelligence (AI) and multimodal human-AI interactions.
“Prize competitions provide an agile science experimentation framework for researchers and students encouraging them to explore transformational ideas at the boundaries of what is achievable,” said Reza Ghanadan, senior principal scientist with Alexa AI and head of Alexa Prize. “We have developed the CoBot platform and tools to lower the barriers to AI innovation for both the academic research community and students interested in conversational AI assistants. These tools allow students to quickly deploy their solutions at scale in the real world with Alexa, then observe, evaluate, and enhance their research results using feedback from Alexa customers.”
The teams selected for the challenge, which began in January, feature five returning entrants — including the top three finishers in the most recent challenge — and five new universities.
Team | University | Faculty advisor |
Returning | ||
| TWIZ | NOVA School of Science and Technology | João Magalhães |
| EvoquerBOT | Penn State University | Rui Zhang |
| Taco 2.0 | The Ohio State University | Huan Sun |
| GRILL | University of Glasgow | Jeff Dalton |
| Maruna | University of Massachusetts Amherst | Hamed Zamani |
New | ||
| BoilerBot | Purdue University | Julia Rayz |
| DiWBot | Rutgers University | Matthew Stone |
| Sage | University of California, Santa Cruz | Xin (Eric) Wang |
| ISABEL | University of Pittsburgh | Malihe Alikhani |
| PLAN-Bot | Virginia Tech | Ismini Lourentzou |
The prizes for overall performance in the competition will be $500,000 for the first-place team, $100,000 for second, and $50,000 for third. Those prizes will be paid out to the students on the teams with the best overall performance.
“I am delighted to see that new teams are joining the second year of the competition together with returning teams, who, by competing again, are signaling to us that they found value in the TaskBot challenge, said Yoelle Maarek, vice president research and science for Amazon Shopping.
“We expect these talented graduate students to continue surprising us, as well as Amazon customers, this year. Connecting academia, Amazonians, and actual customers experimenting with taskbots, is a winning combination to keep pushing the boundaries of science in conversational AI for Alexa to delight and ease the lives of millions of customers.”
The Alexa Prize is a competition for university students dedicated to advancing the field of conversational AI. Launched in 2016, the program was created to recognize students from around the globe who are changing the way we interact with technology.
TaskBot Challenge 2 teams are working to address one of the hardest problems in conversational AI — creating next-generation conversational AI experiences that delight customers by addressing their changing needs as they complete complex tasks. This challenge builds upon the Alexa Prize’s foundation of providing universities a unique opportunity to test cutting-edge machine learning models with actual customers at scale.
The Alexa Prize TaskBot challenge provides a realistic scenario with real-user multimodal interactions, making this the perfect setting to observe and measure human-bot conversations and AI algorithms in a groundbreaking setting.
Our vision of EvoquerBOT combines improving task completion rates and elevating user satisfaction. To this end, we deliver innovative solutions to fundamental NLP challenges.
We are especially interested in developing innovative ways to achieve successful coordination of multiple modalities, such as visual and verbal elements, and create a more engaging and intuitive user experience.
The GRILL team is excited to continue bringing cutting-edge AI research to improve people’s lives. Our research team works on new capabilities of foundation models that understand text, images, and the surrounding world.
The competition lets us create interfaces for the general public in a production environment – it’s a unique opportunity to connect our research with our career goals.
We are very excited to be part of the community and look forward to working with the Alexa team and other teams.
The Alexa Prize TaskBot Challenge combines a vast range of tasks over multiple domains with multimodal outputs. This is the ultimate test for any moonshot concept, and we can't wait to see what the real world has in store for us.
Participating in this competition is an incredible opportunity that will allow us to do applied research and ship it to real users.
Although artificial intelligence has experienced explosive development in the past decade, there is still a gap between research and real-world application. The TaskBot Challenge provides us with a unique opportunity to explore multimodal AI in practical situations.
Our bot will make adaptable conversation a reality by allowing customers to follow personalized decisions through the completion of multiple, sequential sub-tasks and adapt to the tools, materials, or ingredients available to the user by proposing appropriate substitutes and alternatives.
TaskBot is the first conversational AI challenge to incorporate multimodal customer experiences, so in addition to receiving verbal instructions, customers with Echo Show or Fire TV devices, can also be presented with step-by-step instructions, images, or diagrams that enhance task guidance.
This year’s challenge has been expanded to include more hobbies and at-home activities. Participating teams were asked to propose interesting ways to incorporate visual aids into every conversation turn when a screen is available. Innovative ideas on improving the presentation of visual aids, as well as the coordination of visual and verbal modalities, were part of the team selection criteria.
Each university selected for the challenge receives a $250,000 research grant, Alexa-enabled devices, free Amazon Web Services (AWS) cloud computing services to support their research and development efforts, access to Amazon scientists, the CoBot (conversational bot) toolkit and other tools such as automated speech recognition through Alexa, neural detection and generation models, conversational data sets, and design guidance and development support from the Alexa Prize team.
"Alexa, let's work together"
The university teams’ taskbots will be available for Alexa customers to engage with in May 2023 with a finals event being held in September, and winners announced later that month.
As with the previous challenge, Alexa customers can engage in conversation with teams’ taskbots when they become available in May by saying, “Alexa, let’s work together.” Until then, “Alexa, let’s work together” will direct you to conversations with the previous challenge winners of 2022 and the Alexa Prize TaskBot.
After initiating the interaction, Alexa customers then receive a brief message informing them that they are interacting with an Alexa Prize university taskbot before being randomly connected to one of the participating taskbots.
After exiting the conversation with the taskbot, which customers can do at any time, the customer is prompted for a verbal rating, followed by an option to provide additional feedback. The interactions, ratings, and feedback are shared with the teams to help them improve their taskbots. Customer ratings are also used to determine which university teams will move on to the semifinals and finals.
Our goal is to contribute to the multimodal conversational AI field and move it closer to the way humans perceive, reason, and communicate through multimodal information.
We look forward to the Challenge because it is the perfect platform to create multimodal, tasked-oriented dialogue systems that elevate user experience and engagement.
Through this TaskBot Challenge, we hope our work can expand the horizon of conversational AI along dimensions like dialogue depth, multi-modal coordination, commonsense reasoning, and learning from use.
The GRILL team is creating the next generation of open assistants that understand and use knowledge about the world and can communicate effectively to inform and educate.
Our TaskBot will help people get things done through personalized, adaptive, and context-aware conversational interaction by combining our research results with the state-of-the-art capabilities of Alexa devices.
We work towards making conversational AI technology more inclusive and collaborative. Inclusive Alexa can collaborate with users from diverse cultures and with different communication capabilities and preferences.
We hope to develop a task-oriented system that can interact with users based on their level of knowledge, experience, and communication preference.
Success in the previous TaskBot Challenge required teams to address many difficult AI obstacles. The challenge required the fusion of multiple AI techniques including knowledge representation and inference, commonsense and causal reasoning, and language understanding and generation.
The “GRILLBot” team from University of Glasgow won the TaskBot 1 Challenge, earning a $500,000 prize for its performance. Teams from NOVA School of Science and Technology (Portgual) and The Ohio State University earned second- and third-place prizes, respectively.
Research papers from Amazon’s Alexa Prize team, and each of the competing teams, can be viewed and downloaded here.
