Amazon at ACL: How to teach machines to reason

Amazon’s Dan Roth on a hot new research topic — that he’s been studying for more than 25 years.

As a senior area chair at this year’s meeting of the Association for Computational Linguistics (ACL), Dan Roth, who recently joined Amazon Web Services’ AI organization as science lead for natural-language processing, has a good vantage on paper submissions to the conference. On this year’s program, one theme leaped out at him.

Dan Roth.jpg
Dan Roth, science lead for natural-language processing in Amazon Web Services’ AI organization and the Glandt Distinguished Professor in the University of Pennsylvania’s Department of Computer and Information Science.

“I looked at some statistics of papers in ACL, and I saw that there are dozens of papers now that have ‘reasoning’ in the title,” says Roth, who is also the Glandt Distinguished Professor in the University of Pennsylvania’s Department of Computer and Information Science. “The title ‘learning to reason’ is now becoming sort of hot. I think a lot of AI is going in that direction.”

Machine reasoning, Roth says, is “the ability to make inferences, especially in ‘sparse’ situations that are unlikely to have been observed before”. The classic example is deduction: from the facts that all women are mortal and that Sappho is a woman, a machine reasoning system should infer that Sappho is mortal.

Roth is well situated to review recent progress in the field, as it’s been a topic of his own research for more than 25 years. 

“This was actually my PhD work,” he says. “Learning theory was an emerging field at that time. The questions were basically, How can we formalize learning, and what does it mean that something is learnable or not learnable? What are the computational-complexity issues in learning? I was trying to move this towards questions in reasoning, which were never studied from a theoretical perspective or computational-complexity perspective.

“The assumption was that someone gives you an input — a knowledge base, for example — and you present reasoning queries to it, and in this context you want to show what can be computed. My PhD thesis was about showing that if you don't start from a knowledge base, but you jointly do learning from data and reasoning from the resulting, intermediate representation, it’s easier than doing each one of them separately. You could say that end-to-end learning today is an instantiation of this learning-to-reason process, although just conceptually. Technically, the things are very, very different.”

Compositionality

Even though Roth is, in a sense, a pioneer of end-to-end reasoning models, he believes that more-complex reasoning problems will require more-complex modeling.

“We have a lot of hard problems that we are far from being able to address using just one model,” he says. “A lot of the problems will require thinking about things in a modular way. 

Amazon at ACL

Learn more about Amazon's involvement at ACL 2021 — research papers, workshops and tutorials, and committee memberships.

“I'll give you a simple example. I want to ask my virtual assistant, ‘Are we going to make it to dinner before the movie?’ What does this assistant need to do in order to respond to my question? It needs to know where I am now, where the movie is, how long it's going to take to get there — that's easy to do today. How long is dinner? I didn't say anything about it, but we have some idea of the typical length of dinner, maybe as a function of where dinner is. Do I need to find parking? I didn't mention parking. It's an implicit event, but we know that I have to park, maybe next to the dinner place, maybe next to the movie. I have to factor this in.

“So I have to have models that know how to compute things, have some common sense — typical time of dinner, typical time of finding parking, driving between these places. And then I need a model that knows how to put this together. It's not going to be the same model, because I'm not going to train on each question. Many of the problems that we want to address are like that, where there's modularity, and we will never be able to move forward without realizing that there is modularity.”

Symbolic reasoning

Moreover, Roth says, the systems that integrate these separate modules will almost certainly need to use symbolic reasoning, or rule-based manipulation of symbolic representations.

“The growth and the excitement around neural networks has left symbols behind,” Roth says. “Some people think that symbols are an evil invention of the old AI people. But symbols were invented because they’re useful, necessary abstractions. And also, explanations are symbolic, right? When you ask me, ‘Why did you decide this?’ or ‘Why is this implied by that?’, I need to explain it to you, and I need to use symbols when I do this. So I think we are beginning to explore this interesting space between models that are continuous, if you like, and interactions that are largely symbolic.

Some people think that symbols are an evil invention of the old AI people. But symbols were invented because they’re useful, necessary abstractions
Dan Roth

“I'll give you an example. I've worked a lot on reasoning about time, as expressed in natural-language text. If you want to reason about events, you have to use the fact — and people do it all the time — that time is transitive. If A happens before B, and B happens before C, then A happens before C. This will never be written explicitly. So we kind of tell our models ‘Time is transitive’, and we can show that this helps a lot.”

The transitivity of time, however, is something that can be represented in the architecture of a neural network. That won’t always be the case, Roth explains.

“There are some cases where only in postprocessing are you aware of some declarative constraints,” Roth says. “Once you evaluate your model, once you decode, once you make the decision — only then do you want to impose a declarative constraint. Sometimes there are constraints that I was unaware of while I was training the model: the model is fixed, I trained it yesterday, but now I'm using it in a given situation where I'm aware of a constraint, and I want to be able to impose it. And there is very interesting theoretical work that people are doing now on trying to understand the advantages and disadvantage of these two paradigms — when which one is better. But the fact of the matter is that we need both.”

“In the last five years, deep neural networks have had a huge impact, especially in the context of natural language,” Roth adds. “There's a lot of excitement, for good reason. But sooner or later, people get to the realization that that's not sufficient. I think today, more and more people are beginning to think about reasoning problems and the need to decompose and compose to address them.”

Related content

US, WA, Seattle
Job summaryEmployer: Amazon.com Services LLCPosition: Data Scientist IILocation: Seattle WAMultiple Positions Available:1. Perform analytical tasks on vast amounts of structured and unstructured data to extract actionable business insights2. Develop algorithms using advanced mathematical and statistical techniques including machine learning to predict business outcomes and recommend optimal actions to management3. Use advanced algorithms to solve business problems, and transform one-off models into automated systems4. Analyze and validate data to ensure high data quality and reliable insights5. Run analytical experiments in a methodical manner to find opportunities for product and process optimization6. Communicate analytical output and business insights to management using visualization techniques and data storytelling7. Write concise documents communicating results to stakeholders and visualize data to drive decision making#0000
US, VA, Arlington
Job summaryAmazon is looking for a passionate, talented, and inventive Research Scientists with a strong machine learning background to help build industry-leading Speech and Language technology. This includes delivering innovative HR experiences for our employees. To get there, we need exceptionally talented, bright, and driven people. The Employee Services Technology (ES Tech) is growing rapidly and is looking for a talented senior research scientist to create massive scale and powerful new solutions that our customers need to be successful. In this role, you will work to establish world class data science, analytics and reporting for Amazonians as part of building the Personalization Engine for myHR. This key role will work closely with internal partners to assist in developing and managing solutions for ES Tech. We are looking for candidates who want to invent the future for employee engagement.This role will play critical part in forecasting initiatives, responsible for building models and prototypes for ES Tech division, and will require close collaboration with other scientists on the team that are developing state-of-the-art optimization algorithms to scale. This team plays a significant role in various stages of the innovation pipeline from identifying employee needs, developing new algorithms, prototyping/simulation, to implementation by working closely with colleagues in engineering, product management and employee benefits.We are seeking a Senior Research Scientist with expertise in mixed-methods research, preferably in social science and behavioral research. In this role, you will lead and support research efforts within the Recruiting, Talent Management, and Leadership & Development space.You will help set the direction for science and research in the organization. You will be a thought leader on the team, partnering with a diverse set of stakeholders to identify and develop impactful areas for novel research about talent and recruiting outcomes, mechanisms, and programs. You will mentor and provide scientific expertise/peer review to other scientists and analysts on the team.The ideal candidate should have strong problem solving skills, excellent business acumen, as well as an expertise in both qualitative and quantitative methods. This role will need to navigate complex and ambiguous business challenges by asking the right questions, understanding what methodologies to employ, and communicating results to multiple audiences (e.g., technical peers, functional teams, business leaders). Translating business and stakeholder needs into realistic and actionable scientific research will be a regular challenge in this role.Responsibilities include:· Leading scientific direction, providing consult, mentorship, and peer review.· Partnering closely and driving effective collaborations across multi-disciplinary science, analytics, HR, and business teams. Reviewing and scoping research requests, and recommending appropriate scientific methodologies.· Designing, developing, and executing qualitative and quantitative data collection, research, and experimentation. Strong experience with statistical analysis required. Experience with survey development and experimental design (including quasi-experimental) preferred.· Communicating findings and business impact effectively (written and verbally) with both technical and non-technical stakeholders
US, VA, Arlington
Job summaryAmazon is looking for a passionate, talented, and inventive Research Scientists with a strong machine learning background to help build industry-leading Speech and Language technology. This includes delivering innovative HR experiences for our employees. To get there, we need exceptionally talented, bright, and driven people. The Employee Services Technology (ES Tech) is growing rapidly and is looking for a talented senior research scientist to create massive scale and powerful new solutions that our customers need to be successful. In this role, you will work to establish world class data science, analytics and reporting for Amazonians as part of building the Personalization Engine for myHR. This key role will work closely with internal partners to assist in developing and managing solutions for ES Tech. We are looking for candidates who want to invent the future for employee engagement.This role will play critical part in forecasting initiatives, responsible for building models and prototypes for ES Tech division, and will require close collaboration with other scientists on the team that are developing state-of-the-art optimization algorithms to scale. This team plays a significant role in various stages of the innovation pipeline from identifying employee needs, developing new algorithms, prototyping/simulation, to implementation by working closely with colleagues in engineering, product management and employee benefits.We are seeking a Senior Research Scientist with expertise in mixed-methods research, preferably in social science and behavioral research. In this role, you will lead and support research efforts within the Recruiting, Talent Management, and Leadership & Development space.You will help set the direction for science and research in the organization. You will be a thought leader on the team, partnering with a diverse set of stakeholders to identify and develop impactful areas for novel research about talent and recruiting outcomes, mechanisms, and programs. You will mentor and provide scientific expertise/peer review to other scientists and analysts on the team.The ideal candidate should have strong problem solving skills, excellent business acumen, as well as an expertise in both qualitative and quantitative methods. This role will need to navigate complex and ambiguous business challenges by asking the right questions, understanding what methodologies to employ, and communicating results to multiple audiences (e.g., technical peers, functional teams, business leaders). Translating business and stakeholder needs into realistic and actionable scientific research will be a regular challenge in this role.Responsibilities include:· Leading scientific direction, providing consult, mentorship, and peer review.· Partnering closely and driving effective collaborations across multi-disciplinary science, analytics, HR, and business teams. Reviewing and scoping research requests, and recommending appropriate scientific methodologies.· Designing, developing, and executing qualitative and quantitative data collection, research, and experimentation. Strong experience with statistical analysis required. Experience with survey development and experimental design (including quasi-experimental) preferred.· Communicating findings and business impact effectively (written and verbally) with both technical and non-technical stakeholders
US, CA, San Francisco
Job summaryAbout Us:Launched in 2011, Twitch is a global community that comes together each day to create multiplayer entertainment: unique, live, unpredictable experiences created by the interactions of millions. We bring the joy of co-op to everything, from casual gaming to world-class esports to anime marathons, music, and art streams. Twitch also hosts TwitchCon, where we bring everyone together to celebrate, learn, and grow their personal interests and passions. We're always live at Twitch. Stay up to date on all things Twitch on Linkedin, Twitter and on our Blog.About the Role:Amazon Interactive Video Service (IVS) is a managed live video service, within Amazon Web Services (AWS), built on top of Twitch Video. It has one of the world's largest live video networks. IVS powers the many companies that are using live video to uniquely engage and build communities, just as it does for Twitch, where more than a billion hours are streamed live each month.The IVS Video Intelligence team, a central data team for all of IVS, is looking for a principal applied scientist to create a strategic impact for the entire IVS business. As part of the team, you will shape the data and science strategy for the whole of IVS. You will both invent new solutions and bring your knowledge of ML ops best practices to establish cross-company standards. You will oversee our novel end-to-end predictive simulation of our video streaming infrastructure to test product hypotheses, forecast demand and capacity needs, and cover failure scenarios. You will report to the Video Intelligence team lead.If you are experienced in predictive analytics and have a passion for deep technical problems at consumer web scale, come help us grow an enterprise cloud service. You will work in our San Francisco office, but we will also consider other locations including Seattle (WA), Salt Lake City (UT), or US-Virtual.You Will:• Create the organizational science strategy.• Invent new technologies to give live video broadcasters and viewers a better overall experience.• Become a domain expert real-time video streaming tracking and measurement, building trust with your product and infrastructure partners.• Mentor junior team members and promote analytics and experimentation best practices throughout the company.
US, WA, Bellevue
Job summaryAt Amazon, we're working to be the most customer-centric company on earth. To get there, we need exceptionally talented, bright, result oriented, and driven people. Amazon is seeking a Simulation Engineer to assist in designing and optimizing the middle mile concepts and process improvements using discrete event simulations. Successful candidates will be natural self-starters who have the drive to design, model, and simulate new concepts and processes. The Simulation Engineer will be expected to deep dive problems and drive relentlessly towards creative solutions. This individual needs to be comfortable interfacing and driving various functional teams and individuals at all levels of the organization in order to be successful. Perform process modelling and simulation using discrete event simulation software’s, process optimization, statistical data analysis, and Design of Experiments (DOE) etc. to drive decisions on process and designs. Need based remote work option is available.Responsibilities:· Lead system level complex Discrete Event Simulation (DES) projects to build , simulate, and optimize the fulfillment center operational process flow models using FlexSim, Demo 3D, AnyLogic or any other Discrete Event Simulation (DES) software packages· Understand process flows , analyze data, perform Design of Experiments and effectively represent in simulation model to achieve better correlation and process improvements· Manage multiple DES simulation projects and tasks simultaneously and effectively influence, negotiate, and communicate with internal and external business partners, contractors and vendors.· Facilitate process improvement initiatives among site operations, engineering, and corporate systems groups.· Utilize code (python or another object oriented language) for data analysis and modeling algorithms· Analyze historical data to identify trends and support decision making using Statistical Techniques· Lead and coordinate simulation efforts between internal teams and outside vendors to develop optimal solutions for the network, including equipment specification, material flow, process design, and site layout.· Deliver results according to project schedules and quality· Provide written and verbal presentations to share insights and recommendations to audiences of varying levels of technical sophistication.· Make technical trade-offs for long term/short-term needs considering challenges in business area by applying relevant data science disciplines, and interactions among systems.
US, PA, Pittsburgh
Job summaryAmazon is looking for passionate Applied Scientists with expertise in Machine Translation and Information retrieval to help us expand our MT and NLP technology for search.Amazon’s International Search team owns defining and delivering Amazon’s cutting-edge Search and Machine Translation initiatives to customers and cultures across multiple regions. We do this by developing our own customer-facing features, services and platforms based on customer needs, and through partnerships with Amazon technical teams around the world including Central organizations.You will work with the largest online retail search application in the world, both in terms of users, catalogue size, and computing resources, and your work will directly impact millions of our customers. You will collaborate with recognized experts in business, science and engineering.Our mission is to enable a superior experience for all of Amazon’s customers in their native language by developing and deploying state-of-the art technology and applications in Machine Translation (MT), Natural Language Processing (NLP) and Machine Learning (ML). The global scale of Amazon means true big data problems, big data opportunities, and building for global customers. Machine Translation allows Amazon to reach people in their preferred language. We are a team tackling the tough language problems that general-purpose MT cannot solve.As part of our MT research & development team, you will work on developing, deploying, maintaining and supporting our fleet of Machine Translation systems, particularly focused on Search. You will collaborate with recognized experts in science and engineering to develop novel algorithms and modeling techniques to advance the state-of-the-art in MT, search and information retrieval. You will partner closely with our engineering teams, publish scientific papers and apply for patents for your inventions. Your work will directly impact millions of our customers in the form of products and services that make use of our MT and NLP technology. You will gain hands-on experience with Amazon’s heterogeneous language data sources and large-scale computing resources to accelerate advances in MT and NLP.Key job responsibilities* Drive scientific exploration and innovation for MT improvements and evaluation techniques* Train and evaluate neural MT models using Amazon's proprietary MT technology* Analyze the accuracy and performance of MT models for search* Assist in deploying, maintaining, and supporting Amazon's fleet of dedicated MT systems in production* Design and implement software to collect, analyze and prepare large natural language data sets
DE, BY, Munich
Job summaryGlobal Workforce Staffing (WFS), a division of Amazon’s Worldwide Operations People Experience and Technology (PxT, aka Human Resources in other companies), manages Amazon’s Tier 1 talent supply chain. We attract, hire, and onboard the associates who, by fulfilling orders at the frontlines of the company, make Amazon a global leader in delivery and logistics. The Market Intelligence ensures that Amazon can deliver an industry leading customer delivery experience while raising the bar for its largest candidate and employee population--Tier 1 Associates. We ensure that Amazon can take scale globally in a consistent manner, accounting for the specific regional needs and characteristics of European countries.We are seeking a Manager, Data Science, with a heavy focus on quantitative data analysis and evaluation, and a deep focus on understanding European labor markets. You will be responsible for leading a new global expansion team from the ground up, develop research roadmaps, run experiments, and drive business impact through your research at global scale.The ideal candidate should be well versed in quantitative methods, including classical statistics and machine learning approaches. Competitive candidates will be very comfortable with at least one computational language (e.g., Python, R). Candidates should be comfortable selecting and leading their team through deployment of the best fit computational models and machine learning algorithms for analyzing their output.Candidates should have demonstrated experience leading data science and analytics projects related to labor market research and analysis, including research on wage sensitivity and elasticity, addressable workforce sizing, competition, and other factors.A day in the lifeYou will lead a team of Data Scientists and BI Engineers, and partner with Data and Software Engineers in scaling intelligence by country pertaining to market presence (site selection), launch risk, and hiring risk. You will build strong trusting relationships with key business partners to influence and drive success in their operations. You will continually coach and mentor a team to be successful in agile research iterations, working hand-in-hand with subject matter experts as humans-in-the-loop refining and ensuring the adoption of your models and products.
GB, MLN, Edinburgh
Job summaryAre you a MS or PhD student interested in a 2022 Applied Science Internship in the fields of Speech, Robotics, Computer Vision, or Machine Learning/Deep Learning?Do you enjoy diving deep into hard technical problems and coming up with solutions that enable successful products that improve the lives of people in a meaningful way?If this describes you, come join our research teams at Amazon. As an Applied Science Intern, you will have access to large datasets with billions of images and video to build large-scale machine learning systems. Additionally, you will analyze and model terabytes of text, images, and other types of data to solve real-world problems and translate business and functional requirements into quick prototypes or proofs of concept.We are looking for smart scientists capable of using a variety of domain expertise combined with machine learning and statistical techniques to invent, design, evangelize, and implement state-of-the-art solutions for never-before-solved problems.Machine Learning Science:Amazon has multiple positions available for Applied Scientists in Berlin, Munich, Tuebingen, Cambridge, Edinburgh, London, Iasi and Barcelona..A few of the teams that are hiring include:· Core AI· · Amazon Search· · AWS AI· · Advertising Technologies· · Community Shopping· · Prime VideoSpeech and Language Technology:We are hiring in all areas of spoken language understanding: ASR, NLP, NLU, text-to-speech (TTS), and Dialog Management. Amazon has multiple positions available for Speech Scientists in Aachen, Barcelona, Berlin, Cambridge, Edinburgh, Gdansk, Haifa, Tel Aviv and Turin.A few of the teams that are hiring currently include:· Alexa ML· · Alexa Brain· · Alexa Shopping· · Amazon Search· · CS TechnologyComputer Vision:Amazon has multiple positions available for Computer Vision Scientists in locations such as Berlin, Barcelona, Tuebingen, Haifa and Tel Aviv.We are currently hiring for multiple teams including:· Visual Search· Amazon AI (AWS Rekognition)· Amazon Go· Lab126
TW, TPE, Taipei
Job summaryThere is nothing COOLER than thinking and developing a brand-new IoT protocol like Sidewalk.Amazon Sidewalk is looking for a Sr. Data Scientist to help build and maintain analytics platforms in Sidewalk Network Health and Analytics team.About the RoleAs a Sr. Data Scientist, you will work closely with internal teams to improve and solve large scale analytics. You will leverage your analytical skills to provide insights on the end-to-end architecture and drive optimization. You must be willing to own, you will be responsive, flexible and able to succeed within an open collaborative peer environment. You must also be able and willing to multi-task and open to learn new technologies quickly. You are comfortable with proactive outward communication and technical leadership and you never shy away from a challenge.Ideally, you have hands-on experience with python and jupyter notebooks, you have familiarity with AWS infrastructure and running cloud operations.A successful candidate will need excellent communication skills and the ability to interact with software developers and managers that will be contributing and leveraging the analytics platform. If you are interested on building and managing large scale platforms, this position will provide you with a unique opportunity to deploy analytics at scale in a platform for real-time operations.About the team : Amazon SidewalkAmazon Sidewalk is a new long-term effort to greatly extend the working range of low-bandwidth, low-power, smart lights, sensors, and other low-cost devices customers install at the edge of their home network. We believe customers shouldn’t have to settle for connected devices that lose functionality past the front door. With Amazon Sidewalk, customers will be able to place smart devices anywhere on their property and know the devices will work, even in dead spots where Wi-Fi and Bluetooth don’t reach, helping bridge the connectivity gaps around our homes!Key job responsibilities· Broad cross-functional engineering interaction to deliver analytics real-time for Amazon Sidewalk· Strong analytical and quantitative data-analysis; use hard data and metrics to back up assumptions, recommendations, and drive actions· Drive system level integration of telemetry and analytics.
IT, Turin
Job summaryAmazon is looking for a Data Scientist with a passion for languages to join our Alexa AI - Natural Understanding Turin team. We are seeking a candidate with strong analytical skills and Natural Language Processing (NLP) experience to help us develop language components for a variety of Alexa products. Come join the Alexa team, building the speech and language solutions behind Alexa, Amazon Echo and other Amazon products and services! You will help us invent the future. As a NLP Data Scientist of the Alexa AI - NU Team, you will work close with Language Engineers to build and releases NLU models in production and improve them. You will gain hands-on experience with Amazon’s heterogeneous structured data sources; as well as large-scale computing resources to accelerate advances in training deep neural networks for natural language understanding. You will take lead on solving highly visible and impactful business problems in areas of automation, self-service solution and quality improvement to continue delight Alexa customers and help driving Amazon business performance. The ideal candidate is clearly passionate about delivering experiences that delight customers and creating solutions that are robust. Creating reliable, scalable and high performance products requires exceptional technical expertise, and a sound understanding of the fundamentals of Machine Learning, NLP, Linguistic and Problem solving. This role requires working closely with business, engineering and other scientists within the team and across Amazon to raise the bar in operational excellence, improving tools and automating workflows. You will lead high visibility and high impact programs collaborating with various teams across Amazon. You will focus on deliver results with the right quality and in a timely fashion. Your bias for action will be critical to move quickly on projects, with calculated risk taking.