AVA: an automatic eValuation approach for question answering systems

2021
Download
Abstract
We introduce AVA, an automatic evaluation approach for Question Answering, which given a set of questions associated with Gold Standard answers (references), can estimate system Accuracy. AVA uses Transformer-based language models to encode question, answer, and reference texts. This allows for effectively assessing answer correctness using similarity between the reference and an automatic answer, biased towards the question semantics. To design, train, and test AVA, we built multiple large training, development, and test sets on public and industrial benchmarks. Our innovative solutions achieve up to 74.7% F1 score in predicting human judgment for single answers. Additionally, AVA can be used to evaluate the overall system Accuracy with an error lower than 7% at 95% of confidence when measured on several QA systems.
Download

Work with us

See more jobs
GB, Cambridge
Do you want to work on some of the most innovative Text-to-Speech technology in the industry? Well, you’ve come to the right place.We are looking for a passionate, talented, and inventive Machine Learning Scientist with a strong background in Machine/Deep Learning/NLP/Speech and Software Development to help build industry-leading Speech and Language technology. Our mission is to push the envelope in Text-to-Speech (TTS) in order to provide the best-possible experience for our customers. Your role is to leverage your strong background in Computer Science and Machine Learning to help build the next generation of our synthetic voices used by millions of customers every day.A day in the lifeNo two days are the same as a Applied Scientist in TTS Research!You will be working in a dynamic, fast-moving environment. You’ll be developing new Deep learning models to improve the quality of our speech models, running experiments and writing papers and patents or making prototype solutions production ready.About the hiring groupThis role sits with TTS, and the goal of the team is to create natural, friendly and emotional voices for Alexa.Job responsibilitiesAs an Machine Learning Scientist on our team you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in speech synthesis. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to build Machine Learning models for their application in speech synthesis.This role requires a pragmatic technical leader comfortable with ambiguity, capable of summarizing complex data and models through clear visual and written explanations.The ideal candidate will have experience with machine learning models and their application in AI systems. We are particularly interested in experience applying natural language processing, deep learning, and speech generation at scale. Additionally, we are seeking candidates with strong interest in applied sciences and engineering, creativity, curiosity, and great judgment.Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice to know more about how we collect, use and transfer the personal data of our candidates.
IN, KA, Bangalore
The Amazonian Experience and Technology (AET) team is looking for a passionate Applied Scientist with experience in Natural Language Processing to help us fulfill our goal of becoming the most scientific people experience and technology organization in the world. In this role, you will work with service owners across the globe to help automate manual processes to help us solve employee problems faster and more efficiently. You will come across challenging problems, using data to solve issues such as contact classification, automatic translation, intelligent routing, and text autocompletion that will be used by employees and AET agents to improve the experience of Amazonians reaching out to HR.About the hiring groupThe Data and Analytics Services, Research and Applied Science team provides rigorous scientific support to business partners across the globe within the Amazonian Experience and Technology organization. Our goal is to make operations as efficient as possible by leveraging science to make sure the experience of Amazonians when contacting HR is as frustration free as possible. In this team, you will work with Economists, Applied Scientists, Data Scientists and Software Development Engineers to help solve complicated problems for Amazon's ever growing employee population.Job responsibilities· Analyze data and metrics generated by Amazonian interactions with Human Resources to find opportunities to improve our services.· Design, build, and deploy effective and innovative Machine Learning and Natural Language solutions to improve various components of our processes, like contact routing, case classification, translation, and fraud detection.· Evaluate the proposed solutions via offline benchmark tests as well as online A/B tests in production.· Publish and present your work at internal and external scientific venues
US, WA, Seattle
Amazon’s mission is to be the most customer-centric company in the world. The vision of Workforce Intelligence is to design the ideal workforce to meet the customer promise anywhere. This organization leads and influences global workforce strategies that enable Amazon to scale operations more efficiently while also providing a unique voice for the hourly workforce. This is accomplished through a variety of science driven initiatives, experimentations, ML driven modeling, and data engineering. Workforce Intelligence is on the front lines of that mission by providing robust research, data science and analytics to fill our jobs across the globe.Key responsibilities include:· As a scientist in our team, you will conduct primary research to capture stated candidate preferences and explore candidate revealed preferences.· Conduct secondary research to generate actionable insights by analyzing historical data from databases (Redshift, SQL Server, and Salesforce).· Conduct interviews, focus groups, and observations to deeply understand our candidates.· Develop and own models of candidate decision making and conduct experiments to validate and refine those models.Inclusive Team CultureHere at Amazon, we embrace our differences. We are committed to furthering our culture of inclusion. We have 12 affinity groups (employee resource groups) with more than 87,000 employees across hundreds of chapters around the world. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which reminds team members to seek diverse perspectives, learn and be curious, and earn trust.FlexibilityIt isn’t about which hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We offer flexibility and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthWe care about your career growth too. Whether your goals are to explore new technologies, take on bigger opportunities, or get to the next level, we'll help you get there. Our business is growing fast and our people will grow with it.
US, NY, New York City
Description:Take Earth's most customer-centric company. Mix in hundreds of millions of shoppers spending tens of billions of dollars annually, an exciting opportunity to build next-generation shopping experiences, Amazon’s tremendous computational resources, and our extensive e-Commerce experience. What do you get? The most exciting position in the industry.About our team:Our mission is to delight every Amazon customer with a personalized shopping experience. We achieve our mission through investments in UX, Science, and Systems with the purpose of delivering the future of shopping on Amazon. We are seeking an Applied Scientist to work on step function science improvements across the recommendations space.About our organization:Our team is part of Amazon’s Personalization organization, a high-performing group that leverages Amazon’s expertise in machine learning, big data, distributed systems, and user experience design to deliver the best shopping experiences for our customers. We run global experiments and our work has revolutionized e-commerce with features such as “Customers who bought this item also bought”, “Frequently bought together”, and "Keep shopping for ...". Amazon’s internal surveys regularly recognize us as one of the best engineering organizations to work for in the company, with visible high-impact work, low operational load, respectful work-life balance, and continual opportunity to learn and grow.About the role:As an Applied Scientist on the team you will be working on cutting edge ways to help customers find the right products and content on their shopping journey. Our goal is to make step function improvements in the use of advanced Machine Learning (ML) on very large scale datasets. This is an exciting initiative to develop cutting edge ML solutions and apply them to a problem of this magnitude. You will be part of a multidisciplinary team, working on one of the largest scale machine learning systems in the company.You will hone your skills in areas such as deep learning and reinforcement learning while building scalable industrial systems. As a member of a highly leveraged team of talented engineers and ML scientists, you will have a unique opportunity to help build infrastructure that accesses terabytes of data to produce and deliver models with low latency and high reliability.The goal is to innovate new features and models that can make a huge impact on the customer experience.You will play a critical role in ideation for the team. We are building the next generation ML system that powers the biggest internet retailer on earth, and we hope you will join us!Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.
US, CA, San Diego
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLC, an Amazon.com companyTitle: Applied Scientist IILocation: San Diego, CAPosition Responsibilities:Participate in the design, development, and evaluation of highly innovative models for risk management. Use statistical and machine learning techniques to create scalable risk management systems. Analyze and understand Amazons historical business data for specific instances of risk and/or broader risk trends. Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation, and model implementation. Track general business activity and provide clear, compelling management reporting. Research and implement novel ML and statistical approaches to add value to the business. Work closely with software engineering teams to drive real-time model implementations and new feature creations. Work closely with operations staff to optimize risk management operations.Amazon.com is an Equal Opportunity-Affirmative Action Employer - Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.#0000
US, CA, Palo Alto
MULTIPLE POSITIONS AVAILABLEEntity: Amazon.com Services LLC, an Amazon.com CompanyTitle: Research Scientist IIIWorksite: Palo Alto, CAPosition Responsibilities:Interact with various software and business groups to develop an understanding of their business requirements and operational processes. Utilize acquired knowledge and business judgment to build decision-supporting and operational tools to improve the bottom line. Build quantitative mathematical models to represent a wide range of supply chain, transportation and logistics systems. Implement these models and tools through the use of modeling languages and engineering code in software languages such as Java, C++, C# or C. Gather required data for analysis and mathematical model building by writing ad-hoc scripts and database queries. Perform quantitative, economic, and numerical performance analyses of these systems under uncertainty using statistical and optimization tools. Create computer simulations to support operational decision-making. Identify areas with potential for improvement and work with internal teams to generate requirements to realize improvements. Design optimal or near optimal solution methodologies to be used by in-house decision support tools and software. Create software prototypes to verify and validate the devised solutions methodologies. Integrate prototypes into production systems using standard software development tools and methodologies.Amazon.com is an Equal Opportunity-Affirmative Action Employer - Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation.#0000
CA, BC, Vancouver
Amazon Brand Protection organization focuses on building trust with all brands by accurately representing and completely protecting their brands on Amazon. We strive to be the most trusted thought leader in the space and ensure that public perception mirrors the trustworthy experience we deliver. The Brand Protection machine learning (ML) team is responsible to provide data driven long term strategies and solutions. The team is responsible to develop the state of art ML algorithms to ensure each product is brand authentic and to ensure no abuse or infringements on any brands. The ML team faces the challenges to work with huge amount of structured and unstructured data including images and product descriptions and to develop ML solutions that can scale to protect millions of brands and billions of products worldwide. The team also faces the challenge to fast update our ML systems to stay ahead of bad actors who constantly circumvent our algorithms. If you are excited at these responsibilities and challenges and if you love data and machine learning, we have a position for you. We are looking for a strong manager to manage the ML science team in Vancouver.As the manager, you will hire and develop ML talents. You will design long terms plans and define SMART goals. You will build roadmaps to achieve team’s vision and goals. You will lead the ML directions. You will lead roadmap and plan executions. You will be able to deep dive and guide your team both in directions and in details. You understand ML cycles and advocate ML best practices. You will keep abreast with new ML technologies.Major responsibilities:· Work with business/tech teams to identify opportunities, design solution, implement and monitor ML models.· Understand business challenges by analyzing data and customer feedback· Guide team members on model building strategies and model experiment, implementation, measurement and continuous improvement· Build and manage team roadmaps· Create long term plans to address complicated business problems at scale using ML· Deep dive to provide business insights· Create business and analytics reports and present to the senior management teams· Lead research and implement novel machine learning and statistical approaches
US, WA, Seattle
Do you want to join an innovative team of scientists who invent and apply the most advanced machine learning, NLP and conversational AI techniques to create the best customer engagement experience on the earth? Do you want to revolutionize the way customers solve their issues and get their questions answered? Do you want to break language barriers so that Amazon customers can interact seamlessly with customer service? At Customer Engagement Technology, we develop specialized products that help customers solve problems. Our team leads the technical innovations in these spaces and set the bar for every other company that exists. We innovate on behalf of customers, developing chat bot, self-service, and associate-facing products that delight customers and support our world-class customer service workforce. We leverage big data, NLP, ML, and a focus on continuous innovation to create an amazing experience for customers as we scale to meet business growth each year.If you like to solve end-to-end business problems with machine learning and have a direct impact on the bottom line of Amazon’s business, if you love to innovate, discover knowledge from big structured and unstructured data and deliver cutting-edge technology that creates superior customer experience, then we want you on our team.Major responsibilities:· Use machine learning, NLP and statistical techniques to create scalable solutions for business problems· Analyze and extract relevant information from large amounts of both structured and unstructured data to help automate and optimize key processes· Design, experiment and evaluate highly innovative models for predictive learning· Work closely with software engineering teams to drive real-time model experiments, implementations and new feature creations· Work closely with business staff to optimize various business operations· Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation· Track general business activity and provide clear, compelling management reporting on a regular basis· Research and implement novel machine learning and statistical approaches
US, VA, Arlington
***This role is open to the following locations: Arlington VA, Austin TX, Las Vegas NV, or Boston MA***This new role in the Amazon Benefits Department will establish world class data science/business intelligence, analytics and reporting for the Amazon international benefit plans. This key role will work closely with internal and external partners to assist in developing and managing health care and population health management solutions for the Amazon Benefits Department. The Data Scientist will be expected to have a deep understanding of health plans, and will analyze data to help identify opportunities to improve employee wellbeing and health care outcomes, while reducing medical expenditures.Core Responsibilities:· Manage and execute entire projects or components of large projects from start to finish including data gathering and manipulation, synthesis and modeling, problem solving, and communication of insights and recommendations.· Oversee the development and implementation of data integration and analytic strategies to support population health initiatives.· Leverage data to explore and introduce areas of health care analytics and technologies.· Analyze data to identify opportunities to impact populations.· Perform advanced integrated comprehensive reporting, consultative, and analytical expertise to provide healthcare cost and utilization data and translate findings into actionable information for internal and external stakeholders.· Oversee the collection of data, ensuring timelines are met, data is accurate and within established format.· Act as a data and technical resource and escalation point for data issues, ensuring they are brought to resolution.· Serve as the subject matter expert on health care benefits data modeling, system architecture, data governance, and business intelligence tools.
US, WA, Seattle
Prime Video is shaping the future of digital video entertainment. Our vision is to provide the most comprehensive global video service that customers choose first and spend the most time engaged with. Our Amazon Prime subscription streaming service provides a broad selection of shows, movies and live sports included with customers’ subscription to Amazon Prime. With TVOD (Rent or Buy) and Channels (subscribe to premium channels such as HBO or Starz), we augment our proposition by giving customers immediate access to the broadest selection of video content available anywhere (nearly 200,000 titles), and we drive monetization opportunities for our business.The Worldwide Marketplace Science Team within Amazon Prime Video builds commercial optimization tools for our TVOD and Channels businesses. In this role, you will invent science and systems for TVOD and Channels, including ML-based pricing, personalized promotions, and subscription bundles, among others. You will work with a team of economists and machine learning scientists to design commercial products, and you will work with technology teams to productize and maintain the associated solutions.We are looking for the next outstanding data scientist to join our interdisciplinary team of analysts, data scientists, economists, and engineers. The ideal candidate combines data-science acumen with strong business judgment. You have versatile modeling skills and are comfortable extracting insights from observational and experimental data. You translate insights into action through proofs-of-concept and partnerships with engineers to productionize. You are excited to learn from and alongside seasoned analysts, scientists, engineers, and business leaders. You are an excellent communicator and effectively translate business ideas and technical findings into business action (and customer delight).Key Responsibilitieso Partner with business, finance, and tech teams to build models and productized science for TVOD and Channels, such as:o Targeting and personalizing promotionso Optimize content lifecycling and pricing across lines of businesso Valuing portfolios of TVOD contento Serve as a science / strategy advisor to global TVOD and Channels businesses
US, WA, Seattle
Prime Video is shaping the future of digital video entertainment. Our vision is to provide the most comprehensive global video service that customers choose first and spend the most time engaged with. Our Amazon Prime subscription streaming service provides a broad selection of shows, movies and live sports included with customers’ subscription to Amazon Prime. With TVOD (Rent or Buy) and Channels (subscribe to premium channels such as HBO or Starz), we augment our proposition by giving customers immediate access to the broadest selection of video content available anywhere (nearly 200,000 titles), and we drive monetization opportunities for our business.Worldwide Marketplace Science (within Amazon Prime Video) builds production systems to transform how our TVOD and Channels businesses grow and monetize. In this role, you will invent science and systems for TVOD and Channels, including ML-based pricing, personalized promotions, and customized subscription bundles, among others. You will work with a team of economists and machine learning scientists to design customer-facing products, and you will work with technology teams to productize and maintain the associated solutions.We are looking for the next outstanding data scientist to join our interdisciplinary team of analysts, data scientists, economists, and engineers. The ideal candidate combines data-science acumen with strong business judgment. You have versatile modeling skills and are comfortable extracting insights from observational and experimental data. You translate insights into action through proofs-of-concept and partnerships with engineers to productionize. You are excited to learn from and alongside seasoned analysts, scientists, engineers, and business leaders. You are an excellent communicator and effectively translate business ideas and technical findings into business action (and customer delight).Key Responsibilities· Research, develop, and implement scalable solutions for targeted and personalized promotions, recommended actions for studio partners, and more!· Coordinate with business, finance, and tech teams to expand understanding of what is possible and ensure focus on the right problems· Collaborate with tech and engineering teams to develop and optimize production systems· Work with other scientists to raise the science bar across Prime Video
IN, KA, Bangalore
Safety is paramount for Amazon. In his 2020 Letter to Shareholders, Jeff Bezos has included a commitment on making Amazon Earth’s Safest Place to Work. Workplace Health and Safety (WHS) organization is helping Amazon fulfill this commitment.And at WHS-Tech, we are exploring possibilities for technology to influence this course. From using devices to monitor and improve environment/health, to linking together data from our buildings and devices to plan safety-related events—the opportunity set is huge for us to further enhance the safety and wellbeing of our associates. Our team seeks to reinvent the way Amazon anticipates, assesses, and manages workplace hazards and risks. So, if creating industry-first products and solutions interests you, then this is the team for you.Our problem space is huge, diverse, and exciting, given the impact we can create on employee safety. As our programs have evolved, so has the need for advanced methods to measure the impact and uncover trends/insights that can help boost program (and Safety) performance. On the same lines—harnessing the huge volume of data to understand what drives safety is another priority. Safety Science team also partners closely with the Software Development team to develop AI components for technology products being developed for field safety.
US, KY, Hebron
We are a team of doers working passionately to apply cutting-edge advances in diagnostics technology to solve real-world problems. As a senior assay development research scientist, you will work with a unique and gifted team developing exciting medical diagnostic products and collaborate with cross-functional teams. Our team rewards intellectual curiosity while maintaining a laser-focus in bringing products to market. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the cutting edge of both academic and applied research in this product area, you have the opportunity to work together with some of the most talented scientists, engineers, and product managers.The Senior Research Scientist should be an expert in using molecular tools to develop assays leading to commercially viable products. They should have a deep understanding of underlying molecular processes and biochemical components within PCR, qPCR, microarrays, NGS, and WGS assays. This is someone capable of diving deep into the data (requires in-depth understanding of the lab processes, for example to do root cause analysis of experimental outcome based on the data), and someone who can work independently.Job responsibilities· Research and develop both existing and new cutting edge molecular and/or immunoassay technologies· Carefully execute laboratory experiments and provide guidance to teams of scientists and engineers through the writing of clear standard operating protocols.· Assist with assay process transfer to production environments and design and execute verification/validation plans· Work with a team of scientists and engineers to communicate experiments and conclusion for review· Work closely with clinical, regulatory and quality stakeholders· Follow literature in the field
US, CA, Santa Clara
Interested in machine learning and AI? Can you envision a future where technology is driven primarily by smarter machines?The mission of AWS AI is to make machine learning easy, fast, and universal across all of our customers. Our world class platform provides the services that runs 85% of all on-cloud machine learning through performance optimizations, machine learning tools, and SDKs to democratize machine learning. Our customers include scientists, data analysts, and ML engineers all building and deploying models through either SageMaker or self-managed instances on EC2/EKS.The AWS Machine Learning Services group in Amazon AI is seeking applicants for an applied scientist position in the area model security and privacy. As an applied scientist in AWS AI, you will be driving many scientific breakthroughs to help our customers understand and trust ML-based outcomes, and also working with software engineers to deploy these solutions to AWS customers. You will be working on the ML platforms and services that power over 85% of machine learning in the cloud, and help make machine learning fast, accessible, universal, and trustworthy for all our customers.We look for candidates with PhD in a relevant technical field (such as Algorithms, Machine Learning, AI, and Mathematics), preferably with interest / experience in model security and privacy. We also encourage graduating PhD students as well as recent PhD graduates to apply. You will be working closely with software engineers so experience with cloud systems is a benefit. Our customers are deeply technical and the solutions we build for them are strongly coupled to technical feasibility. You must be able to thrive and succeed in an entrepreneurial environment, and not be hindered by ambiguity or competing priorities. This means you are not only able to develop and drive high-level strategic initiatives, but can also roll up your sleeves, dig in and get the job done. Ownership, high judgment, negotiation skills, ability to influence, analytical talent and leadership are essential to success in this role.For more information about AWS AI team, please see https://aws.amazon.com/machine-learning and https://aws.amazon.com/sagemaker.About UsInclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.
US, CA, Sunnyvale
Interested in Amazon Alexa? We’re building the speech and language solutions behind Amazon Alexa and Amazon products and services such as the Amazon Echo and Dot. Come join us!As a key member of the team, you will provide machine learning expertise that helps accelerate the business. You will build various data and machine learning models that help us innovate different ways to enhance the customer experience. You will need to be entrepreneurial, wear many hats, and work in a highly collaborative environment across engineer, product, and BI. Your work will directly impact our customers in speech and language technology. We like to move fast, experiment, iterate and then scale quickly, thoughtfully balancing speed and quality. An ideal candidate will be an expert in the areas of machine learning and statistics who will have expertise in applying theoretical models in an applied environment. The candidate will be expected to work on numerous aspects of Machine Learning such as feature engineering, probabilistic modeling, hyper-parameter tuning and deep learning.Job responsibilities:1. As an Data Scientist on our team you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art NLU (Natural language understanding) developments.2. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to traing Machine Learning models for their application in NLU.3. This role requires a pragmatic technical leader comfortable with ambiguity, capable of summarizing complex data and models through clear visual and written explanations.4. The ideal candidate will have experience with machine learning models and their application in AI systems. We are particularly interested in experience applying natural language processing, deep learning at scale. Additionally, we are seeking candidates with strong interest in data/research sciences and engineering, creativity, curiosity, and great judgment.5. You will interact with various stake holders: product leaders, program managers, other domain managers and developers on regular basis for requirement collections, deliveries, and other related communicationBASIC QUALIFICATIONS* Bachelor or Master's degree in highly quantitative field (CS, machine learning, mathematics, statistics) or equivalent experience.* Experience applying various machine learning techniques, and understanding the key parameters that affect their performance.* Experience developing experimental and analytic plans for data modeling processes, use of strong baselines, and the ability to accurately determine cause and effect relationships.* Have a history of building systems that capture and utilize large data sets in order to quantify performance via metrics or KPIs.* Understanding of relevant statistical measures such as confidence intervals, significance of error measurements, development and evaluation data sets, etc.* Experienced in using multiple data science methodologies to solve complex business problems.* Experienced in handling large data sets using SQL and databases in a business environment.* Excellent verbal and written communication.* Strong troubleshooting and problem solving skills.* Thrive in a fast-paced, innovative environment.PREFERRED QUALIFICATIONS* Master's degree in Statistics, Applied Math, Operations Research, Economics, or a related quantitative field with at least 4+ years of working experience as a Data Scientist.* Experienced in writing academic-styled papers for presenting both the methodologies used and results for data science projects.* Hypothesis testing (including statistical hypothesis testing) and experimental design* Experience with unstructured textual dataAmazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
GB, Cambridge
Are you interested in delighting Alexa customers around the world? Come join us and help shape Alexa's future. We have a position in the Alexa AI Knowledge group in Cambridge to solve challenging problems in the knowledge extraction and acquisition space.Our challenge is to ensure Alexa understands and answers every question on any topic at any time, from popular topics such as "How old is Dresden Cathedral?", " "How long till Diwali?" to those where intelligent and common-sense reasoning are needed, such as “If Vettel wins Monaco, can Lewis still win the championship?”, “Who won the football match?”. In the Automated Knowledge Extraction team, we aim to answer these questions by retrieving relevant and up-to-date knowledge to teach Alexa about what is happening in the world every day.We are looking for a Machine Learning Scientist to research, build and test novel techniques (for example, Deep Neural Network architectures) to acquire knowledge across a range of diverse, structured data sources, at web scale. This requires methods that lie beyond the cutting edge academic and industrial research of today. The ideal candidate is a creative problem solver, and will have industry and academic experience in Natural Language Processing (NLP) and Machine Learning (ML).This job will test your skills across the board. You will research the latest knowledge extraction techniques and understand trade-offs between competing approaches to identify the ones that are likely to have real impact on our customers. You will help us continually re-think the problem and improve the state-of-the-art, using data-driven arguments in writing to inspire and influence a range of audiences. You will leverage your programming skills to implement and improve training recipes, model building and production prototypes. In return, you will have the opportunity to access large data sets across Amazon, guide and mentor junior Scientists, and learn from some of the best senior Scientist in the field. You will also work alongside talented Software Development Engineers to deploy advancements in the state-of-the-art in production, and be challenged by the engineering discipline needed to enable Alexa to operate at a global scale and impact millions of customers.And while the challenges are big and the pace fast, we have fun while we’re at it. It’s a high-energy, good-natured, and inclusive environment. If you think you have what it takes to change the world, then apply to join us. We work hard, have fun, and make history at Amazon!We believe passionately that employing a diverse workforce is central to our success and we make recruiting decisions based on your experience and skills. We welcome applications from all members of society irrespective of age, gender, disability, sexual orientation, race, religion or belief.Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice to know more about how we collect, use and transfer the personal data of our candidates.
US, WA, Bellevue
The Alexa Experience organization owns a number of programs and domains which drive high customer engagement and feature discovery, maintains customer trust, understands customers’ feedback, develops new features that provide utility value for customers with special needs, and develops locally relevant experiences. Our team develop and train natural language understanding models to improve these programs and to create new delighting Alexa experiences for customers.We’re looking for a passionate scientist to help design, develop and release to production NLU models that provide the best-possible experience for our customers. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to develop new features and predict key user behaviors, both offline and in real time.About the hiring groupWe are a multi-disciplinary science team working on solving business needs leveraging state of the art approaches in natural language understanding (NLU) models, deep learning, reinforcement learning, causal inferences, and more, creating a rich and collaborative science environment.Job responsibilitiesWhat You Will Do:· Be the subject matter expert in NLU model authoring process, guiding multiple product teams toward delivering best customer experiences on Alexa.· Contribute to the design of new features by extending existing annotation schemas to cover new areas· Develop and release to production Alexa NLU models· Measure and analyze accuracy of models; develop solutions to improve performances· Contribute to the research vision of the team, develop and execute on a roadmap that addresses the technical challenges our product teams face· Raise the bar for research quality and impact. Stay up to date on research results both within and complementary to the Alexa technology space and helping your team and its partner to utilize tried patterns and state of the art results to work smarter and faster.· Deliver reports and/or models to influence technical directions.· Design processes and tools to simplify Alexa feature expansion for your team and partners.· Perform data analysis, A/B experimentation and modeling with large data sets to develop insights that increase device usage and customer experience· Collaborate with peers in data analysis and feature design effortsAmazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
GB, MLN, Edinburgh
About the teamKey job responsibilitiesDo you want to make a real difference to real people's lives? Want to design and build fair and explainable systems which automate recruitment processes across Amazon? Come and be part of a team that develops new machine learning (ML) technologies, which help Amazon scale for its customers by recruiting diverse teams. You will work as an ML scientist in a team of other scientists and software developers. You will primarily be writing solutions in Python and will be using the latest technologies including AWS (e.g. Sagemaker). You will be contributing regularly to the code base as this is an applied role with the expectation of 50% of your time spent on the code. Your solutions will meet remarkably high standards of performance and reliability, and will operate at massive scale. You will work as part of a sustainably paced agile team. You will play a hands on leadership role in your team giving you the responsibility, authority, and autonomy to ensure success. You will be involved in every aspect of the process - from idea generation, customer engagement, business analysis and scientific design through to software development and operations. Join a team full of talented people who come from all over the world. Enjoy the chance to work in a relaxed setting with a good social life. The team, primarily based in Edinburgh, Scotland, is rapidly expanding. We are looking for ML scientists who can delight our customers by continually learning and inventing. Our ideal candidate is an experienced ML scientist who has a track-record of statistical analysis and building models to solve real business problems, who has great leadership and communication skills, and who has a passion for fairness and explainability in ML systems. The role offers an exceptional opportunity for growth and to make a real difference to Amazon recruitment. If you are selected, you have the opportunity to really impact our business by inventing, improving, and building world class systems, delivering results, working on exciting and challenging projects. Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Please let us know if you have any special requirements in relation to this recruitment process.A day in the lifeAmazon is looking for an Applied Scientist. As an scientist working at Amazon, you will play a key role in identifying business opportunities, measuring opportunity, inventing and prototyping solutions. You will use a wide range of technologies, programming languages and systems, and work alongside an experienced engineering team. You will have the freedom and encouragement to explore your own ideas and the reward of seeing your contributions benefit millions of customers worldwide.About the hiring groupJoin a team full of talented people who come from all over the world. Enjoy the chance to work in a relaxed setting with a good social life. The team, primarily based in Edinburgh, Scotland, is rapidly expanding.Job responsibilitiesYou will work as an ML scientist in a team of other scientists and software developers. You will primarily be writing solutions in Python and will be using the latest technologies including AWS (e.g. Sagemaker). You will be contributing regularly to the science research and the code base as this is an applied role. Your solutions will meet remarkably high standards of performance and reliability, and will operate at massive scale.You will work as part of a sustainably paced agile team. You will play a hands on role in your team giving you the responsibility, authority, and autonomy to ensure success. You will be involved in every aspect of the process - from idea generation, customer engagement, business analysis and scientific design through to software development and operations.We are looking for ML scientists who can delight our customers by continually learning and inventing and who want to use their expertise to solve real business problems. Our ideal candidate has a strong scientific foundation and experience of statistical analysis and model building and has a passion for fairness and explainability in ML systems.The role offers an exceptional opportunity for growth and to make a real difference to Amazon recruitment. If you are selected, you have the opportunity to really impact our business by inventing, improving, and building world class systems, delivering results, working on exciting and challenging projects.Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice to know more about how we collect, use and transfer the personal data of our candidates.
US, WA, Seattle
Come build the future of FinTech with us! The WW Installments team within Consumer Payments (CP) brings together the best of machine learning, econometrics, tech, business, and finance to deliver for customers across the world. We build the foundational systems and products to enable Amazon customers with installment and related payment methods. Our mission is to delight our customers by building payment experiences and financial services that are trusted, valued and easy to use from anywhere in any way. Our products are growing rapidly and we are continuously adding new market-leading features and launching new customer facing ML solutions. Our team of high caliber software developers, statisticians, analysts and product managers use rigorous quantitative approaches to ensure that we target the right product to the right customer at the right moment, managing tradeoffs between click through rate, approval rates, customer friction, economic profit, and loss rates. We leverage the wealth of Amazon’s information to build a wide range of probabilistic models, set up experiments that ensure that we are thriving to reach global optimums and leverage Amazon’s technological infrastructure to display the right offerings in real time. We own our own Science business and work across Amazon to deliver algorithms to partner scenarios and their customers. Our goal is to delight our customers with their purchasing experience and our solutions improve the shopping experience for hundreds of millions of consumers worldwide accordingly. Those of us who love to work with data see this as the pinnacle of opportunities within FinTech and the BNPL/Installment space that you cannot find anywhere else in the world.We are seeking an exceptionally talented individual to improve our analytic capabilities. This is an opportunity to join a group with a broad charter and stakeholders across Amazon. In this role, you will be working in one of the world's largest and most complex data warehouse environments. You should be passionate about working with huge datasets and be someone who loves to bring data together to answer business questions. You should have deep expertise in creation and management of datasets and the proven ability to translate the data into meaningful insights. You will have to work with a group of applied and data scientists and play an integral role in strategic decision-making. The right candidate will possess excellent business and communication skills, define business objectives and prioritize work across the team to support business outcomes, and develop solutions to key business questions. You should have a solid understanding of efficient and scalable data mining and an ability to use the data in financial and statistical modeling.
US, VA, Herndon
Are you excited to help the Intelligence Community to leverage massive amounts of data in developing Machine Learning (ML) and Deep Learning (DL) models and adopting Artificial Intelligence (AI)? Eager to learn from many different enterprise’s use cases of AWS ML and DL? Thrilled to be key part of Amazon, who has been investing in Machine Learning for decades, pioneering and shaping the world’s AI technology?At Amazon Web Services (AWS), we are helping the federal government build ML and DL models on the AWS Cloud. We are applying predictive technology to large volumes of data and against a wide spectrum of problems. Our Professional Services organization works together with our AWS customers to address their business needs using AI.AWS Professional Services is a unique consulting team. We pride ourselves on being customer obsessed and highly focused on the AI enablement of our customers. If you have experience with AI, including building ML or DL models, we’d love to have you join our team. You will get to work with an innovative company, with great teammates, and have a lot of fun helping our customers.In this role, you will be able to dive deep into data, discovering root causes, and designing long-term solutions. It will be a person who likes to have fun, loves to learn, and wants to innovate in the world of AI.Major responsibilities include:· Understand the customer’s business need and guide them to a solution using our AWS AI Services, AWS AI Platforms, AWS AI Frameworks, and AWS AI EC2 Instances .· Assist customers by being able to deliver a ML / DL project from beginning to end, including understanding the business need, aggregating data, exploring data, building & validating predictive models, and deploying completed models to deliver business impact to the organization.· Use Deep Learning frameworks like MXNet, Caffe 2, Tensorflow, Theano, CNTK, and Keras to help our customers build DL models.· Use SparkML and Amazon Machine Learning (AML) to help our customers build ML models.· Work with our Professional Services Big Data consultants to analyze, extract, normalize, and label relevant data.· Work with our Professional Services DevOps consultants to help our customers operationalize models after they are built.· Assist customers with identifying model drift and retraining models.· Research and implement novel ML and DL approaches, including using FPGAThis position requires that the candidate selected be a U.S. citizen and must currently possess and maintain an active TS/SCI security clearance with polygraph.**For more information, please contact Renee Taylor at tarenee@amazon.com**Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and we host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.We're dedicated to supporting new team members. Our team has a broad mix of experience levels and Amazon tenures, and we’re building an environment that celebrates knowledge sharing and mentorship.Our team also puts a high value on work-life balance. Striking a healthy balance between your personal and professional life is crucial to your happiness and success here, which is why we aren’t focused on how many hours you spend at work or online. Instead, we’re happy to offer a flexible schedule so you can have a more productive and well-balanced life—both in and outside of work.