Building systems that automatically adjust to workloads and data

Tim Kraska, who joined Amazon this summer to build the new Learned Systems research group, explains the power of “instance optimization”.

As an associate professor of electrical engineering and computer science at MIT, Tim Kraska researched instance-optimized database systems, or systems that can automatically adapt to new workloads with minimal human involvement.

Tim Kraska.png
Tim Kraska, an associate professor of electrical engineering and computer science at MIT and director of applied science for Amazon Web Services.

Earlier this year, Amazon hired Kraska and his team to further develop this technology. Currently, Kraska is on leave from MIT, and as director of applied science for Amazon Web Services (AWS), he is helping establish Amazon’s new Learned Systems Group (LSG), which will focus on integrating machine learning (ML) into system design. The group’s first project is to bring instance optimization to AWS’s data warehousing service, Amazon Redshift. Kraska spoke with Amazon Science about the value of instance optimization and the attraction of doing research in an industrial setting.

  1. Q. 

    What is instance optimization?

    A. 

    If you develop a system from scratch for a particular use case, you are able to get orders of magnitude better performance, as you can tailor every system component to that use case. However, in most cases you don't want to do that, because it's a huge effort. In the case of databases, the saying is that it normally takes at least seven years to get the system so that it's usable and stable.

    The idea of instance optimization is that, rather than build one system per use case, we build a system that self-adjusts — instance-optimizes itself — to a particular scenario to get as close as possible to a hand-tuned solution.

  2. Q. 

    How does it do that?

    A. 

    There are different ways to achieve the self-adjustment. With any system, you have a bunch of knobs and a bunch of design choices. If you take Redshift, you can tune the buffer size; you can create materialized views; you can create different types of sort orders. And database administrators can adjust these knobs and make design choices, based on their workloads, to get better performance.

    Related content
    Two authors of Amazon Redshift research paper that will be presented at leading international forum for database researchers reflect on how far the first petabyte scale cloud data warehouse has advanced since it was announced ten years ago.

    The first form of self-adjustment is to make those decisions automatically. You have, let's say, a machine learning model that observes the workload and figures out how to adjust these knobs and what materialized views and sort keys to create. Redshift already does this, for example, with a feature called Automated Materialized Views, which accelerates query performance.

    The next step is that in some cases it's possible to replace components through novel techniques that allow either more customization or tuning in ways that weren’t previously possible.

    To give you an example, in the case of data layouts, current systems mainly support partitioning data by one attribute, which could be a composite key. The reason is that the developers of these systems always thought that someone has to eventually make these design choices manually. Thus, in the past, the tendency was to reduce the number of tuning parameters as much as possible.

    Related content
    Amazon researchers describe new method for distributing database tables across servers.

    This, of course, changes the moment you have automatic tuning techniques using machine learning, which can explore the space much more efficiently. And now maybe the opposite is true: providing more degrees of freedom and more knobs is a good thing, as they offer more potential for customization and, thus, better performance.

    The third self-adjustment method is where you deeply embed machine learning models into a component of the system to give you much better performance than is currently possible.

    Every database, for example, has a query optimizer that takes a SQL query and optimizes it to an execution plan, which describes how to actually run that query. This query optimizer is a complex piece of software, which requires very carefully tuned heuristics and cost models to figure out how best to do this translation. The state of the art now is that you treat this as a deep-learning problem. So we talk at that stage about learned components.

    Query patterns.png
    A comparison of two different approaches to learning to detect query patterns, using graph convolution networks (top) and tree convolution networks (bottom). From “LSched: A workload-aware learned query scheduler for analytical database systems”.

    The ultimate goal is to build a system out of learned components and to have everything tuned in a holistic way. There's a model monitoring the workload, watching the system, and making the right adjustments — potentially in ways no human is able to.

  3. Q. 

    Is it true that you developed an improved sorting algorithm? I thought that sorting was pretty much a solved problem.

    A. 

    That's right. It's still surprising. The way it works is, you learn a model over the distribution of the data — the cumulative distribution function, or CDF, which tells you where an item falls into the probability mass. Let's assume that in an e-commerce database, you have a table with orders, each order has a date, and you want to sort the table by date. Now you can build the CDF over the date attribute, and then you can ask a question like “How many orders happened before January 1st, 2021?”, and it spits out the probability.

    The nice thing about that is that, essentially, the CDF function allows you to ask, “Given an order date, where in the sorted order does it fit?” Assuming the model is perfect, it suddenly allows you to do sorting in O(n). [I.e., the sorting time is proportional to the number of items being sorted, n, not n2nlogn, or the like.]

    Learned sorting.png
    Recursively applying the cumulative distribution function (CDF) to sort items in an array in O(n) time. From “The case for a learned sorting algorithm”.

    Radix sort is also O(n), but it can be memory intensive, as the efficiency depends on the domain size — how many unique values there could possibly be. If your domain is one to a million, it might still be easily do-able in memory. If it's one to a billion, it already gets a little bit harder. If it's one to — pick your favorite power of ten — it eventually becomes impossible to do it in one pass.

    The model-based approach tries to overcome that in a clever way. You know roughly where items land, so you can place them into their approximate position and use insertion sort to correct for model errors. It’s a trick we used for indexes, but it turns out that you can use the same thing for sorting.

  4. Q. 

    For you, what was the appeal of doing research in the industrial setting?

    A. 

    One of the reasons we are so attracted to working for Amazon is access to information about real-world workloads. Instance optimization is all about self-adjusting to the workload and the data. And it's extremely hard to test it in academia.

    There are a few benchmark datasets, but internally, they often use random-number generators to create the data and to determine when and what types of queries are issued against the system.

    We fundamentally have to rethink how we build systems. ... Whenever a developer has to make a trade-off between two techniques or defines a constant, the developer should think about if this constant or trade-off shouldn’t be automatically tuned.
    Tim Kraska

    Because of this randomness, first of all, there are no interesting usage patterns — say, when are the dashboarding queries running, versus the batch jobs for loading the data. All that is gone. Even worse, the data itself doesn’t contain any interesting patterns, which either makes it too hard, because everything is random, or too easy, because everything is random.

    For example, when we tested our learned query optimizer on a very common data-warehousing benchmark, we found that we barely got any improvements, whereas for real-world workloads, we saw big improvements.

    We dug in a little bit, and it turns out that for common benchmarks, like TPC-H, every single database vendor makes sure that the query plans are close to perfect. They manually overfit the system to the benchmark. And this translates in no way to any real-world customer. No customer really runs queries exactly like the benchmark. Nobody does.

    Working with Redshift’s amazing development team and having access to real-world information provides a huge advantage here. It allows us not only to evaluate if our previous techniques actually work in practice, but it also helps us to focus on developing new techniques, which actually make a big difference to users by providing better performance or improved ease of use.

  5. Q. 

    So the collaboration with the Redshift team is going well?

    A. 

    It has been great and, in many ways, exceeded our expectations. When we joined, we certainly had some anxiety about how we would be working with the Redshift team, how much we would still be able to publish, and so on. For example, I know many researchers in industry labs who struggle to get access to data or have actual impact on the product.

    None of these turned out to be a real concern. Not only did we define our own research agenda, but we are also already deeply involved with many exciting projects and have a whole list of exciting things we want to publish about.

  6. Q. 

    Do you still collaborate with MIT?

    A. 

    Yes, and it is very much encouraged. Amazon recently created a Science Hub at MIT, and as part of the hub, AWS is also sponsoring DSAIL, a lab focused on ML-for-systems research. This allows us to work very closely with researchers at MIT.

  7. Q. 

    Some of the techniques you’ve discussed, such as sorting, have a wide range of uses. Will the Learned Systems Group work with groups other than Redshift?

    A. 

    We decided to focus on Redshift first as we had already a lot of experience with instance optimization for analytical systems, but we’ve already started to talk to other teams and eventually plan to apply the ideas more broadly.

    I believe that we fundamentally have to rethink how we build systems and system components. For example, whenever a developer has to make a trade-off between two techniques or defines a constant, the developer should think about if this constant or trade-off shouldn’t be automatically tuned. In many cases, the developer would probably approach the design of the component completely differently if she knows that the component is expected to self-adjust to the workload and data.

    Related content
    Optimizing placement of configuration data ensures that it’s available and consistent during “network partitions”.

    This is true not only for data management systems but across the entire software stack. For example, there has been work on improving network packet classification using learned indexes, spark scheduling algorithms using reinforcement learning, and video compression using deep-learning techniques to provide a better experience when bandwidth is limited. All these techniques will eventually impact the customer experience in the form of performance, reduced cost, or ease of use.

    For good reason, we already see a lot of adaptation of ML to improve systems at Amazon. Redshift, for example, offers multiple ML-based features — like Automated Materialized Views or automatic workload management. With the Learned Systems Group, we hope to accelerate that trend, with fully instance-optimized systems that self-adjust to workloads and data in ways no traditional system can. And that will provide better performance, cost, and ease of use for AWS customers.

Related content

GB, London
"Are you a MS or PhD student interested in the fields of Computer Science or Operational Research? Do you enjoy diving deep into hard technical problems and coming up with solutions that enable successful products? If this describes you, come join our research teams at Amazon. " Key job responsibilities The candidate will be responsible for the design and implementation of optimization algorithms. The candidate will translate high-level business problems into mathematical ones. Then, they will design and implement optimization algorithms to solve them. The candidate will be responsible also for the analysis and design of KPIs and input data quality. About the team ATS stands for Amazon Transportation Service, we are the middle-mile planners: we carry the packages from the warehouses to the cities in a limited amount of time to enable the “Amazon experience”. As the core research team, we grow with ATS business to support decision making in an increasingly complex ecosystem of a data-driven supply chain and e-commerce giant. We take pride in our algorithmic solutions: We schedule more than 1 million trucks with Amazon shipments annually; our algorithms are key to reducing CO2 emissions, protecting sites from being overwhelmed during peak days, and ensuring a smile on Amazon’s customer lips. We do not shy away from responsibility. Our mathematical algorithms provide confidence in leadership to invest in programs of several hundreds millions euros every year. Above all, we are having fun solving real-world problems, in real-world speed, while failing & learning along the way. We employ the most sophisticated tools: We use modular algorithmic designs in the domain of combinatorial optimization, solving complicated generalizations of core OR problems with the right level of decomposition, employing parallelization and approximation algorithms. We use deep learning, bandits, and reinforcement learning to put data into the loop of decision making. We like to learn new techniques to surprise business stakeholders by making possible what they cannot anticipate. For this reason, we work closely with Amazon scholars and experts from Academic institutions. We are open to hiring candidates to work out of one of the following locations: London, GBR
US, MA, Boston
The Artificial General Intelligence (AGI) team is looking for a highly-skilled Senior Applied Scientist, to lead the development and implementation of cutting-edge algorithms and push the boundaries of efficient inference for Generative Artificial Intelligence (GenAI) models. As a Senior Applied Scientist, you will play a critical role in driving the development of GenAI technologies that can handle Amazon-scale use cases and have a significant impact on our customers' experiences. Key job responsibilities - Design and execute experiments to evaluate the performance of different decoding algorithms and models, and iterate quickly to improve results - Develop deep learning models for compression, system optimization, and inference - Collaborate with cross-functional teams of engineers and scientists to identify and solve complex problems in GenAI - Mentor and guide junior scientists and engineers, and contribute to the overall growth and development of the team We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA | Boston, MA, USA | New York, NY, USA
US, WA, Bellevue
We are looking for detail-oriented, organized, and responsible individuals who are eager to learn how to work with large and complicated data sets. Knowledge of econometrics, and basic familiarity with Python or R, is necessary. Experience with SQL is a plus. These are full-time positions at 40 hours per week, with compensation being awarded on an hourly basis. You will learn how to build data sets and apply econometric methods to support business decisions, collaborating with economists, scientists, and product managers. These skills will translate well into writing applied chapters in your dissertation and provide you with work experience that may help you with placement. Roughly 85% of previous cohorts have converted to full time economics employment at Amazon. If you are interested, please send your CV to our mailing list at econ-internship@amazon.com. Key job responsibilities Collaborate with business and science colleagues to understand the business problem and collect relevant data. Provide statistically rigorous analysis of data that contributes to business decision-making. Effectively communicate your results to colleagues and business leaders. A day in the life Meet with colleagues to discuss how the business currently works. Discuss ways in which the customer experience could be improved, and what data you'd need to test your hypotheses. Meet with data and business intelligence engineers to build an efficient data pipeline using SQL, spark and other big data tools. Propose and execute a plan to analyze your data, working closely with your econ colleagues. Use Amazon's development tools, coding your estimators in Python or R. Draft your findings for an internal knowledge sharing session. Iterate to improve your work and communicate your final results in a business document. About the team We are a team of four economists that works within the delivery experience org. Our goal is to improve the delivery experience for our customers while reducing costs. This mission puts us in a unique position to influence both the front end customer experience and the supply chain that ultimately places constraints on that experience. This means we often work with and influence teams outside of our own organization. As a result, we have the privilege of working with a diverse group of experts, including those in supply chain, operations, capacity management, and user experience. We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA
ES, B, Barcelona
"Are you a MS or PhD student interested in the fields of Computer Science or Operational Research? Do you enjoy diving deep into hard technical problems and coming up with solutions that enable successful products? If this describes you, come join our research teams at Amazon. " Key job responsibilities The candidate will be responsible for the design and implementation of optimization algorithms. The candidate will translate high-level business problems into mathematical ones. Then, they will design and implement optimization algorithms to solve them. The candidate will be responsible also for the analysis and design of KPIs and input data quality. About the team ATS stands for Amazon Transportation Service, we are the middle-mile planners: we carry the packages from the warehouses to the cities in a limited amount of time to enable the “Amazon experience”. As the core research team, we grow with ATS business to support decision making in an increasingly complex ecosystem of a data-driven supply chain and e-commerce giant. We take pride in our algorithmic solutions: We schedule more than 1 million trucks with Amazon shipments annually; our algorithms are key to reducing CO2 emissions, protecting sites from being overwhelmed during peak days, and ensuring a smile on Amazon’s customer lips. We do not shy away from responsibility. Our mathematical algorithms provide confidence in leadership to invest in programs of several hundreds millions euros every year. Above all, we are having fun solving real-world problems, in real-world speed, while failing & learning along the way. We employ the most sophisticated tools: We use modular algorithmic designs in the domain of combinatorial optimization, solving complicated generalizations of core OR problems with the right level of decomposition, employing parallelization and approximation algorithms. We use deep learning, bandits, and reinforcement learning to put data into the loop of decision making. We like to learn new techniques to surprise business stakeholders by making possible what they cannot anticipate. For this reason, we work closely with Amazon scholars and experts from Academic institutions. We are open to hiring candidates to work out of one of the following locations: Barcelona, B, ESP
IN, TN, Chennai
DESCRIPTION The Digital Acceleration (DA) team in India is seeking a talented, self-driven Applied Scientist to work on prototyping, optimizing, and deploying ML algorithms for solving Digital businesses problems. Key job responsibilities - Research, experiment and build Proof Of Concepts advancing the state of the art in AI & ML. - Collaborate with cross-functional teams to architect and execute technically rigorous AI projects. - Thrive in dynamic environments, adapting quickly to evolving technical requirements and deadlines. - Engage in effective technical communication (written & spoken) with coordination across teams. - Conduct thorough documentation of algorithms, methodologies, and findings for transparency and reproducibility. - Publish research papers in internal and external venues of repute - Support on-call activities for critical issues BASIC QUALIFICATIONS - Experience building machine learning models or developing algorithms for business application - PhD, or a Master's degree and experience in CS, CE, ML or related field - Knowledge of programming languages such as C/C++, Python, Java or Perl - Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing - Proficiency in coding and software development, with a strong focus on machine learning frameworks. - Understanding of relevant statistical measures such as confidence intervals, significance of error measurements, development and evaluation data sets, etc. - Excellent communication skills (written & spoken) and ability to collaborate effectively in a distributed, cross-functional team setting. PREFERRED QUALIFICATIONS - 3+ years of building machine learning models or developing algorithms for business application experience - Have publications at top-tier peer-reviewed conferences or journals - Track record of diving into data to discover hidden patterns and conducting error/deviation analysis - Ability to develop experimental and analytic plans for data modeling processes, use of strong baselines, ability to accurately determine cause and effect relations - Exceptional level of organization and strong attention to detail - Comfortable working in a fast paced, highly collaborative, dynamic work environment We are open to hiring candidates to work out of one of the following locations: Chennai, TN, IND
US, VA, Arlington
Machine learning (ML) has been strategic to Amazon from the early years. We are pioneers in areas such as recommendation engines, product search, eCommerce fraud detection, and large-scale optimization of fulfillment center operations. The Generative AI team helps AWS customers accelerate the use of Generative AI to solve business and operational challenges and promote innovation in their organization. As an applied scientist, you are proficient in designing and developing advanced ML models to solve diverse challenges and opportunities. You will be working with terabytes of text, images, and other types of data to solve real-world problems. You'll design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. We’re looking for talented scientists capable of applying ML algorithms and cutting-edge deep learning (DL) and reinforcement learning approaches to areas such as drug discovery, customer segmentation, fraud prevention, capacity planning, predictive maintenance, pricing optimization, call center analytics, player pose estimation, event detection, and virtual assistant among others. AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success. AWS Support also partners with a global list of customers that are building mission-critical applications on top of AWS services. Key job responsibilities The primary responsibilities of this role are to: - Design, develop, and evaluate innovative ML models to solve diverse challenges and opportunities across industries - Interact with customer directly to understand their business problems, and help them with defining and implementing scalable Generative AI solutions to solve them - Work closely with account teams, research scientist teams, and product engineering teams to drive model implementations and new solution About the team About AWS Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. We are open to hiring candidates to work out of one of the following locations: Arlington, VA, USA | Atlanta, GA, USA | Austin, TX, USA | Houston, TX, USA | New York, NJ, USA | New York, NY, USA | San Francisco, CA, USA | Santa Clara, CA, USA | Seattle, WA, USA
US, WA, Seattle
Prime Video offers customers a vast collection of movies, series, and sports—all available to watch on hundreds of compatible devices. U.S. Prime members can also subscribe to 100+ channels including Max, discovery+, Paramount+ with SHOWTIME, BET+, MGM+, ViX+, PBS KIDS, NBA League Pass, MLB.TV, and STARZ with no extra apps to download, and no cable required. Prime Video is just one of the savings, convenience, and entertainment benefits included in a Prime membership. More than 200 million Prime members in 25 countries around the world enjoy access to Amazon’s enormous selection, exceptional value, and fast delivery. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video technologist, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! Key job responsibilities As a Data Scientist at Amazon Prime Video, you will work with massive customer datasets, provide guidance to product teams on metrics of success, and influence feature launch decisions through statistical analysis of the outcomes of A/B experiments. You will develop machine learning models to facilitate understanding of customer's streaming behavior and build predictive models to inform personalization and ranking systems. You will work closely other scientists, economists and engineers to research new ways to improve operational efficiency of deployed models and metrics. A successful candidate will have a strong proven expertise in statistical modeling, machine learning, and experiment design, along with a solid practical understanding of strength and weakness of various scientific approaches. They have excellent communication skills, and can effectively communicate complex technical concepts with a range of technical and non-technical audience. They will be agile and capable of adapting to a fast-paced environment. They have an excellent track-record on delivering impactful projects, simplifying their approaches where necessary. A successful data scientist will own end-to-end team goals, operates with autonomy and strive to meet key deliverables in a timely manner, and with high quality. About the team Prime Video discovery science is a central team which defines customer and business success metrics, models, heuristics and econometric frameworks. The team develops, owns and operates a suite of data science and machine learning models that feed into online systems that are responsible for personalization and search relevance. The team is responsible for Prime Video’s experimentation practice and continuously innovates and upskills teams across the organization on science best practices. The team values diversity, collaboration and learning, and is excited to welcome a new member whose passion and creativity will help the team continue innovating and enhancing customer experience. We are open to hiring candidates to work out of one of the following locations: Seattle, WA, USA
US, NJ, Newark
Employer: Audible, Inc. Title: Data Scientist II Location: 1 Washington Street, Newark, NJ, 07102 Duties: Design and implement scalable and reliable approaches to support or automate decision making throughout the business. Apply a range of data science techniques and tools combined with subject matter expertise to solve difficult business problems and cases in which the solution approach is unclear. Acquire data by building the necessary SQL/ETL queries. Import processes through various company specific interfaces for accessing RedShift, and S3/edX storage systems. Build relationships with stakeholders and counterparts, and communicate model outputs, observations, and key performance indicators (KPIs) to the management to develop sustainable and consumable products. Explore and analyze data by inspecting univariate distributions and multivariate interactions, constructing appropriate transformations, and tracking down the source and meaning of anomalies. Build production-ready models using statistical modeling, mathematical modeling, econometric modeling, machine learning algorithms, network modeling, social network modeling, natural language processing, or genetic algorithms. Validate models against alternative approaches, expected and observed outcome, and other business defined key performance indicators. Implement models that comply with evaluations of the computational demands, accuracy, and reliability of the relevant ETL processes at various stages of production. Position reports into Newark, NJ office; however, telecommuting from a home office may be allowed. Requirements: Requires a Master’s in Statistics, Computer Science, Data Science, Machine Learning, Applied Math, Operations Research, Economics, or a related field plus two (2) years of Data Scientist or other occupation/position/job title with research or work experience related to data processing and predictive Machine Learning modeling at scale. Experience may be gained concurrently and must include: Two (2) years in each of the following: - Building statistical models and machine learning models using large datasets from multiple resources - Non-linear models including Neural Nets or Deep Learning, and Gradient Boosting - Applying specialized modelling software including Python, R, SAS, MATLAB, or Stata. One (1) year in the following: - Using database technologies including SQL or ETL. Alternatively, will accept a Bachelor's and five (5) years of experience. Multiple positions. Apply online: www.amazon.jobs Job Code: ADBL135. We are open to hiring candidates to work out of one of the following locations: Newark, NJ, USA
US, WA, Bellevue
The Artificial General Intelligence (AGI) team is looking for a passionate, talented, and inventive Applied Scientist with a strong deep learning background, to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems. Key job responsibilities As an Applied Scientist with the AGI team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs. Your work will directly impact our customers in the form of products and services that make use of audio technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate advances in AGI in audio domain. About the team Our team has a mission to push the envelope of AGI in audio domain, in order to provide the best-possible experience for our customers. We are open to hiring candidates to work out of one of the following locations: Bellevue, WA, USA | Boston, MA, USA
DE, BE, Berlin
Are you fascinated by revolutionizing Alexa user experience with LLM? The Artificial General Intelligence (AGI) team is looking for an Applied Scientist with background in Large Language Model, Natural Language Process, Machine/Deep learning. You will be at the heart of the Alexa LLM transition working with a team of applied and research scientists to bring classic Alexa features and beyond into LLM empowered Alexa. You will interact in a cross-functional capacity with science, product and engineering leaders. Key job responsibilities * Work on core LLM technologies (supervised fine tuning, prompt optimization, etc.) to enable Alexa use cases * Research and develop novel metrics and algorithms for LLM evaluation * Communicating effectively with leadership team as well as with colleagues from science, engineering and business backgrounds. We are open to hiring candidates to work out of one of the following locations: Berlin, BE, DEU