Foundation Model Development call for proposals — Winter 2024
Advancing the frontiers of foundation model development.
About this CFP
AWS offers a broad and deep set of services for organizations to create meaningful machine learning solutions faster. Our mission is to share our learnings and ML capabilities as fully managed services, and put them into the hands of every scientist and developer.
AWS is soliciting funding proposals related to foundation model (FM) development, including enhancements for dialogue, factual questioning and answering, text generation, document summarization, image generation, and more. Proposals should be expanding the state of the art in terms of training methods, with a focus on one of the following areas: 1) to reduce incorrect or nonsensical answers, 2) to reduce the sensitivity to tweaks to the input prompt, 3) ask clarifying questions when facing ambiguous questions, 4) develop efficiency enhancements across the training and hosting FM lifecycle.
We are inviting work that both critiques and enhances the state-of-the-art in FM, including but not limited to the following topics:
- Reinforcement learning with human feedback
- Scaling laws and their inverse, including for model fine-tuning
- Novel datasets and training methods
- Distillation with enhanced reasoning
- Foundation models in novel modalities and domains, such as biology, manufacturing, fashion, etc.
- Bias detection and mitigation throughout the foundation model lifecycle
AWS aims to advance foundation model development by funding the creation of open-source artifacts, datasets and code that benefit the research community at large, or impactful research that uses machine learning tools. AWS will provide credits for use on EC2 instances powered by AWS Trainium chips to help researchers develop and test solutions at scale. In addition to AWS Trainium, proposed research can be conducted using AWS AI Services, AWS ML Services (Amazon SageMaker, Amazon SageMaker Ground Truth, Amazon SageMaker Neo, and Amazon Augmented AI), and Amazon Bedrock.
- Submission period: January 17 - March 6, 2024 (11:59PM Pacific Time)
- Decision letters will be sent out in May 2024
Selected Principal Investigators (PIs) may receive the following:
- $250,000 in AWS Promotional Credits, which can be used on generally available EC2 instances that are powered by AWS Trainium chips
- AWS Trainium training resources, including AWS tutorials and hands-on sessions with Amazon scientists and engineers
Awards are structured as one-year unrestricted gifts. The budget should include a list of expected costs specified in USD, and should not include administrative overhead costs.
Your receipt and use of AWS Promotional Credits is governed by the AWS Promotional Credit Terms and Conditions, which may be updated by AWS from time to time.
Please refer to the ARA Program rules on the Rules and Eligibility page.
PIs are encouraged to exemplify how their proposed techniques or research studies expand the body of knowledge on pretraining and evaluation methods. PIs should either include plans for open source contributions or state that they do not plan to make any open source contributions (data or code) under the proposed effort. Proposals for this CFP should be prepared according to the proposal template and are encouraged to be a maximum of 5 pages, not including Appendices.
Funding decisions are based on the creativity and quality of the scientific content, and potential impact to the research community and society at large. AWS may solicit feedback on proposal title and abstracts from select AWS customers, but final award decisions will be made solely by AWS. Proposals will be evaluated based on their express interest in open-sourcing their model artifacts, datasets, and development frameworks. Although usage of Trainium is not a requirement, proposals will be evaluated on their intention to use and explore novel hardware for AI/ML, such as Trainium.
Expectations from recipients
To the extent deemed reasonable, Award recipients should acknowledge the support from ARA. Award recipients will inform ARA of publications, presentations, code and data releases, blogs/social media posts, and other speaking engagements referencing the results of the supported research or the Award. Award recipients are expected to engage with AWS throughout the duration of their award year including, but not limited to, providing reports on the status of their research or updates and feedback to ARA via surveys. Award recipients will have the opportunity to work with ARA on an informational statement about the awarded project that may be used to generate visibility for their institutions and ARA.