Prime Video - Automating Quality Analysis & Delivery call for proposals - fall 2021
Solving audio/video challenges with machine learning
About this CFP
Prime Video wants to deliver the best video watching experience for our customers. Our mission is to provide a flawless, high quality, engaging streaming content, while using the fewest bits possible across all content types (VOD, Live, Linear). It is challenging to deliver high-quality video streaming globally across a wide range of device models, different network technologies, and varying home network conditions. Our scientists apply machine learning to optimise the video quality for our customers and automate video analysis so we can operate at scale.
Theme 1: Video Analysis: we want to detect issues that may spoil the experience that customers see or hear in the content they consume. Machine learning allows us to extend monitoring beyond signal processing techniques. We welcome proposals on the following topics:
- In Prime Video content audiovisual synchronization errors happen very rarely. Therefore we are working in a domain with extreme class imbalance. How can we detect audiovisual synchronization errors with high recall and high precision under this extreme class imbalance?
- In live sport content talking heads are sparse making lip-synchronization algorithms of limited benefit. Instead can we identify audiovisual synchronization errors from the context of the content?
- Live sport video is usually accompanied by an audio commentary track. This can lead to false correspondence between talking heads in the video and speech in the audio. How can we distinguish speakers in shot from voice-over content (commentary) for Lip Sync detection?
Theme 2: Adaptive Bitrate Streaming (ABR): the algorithms we use, whether for VOD or Live, have a significant impact on the quality of streaming experience delivered to Prime Video customers. While we have made much progress on developing and deploying new ABR algorithms, there is an outstanding opportunity to innovate in this space by leveraging modern ML methods. We welcome proposals on the following topics:
- Design and implementation of new ABR algorithms using smoothed online convex optimization (SOCO) approaches
- Understanding Quality of Experience (QoE) from the customers’ perspective using quasi-experimental design (QED)
- ML driven auto-tuning and self-learning of ABR algorithms to maximize QoE
Submission period: August 16 - October 8, 2021
Decision letters will be sent out February 2022
Selected Principal Investigators (PIs) may receive the following:
- Unrestricted funds, no more than $80,000 USD on average
- AWS Promotional Credits, no more than $40,000 USD on average
- Training resources, including AWS tutorials and hands-on sessions with Amazon scientists and engineers
Awards are structured as one-year unrestricted gifts. The budget should include a list of expected costs specified in USD, and should not include administrative overhead costs. The final award amount will be determined by the awards panel.
Please refer to the ARA Program rules on the FAQ page.
Proposals should be prepared according to the proposal template. In addition, to submit a proposal for this CFP, please also include the following information:
- Please list the open-source tools you plan to contribute to.
- Please list the AWS ML tools you plan to use and data you plan to obtain.
- Please list the content sets you will use to train and evaluate your results.
ARA will make the funding decisions based on the potential impact to the research community and quality of the scientific content.
Expectations from recipients
To the extent deemed reasonable, Award recipients should acknowledge the support from ARA. Award recipients will inform ARA of publications, presentations, code and data releases, blogs/social media posts, and other speaking engagements referencing the results of the supported research or the Award. Award recipients are expected to provide updates and feedback to ARA via surveys or reports on the status of their research. Award recipients will have an opportunity to work with ARA on an informational statement about the awarded project that may be used to generate visibility for their institutions and ARA.