On the accuracy of video quality measurement techniques

With the massive growth of Internet video streaming, it is critical to accurately measure video quality subjectively and objectively, especially HD and UHD video which is bandwidth-intensive. We summarize the creation of a database of 200 clips, with 20 unique sources tested across a variety of devices. By classifying the test videos into 2 distinct quality regions SD and HD, we show that the high correlation claimed by objective video quality metrics is led mostly by videos in the SD quality region. We perform detailed correlation analysis and statistical hypothesis testing of the HD subjective quality scores and establish that the commonly used ACR methodology of subjective testing is unable to capture significant quality differences, leading to poor measurement accuracy for both subjective and objective metrics even on large-screen display devices.