MintMOS: Inferring video QoE in real time



Results from a survey with video clips and human subjects to capture the accuracy on MintMOS projections.



MintMOS Overview

Inferring the subjective perception of a video stream in real time at arbitrary nodes in the Internet continues to be a stiff problem. This is because computing a mean opinion score (MOS) in real time is both cumbersome and infeasible if the original frames are required for reference to infer quality. This project presents MintMOS: a lightweight, no-reference, loadable kernel module to infer the QoE of a video stream in transit and offer suggestions to improve it. MintMOS revolves around a one-time offline construction of a k-dimensional space, which we call the QoE space. A QoE space is a known characterization of subjective perception for any k-parameters (network dependent/independent) that affect it. We create N partitions of the QoE space by generating N video samples for various values of the k-parameters and conduct subjective surveys using them. Every partition then has an expected QoE associated with it. Instantaneous parameters of a video stream in transit are compared to the pre-computed QoE space to infer QoE and offer suggestions to improve it. Inferring QoE is a lightweight algorithm that is computationally inexpensive.

We build a QoE space with four parameters: encoding bitrate, motion complexity, severity of impact, and loss fraction. We create 54 partitions of this 4-dimensional space, and generate 54 video clips for various values of the four parameters. These clips were shown to 77 subjects in a lab and 143 additional online surveys to create a QoE index for every partition.

We validate the effectiveness of MintMOS's predictions using this QoE space by implementing it on a 22-node wide area overlay using PlanetLab. We streamed packets using IP-traces of a variety of low and high motion clips for seven days and used MintMOS to predict perceptual quality. To verify MintMOS's prediction accuracy, a set of reconstructed clips were shown to 49 human subjects whose ratings were matched with ours. Results show that our MOS predictions are in close agreement with subjective perceptions.

We tested MintMOS both as a user level program and a kernel level module on standard, off-the-shelf, Linux terminals. Testbed experiments show that we can perform 20 MOS calculations per second with 11 parameters and 100 partitions of the QoE space, and upto 4 MOS calculations per second for 1 million partitions of the QoE space.

In the end, we show that MintMOS can become a valuable tool for detecting outages on a path, and can be suitably used as an input for various overlay networks, CDNs, VoD and IPTV.


Related Publications