EPFL-PoliMI video quality assessment database

Introduction: video quality monitoring over IP networks

The use of IP networks for the delivery of multimedia contents is gaining an increasing popularity as a mean of broadcasting media files from a content provider to many content consumers. In the case of video, for instance, packet-switched networks are used to distribute programs in IPTV applications. Typically, these kinds of networks provide only best-effort services, i.e. there is no guarantee that the content will be delivered without errors to the final users. On the other hand, it is interest of the service providers to offer services which satisfy user's expectation in terms of quality. For this reason, it is fundamental in IP networks in particular, and video broadcasting applications in general, to assess the visual quality of distributed video contents.

The received video sequences may be a degraded versions of the original ones. Besides the distortion introduced by lossy coding, user's experience might be affected by channel induced distortions. In fact, the channel might drop packets, thus introducing errors that propagate along the decoded video content because of the predictive nature of conventional video coding schemes, or it might cause jitter delay, due to decoder buffer underflows determined by network latencies.

Objective and subjective Quality Assessment (QA)

Many algorithms have been developed by the research community, in order to compute objective metrics which try to automatically and reliably predict the quality of the multimedia content as perceived by a human end-user. Objective quality metrics available in literature can be divided in three different categories according to the availability of the original, i.e. reference, signal: Full-Reference (FR) metrics, when both original and processed signals are available; Reduced-Reference (RR) metrics, when besides the processed signal, a description of the original signal and some parameters are available; No-Reference (NR) metrics, when only the processed signal is available. When real-time in-service quality evaluation is needed, only NR metrics can be applied because the unimpaired signal is unavailable at the receiver end.

Although human beings can rather easily make judgments about the quality of multimedia contents, predict the subjective assessment by mean of objective algorithms represents a much harder task. The prediction accuracy of the objective metrics is measured by comparing the values of the objective measures with the results of subjective experiments. In fact, subjective tests are usually performed, where a group of individuals is asked to rate the quality of the digital material or the overall multimedia experience. These results are used as benchmark for the objective metrics.

EPFL-PoliMI Subjective Video Quality Assessment Database