EPFL-PoliMI video quality assessment database

 

Introduction: video quality monitoring over IP networks

The use of IP networks for the delivery of multimedia contents is gaining an increasing popularity as a mean of broadcasting media files from a content provider to many content consumers. In the case of video, for instance, packet-switched networks are used to distribute programs in IPTV applications. Typically, these kinds of networks provide only best-effort services, i.e. there is no guarantee that the content will be delivered without errors to the final users. On the other hand, it is interest of the service providers to offer services which satisfy user's expectation in terms of quality. For this reason, it is fundamental in IP networks in particular, and video broadcasting applications in general, to assess the visual quality of distributed video contents.

The received video sequences may be a degraded versions of the original ones. Besides the distortion introduced by lossy coding, user's experience might be affected by channel induced distortions. In fact, the channel might drop packets, thus introducing errors that propagate along the decoded video content because of the predictive nature of conventional video coding schemes, or it might cause jitter delay, due to decoder buffer underflows determined by network latencies.

Objective and subjective Quality Assessment (QA)

Many algorithms have been developed by the research community, in order to compute objective metrics which try to automatically and reliably predict the quality of the multimedia content as perceived by a human end-user. Objective quality metrics available in literature can be divided in three different categories according to the availability of the original, i.e. reference, signal: Full-Reference (FR) metrics, when both original and processed signals are available; Reduced-Reference (RR) metrics, when besides the processed signal, a description of the original signal and some parameters are available; No-Reference (NR) metrics, when only the processed signal is available. When real-time in-service quality evaluation is needed, only NR metrics can be applied because the unimpaired signal is unavailable at the receiver end.

Although human beings can rather easily make judgments about the quality of multimedia contents, predict the subjective assessment by mean of objective algorithms represents a much harder task. The prediction accuracy of the objective metrics is measured by comparing the values of the objective measures with the results of subjective experiments. In fact, subjective tests are usually performed, where a group of individuals is asked to rate the quality of the digital material or the overall multimedia experience. These results are used as benchmark for the objective metrics.

EPFL-PoliMI Subjective Video Quality Assessment Database

In order to provide a balanced and comprehensive database to enable reproducible research in the field of video quality assessment, we provide a publicly available database of 156 video streams, encoded with H.264/AVC and corrupted by simulating the packet loss due to transmission over an error-prone network. The material includes one set of 78 video sequences at CIF spatial resolution and another set of 78 sequences at 4CIF spatial resolution. We also include the results of the subjective tests carried out at the premises of 2 academic institutions, Politecnico di Milano - Italy and Ecole Polytechnique Fédérale de Lausanne - Switzerland. Fourty naive subjects took part to the subjective tests.

For a detailed description of the database and of the subjective test campaign please refer to the papers:

[1] "SUBJECTIVE ASSESSMENT OF H.264/AVC VIDEO SEQUENCES TRANSMITTED OVER A NOISY CHANNEL", authored by Francesca De Simone, Matteo Naccari, Marco Tagliasacchi, Frederic Dufaux, Stefano Tubaro, and Touradj Ebrahimi, accepted at First International Workshop on Quality of Multimedia Experience (QoMEX 2009), July 29–31 2009, San Diego, California, U.S.A.


[2] "H.264/AVC VIDEO DATABASE FOR THE EVALUATION OF QUALITY METRICS", authored by Francesca De Simone, Marco Tagliasacchi, Matteo Naccari, Stefano Tubaro, and Touradj Ebrahimi, submitted to 35th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), March 14–19 2010, Dallas, Texas, U.S.A.


The data and tools made available on this web site are briefly described in the links below. All executables and scripts are provided for Windows operating systems only. In order to  obtain the password to download the data please send an email to francesca.desimone@epfl.ch or marco.tagliasacchi@polimi.it. If you download the data, it is assumed that you agree to the copyright notice reported below. If you use our data for your own publications please do not forget to reference this website and our papers [1] [2].


All inclusive zip file

All the data are available in one single zip file here.

Acknowledge

This work was partially supported by the VISNET II European Network of Excellence (http://www.visnet-noe.org), funded under the European commision IST FP6 programme, and by the Swiss National Foundation for Scientific Research in the framework of NCCR Interactive Multimodal Information Management (IM2).


                                   

Contact information

If you encounter any problem in downloading the data or you have comments please write to francesca.desimone@epfl.ch or marco.tagliasacchi@polimi.it

Copyright notice

Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy, modify, and distribute the data provided and its documentation for research purpose only. The data provided may not be commercially distributed. In no event shall the Ecole Polytechnique Federale de Lausanne and the Politecnico di Milano be liable to any party for direct, indirect, special, incidental, or consequential damages arising out of the use of the data and its documentation. The Ecole Polytechnique Federale de Lausanne and the Politecnico di Milano specifically disclaim any warranties. The data provided hereunder is on an "as is" basis and the Ecole Polytechnique Federale de Lausanne and the Politecnico di Milano have no obligation to provide maintanance, support, updates, enhancements, or modifications.