A crowdsourceable QoE evaluation framework for multimedia content

Until recently, QoE (Quality of Experience) experiments had to be conducted in academic laboratories; however, with the advent of ubiquitous Internet access, it is now possible to ask an Internet crowd to conduct experiments on their personal computers. Since such a crowd can be quite large, crowdsourcing enables researchers to conduct experiments with a more diverse set of participants at a lower economic cost than would be possible under laboratory conditions. However, because participants carry out experiments without supervision, they may give erroneous feedback perfunctorily, carelessly, or dishonestly, even if they receive a reward for each experiment. In this paper, we propose a crowdsourceable framework to quantify the QoE of multimedia content. The advantages of our framework over traditional MOS ratings are: 1) it enables crowdsourcing because it supports systematic verification of participants’ inputs; 2) the rating procedure is simpler than that of MOS, so there is less...

• Type: Conference
• Year: 2009
• Where: MM
• Authors: Kuan-Ta Chen, Chen-Chi Wu, Yu-Chun Chang, Chin-Laung Lei
• Keywords: Internet Crowd, Lower Economic Cost, MM 2009, Multimedia, Ubiquitous Internet Access