Item Familiarity Effects in User-Centric Evaluations of Recommender Systems

Laboratory studies are a common way of comparing recommendation approaches with respect to different quality dimensions that might be relevant for real users. One typical experimental setup is to first present the participants with recommendation lists that were created with different algorithms and then ask the participants to assess these recommendations individually or to compare two item lists. The cognitive effort required by the participants for the evaluation of item recommendations in such settings depends on whether or not they already know the (features of the) recommended items. Furthermore, lists containing popular and broadly known items are correspondingly easier to evaluate. In this paper we report the results of a user study in which participants recruited on a crowdsourcing platform assessed system-provided recommendations in a between-subjects experimental design. The results surprisingly showed that users found non-personalized recommendations of popular items t...

• Type: Journal
• Year: 2015
• Where: RECSYS
• Authors: Dietmar Jannach, Lukas Lerche, Michael Jugovac
• Keywords: Control Systems, RECSYS 2015