During a three-day workshop in February 2012, 45 Information Retrieval researchers met to discuss long-range challenges and opportunities within the field. The result of the works...
James Allan, W. Bruce Croft, Alistair Moffat, Mark...
Abstract. The notion of having a “living lab” to undertaken evaluations has been proposed by a number of proponents within the field of Information Retrieval (IR). However, wh...
In this SIG, we will present scenarios that exemplify many of the key challenges of doing user research and usability evaluation internationally. We will use these to stimulate di...
Testing and evaluating new architectural propositions is a challenge. Given the usual variety of technologies and scales involved in the necessary evaluation, a one-size-fits-all a...
The paper makes three points of significance for IR research: (1) The Cranfield paradigm of IR evaluation seems to lose power when one looks at human instead of system performance....