Collaborative information filtering techniques play a key role in many Web 2.0 applications. While they are currently mainly used for business purposes such as product recommendation, collaborative filtering also has potential for usage in eLearning applications. The quality of a student provided solution can be heuristically determined by peers who review the solution, thus effectively disburdening the workload of tutors. This paper presents a collaborative filtering approach which is specifically designed for eLearning applications. A controlled lab study with the system confirmed that the underlying algorithm is suitable as a diagnostic tool: The system-generated quality heuristic correlated highly with an expert-provided manual grading of the student solutions. This was true independent of whether the students provided fine-grained or coarsegrained evaluations of peer solutions, and independent of the task type that the students worked on. Further, the system required only few pee...