Determining the Significance of Item Order In Randomized Problem Sets

Researchers who make tutoring systems would like to know which sequences of educational content lead to the most effective learning by their students. The majority of data collected in many ITS systems consist of answers to a group of questions of a given skill often presented in a random sequence. Following work that identifies which items produce the most learning we propose a Bayesian method using similar permutation analysis techniques to determine if item learning is context sensitive and if so which orderings of questions produce the most learning. We confine our analysis to random sequences with three questions. The method identifies question ordering rules such as, question A should go before B, which are statistically reliably beneficial to learning. Real tutor data from five random sequence problem sets were analyzed. Statistically reliable orderings of questions were found in two of the five real data problem sets. A simulation consisting of 140 experiments was run to valida...

• Type: Journal
• Year: 2009
• Where: EDM
• Authors: Zachary A. Pardos, Neil T. Heffernan
• Keywords: Data Mining, EDM 2009, Item Order Effects, Problem Sets, Random Sequence