Predicting Individual Differences in Student Learning via Collaborative Filtering

Effective teaching requires an understanding of a student’s knowledge state—what material the student has and has not mastered and what material is fragile and easily lost. To facilitate automated teaching, our goal is to construct models that infer the knowledge state of individual students for specific elements of knowledge. The challenge to inference is that the available evidence is quite weak. For example, suppose that a student solved four out of five specific long-division problems correctly on a quiz; how well would you expect the student to do on a particular long-division problem assigned a month later? To overcome the sparsity of observations, we use a collaborative filtering approach that leverages information about a population of students studying a population of items (elements of knowledge) to infer how well a specific student has learned a specific item. We extend item-response theory, a traditional class of models that recover latent traits of students and items, to address the facts that knowledge state is nonstationary and that both observations and predictions may span a broad range of time. This extension is based on a psychological model of memory that can take into account dynamic information about study history. We evaluate three alternative models whose latent variables are determined either via maximum likelihood estimation or a hierarchical Bayesian approach. We show for two different student-learning data sets that, when we combine multiple weak sources of information from the population, we can make strong inferences about an individual student’s knowledge and performance.

[1]  Shana K. Carpenter,et al.  The Wickelgren Power Law and the Ebbinghaus Savings Function , 2007, Psychological science.

[2]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[3]  Timothy C. Rickard,et al.  Spacing and the transition from calculation to retrieval , 2008, Psychonomic bulletin & review.

[4]  J. Wixted The psychology and neuroscience of forgetting. , 2004, Annual review of psychology.

[5]  P. Boeck,et al.  Explanatory item response models : a generalized linear and nonlinear approach , 2004 .

[6]  John R. Anderson,et al.  Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect , 2005, Cogn. Sci..

[7]  J. Fox Bayesian Item Response Modeling: Theory and Applications , 2010 .

[8]  Kelli M Taylor,et al.  The effects of overlearning and distributed practise on the retention of mathematics knowledge , 2006 .

[9]  Joel D. Martin,et al.  Student assessment using Bayesian nets , 1995, Int. J. Hum. Comput. Stud..

[10]  John R. Anderson,et al.  Skill Acquisition and the LISP Tutor , 1989, Cogn. Sci..

[11]  J. Templin,et al.  Skills Diagnosis Using IRT-Based Latent Class Models , 2007 .

[12]  W. Marsden I and J , 2012 .

[13]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[14]  H. Pashler,et al.  Distributed practice in verbal recall tasks: A review and quantitative synthesis. , 2006, Psychological bulletin.

[15]  Phillip J. Grimaldi,et al.  Normative multitrial recall performance, metacognitive judgments, and retrieval latencies for Lithuanian—English paired associates , 2010, Behavior research methods.

[16]  D. Andrade,et al.  Item response theory for longitudinal data: population parameter estimation , 2005 .