Maximum likelihood estimation based DINA model and Q-matrix learning

Cognitive diagnosis models (CDMs) are of growing interest in test development and in the measurement of human performance. The DINA (deterministic input, noisy, and gate) model has been widely used in cognitive diagnosis tests and in the process of test development. Central to many such models, the well-known Q-matrix [1], which specifies the item-attribute relationships and two noise parameters in DINA model related to item response functions are termed as slip and guessing. Slip parameter indicates a student with mastery of all of the attributes that an item requires fails to answer the item correctly. In contrast, guessing parameter indicates a student lacks the attributes that are required by an item but succeeds to answer the item correctly. In this paper, we developed a new method and presented an alternate recursive algorithm to learn Q-matrix and uncertainty variables slip and guessing based on Boolean Matrix Factorization (BMF) and Maximum Likelihood Estimation (MLE) respectively for DINA model of CDM. In particularly, we spontaneously transferred the deterministic Q-matrix learning problem into BMF problem. Because BMF is an NP-hard problem [2], we proposed an alternate recursive method to find approximate solution by adding one dimension in attribute latent space in each step. On the other hand, we analytically estimated the slip and guessing parameters through the maximum likelihood of uncertainty variables. The optimum process is alternate recursive between latent attribute space and uncertainty variable space. Simulation results show that the MLE Q-matrix learning algorithm has fast convergence to the optimal solution under suitable initial values of Qinit and Ainit. This is extremely important and applicable when the method is extended to big data.

[1]  Kenneth R. Koedinger,et al.  Automated Student Model Improvement , 2012, EDM.

[2]  Jimmy de la Torre,et al.  An Empirically Based Method of Q‐Matrix Validation for the DINA Model: Development and Applications , 2008 .

[3]  Curtis Tatsuoka,et al.  Data analytic methods for latent partially ordered classification models , 2002 .

[4]  L. T. DeCarlo.  Recognizing Uncertainty in the Q-Matrix via a Bayesian Extension of the DINA Model , 2012 .

[5]  Václav Snásel,et al.  Implementing Boolean Matrix Factorization , 2008, ICANN.

[6]  J. Templin,et al.  The Effects of Q-Matrix Misspecification on Parameter Estimates and Classification Accuracy in the DINA Model , 2008 .

[7]  Vilém Vychodil,et al.  Discovery of optimal factors in binary data via a novel method of matrix decomposition , 2010, J. Comput. Syst. Sci..

[8]  Jonathan Templin,et al.  Diagnostic Measurement: Theory, Methods, and Applications , 2010 .

[9]  Manolis Mavrikis,et al.  Proceedings of the 7th International Conference on Educational Data Mining. , 2014, EDM 2014.

[10]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[11]  R. Xiang.  Nonlinear penalized estimation of true Q-matrix in cognitive diagnostic models , 2013 .

[12]  Jingchen Liu,et al.  Data-Driven Learning of Q-Matrix , 2012, Applied psychological measurement.

[13]  Michel C. Desmarais,et al.  A Matrix Factorization Method for Mapping Items to Skills and for Enhancing Expert-Based Q-Matrices , 2013, AIED.

[14]  L. T. DeCarlo.  On the Analysis of Fraction Subtraction Data: The DINA Model, Classification, Latent Class Sizes, and the Q-Matrix , 2011 .

[15]  Author,et al.  Item to skills mapping : Deriving a conjunctive Q-matrix from data , 2012 .

[16]  Tiffany Barnes.  Novel Derivation and Application of Skill Matrices: The q-Matrix Method , 2010 .

[17]  Michel C. Desmarais,et al.  Mapping question items to skills with non-negative matrix factorization , 2012, SKDD.

[18]  Radim Belohlávek,et al.  Impact of Boolean factorization as preprocessing methods for classification of Boolean data , 2014, Annals of Mathematics and Artificial Intelligence.

[19]  Jeffrey A Douglas,et al.  Higher-order latent trait models for cognitive diagnosis , 2004 .

[20]  Rudolf Wille,et al.  Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts , 2009, ICFCA.

[21]  Pauli Miettinen,et al.  The Discrete Basis Problem , 2006, IEEE Transactions on Knowledge and Data Engineering.

[22]  Kenneth R. Koedinger,et al.  A Machine Learning Approach for Automatic Student Model Discovery , 2011, EDM.

[23]  Jaideep Vaidya,et al.  Boolean Matrix Decomposition Problem: Theory, Variations and Applications to Data Engineering , 2012, 2012 IEEE 28th International Conference on Data Engineering.

[24]  J. Templin,et al.  Unique Characteristics of Diagnostic Classification Models: A Comprehensive Review of the Current State-of-the-Art , 2008 .

[25]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .