Psychological Science 2018, (2) 466-474 DOI:     ISSN: 0412-1961 CN: 21-1139/TG

Current Issue | Archive | Search                                                            [Print]   [Close]
Information and Service
This Article
Supporting info
PDF(1294KB)
[HTML]
Reference
Service and feedback
Email this article to a colleague
Add to Bookshelf
Add to Citation Manager
Cite This Article
Email Alert
Keywords
cognitive diagnosis
Q-matrix
Item Consistency Criterion
DINA model
Authors
WANG Tai-Xun
GAO Xu-Liang
SA Yan
CHU Dong-Bei
PubMed
Article by Wang,T.X
Article by Gao,X.L
Article by Sa,y
Article by Chu,D.B

A New Q-matrix Estimation Method: ICC based on Ideal Response

1, 1,Tu Dong-Bo

Abstract

Abstract Nowadays, we are not satisfied with a total score from measurement, but hope to get a informative report. As the core of new generation test theory, cognitive diagnosis(CD) attracts more and more people's attention. Since it can reveal the result form a microscopic perspective, such as individuals’ knowledge structures, processing skills and cognitive procedure etc, it would help us to take individualized teaching and promote students ' development. Cognitive diagnosis assessments infer the attribute mastery pattern of respondents by item responses based on Q-matrix. The Q-matrix plays the role of a bridge between items and respondents. Many studies have shown that misspecification of the Q-matrix can affect the accuracy of model parameters and result in the misclassification of respondents. In practice, Q_matrix is established by experts. However , different experts may provided different Q_matrices. To avoid the subjectivity from experts in Q-matrix specification and ensure the correct of Q_matrix, researchers are trying to look for objective methods. Nevertheless,existing methods need information from parameter and a large amount of computation. To simplify the method of Q-matrix estimation, this article introduces a new Q-matrix estimation methods based on ICC(Item Consistency Criterion).The logic of the method as follow: If the measurement pattern of the item A is a subset of the item B.The logic of the ICC method is that it is impossible a person get “0” score on item A,but get “1”on item B. Of course,if item A and item B have same measurement pattern. It is impossible that a person get “1”score on item A,but get “0”on item B(or, the other way around). From this logic we come up with Item Consistency Criterion. In order to improve the effect of ICC method,we come up with ICC-IR (ICC based on ideal response)method. In order to explore the effect of this method, we considered different number of participants, different number of base items and different Q-matrix whose attribute number is different. The item parameters and attribute mastery pattern of respondents are obeyed a uniform distribution. In addition, we compared with the Likelihood D2 Statistic. The Monte Carlo simulation study and real data study showed that: generally, the ICC-IR method can recover the real Q-matrix with a high rate of success. Compared with the Likelihood D2 Statistic, the ICC-IR method is better. Furthermore, the ICC-IR method is easier to understand and needs less computation. The real data study also showed that the ICC-IR method can estimate the Q_matrix with a high success rate. Besides, without the needs of parameters estimation, the method is not affected by the deviation caused by the misfit between model and data. In a word, the method is simple and effective in Q-matrix Estimation, what is meaningful to the simplification of cognitive diagnosis.

Keywords cognitive diagnosis   Q-matrix   Item Consistency Criterion   DINA model  
Received 2017-05-06 Revised 2017-07-30 Online: 2018-03-20 
DOI:
Fund:
Corresponding Authors: Tu Dong-Bo
Email: tudongbo@aliyun.com
About author:

References:
Similar articles
1..Research on MCMC Parameter Estimation and Properties of High Order DINA Model[J]. Psychological Science, 2011,34(6): 1476-1481
2.Tu Dong-Bo.The development of cognitive diagnosis test based on automated item generation[J]. Psychological Science, 2013,36(1): 210-215
3..Item Selection Strategies and Initial Items Selection Methods of CD_CAT[J]. Psychological Science, 2013,36(2): 469-474
4.wenyi Wang Shu-Liang DING.An Item Discrimination Index and its Application in Cognitive Diagnostic Assessment on a Classification-Oriented View[J]. Psychological Science, 2018,(2): 453-458

Comment for this article:

Copyright by Psychological Science