关键词: 认知诊断评估, 拓展Q矩阵理论, 可达阵, 扩张算法, Q矩阵标定方法

Abstract: Cognitive diagnostic assessment has two phases, like a statistical pattern recognition and classification methodology. The first phase is feature generation, followed by classification stage. Q-matrix corresponds to the feature generation phase in statistical pattern recognition. Feature generation is of paramount importance in any pattern recognition task. Therefore, Q-matrix plays a very important role in establishing a relation between latent attribute patterns and ideal response patterns. In practice, a Q-matrix is difficult to specify correctly in cognitive diagnostic assessment and misspecification of the Q-matrix can seriously affect the accuracy of both item parameter estimates and the classification of examinees. The existing methods including the δ method, the γ method, the Q-matrix refinement method, and maximum likelihood estimation method relies on estimates of examinees’ attribute patterns and its classification accuracy. It is not suitable for the case of a test with a short test length because the short test is seldom used to obtain high classification accuracy. The purpose of this study is to propose a method for Q-matrix specification. We assume that a cognitive requirement for multiple skills within an item is conjunctive - that is, answering the item correctly requires mastery of all the skills required by that item. We consider two expected or ideal response matrices, denoted by ERMQ and ERMR. ERMQ or ERMR can be generated from a reduced Q-matrix or a reachability matrix and the universal set of attribute patterns under the conjunctive assumption. Then any column of the ERMQ can be expressed by the columns of the ERMR under the logical AND operation. This is because the augment algorithm in the generalized Q-matrix theory provides the useful fact that any column of the reduced Q-matrix can be expressed by the columns of the reachability matrix under the logical OR operation. A simulation study was conducted to investigate the performance of the new method under three factors (sample size, item parameters in the reachability matrix, and item parameters for the raw items with unknown q-vector) under the deterministic inputs, noisy “and” gate (DINA) model. Simulation results show that the performance of the new method is promising in terms of correct recovery rates of q-entries and correct classification rates of examinees’ attributes. There listed some major results: (a) the average correct recovery rates of q-entries is above 0.90, when guessing and slipping parameters of items in the reachability matrix and raw items are less than 0.20 and 0.3, respectively, (b)the average difference is very small between the correct classification rates of attributes obtained from the nonparametric classification approach based on the true or simulated Q-matrix and the estimated Q-matrix, (c)for an independent structure with 5 attributes, a relatively small sample size of 120 is required through random sampling that is often easy to attain, (d)the new method only needs subject matter experts to specify a Q-matrix for a part of test items which corresponds to the reachability matrix. One conclusion of this study is that the new method will play a very important role in assisting subject matter experts in Q-matrix specification.

Key words: cognitive diagnostic assessment, the generalized Q-matrix theory, the argument algorithm, Q-matrix specification, the reachability matrix