关键词: 认知诊断模型, 知识状态, MLE, MAP, EAP

Abstract: Cognitive diagnostic models (CDMs), which are also referred to diagnostic classification models (Rupp et al., 2010), are multiple discrete latent-variable models. In the past few decades or even earlier, CDMs have become a popular method in many fields, such as psychological and educational measurement, psychiatric evaluation, and other disciplines. Arguably, to offer fine-grained differentiated diagnostic information based on the examinees' observed response data to further help teachers and clinicians taking individualized instructions or interventions is one of the ultimate purposes of CDMs. Three examinee parameter estimation methods have been proposed to classify examinees into a group of latent classes in CDMs, including the maximum likelihood estimation (MLE; Birnbaum, 1968), maximum a posteriori (MAP; Samejima, 1969) and expected a posteriori (EAP; Bock & Mislevy, 1982). Huebner and Wang (2011) investigated the performance of MLE, MAP, and EAP for classifying examinees within the DINA model framework. They found that MLE/MAP had a higher correct classification rate on all K skills. In their study, however, the item parameters and structural parameters were assumed to be known. Although the previous study compared the performance of the MLE, MAP and EAP, the choice of the most suitable examinee parameter estimation methods in CDMs still tend to be a problem.
In this study, we proposed that the main difference between MLE, MAP and EAP is that the last two methods consider the latent knowledge state distribution. Thus, a simulation study was conducted to investigate the impact of latent knowledge state distribution on the classification accuracy of MLE, MAP and EAP. Five factors were manipulated: the attribute tetrachoric correlation (0, .5 and .8), number of sample size (300, 1,000 and 5,000), number of attributes (3 and 5), data-generated models (DINA, DINO, A-CDM and G-DINA) and the types of Q-matrices (correctly and incorrectly). Four evaluation criteria were pattern correct classification rate (PCCR), attribute correct classification rate (ACCR), the classification rate for each skill (Skill_k) and the average of the classification rate for all skill (Total). The classification results for all four criteria were averaged over the 1000 replications. Results showed that, (1) When the attribute tetrachoric correlation was zero, MLE produced the highest correct classification rate with the criteria of PCCR; (2) When the attribute tetrachoric correlation was moderate or high, the EAP and MAP generally yielded higher classification rate than that of the MLE; (3) The correct classification rate increased as the attribute correlation and item quality increased; (4) The correct classification rate of the misspecification of Q-matrix were worse than those in true Q-matrix and items with more attributes had lower accuracy; (5) The DINA and DINO models yielded more accurate classification rate than the G-DINA and A-CDM models.
Overall, choosing the most appropriate knowledge state estimation method is of theoretical and practical importance. The results of this study indicated that the classification accuracy of MLE, MAP and EAP were affected by the latent knowledge state distribution, we recommend using EAP/MAP as an estimation method in practice to ensure the accuracy of estimation.

Key words: cognitive diagnostic model, knowledge state, MLE, MAP, EAP