关键词: CD-CAT, 建立题库, 认知诊断模型, 混合模型

Abstract: Cognitive diagnostic computerized adaptive testing (CD-CAT) combines the advantages of both cognitive diagnosis and CAT, which can make adaptive diagnosis for different individuals and provide more detailed diagnostic information on the knowledge competence of the examinees. Currently, CD-CAT has been a promising research area and gained more and more attention. The first step in the implementation of CD-CAT is to build a high quality item bank, and one difficulty that practitioners face is that of how to select the most appropriate cognitive diagnostic model (CDM) from such a large number of models. A wide array of CDMs have been developed based on different assumptions, for example, some reduced CDMs include the Deterministic Inputs, Noisy And Gate (DINA) model, the Deterministic Inputs, Noisy “Or” Gate (DINO; Templin & Henson, 2006) model, the Additive Cognitive Diagnostic Model (ACDM; de la Torre, 2011), the Linear Logistic Model (LLM) and the Reduced Reparametrized Unified Model (RRUM; Hartz, 2002). Apart from these reduced CDMs, some generalized models have also been developed, including the generalized DINA (G-DINA; de la Torre, 2011) model, the general diagnostic model (GDM; von Davier, 2005), and the log-linear CDM (LCDM; Henson, Templin, & Willse, 2009). Compared with the reduced CDMs, generalized CDMs are more complex and require a larger sample size to yield accurate estimates. In addition, compared with the complex generalized CDMs model, using reduced models can improve the accuracy of diagnostic test and may lead to more straightforward and meaningful interpretations. However, almost of all the research and application of CD-CAT has been conducted with the using only one CDM to estimate the item parameters. Analysis of real test data indicated that no single reduced model can be expected to satisfactorily fit all the items. The Wald test was developed as an item-level statistical test to examine whether the G-DINA model can be replaced by a reduced CDM without losing model data fit significantly in order to select an appropriate CDM for each item. The study developed a new selecting model method, namely, mixed models method, to the construction of item bank in CD-CAT. To explore the effectiveness of the mixed model method, three simulation experiments were conducted. The study 1 was aimed to investigate the efficiency of the mixed model method in CD-CAT considering a variety of factors, namely, item selection strategy (SHE and MPWKL), the test length (10, 15, 20 and 25). The number of attributes was fixed to K = 7. An item bank of 360 items was simulated with the highest and lowest probabilities of success, P(1) and P(0), were generated from uniform distributions with U(0.7,0.95) and U (0.05,0.3), respectively. The purpose of study 2 was to compare the efficiency of CD-CAT with the use of G-DINA, DINA, DINO, ACDM, LLM and RRUM model to analyze data generated from mixed model. The last study was to apply the mixed model method to an empirical data. Simulations results showed that the mixed CDMs can be used in the construction of CD-CAT and can improve both the validity and reliability of the test scores from a CD-CAT program.

Key words: CD-CAT, Item bank, CDMs, Mixed-CDMs