Psychological Science 2018, (2) 459-465 DOI:     ISSN: 0412-1961 CN: 21-1139/TG

Current Issue | Archive | Search                                                            [Print]   [Close]
Information and Service
This Article
Supporting info
PDF(510KB)
[HTML]
Reference
Service and feedback
Email this article to a colleague
Add to Bookshelf
Add to Citation Manager
Cite This Article
Email Alert
Keywords
CD-CAT
multiple-strategy cognitive diagnosis
multiple-strategy reduced reparameterized unified model (MS-RRUM)
two-step item selection method
Authors
DAI Bu-Yun
ZHANG Min-Jiang
LI Guang-Meng
WANG Xin-Guang
HU Pan
PubMed
Article by Dai,B.Y
Article by Zhang,M.J
Article by Li,G.M
Article by Wang,X.G
Article by Hu,p

A CD-CAT Item Selection Method Containing Strategy, Knowledge State and Ability

1,2, Guang MingLIXin GuangWANG2,shan HU4

Abstract

Cognitive diagnostic computerized adaptive testing (CD-CAT) has become an increasingly important testing mode. Item selection methods are currently one of the most pressing issues in the field of CD-CAT research. For some tasks, not only a participant’s strategy and knowledge state (KS, i.e., attribute mastery pattern) need to be diagnosed, but also his/her macro ability needs to be assessed in CD-CAT. For the sake of high efficient, item selection methods for this type of CD-CAT have to consider the above three purposes. The Multiple-Strategy Shannon Entropy (MSSHE) index (Dai et al., 2015) and Fisher information index were combined to produce two new synthetic item selection methods in this paper. The first method was the “multiple-strategy dapperness with information” (MSDWI) index method, in which the MSDWI index was the Fisher information index Multiplied by the reciprocal of the MSSHE index. The second method was the “two-step item selection method”, in which items were first selected using the MSSHE index and then selected using the Fisher information index when the test length reached a certain value. Based on the multiple-strategy reduced reparameterized unified model (MS-RRUM) and independence attributes, under different lengths of fixed-length CD-CAT and with various numbers of knowledge attributes, three selection item methods were systematically compared in the present study: the above two methods and random method. The strategy, KS and macro ability recoveries were considered as evaluation indices. Experiments were repeated 20 times using the MATLAB 2012b software package. Groups of examinees were simulated, and each group was composed of 1000 examinees. In the simulation, examinees answered the items that were selected by a certain item selection method, and their strategies, KSs and abilities were estimated. Then, the estimated values were compared with the true values. Results showed that with respect to strategy, KS and ability estimations, the two-step method always yielded the best performance when the attribute number was 4 or 6. When the attribute number was 8, the two-step method yielded the best performance with respect to strategy and KS estimations; however, the random method yielded the best performance with respect to ability estimation. In single-strategy CD-CAT which considering KS and ability estimations meanwhile, the “dapperness with information” (DWI) index method which was “in one step” was proper (Dai, Zhang, & Li, 2016). However, according to the current study, in multiple-strategy CD-CAT which considering strategy, KS and ability estimations meanwhile, the MSDWI method which was “in one step” was inappropriate. Instead, the two-step method was better. Besides, for this context, it is not proper to involve too much attributes in a single test. This study performs the CD-CAT which considering three purposes meanwhile, and provides a theoretical foundation for more complicated cognitive diagnostic modeling in future studies. A proper item selection method in this CD-CAT context has been proposed, making item selection efficient and saving item bank.

Keywords CD-CAT   multiple-strategy cognitive diagnosis   multiple-strategy reduced reparameterized unified model (MS-RRUM)   two-step item selection method  
Received 2017-01-22 Revised 2017-11-04 Online: 2018-03-20 
DOI:
Fund:
Corresponding Authors:
Email: 2640726401@qq.com
About author:

References:
Similar articles
1..The Study of the Impact of the Structure of Item Bank on On-Line Raw Item Attributes Identification Accuracy[J]. Psychological Science, 2012,35(2): 452-456
2..Item Selection Methods for Balancing Test Efficiency with Item Bank Usage Efficiency in CD-CAT[J]. Psychological Science, 2014,37(1): 212-216

Comment for this article:

Copyright by Psychological Science