心理科学

心理科学 ›› 2023, Vol. 46 ›› Issue (6): 1495-1503.DOI: 10.16719/j.cnki.1671-6981.20230628

• 统计、测量与方法 • 上一篇    下一篇

基于称名反应模型的CD-CAT选题方法比较研究 *

张杰1, 罗照盛**1, 喻晓锋**1, 秦春影1,2   

  1. 1江西师范大学心理学院, 南昌, 330022;
    2南昌师范学院数学与信息科学学院, 南昌, 330032
  • 发布日期:2023-12-19
  • 通讯作者: **喻晓锋,E-mail：xyu6@jxnu.edu.cn;罗照盛,E-mail：luozs@jxnu.edu.cn
  • 基金资助:
    *本研究得到江西省教育科学十四五规划 2021 课题（21YB027）和国家自然科学基金项目（32360208）的资助

A Comparative Study of Item Selection Methods in CD-CAT based on Nominal Response Model

Zhang Jie1, Luo Zhaosheng1, Yu Xiaofeng1, Qin Chunying1,2   

  1. 1School of Psychology, Jiangxi Normal University, Nanchang, 330022;
    2School of Mathematics and Information Science,Nanchang Normal University, Nanchang, 330032
  • Published:2023-12-19

PDF

摘要： 当前大多数CD-CAT有关的研究都是基于0-1计分的数据资料展开的,而在实际的教育与心理测验中,还包含大量称名反应数据。本文基于称名反应认知诊断模型（NR-cRUM）开发了适用于称名反应数据的CD-CAT（以下简称NCD-CAT）,并将7种0-1计分CD-CAT的选题方法引入NCD-CAT中。比较不同条件下,不同选题方法对被试判准率和测验效率的影响。结果表明NR_PWCDI、NR_MPWKL等PWKL系新方法和NR_SHE/MI方法能较好地适用于NCD-CAT,且在大多数条件下优于基线方法NR_PWKL。研究拓展了称名多级计分CD-CAT的选题方法。

关键词: CD-CAT, 称名反应, NR-cRUM, 选题方法, 多项选择题

Abstract:

Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) combines the advantages of cognitive diagnosis and CAT, which could improve the efficiency and accuracy of CD-CAT. CD-CAT can be divided into two types, including dichotomous and polytomous. Presently, the majority of researches on CD-CAT are based on dichotomous CD-CAT. However, among the practical tests in psychology and education, there are many polytomous items, which can be further divided into nominal polytomous and ordinal polytomous items according to whether there is an order or grade between every response category. Nominal polytomous items are items whose response categories are independent and without orders or grades between every response category. Although researchers have developed (ordinal) polytomous CDMs and corresponding CD-CAT, few nominal CDMs and CD-CAT are based on nominal responses.

This study introduces seven commonly used item selection methods in dichotomous CD-CAT into NCD-CAT (CD-CAT based on nominal response models). PMR (pattern match ratio) and test efficiency index are evaluated under different conditions between these item selection methods. Here are details of two simulation studies below.

Study 1 compared the performance of NR_PWKL, NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, NR_MI, and NR_GDI methods under different test lengths (5, 10, 15, 20) and item pool qualities (high and low) in NCD-CAT. Results showed that: (1) the PMRs of NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI are higher than or equal to that of NR_PWKL, especially in short tests. (2) As test length gets longer, that PMR advantage is missing, which is the same as the results of Zheng and Chang (2016). (3) Compared to test length, item quality has a greater impact on PMR. For instance, with item quality descending, the PMR declined about 30% among all conditions.

Study 2 was an experiment study on variable-length NCD-CAT that was conducted to compare the performance of each item selection method under the conditions of three maximum posterior probabilities ( .8, .85, .9) and two item qualities (high and low). The results showed that: (1)under all experimental conditions the average test lengths of NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI are shorter than that of NR_PWKL; the difference is more than .738; (2)affected by item quality, the average length of NR_GDI is smaller than that of NR_PWKL under high-quality conditions and larger than it under low-quality conditions.

To sum up, this study compared the performance of 7 commonly used item selection methods of dichotomous CD-CAT in NCD-CAT with different conditions (fixed and variable length). The simulation study showed that, under most conditions, the NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI methods performed well when compared to baseline algorithm NR_PWKL. This study has expanded the alternatives of item selection methods in NCD-CAT.

Key words: CD-CAT, nominal responses, NR-cRUM, item selection methods, multiple-choice items