Psychological Science ›› 2012, Vol. 35 ›› Issue (2): 452-456.

Previous Articles     Next Articles

The Study of the Impact of the Structure of Item Bank on On-Line Raw Item Attributes Identification Accuracy

  

  • Received:2011-01-05 Revised:2012-02-13 Online:2012-03-20 Published:2012-03-20

题库结构对原始题在线属性标定准确性之影响研究

汪文义1,丁树良2   

  1. 1. 江西师范大学心理学院
    2. 江西师范大学
  • 通讯作者: 丁树良
  • 基金资助:
    国家自然科学基金青年基金项目;江西省教育厅青年科学基金项目;全国教育考试”十一五”科研规划课题;教育部人文社科项目;江西省高等学校教学改革研究课题;江西师范大学研究生创新基金;江西省研究生创新基金

Abstract: Cognitive Diagnostic Assessment is based on the incidence Q-matrix (Tatsuoka, 2009). The entries of Q-matrix indicate which skills and knowledge are involved in the solution of each item. In real situations, whether the items have or have not been identified attributes before its construction, it will cost a lot of money, require more efforts to identify attributes through specialists according the special procedure and yet can’t completely assume the correctness due to the subjectivity. On-line item attributes identification is a new field and study of the impact of item bank hasn’t been found in the literature. So this study is concerned with the impact of item bank on on-line item attributes identification in cognitive diagnostic computerized adaptive testing (CD-CAT), especially when the item bank doesn’t include the whole reachability matrix. The study describes the impact of knowledge states’ equivalent classes on the item attributes vectors’ equivalent classes. Some of those are called the discriminate item attributes vector when the item attributes vectors’ equivalent classes only include one item attributes vector; the others are called indiscriminate item attributes vector. Moreover, the study introduces Marginal Maximum Likelihood Estimation (MMLE) for on-line item attributes identification, which integrates the uncertainty of estimate knowledge states in the procedure of identification.To explore whether the accurary of discriminate item attributes vectors is better than that of indiscriminate item attributes vectors, and whether the columns of reduced Q matrix except the columns of reachability matrix can provide the reasonable accurary of attributes identification. Considering six attributes under the unstructured condition, two simulation experiments are conducted using deterministic inputs,noisy “and” gate model (DINA). The simulation results show that log odds ratios are almost all above zero. It indicates that the correct classification rates (the proportion of times a item is correctly classified on a attributes vector) of the discriminate item attributes vector is significantly better than indiscriminate item attributes vector. The more number of the items in reduced Q matrix except the whole reachability matrix counld compensate the insufficient item bank to some extent. It also demonstrates that the reachability matrix is important for item bank designed for cognitive diagnostic computerized adaptive testing. Other areas of applications of the reachability matrix , including test construction and test equating, are worth further consideration.

Key words: the reachability matrix, CD-CAT, On-Line identification, MMLE, DINA

摘要: 目前已有研究证明可达阵在认知诊断测验编制中起重要作用,但迄今为止并没有引起普遍注意。本文主要讨论当题库缺少某些可达阵对应的项目类,对原始题的属性向量在线标定的准确性的影响。本文对含6个属性的独立型结构进行了模拟试验,结果显示:如果题库不充要,原始题的属性标定准确性受到影响,题库中非可达阵中项目对标定有一定的弥补作用。间接印证了可达阵在认知诊断题库起到非常重要的作用。

关键词: 可达阵, 计算机化自适应诊断测验, 属性向量标定, MMLE, DINA

CLC Number: