Journal of Psychological Science ›› 2022, Vol. 45 ›› Issue (6): 1466-1474.

Previous Articles     Next Articles

Item Attribute Vector Balance Strategy in Cognitive Diagnostic Test Design

Xiao-Juan TANG1,Shu-Liang DINGZonghuo YU1   

  • Received:2021-01-18 Revised:2021-10-31 Online:2022-11-20 Published:2022-12-11
  • Contact: Shu-Liang DING

题目属性向量平衡策略的认知诊断测验设计

唐小娟,丁树良,俞宗火   

  1. 江西师范大学
  • 通讯作者: 丁树良

Abstract: CD(Cognitive Diagnosis) can provide personalized diagnostic results for subjects. The selection strategy of CD and CD-CAT (Cognitive Diagnostic Computerized Adaptive Testing) is also a test design in essence, which can provide references for the design of cognitive diagnostic tests. Some studies believe that a high accuracy rate can be obtained only when each attribute is fully measured. Therefore, in CD-CAT, the Attribute Balance(AB) strategy was proposed. Combined with the attribute balance index, MMGDI (Modified Maximum Global Discrimination Index) had a high attribute and pattern accuracy rate. However, when all attributes in projects were constrained by the hierarchical relationship, and the attribute balance strategy was difficult to implement. Therefore, it is more reasonable to consider the item attribute vector balance strategy. It is called the Item Attribute Vector Balance (IAVB) matrix if the attribute hierarchy relationship contained in the test Q matrix which is consistent with the theoretical attribute and hierarchy relationship, and the test Q matrix is composed of different item attribute vectors with the same number of uses.IAVBwas the extension of the AB. In this paper, the study was conducted in two sub-studies. The comparison of IAVB and ABunder the conditions of 3 and 5 attributes with independent structure and 5 attributes with unstructured. The total number of participants in each study was 30 times of theknowledge state categories. The error and guessing parameters of DINA model were fixed at 0.05. Based on the expected response mode, for each score x, a random number r which follows the U (0,1) distribution was generated. If r>0.05, the score was still x; otherwise, the score was 1-x.. The following conclusions could be obtained through the simulation studys : (1) when the numbers of item attributes examined were approximately the same, PMR(Pattern Match Ratio)and other indicators of theIAVB tests were higher than others. Compared with other tests, PMR of the IAVB tests in Study 1 increased by more than 17%. PMR of the IAVB tests in study 2 increased by more than 15%. (2) When the numbers of attributes were different between items, the less the number of attributes was, the higher PMR and other indicators were; The more attributes the item examined, the less information it provided; (3) With independent structure, when the numbers of attributes examied in each item were the same and the times of each attribute examied in a test were the same, each index of the IAVB tests were higher than the AB tests, and the indicators of the AB testswere higher than other tests. For example, in Study 1, compared with the AB tests, PMR of the IAVB test Q1 increased by 14.70% and PMR of Q4 increased by 23.54%. Compared with other tests , PMR of the AB tests increased by 3.98%. In the second study, compared with the AB test Q51, PMR of the IAVB test Q5 increasedby 24.08%, and PMR of Q51 was higher than other tests of Q52 and Q53 30.19%. In conclusion, IAVB strategy, which considers both the attribute hierarchy and the sufficient measurement of each attribute, is the extension of AB strategy. In the design of cognitive diagnostic tests, more items should be selected which have fewer attributes and meet the condition of IAVB. This strategy can be applied not only to small tests, but also to large tests (such as PISA). The design of large test can include as many items as possible with fewer attributes (reusable), and satisfy the item attribute vector balance.

Key words: Cognitive diagnostic test design, IAVB, AB, PMR

摘要: 为组卷制定的题目属性向量平衡(IAVB)策略强调试卷必须体现认知模型,并将题目属性向量而不是以单个属性作为考察单位。该策略克服严格属性平衡(AB)策略仅适用于独立结构的不足,且在每个题目考察属性个数(大致)相同的条件下,以模式判准率(PMR)为衡量标准,该策略优于非IAVB策略。特别地,若属性层级结构为独立结构时,IAVB策略最优,严格属性平衡策略次之,两种策略均未采用的,则最差。在题目属性数一定条件下,IAVB矩阵提高PMR更显著。

关键词: 认知诊断测验设计, IAVB, AB, 模式判准率