关键词: 过程性数据, 题目作答时间, 认知诊断模型, MCMC算法, PISA2012

Abstract: The development of computer-based assessments can collect process data for different test evaluation purposes. In current psychological measurement modeling, response time is the most frequently studied process data in log files. The use of response time in cognitive diagnosis assessment can further improve the accuracy of diagnosis. However, almost all of the cognitive diagnosis models (CDMs), particularly those involve response time, pay little attention to process data. Recently, a few CDMs, concerning item response times, have been developed, such as the JRT-DINA model (Zhan, Jiao, & Liao, 2018). In this paper, a more concise CDM, named the RRT-DINA (Reduced Response Times DINA) model was proposed via constructing a direct relationship between the probability of correct responses and person speed parameter. And the MCMC algorithm was applied to estimate both item and person parameters of the RRT-DINA model. Firstly, an empirical study was conducted to compare the performances, which are evaluated by the PISA2012 data, of the RRT-DINA model and JRT-DINA model respectively. The results showed that comparing with JRT-DINA model, the new model had lower -2LL, AIC, BIC and DIC values, which indicated that the new model fitted this empirical data better. Secondly, to further verify the performance of the new model, four factors were investigated in the simulation study, i.e., the sample size (500 examines, 1000 examines, 2000 examines), the test length(15 test items,30 test items), the number of cognitive attributes (3, 5, 7) and the types of data generation models (RRT-DINA and JRT-DINA). The results of simulation studies revealed that: (1) When using the RRT-DINA model to generate the data, the recovery of item parameters and person speed parameters of the new model was decent. And the accuracy of parameter estimation in both CDMs will be higher with the increase of the test length, on the whole. Meanwhile, the AACCR and PCCR values obtained from the RRT-DINA model were much more precise than those got from the JRT-DINA model. Under the biggest gap, the AACCR and PCCR of RRT-DINA model were 11.3% and 20.01% higher than JRT-DINA model respectively. Furthermore, the AACCR and PCCR values of the two models were more accurate as the test length increased ; (2) When using the JRT-DINA model to generate the data, the recovery of item parameters and person speed parameters of the new model was slightly inferior to the JRT-DINA model, but the differences in terms of the recovery between the two were very close to those got from the JRT-DINA model. Under the biggest gap, the AACCR and PCCR of JRT-DINA model were only 2.88% and 11% higher than RRT-DINA model respectively. Similarly, the AACCR and PCCR values of the two models were more accurate as the test length increased. In addition, the authors discussed the reasons for choosing response time as a representative of process data, the influence of Q matrix on model parameter estimation and the question of how to choose between the RRT-DINA model and JRT-DINA model. Finally, this paper ends with the prospects of future researches: (1) Introduction of more types of process data; (2)Extension of the speed parameter to multiple dimensions. In general, this paper proposed a simplified and better-performed CDM which could utilize the response times information. And the new model is based on a more direct modeling method rather than the hierarchical modelling framework.

Key words: process data, response times, cognitive diagnosis model, MCMC algorithm, PISA2012