Psychological Science ›› 2015, Vol. ›› Issue (2): 452-456.

Previous Articles     Next Articles

Research Progress in Computerized Multistage Adaptive Testing

  

  • Received:2014-02-13 Revised:2014-06-21 Online:2015-03-20 Published:2015-03-20

计算机化多阶段自适应测验研究述评

王钰彤1,罗照盛2,王睿1   

  1. 1. 江西师范大学
    2. 江西师范大学 心理学院
  • 通讯作者: 罗照盛

Abstract: Abstract Computerized multistage adaptive testing (MST) is a kind of test forms based on computerized technology, consists of sets of items scored and administered as a unit. These sets of items are called modules or testlets. They are a number of short linear tests, which provide with a certain percentage of test information to reduce the measurement errors. Items in a module may centre on one or several common stems, such as paragraph and diagram, or they may have no relevance with each other. In the MST, adaptations occur at the items sets level, based on the cumulative performance of previous items, then select the next module. MST has fewer adaptations than item level computerized adaptive test (CAT), but more adaptations than conventional paper-and-pencil (P&P) test. It combines the components of conventional P&P test with the adaptive character of CAT. And the advantage of these two test forms yet overcome the disadvantages of them. Thus there is no doubt that it is a compromise of the two tests forms How to build a MST? This is the first thing test developers should consider. The number of stages, modules in every stage, and items in every module, all these must have been decided before the test has been built. Target statistics, and qualitative specification also should be considered before the test has been built. The ways of score, adapt and assemble the test are all the components as vital as listed before. As the test has already been built before it has been taken, test developers could check the items for non-statistical properties, including content balance, ordering and the potential for context effects, cognitive level, item format, answer key position, word count, and any other characteristics of interest or concern in developing the modules. MST may assure the item response theory (IRT) assumptions of local independence and unidimensionality among modules. Items in one stem which violates local independence assumptions are treated as polytomous ones. Therefore all modules should be allocated optimally. When subjects take the test, they can preview and review items in a module, and modify the false. Then, the subjects may operate the modules optimally. Both the test developers and subjects could operate the module optimal, in order to obtain a better result in the exam. MST appeared to provide with the opportunity to improve the quality of examination. It has already been used in many large evaluation tests, such as Uniform CPA Examination and Graduate Record Examination (GRE). Along with the study of various tests, we can find that compared with conventional P&P test and CAT, MST is obviously of the superiority. Compared with conventional P&P test, it superiors in parameter invariance, time saving, feedback in time, estimating more accurately, and so on. Compared with CAT, it superiors in controlling of non-statistical properties, controlling of item exposure, having opportunity to check the items, etc. The direction of future research is how to minimize measurement errors, in order to make the application of MST more convenient and effective.

Key words: Key words Computerized multistage adaptive testing (MST), paper-and-pencil (P&P) test, computerized adaptive test (CAT), stage, module

摘要: 摘 要 计算机化多阶段自适应测验是基于计算机技术的测验形式,它将题目集合作为测试单元,通过多阶段自适应的形式对被试进行测试和评分。近年来通过研究各种测验形式,发现其比计算机化自适应测验和传统纸笔测验突显出更大优势。与传统纸笔测验相比,其具有参数不变性、能力估计更精确等优势。与计算机化自适应测验相比,其具有可控制题目特性、被试可检查题目等优势。如何减小测量误差,使其应用更加便捷、有效,是未来研究的发展方向。

关键词: 关键词 计算机化多阶段自适应测验 传统纸笔测验 计算机化自适应测验 阶段 模块