Abstract:

Abstract Multi-stage Testing is a computerized testing which aims to improve the disadvantages of CAT, it incorporates most of the advantages from CAT and linear testing. It has some advantages, such as reducing the test length, efficient for measurement, allowing test takers item review, easier to implement, easier to assemble, flexible test schedule for test takers and so on. Cognitive Diagnosis Assessment aims to determine whether or not examinees have each of many attributes or skills underlying responses to items. In contrast to IRT, CDA provides a more detailed evaluation of the strengths and weaknesses of students. Zheng & Chang (2014) presented a new adaptive testing design called on-the-fly assembled multistage adaptive testing (OMST), which combined the benefits of CAT and MST and offsets their limitations. Like MST, OMST is administered in stages and only adapts between stages. But unlike MST, where the modules in every stage are all pre-assembled before administration, the stages in OMST are assembled on the fly. This study presents Cognitive Diagnostic Multi-stage Testing based on the thoughts of MST、CDA、CD-CAT and OMST, in order to increase the application of CDA. CD-MST is a new idea which combines CD with MST. CD-MST has many advantages. It has the functions of cognitive diagnosis and adaptive; compared with CD-CAT, it has speed superiority, and allows the examinees to go back to check and revise; it is also a flexible testing, the test takers can arrange the testing designs according to their demand. CD-MST can solve some practical problems, it has theoretical and practical values. Among the main components of CD-MST, stage item selection strategies can influent the estimation of the examinee’s attribute vectors. Here we use some CD-CAT’s item selection indices to choose stage items of CD-MST. In CD-CAT, these indices are used to choose item, the largest one or the smallest one is chosen. But in CD-MST, if the stage has five items, it is needed to choose the largest five items or the smallest five items, so it is called stage item selection strategies. The research used two simulations to illustrate the CD-MST. First, we tried to find out if the testing designs of CD-MST could influence the results of CD-MST, two contributory factors were considered, the quality of item bank and the item selection strategy. Second, we wanted to contract the CD-MST and the CD-CAT, the following value indexes were considered, the ACCR and the PCCR. We also investigated the speed of CD-MST and CD-CAT. The results showed that compared with MPWKL, GDI and SHE, the recovery rate of PWKL was poorer. This is because PWKL relies on the estimate of knowledge state α_i^((t)). α_i^((t)) is inaccurate in initial stage of testing, PWKL would be influenced. CD-MST has fewer adaptive frequencies, so PWKL is not fit CD-MST. Compared with CD-CAT, CD-MST was more influenced by the quality of item bank. When the item bank had high quality, the recovery rate of CD-MST was as high as CD-CAT. Form the results of the examinees’ average time consuming, we could conclude that CD-CAT cost more time than CD-MST, CD-MST can save the time of 2/3.