关键词: GLMM, 概化理论, 项目反应理论, 心理测量

Abstract: It is important to inspect the quality of psychometric tools (e.g., ability tests and personality scales) before they are applied. Due to the drawbacks of classical test theory, GT (generalizability theory) and IRT (item response theory) are becoming popular. Though some efforts have been made to combine GT and IRT, most of the researches continue to employ GT and IRT separately. That is because previous models such as GRIM (Generalizability in Item Response Theory Modeling) and HRM (Hierarchical Rater Model), are a little complicated and lack program to perform them. Therefore, this paper proposed GLMM (Generalized Linear Mixed Model) to unify GT and IRT. GLMM is an extension of Linear Mixed Model. By emploiting various link functions, response variables are no longer limited by continuous data in GLMM. Therefore, it is suitable to analyze discrete data such as dichotomous data. There are a lot of advantages to unify GT and IRT under the framework of GLMM. First of all, GLMM can provide variance components that are key components in GT as well as difficulty parameters that are necessary in IRT at the same time. Secondly, GLMM is simpler than previous models. In addition, we can perform GLMM in many programs such as lme4 package in R, HLM and so on. Last but not least, compared with EMS (Expected Mean Squares), traditional method to estimating variance components in GT, GLMM can avoid the violation of assumption of interval scale, which improves the reliability of analysis. To illustrate the feasibility and the strengths of GLMM, a simulation study and an empirical study were conducted. In the simulation study, σ_p^2=2×π^2/3, σ_i^2=1×π^2/3, and the reason why σ_p^2 and σ_i^2 were the multiples of π^2/3 was that the default residual variance of binominal GLMM using logit as linking function was π^2/3. Setting true parameters of variance component in this way provided us a simple proportional relationship. Person effect and item effect were randomly drawn from normal distribution with variance of σ_p^2 and σ_i^2 respectively, and the item effect was treated as easiness parameter. By exploiting the inverse logit function, the sum of person effect and item effect was transformed to the probability of a correct response. Then binary response was drawn from Bernoulli distribution with probability calculated from last step. GLMM was exploited to analyze the data. To make comparison, EMS and Rasch function in ltm package were also used. The results showed that GLMM provided more precise estimates of variance component, G coefficient and Φ coefficient than EMS did, while difficulty parameters estimated from GLMM were more precise than their counterparts from ltm package. Empirical data was LSAT dataset from ltm package with 1000 subjects, who answered 5 dichotomous questions. The results showed that the percentages of σ_p^2 from GLMM and EMS were close, but the percentages of σ_i^2 or σ_(pi,e)^2 were quite different between methods. In addition, difficulty parameters estimated through GLMM and traditional Rasch model were close. Compared with traditional GT and IRT, GLMM can produce reliable and precise results, especially no longer rely on the interval scale data assumption as EMS does. Therefore, it is appropriate to combine GT and IRT using GLMM to analyze psychometric tools which offers some special advantages.

Key words: GLMM, Generalizability theory, Item Response Theory, Psychometrics