关键词: 信度, 合成信度, 元分析, 置信区间

Abstract: Meta-analysis has become an indispensable tool for reaching accurate and representative conclusions about topics of interest within a body of literature. Meta-analysis can also play a very important role in designing new researches. Meta-analysis is one way to obtain more narrow confidence intervals. The confidence interval for the average parameter value will often be considerably narrower, and hence more informative, than the parameter confidence interval obtained from a single study. An increase in external validity is an added benefit of averaging parameter estimates from multiple studies (Bonett, 2009). Test reliability is often used to reflect measurement consistency and stability. Haase (1998) referred to meta-analysis methods of reliability estimates as reliability generalization. Meta-analysis of reliability estimates may be used to obtain more accurate reliability estimates (narrower confidence intervals). Composite reliability can better estimate reliability by using confirmatory factor analysis (Bentler, 2009; Green & Yang, 2009; Wen & Ye, 2011). Meta-analysis of composite reliability can better evaluate test quality. There are three statistical models to do meta-analysis: the constant coefficient model, the random coefficient model, and the varying coefficient model. In general, compared to constant coefficient or random coefficient models, varying coefficient model is recommended to do meta-analysis, which can be used in a much wider range of problems. Under varying coefficient model, we proposed a method to compute point estimate and confidence interval for the average composite reliability coefficient of a unidimensional test. To evaluate the confidence interval for the average composite reliability coefficient obtained by our proposed method, a simulation study was conducted to assess the performance of our proposed method under a wide range of conditions. Four factors were considered in the simulation design: (a) the number of study (m=5, 10, and 20); (b) the number of items on each test (k=3, 6, 10, and 15); (c) factor loading (.5–.7，.7–.9, .5–.9); (d) sample size (N=200–500, 500–1000 and 200–1000). In total, 108 treatment conditions were generated in terms of the above 4-factor simulation design (i.e., 108=3×4×3×3). The simulation results indicated that the performance of our proposed method was remarkable in that its true 95% coverage probability was very close to 95% for all of the 108 conditions. In no case did the coverage probability drop below 94.9%. We recommended that our proposed method could be adopted to estimate the confidence interval of composite reliability for meta-analysis. We used an example of a unidimensional test to illustrate the use of our proposed method to obtain a narrow confidence interval for the average composite reliability across the six study populations.

Key words: reliability, composite reliability, meta-analysis, confidence interval