关键词: 问卷数据, 量表, 测量模型, 信度, 效度

Abstract: Questionnaire data have been frequently employed in empirical studies of psychology, as well as in many other behavioral and social science disciplines. This paper discusses preliminary work for modeling questionnaire data, including the data processing which might affect the analysis result. First of all, initial processes of raw data are introduced, including data checking, missing value imputation, and normality test. Then we focus on the questionnaire (scales and items) evaluation based on a measurement model using Confirmatory Factor Analyses (CFA). The construct validity of the scale is acceptable if the measurement model reflecting the hypothetical construct proposed by the theory fits the data with acceptable fit indexes (CFI and TLI > 0.9; RMSEA and SRMR <0.08, say). When item-factor relationship is examined, some items with low loading (e.g., less than 0.4 in the completely standardized solution) are often deleted. It is necessary to consider and explain that the remaining items of the scale are still a representive item sample to measure the latent variable. For a general test, the measurement errors of items are reasonably uncorrelated. If Cronbach's coefficient α is high enough to be accepted, then test reliability is also acceptable. Suppose that the total score of the test is meaningful and employed, it would be better report the composit reliability with a confidence interval. For a multidimensional test, the total score could be employed only when homogeneity reliability is not lower than 0.5. For a reaseach with several latent variables, discriminant validity could be examined by a series of CFA models. One-factor model is the worst fitted whereas the separated-factor model in which one latent variable corresponding to one factor is the best fitted. Discriminant validity is verified if the separated-factor model is obviously better fitted than any other competitive model in the series of CFA models. Then a method factor is added to the separated-factor model as a global factor to set up a bifactor model, and common method bias is not a problem if the bifactor model is not obviously better fitted than the separated-factor model. Structure equation models are frequently applied to analyze questionnaire data. It is suggested that the sample size be large enough so that it is more than 10 times of the nubmer of indicators, or 5 times of the number of parameters which are freely estimated. When the sample size is not large enough, item parceling constitutes a technique of improving the quality of indicators and model fit. The prerequisites for parceling are unidimension and homogeneity, and the applicability of parceling is the analysis of structural models, rather than measurement models. If the scale is multidimensional, internal-consistency approach is recommended so that the items of the same dimension are parceled to one or three indicators for structural equation modeling. When a multiple regression model is involved, multicollinearity could be detected by tolerance or the variance inflation factor (VIF=1/tolerance). Each predictor has a VIF, and a VIF of 5 (or 10) or above indicates a (or serious) multicollinearity problem. A VIF > 5 (or 10) is equivalent to the variance of the predictor is explained by more than 80% (or 90%) by all the other predictors, that is, the coefficient of determination of the regression of the predictor on all the other predictors is larger than 0.8 (or 0.9). For a cross-sectional design of study, it may be an issue how to propose a hypothesis that one variable is a cause of another. The issue could be addressed by domain theory, literature or commonsense. If X is more essential (or more stable, or more objective, or more long-standing, etc.) than Y, X is much more likely to act as a cause than Y. Variable control is necessary for causal inference, in order to eliminate the spurious effect when there exists a common cause of X and Y, or remove the unanalyzed effect when there exists a covariate with which X affects Y. Recently the replication crisis attracted attention and discussion. For a questionnaire data set, different results might arise from different methods of processing data before modeling and analyzing. Appropriate data processing could help obtain a reasonable result and raise the repeatability of the result.

Key words: questionnaire data, scale, measurement model, reliability, validity