Journal of Psychological Science ›› 2023, Vol. 46 ›› Issue (3): 760-767.

Previous Articles    

The inter-coder consistency in qualitative research

Qi Lidong, Yang Liping, Chen Jiamin, Zhang Dengpan, Shu Mei, Li Xihe   

  1. (School of Psychology, Nanjing Normal University, Nanjing, 210097)
  • Received:2022-06-08 Revised:2022-11-20 Online:2023-05-20 Published:2023-05-20
  • Contact: Li-Ping YANG

质性研究的编码者间一致性

亓立东,杨莉萍,陈家敏,张登攀,舒梅,李兮何   

  1. (南京师范大学心理学院,南京,210097)
  • 通讯作者: 杨莉萍

Abstract: Abstract In recent years, there has been a consistent rise in the number of domains that make use of qualitative research methods. More and more psychologists are resorting to qualitative research to get further into their specialized fields of study. They assert that qualitative research is not only more flexible and adaptive but also offers benefits that quantitative research does not. As the number of qualitative studies conducted in psychology continues to skyrocket, researchers have begun paying increased attention to the methods that may be used to evaluate the quality of a study. This pertains to how researchers demonstrate the rigor of their studies. Coding consistency is considered by some to be an important criterion for evaluating the quality of qualitative research. However, there is considerable debate among researchers about whether coding consistency can be used to demonstrate study quality. Therefore, this study synthesizes and discusses the pertinent ideas raised in previous research. Researchers have proposed the concept of "coding consistency" as a reliability factor for coding in qualitative research. There are two forms of coding consistency: intra-coder consistency and inter-coder consistency. The latter type, inter-coder consistency, is a measurement of reliability that is frequently employed by qualitative researchers. The researchers concluded that a higher level of inter-coder consistency suggested a higher level of repeatability in the coding findings as well as a more objective discovery of reality. Nevertheless, this view is questioned by many scholars. They argue that establishing the reproducibility and objectivity of coding should not be the purpose of assessing inter-coder consistency; rather, the goal should be to encourage coders to engage in a negotiated discourse that helps researchers appreciate the study concerns on numerous levels. This topic has been discussed for a considerable amount of time, but no plausible answer has ever been offered. The results showed that many researchers discussed the topics solely from their point of view and did not consider the many distinct components of qualitative research. Inter-coder consistency may be used as a reliability indicator for qualitative research by researchers who pursue reproducibility and objectivity of research results; however, researchers who identify with constructionism suggest employing negotiated coding/critical friends to find many facets of the study issue. In addition, the research summarized many methods for assessing inter-coder consistency, such as percent agreement, category agreement, Cohen's kappa, and other similar techniques. According to the findings of this research, the approach to inquiry is what governs each stage of the qualitative research process. Qualitative researchers must give some thought to the connection that exists between the research approaches that were used and the approach to inquiry. In addition, researchers want to take into consideration the impact that the personal identity aspects of coders have on collaboration. In the not-too-distant future, academics will be tasked with developing procedures for the comprehensive evaluation of qualitative research.

Key words: inter-coder consistency, coding reliability, negotiated consistency, quality evaluation of qualitative research

摘要: 质性研究的编码者间一致性日渐受到研究者关注。部分研究者倾向于将编码者间一致性作为编码信度指标,以表明编码结果的可重复性与客观性。也有研究者反对这样做,他们强调资料分析结果的社会建构性,建议采取协商对话的方式发展对研究资料的多元理解。基于对以上两种态度的分析,本研究进一步讨论了编码者间一致性在质性研究中的适用性、编码者身份对合作编码的影响及质性研究的综合性评价等问题。

关键词: 编码者间一致性, 编码信度, 协商对话, 研究质量