Quantitative Methods for Estimating the Reliability of Qualitative Data

Abstract

Background: Measurement is a indispensable aspect of conducting both quantitative and qualitative research and evaluation. With respect to qualitative research, measurement typically occurs during the coding process.

Purpose: This paper presents quantitative methods for determining the reliability of conclusions from qualitative data sources. Although some qualitative researchers disagree agree with such application, a link between the qualitative and quantitative fields is successfully established through data collection and coding procedures.

Setting: Not applicable.

Intervention: Not applicable.

Research Design: Case study.

Data Collection and Analysis: Narrative data were collected from a random sample of 528 undergraduate students and 28 professors.

Findings: The calculation of the kappa statistic, weighted kappa statistic, ANOVA Binary Intraclass Correlation, and Kuder-Richardson 20 is illustrated through a fictitious example. Formulae are presented so that the researcher can calculate these estimators without the use of sophisticated statistical software.

Keywords: qualitative coding; qualitative methodology; reliability coefficients