Are All Biases Bad? Collaborative Grounded Theory in Developmental Evaluation of Education Policy

Abstract

Background: Using two researchers as independent instruments for interpretation in education policy evaluation, this study applies a collaborative grounded theory approach to qualitative data analysis and theory generation.

Purpose: This study argues that varied perspectives should be a critical component in the methodological and analytical choices of education research, especially when the sought after outcome is deeper understanding of the impact, both positive and negative, of an education program or policy. In this study, rather than using one researcher to confirm the reliability of the other, the study explores the outcome of drawing on the positional reflexivity of two researchers, each with a distinct perspective, as a potential strength to co-generate themes and theory in the evaluation of complex policy or programs.

Setting: The data for this analysis originated from interviews of education leaders (n = 13) from two states with contrasting approaches to teacher evaluation: Kentucky and California.

Intervention: NA

Research Design:  Qualitative developmental evaluation

Data Collection and Analysis: Semi-structured interviews and grounded theory coding

Findings: Results suggest that more robust theory and analysis may result from independent thematic development and converged theory generation when working in a research team, as opposed to early application of inter-rater reliability. The reflexivity of different perspectives was, in part, reflexive of the self—their own biased perspective and prior experience. When merged, their joint interpretation may have unearthed greater dimensionality. Findings from this study can inform future strategies for evaluating qualitative research data, especially within a developmental evaluation approach aimed at understanding system complexity in education policy and practice.