Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory

Main Article Content

Jason S. McIntosh
Jane Buckley
Thomas Archibald

Abstract

Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, until recently little empirical inquiry on the dimensions of evaluative thinking has been conducted.


Purpose: To address that lack, the purpose of the study presented in this paper is to refine the construct of evaluative thinking by exploring its underlying dimensions and to ascertain the internal consistency of an instrument developed to measure evaluative thinking, the Evaluative Thinking Inventory (ETI).


Setting: The ETI was developed as part of an ECB initiative focused on non-formal science, engineering, technology, and math (STEM) education in the United States, and was tested as part of a study focused on evaluating gifted education programs, also in the United States.


Intervention: Not applicable.


Research design: Survey research and exploratory factor analysis (EFA).


Data collection & analysis: The ETI was administered to participants in a study measuring the effectiveness of a tool used to conduct internal evaluations of gifted education programs. SPSS was used to conduct an EFA on 96 completed ETIs. Cronbach’s alpha was used to estimate the internal consistency of the instrument.


Findings: The analysis of the ETI revealed a two-factor model of evaluative thinking (i.e., believe in and practice evaluation and pose thoughtful questions and seek alternatives). This study also provided internal consistency evidence for the ETI showing alpha reliabilities for the two factors ranging from 0.80 to 0.82. The ETI has potentially wide applicability in research and practice in ECB and in the field of evaluation more generally.


Keywords: evaluative thinking; evaluation capacity building; research on evaluation; exploratory factor analysis.

Downloads

Download data is not yet available.

Article Details

How to Cite
McIntosh, J. S., Buckley, J., & Archibald, T. (2020). Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory. Journal of MultiDisciplinary Evaluation, 16(34), 104–117. https://doi.org/10.56645/jmde.v16i34.591
Section
Research Articles
Author Biographies

Jason S. McIntosh, Washington Elementary School District

Jason S. McIntosh is the Coordinator of the Gifted Services Department, Washington Elementary School District in Arizona. 

Jane Buckley, JCB Consulting

Jane Buckley is an independent consultant with a focus on evaluative thinking and the development of learning organizations.

Thomas Archibald, Virginia Tech

Thomas Archibald is an Associate Professor and Extension Specialist in the Department of Agricultural, Leadership, and Community Eduation at Virginia Tech.