Question-Driven Methods or Method-Driven Questions? How We Limit What We Learn by Limiting What We Ask

Abstract

The “methodologically manic-obsessive” evaluation profession and the metrics- and measures-obsessed lay users of evidence have managed to seriously limit the value of what we learn from evaluation. Evaluation questions asked at the front end are limited by the askers’ narrow understanding of what is possible methodologically at the back end. This, alongside the political and psychological forces working against real evaluation, are major drivers of the single narrative thinking that pervades the formulation and evaluation of national and local government policies and initiatives. This paper provides practical suggestions for asking the big-picture questions that really need to be asked, and suggests how real evaluation can step up to the plate, methodologically and otherwise.