Evaluation and Limitations of Social Interventions: The Case of Spain

Abstract

Background: Although the number and practice of evaluation varies enormously among policy areas, there are very few studies about the unequal evaluation development in the different policy sectors.

Purpose:  This article aims to (1) acknowledge the different evaluation development among different policy sectors in Spain (2) identify the factors and causes that provoke this disproportion, and (3) explore potential consequences of this unequal distribution of evaluation studies among policy areas.  

Setting: Spain.

Intervention: Public polices in Spain.

Research Design: A sample of evaluations is classified by policy sector and the number of evaluations in each sector is analyzed and compared. Then, other significant variables are identified for explaining differences among sectors.

Data Collection and Analysis: The cases (evaluation studies) are drawn from two samples: (1) a data base of evaluation studies and (2) a survey to Spanish evaluators held in 2009. The comparison was done with difference in proportions, adjusted standardised residuals and crosstabs.

Findings: Analysis of Spanish evaluations shows that program evaluations are much more frequent in the social policies’ area than in the areas of security, defense or justice. A variable with a high ability to predict whether or not evaluations will be carried out is identified: the selective versus universal nature of the policies being evaluated. Selective interventions are more frequently evaluated than universal policies. This lack of balance makes selective interventions more prone to severe critical analysis. This evaluation bias, in turn, produces a series of perverse effects such as a greater probability of cutting down programs based on selective application strategies.

Keywords: Evaluation, social policies, pubic policy, Spain, policy limitation