Conditions to Consider in the Use of Randomized Experimental Designs in Evaluation

Main Article Content

George Julnes
Melvin Mark
Stephanie Shipman

Abstract

Debates about the role of randomized experiments in evaluation have been heated at times, which likely has not facilitated and possibly has hindered thoughtful judgments about whether and when to use a randomized experimental design. The challenges of thoughtful deliberation may be especially great for funders and others who influence the choice of an evaluation design but are not immersed in methodological literatures. The current paper offers a non-technical summary of general factors to take into consideration when determining the appropriateness of a randomized design in a forthcoming evaluation or set of evaluations. Four general conditions are described that should be considered with respect to the specific context for the upcoming evaluation(s). These are, first, the expected value of the information that a well-implemented experiment can provide in the specific context; second, the legal and ethical issues that apply in the circumstances at hand; third, the practical constraints (or facilitating factors) that would apply to a randomized experiment in that context; and fourth, the likely value of the experimental findings in relation to and as part of a portfolio of evaluative studies in the specific context.

Downloads

Download data is not yet available.

Article Details

How to Cite
Julnes, G., Mark, M., & Shipman, S. (2022). Conditions to Consider in the Use of Randomized Experimental Designs in Evaluation. Journal of MultiDisciplinary Evaluation, 18(42). Retrieved from https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/741
Section
Ideas to Consider

References

AEA. (2019). An evaluation roadmap for a more effective government. Revised. https://www.eval.org

/Portals/0/Docs/AEA%20Evaluation%20Roadmap%202019%20Update%20FINAL.pdf

Bell, S. H., & Peck, L. R. (2016). On the feasibility of extending social experiments to wider applications. Journal of MultiDisciplinary Evaluation, 12(27), 93–111. https://journals.sfu.

ca/jmde/index.php/jmde_1/article/view/452

Julnes, G. (2020). Editor’s Notes: Experimental Methodology. American Journal of Evaluation, 41(4), 486–493. https://doi.org/10.1177/1098214020954305