Empowerment Evaluation of Programs Involving Youth Evaluators’ Perceptions

Main Article Content

Sarah Heath
Katherine Moreau

Abstract

Background: Participatory and collaborative evaluation approaches, including Empowerment Evaluation (EE), are useful for evaluating programs involving youth. Empowerment evaluation involves stakeholders in the evaluation process through a set of structured steps. It is primarily concerned with empowering, illuminating, and building program beneficiaries’ self-determination. Given the emphasis that EE places on inclusivity of stakeholders, it appears to be a good fit for evaluating programs that involve youth.


Purpose: To explore the extent to which evaluators use EE to evaluate programs involving youth as well as what factor(s) facilitate and hinder their use of EE in these programs.


Setting: The study involved evaluators associated with the Collaborative, Participatory and Empowerment Evaluation and Youth-Focused Evaluation Targeted Interest Groups (TIGs) of the American Evaluation Association (AEA) who are involved in evaluating programs targeted at youth.


Intervention: Not applicable.


Research Design: We used a two-phase sequential mixed-methods research design. In Phase 1, we surveyed evaluators. In Phase 2, we interviewed a sample of evaluators from Phase 1. 


Findings: In Phase 1, 41 (53.9%) respondents indicated not using EE to evaluate programs involving youth, 30 (39.5%) had used EE and 5 (6.6%) were unsure. Of those who used EE, they used it to teach youth program stakeholders about evaluation (n=8, 24.2%), produce more authentic results by engaging youth as experts of their lived experience (n=7, 21.2%) or produce more useful results for stakeholders to use (n=6, 18.2%), as well as other less popular reasons. In Phase 2, 12 interviewees raised five factors that facilitate or hinder the use of EE to evaluate programs involving youth including, evaluator perceptions, type of evaluation experience, evaluator knowledge and professional training, guidelines from organizations and funders, and stakeholders and time. Factors that some interviewees viewed as facilitators others viewed as hinderances.


 


Keywords: empowerment evaluation, program evaluation, youth-focused evaluation


 


 

Downloads

Download data is not yet available.

Article Details

How to Cite
Heath, S., & Moreau, K. (2022). Empowerment Evaluation of Programs Involving Youth: Evaluators’ Perceptions. Journal of MultiDisciplinary Evaluation, 18(42). https://doi.org/10.56645/jmde.v18i42.711
Section
Research Articles

References

American Evaluation Association. (2019). Topical interest groups. https://comm.eval.org/communities/allcommunities

Barrington, G. V. (1999). Empowerment goes large scale: The Canada prenatal nutrition experience. The Canadian Journal of Program Evaluation, 14, 179–192.

Beresford, P. (2000). Service users’ knowledges and social work theory: Conflict or collaboration? British Journal of Social Work, 30(4), 489–503. https://doi.org/10.1093/bjsw/30.4.489

Berg, B. (2008). Qualitative research methods for the social sciences (7th ed.). Pearson Education.

Bulanda, J. J., Szarzynski, K., Siler, D., & McCrea, K. T. (2013). “Keeping it real”: An evaluation audit of five years of youth-led program evaluation. Smith College Studies in Social Work, 83(2–3), 279–302. https://doi.org/10.1080/00377317.2013.802936

Checkoway, B., & Richards‐Schuster, K. (2003). Young people as competent citizens. Community Development Journal, 38(4), 298–309. https://doi.org/10.1093/cdj/38.4.298

Chen, P., Weiss, F. L., & Nicholson, H. J. (2010). Girls study Girls Inc.: Engaging girls in evaluation through participatory action research. American Journal of Community Psychology, 46(1–2), 228–237. https://doi.org/10.1007/s10464-010-9328-7

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Rand McNally College Publishing Company.

Cousins, J. B. (2005). Will the real empowerment evaluation please stand up? A critical friend perspective. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation principles in practice (pp. 183–208). Guilford Press.

Cousins, J. B., Donohue, J. J., & Bloom, G. A. (1995). Collaborative evaluation in North America: Evaluators’ self-reported opinions, practices and consequences. Evaluation Practice, 17(3), 207–226. https://doi.org/10.1177/109821409601700302

Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. New Directions for Evaluation, 1998(80), 5–24. https://doi.org/10.1002/ev.1114

Carley-Baxter, L.R., C.A. Hill, D.J. Roe, S.E. Twiddy, R.K. Baxter & Ruppenkamp, J. (2009). Does response rate matter? Journal editors use of survey quality measures in manuscript publication decisions. Survey Practice. 2 (7), 1-7.

Creswell, J. W. (2014). A concise introduction to mixed methods research. SAGE Publications, Inc.

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.

Dillman, D. A. (2011). Mail and internet surveys: The tailored design method (2nd ed.). John Wiley & Sons.

Dold, C. J., & Chapman, R. A. (2012). Hearing a voice: Results of a participatory action research study. Journal of Child and Family Studies, 21(3), 512–519. https://doi.org/10.1007/s10826-011-9505-9

Doyle, L., Brady, A. M., & Byrne, G. (2009). An overview of mixed methods research. Journal of Research in Nursing, 14(2), 175–185. https://doi.org/10.1177/1744987108093962

Fetterman, D. M. (1994). Empowerment evaluation. Evaluation Practice, 15(1), 1–15. https://doi.org/10.1177/109821409401500101

Fetterman, D. M. (2001). Foundations of empowerment evaluation. Sage Publications.

Fetterman, D. M., Rodríguez-Campos, L., & Zukoski, A. (2018). Collaborative, participatory, and empowerment evaluation. Guilford Press.

Fetterman, D. M., & Wandersman, A. (Eds.). (2005). Empowerment evaluation principles in practice. Guilford Press.

Flores, K. S. (2007). Youth participatory evaluation: Strategies for engaging young people. Jossey-Bass.

Fox, J., & Cater, M. (2011). Participatory evaluation: Factors to consider when involving youth. Journal of Extension, 49(2).

Kaplowitz, M. D., Hadlock, T. D. & Levine, R. (2004). A comparison of web and mail survey response rates. Public Opinion Quarterly, 68(1), 94-101.

Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10(2), 307–312. https://doi.org/10.1111/j..2002.384.doc.x

Langhout, R. D., & Fernandez, J. S. (2015). Empowerment evaluation conducted by fourth- and fifth-grade students. In D. M. Fetterman, S. J. Kaftarian, & A. Wandersman (Eds.), Empowerment evaluation: Knowledge and tools for self-assessment, evaluation capacity building, and accountability (2nd ed., pp. 193–232). SAGE Publications, Inc. https://dx.doi.org/10.4135/9781483387079

Lee, S. (2008). Noncoverage. In P. J. Lavrakas (Ed.), Encyclopedia of survey research methods (pp. 520–521). SAGE Publications, Inc. http://dx.doi.org/10.4135/9781412963947.n333

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. SAGE Publications, Inc.

Mark, M. M. (2008). Building a better evidence base for evaluation theory: Beyond general calls to a framework of types of research on evaluation. In N. L. Smith & P. R. Brandon (Eds.), Fundamental issues in evaluation (pp. 111–134). Guilford Press.

Miles, M. B., Huberman, A. M., & Saldana, J. (2020). Qualitative data analysis: A method sourcebook (4th ed.). SAGE Publications, Inc.

Miller, R. L., & Campbell, R. (2006). Taking stock of empowerment evaluation: An empirical review. American Journal of Evaluation, 27(3), 296–319. https://doi.org/10.1177/109821400602700303

Moreau, K. A., & Cousins, J. B. (2012). Program evaluation in family-centred pediatric rehabilitation settings: A review of evaluation studies and the potential use of participatory and collaborative evaluation approaches. Evaluation Journal of Australasia,11(2), 3–13. https://doi.org/10.1177/1035719X1101100202

Patton, M. Q. (1997). Utilization-focused evaluation: The new century text. SAGE Publications, Inc.

Patton, M. Q. (2005). Utilization-focused evaluation. In S. Mathison (Ed.), Encyclopaedia of evaluation (pp. 429–432). SAGE Publications, Inc. https://dx.doi.org/10.4135/9781412950558

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). SAGE Publications, Inc.

Samuelson, B. L., Smith, R., Stevenson, E., & Ryan, C. (2013). A case study of youth participatory evaluation in co-curricular service learning. Journal of the Scholarship of Teaching and Learning, 13(3), 63–81.

Seidman, I. (2013). Interviewing as qualitative research: A guide for researchers in education and the social sciences (4th ed.). Teachers College Press.

Sheldon, J. F. (2016). Evaluation as social intervention: An empirical study of empowerment evaluation practice and principle effects on psychological empowerment and self-determination outcomes [Unpublished doctoral dissertation]. The Claremont Graduate University.

Smith, N. L. (1993). Improving evaluation theory through the empirical study of evaluation practice. Evaluation Practice, 14(3), 237–242. https://doi.org/10.1177/109821409301400302

Upshur, C. C., & Barreto-Cortez, E. (1995). What is participatory evaluation (PE)? What are its roots. The Evaluation Exchange, 1(3/4), 1–7.

Varpio, L., Ajjawi, R., Monrouxe, L. V., O’Brien, B. C., & Rees, C. E. (2016). Shedding the cobra effect: Problematising thematic emergence, triangulation, saturation and member checking. Medical education, 51(1), 40–50. https://doi.org/10.1111/medu.13124

Wandersman, A., Alia, K. A., Cook, B., & Ramaswamy, R. (2015). Integrating empowerment evaluation and quality improvement to achieve healthcare improvement outcomes. British Medical Journal, 24(10), 645–652. https://doi.org/10.1136/bmjqs-2014-003525

Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based approach to accountability. Evaluation and Program Planning, 23(3), 389–395.

Wanzer, D. & Wisner, D. (2020, April 1). Research on evaluation: It takes a village (the problem). Dana Wanzer. https://danawanzer.com/roe-theproblem/

Zeller-Berkman, S., Muñoz-Proto, C., & Torre, M. E. (2015). A youth development approach to evaluation: Critical participatory action research. Afterschool Matters, 22, 24–31. https://files.eric.ed.gov/fulltext/EJ1083955.pdf