Using Dissemination Research Approaches to Understand the Awareness, Adoption, and Use of The Program Evaluation Standards

Main Article Content

Julie Morrison
Kathleen M. W. Cunningham


Background: The adoption and use of effective, legally defensible, and ethically sound practices relies on the successful dissemination of evidence-based practices and professional standards. The field of program evaluation has standards, competencies, and principles, yet little is known about how these are utilized by education-focused program evaluators.

Purpose: The purpose of this study is to examine the dissemination and use of the program evaluation standards established by the Joint Committee on Standards for Educational Evaluation, relative to the dissemination and use of the American Evaluation Association’s (AEA’s) guiding principles and AEA’s evaluator competencies.

Setting: The SIGnetwork, a network of evaluators of State Personnel Development Grants (SPDGs) funded by the U.S. Department of Education, Office for Special Education Programs (OSEP).

Intervention: NA

Research Design:  Descriptive research.

Data Collection and Analysis: Data collection involved administering an online survey to members designated as evaluators in the SIGnetwork directory. Descriptive statistics were used to summarize the data collected via the online survey.

Findings: Using the formative audience research approach to understanding dissemination, the results of the study support previous findings that awareness of the standards was inconsistent among a sample of AEA members. Respondents self-reported low to moderate levels of familiarity with The Program Evaluation Standards and the other two guidance documents: Guiding Principles for Evaluators and AEA Evaluator Competencies. Using the audience segmentation research approach to understanding dissemination, the results of this study indicate that participants who were AEA members were more likely than those who were not members of AEA to report being familiar with the standards and to have earned an advanced degree related to their role as an evaluator.

Keywords: Joint Committee on Standards for Educational Evaluation, American Evaluation Association, program evaluation standards


Download data is not yet available.

Article Details

How to Cite
Morrison, J., & Cunningham, K. (2023). Using Dissemination Research Approaches to Understand the Awareness, Adoption, and Use of The Program Evaluation Standards. Journal of MultiDisciplinary Evaluation, 19(43), 87–102.
Evaluation Standards Scholarship


American Evaluation Association (AEA) (n.d.). Topical Interest Groups.

American Evaluation Association (2018a). Guiding Principles for Evaluators.

American Evaluation Association (2018b). AEA Evaluator Competencies.

Ayoo, S. (2020). Professionalization of program evaluation: A comparative mixed methods case study of Canada and the United States [Unpublished doctoral dissertation]. University of Minnesota, Minneapolis, MN.

Ayoo, S., Wilcox, Y., La Velle, J. M., Podems, D., & Barrington, G. V. (2020). Grounding the 2018 AEA Evaluation Competencies in the broader context of professionalism. New Directions for Evaluation, 168, 13–30.

Baer, W. C. (1986). Expertise and professional standards. Work and Occupations, 13(4), 532–552.

Bunger, A. C., Hanson, R. F., Doogan, N. J., Powell, B. J., Cao, Y., & Dunn, J. (2016). Can learning collaboratives support implementation by rewiring professional networks? Administration and Policy in Mental Health and Mental Health Services Research, 43(1), 79–92.

Christie, C. A., Quiñones, P., & Fierro, L. (2014). Informing the discussion on evaluator training: A look at evaluators’ course taking and professional practice. American Journal of Evaluation, 35(2), 274–290.

Cramer, T., Canto Porto de Moraes, J., McKenna, A., Keays Hagerman, K., & Allen, L. R. (2021). Knowledge dissemination among early childhood staff members: A promising pathway for professional learning. Journal of Early Childhood Teacher Education, 00(00), 1–14.

Davis, R., & MacKay, K. (2014). Evaluator training content and topic valuation in university evaluation courses. American Journal of Evaluation, 35(3), 419–429.

Dewey, J. D., Montrose, B. E., Schröter, D. C., Sullins, C. D., & Mattox, J. R. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268–287.

Fournier, D. M. (1994). [Review of the book The program evaluation standards: How to assess evaluations, by The Joint Committee on Standards for Educational Evaluation]. Journal of Educational Measurement, 4(31), 363–368.

Gagliardi, A. R., Brouwers, M. C., Palda, V. A., Lemieux-Charles, L., & Grimshaw, J. M. (2011). How can we improve guideline use? A conceptual framework of implementability. Implementation Science, 6(1), 1–11.

Gullickson, A. M., King, J. A., La Velle, J. M., & Clinton, J. M. (2019). The current state of evaluation education: A situation analysis and call to action. Evaluation and Program Planning, 75, 20–30.

Harnar, M. (2019). Familiarity with JCSEE program evaluation standards [Presentation] Annual Meeting of the Joint Committee on Standards for Educational Evaluation, Alexandria, VA, United States.

Hoe, S. L., & McShane, S. (2010). Structural and informal knowledge acquisition and dissemination in organizational learning: An exploratory analysis. Learning Organization, 17(4), 364–386.

House, E. R. (1990). Trends in evaluation. Educational Researcher, 19(3) 24–28.

James Irvine Foundation. (2009). The strong field framework: A guide and toolkit for funders and nonprofits committed to large-scale impact: A field assessment guide. The Bridgespan Group.

Joint Committee on Standards for Educational Evaluation. (1981). Standards for evaluations of educational programs, projects, and materials. McGraw-Hill.

Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards (2nd ed.). Sage.

King, J. A. (2003). Evaluating educational programs and projects in the USA. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook on educational evaluation (pp. 721–732). Kluwer Academic Publishers.

King, J. A., & Stevahn, L. (2020). Presenting the 2018 AEA evaluator competencies. New Directions for Evaluation, 168, 49–61.

La Velle, J. M. (2020). Educating evaluators 1976–2017: An expanded analysis of university-based evaluation education programs. American Journal of Evaluation, 41(4), 494–509.

La Velle, J. M., & Donaldson, S. I. (2015). The state of preparing evaluators. New Directions for Evaluation, 145, 39–52.

Madaus, G. F., Scriven, M., & Stufflebeam, D. L. (Eds.). (2012). Evaluation models: Viewpoints on educational and human services evaluation (Vol. 6). Springer Science & Business Media. Kluwet-Nijhoff Publishing.

Mertens, D. M. (1994). Training evaluators: Unique skills and knowledge. New Directions for Evaluation, 62, 17–28.

Michie, S., Pilling, S., Garety, P., Whitty, P., Eccles, M. P., Johnston, M., & Simmons, J. (2007). Difficulties implementing a mental health guideline: An exploratory investigation using psychological theory. Implementation Science, 2(1), 1–8.

Moss, T., & Crewe, S. E. (2020). The Black Perspective: A framework for culturally competent health related evaluations for African Americans. New Directions for Evaluation, 166, 77–87.

Nahikian, A., Sistenich, V., Audet, F., & Bruderlein, C. (2013). Mapping the Dissemination of Innovation and Practice through Humanitarian Professional Networks. Grotius International.

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.

Purtle, J., Marzalik, J. S., Halfond, R. W., Bufka, L. F., Teachman, B. A., & Aarons, G. A. (2020). Toward the data-driven dissemination of findings from psychological science. American Psychologist, 75(8), 1052–1066.

Robertson, M., Scarbrough, H., & Swan, J. (2003). Knowledge creation in professional service firms: Institutional effects. Organization Studies, 24(6), 831–857.

Rogers, A., Kelly, L. M., & McCoy, A. (2021). Using social psychology to constructively involve colleagues in internal evaluation. American Journal of Evaluation, 42(4), 541–558.

Russ-Eft, D., & Preskill, H. (2009). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change (2nd ed.). Basic Books.

Sage, M., Hitchcock, L. I., Bakk, L., Young, J., Michaeli, D., Jones, A. S., & Smyth, N. J. (2020). Professional collaboration networks as a social work research practice innovation: Preparing DSW students for knowledge dissemination roles in a digital society. Research on Social Work Practice, 31(1) 42–52.

Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Sage Publications.

Stevahn, L., Berger, D. E., Tucker, S. A., & Rodell, A. (2020). Using the 2018 AEA Evaluator Competencies for effective program evaluation practice. New Directions for Evaluation, 168, 75–97.

Stockmann, R., & Meyer, W. (2016). The future of evaluations: Global trends, new challenges and shared perspectives. Palgrave Macmillan.

Symonette, H., Miller, R. L., & Barela, E. (2020). Power, privilege, and competence: Using the 2018 AEA evaluator competencies to shape socially just evaluation practice. New Directions for Evaluation, 168, 117–132.

Tucker, S. A., & King, J. A. (2020). Next steps for AEA and the newly created AEA professionalization and competencies working group. New Directions for Evaluation, 168, 149–162.

Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching and Teacher Education, 24(1), 80–91.

Wenger, E. C. (2010). Communities of practice: A brief introduction. Encyclopedia of Knowledge Management, 1, 1–7.

Worthen, B. R. (1994). Is evaluation a mature profession that warrants the preparation of evaluation professionals? New Directions for Evaluation, 62, 3–15.

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage.