What do we know about how the Program Evaluation Standards are used in public health?

Main Article Content

Goldie MacDonald
Kimberly A. Castelin
Naje' George
Asmith Joseph


Background: Released by the Centers for Disease Control and Prevention (CDC), Framework for Program Evaluation in Public Health prominently features the program evaluation standards (1999). The program evaluation standards (PES) include 30 statements in five domains: utility, feasibility, propriety, accuracy, and evaluation accountability. Despite decades of attention to the PES among framework users and others, how public health professionals apply these standards in their work is not well understood.

Purpose: The study sought to identify notable commonalities in how the PES are used in public health.

Setting: Application of the PES in evaluative work in public health and allied fields.

Intervention: Not applicable.

Research Design:  The study included a search of subscription and nonsubscription sources to identify documents that included explicit content concerning use of standards in evaluative work in public health. Documents identified were screened using predetermined criteria to include or exclude each item in the study. Items included were reviewed and coded using codes developed before examining all documents. For each code, reviewers discussed data from all documents to identify commonalities and variations in application of standards.

Findings: The literature search returned 405 documents to be screened (179 from subscription and 226 from nonsubscription sources). Thirty-eight items were included in the study based on initial screening (11 from subscription and 27 from nonsubscription sources). The study revealed that authors discussed standards as a regular component of evaluation work, but precisely how standards were used was not always explained in detail. Also, authors did not always discuss standards statements but sometimes solely focused on general domains (e.g., feasibility or accuracy). When authors discussed specific statements, they were more descriptive in how they applied the PES (i.e., compared with articles that focused on general domains). Overall, authors placed far greater emphasis on Accuracy and Utility standards, compared with Propriety, Evaluation Accountability, or Feasibility. In many cases, authors used the PES in combination with other resources (e.g., checklists, guidelines, or other standards). Although program evaluation is crucial to public health practice, the mechanics of how professionals consider, integrate, or use evaluation standards is not fully understood.

Keywords: program evaluation; program evaluation standards; public health


Download data is not yet available.

Article Details

How to Cite
MacDonald, G., Castelin, K., George, N., & Joseph, A. (2023). What do we know about how the Program Evaluation Standards are used in public health? . Journal of MultiDisciplinary Evaluation, 19(43), 80–86. https://doi.org/10.56645/jmde.v19i43.847
Evaluation Standards Scholarship


American Evaluation Association. (2018). Guiding Principles [Brochure]. https://www.eval.org/Portals/0/Docs/AEA_289398-18_GuidingPrinciples_Brochure_2.pdf

American Public Health Association. (2019). Public health code of ethics. https://www.apha.org/-/media/files/pdf/membergroups/ethics/code_of_ethics.ashx

Anderson, S. B., Braskamp, L. A., Cohen, W. M., Evans, J. W., Gilmore, A., Marvin, K. E., Shipman, V. C., Vanecko, J. J., & Wooldridge, R. J. (1982). Evaluation Research Society standards for program evaluation. New Directions for Evaluation, 1982(15), 7–19. https://doi.org/10.1002/ev.1306

Beerens, R. J. (2021). Improving disaster response evaluations: Supporting advances in disaster risk management through the enhancement of response evaluation usefulness. [Doctoral dissertation, Lund University]. Research Portal.

Bodenhamer, S. A. (2016). Using participatory methods to create a logic model for Healthiest Babies Possible. [Master’s thesis, Simon Fraser University]. Summit Institutional Repository.

Canadian Institute for Health Information. (2009). The CIHI data quality framework. https://secure.cihi.ca/free_products/dq-data_quality_framework_2009_en.pdf

Castrucci, B. C. (2021). The “10 Essential Public Health Services” is the common framework needed to communicate about public health. American Journal of Public Health, 111, 598–599. https://doi.org/10.2105/AJPH.2021.306189

Centers for Disease Control and Prevention (CDC). (1999). Framework for program evaluation in public health. Morbidity and Mortality Weekly Report, 48(RR-11), 1–40. https://www.cdc.gov/mmwr/PDF/rr/rr4811.pdf

Dunet, D. O., Sparling, P. B., Hersey, J., Williams-Piehota, P., Hill, M. D., Hanssen, C., Lawrenz, F., & Reyes, M. (2008). A new evaluation tool to obtain practice-based evidence of worksite health promotion programs. Preventing Chronic Disease, 5(4). https://www.cdc.gov/pcd/issues/2008/oct/07_0173.htm

Fields, J. (2014). A metaevaluation of evaluations of health care programs that employ the chronic care model (Publication No. 241) [Doctoral dissertation, Western Michigan University]. ScholarWorks. https://scholarworks.wmich.edu/dissertations/241

Flay, B. R., Biglan, A., Boruch, R. F., González Castro, F., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–75. https://doi.org/10.1007/s11121-005-5553-y

Glasgow, R. E., Vogt, M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: the RE-AIM framework. American Journal of Public Health, 89(9), 1322–7. https://ajph.aphapublications.org/doi/epdf/10.2105/AJPH.89.9.1322

Gugiu, P. C. (2015). Hierarchy of evidence and appraisal of limitations (HEAL) grading system. Evaluation and Program Planning, 48, 149–59. https://doi.org/10.1016/j.evalprogplan.2014.08.003

International Association for Public Participation. (2007). IAP2 spectrum of public participation. https://www.iap2canada.ca/Resources/Documents/IAP2%20Spectrum_vertical.pdf

Joint Committee on Standards for Educational Evaluation (JCSEE). (n.d.). Program Evaluation Standards. https://evaluationstandards.org/program/

Joint Committee on Standards for Educational Evaluation. (2018). Checklist of the program evaluation standards statements. https://wmich.edu/evaluation/checklists

Kelly, L. M., & Reid, C. (2021). Baselines and monitoring: More than a means to measure the end. Evaluation Journal of Australasia, 21(1), 40–53. https://doi.org/10.1177/1035719X20977522

Organisation for Economic Co-operation and Development (OECD). (2010). Quality standards for development evaluation. https://doi.org/10.1787/9789264083905-en

Public Health National Center for Innovations. (2020). The 10 essential public health services. https://phnci.org/uploads/resource-files/EPHS-English.pdf

Purnell, L. (2002). The Purnell model for cultural competence. Journal of Transcultural Nursing, 13(3), 193–196. https://doi.org/10.1177/10459602013003006

Scriven, M. (2007). Key evaluation checklist. https://wmich.edu/sites/default/files/attachments/u350/2014/key%20evaluation%20checklist.pdf

Smith, M. Y., Russell, A., Bahri, P., Mol, P. G., Frise, S., Freeman, E., & Morrato, E. H. (2018). The RIMES statement: A checklist to assess the quality of studies evaluating risk minimization programs for medicinal products. Drug Safety, 41(4), 389–401. https://doi.org/10.1007/s40264-017-0619-x

Stufflebeam, D. L. (1999). Program evaluations metaevaluation checklist. https://wmich.edu/sites/default/files/attachments/u350/2014/program_metaeval_long.pdf

Tongchiw, T. (2013). The meta-evaluation of revolving funds evaluation in Thailand. [Doctoral dissertation, National Institute of Development Administration]. NIDA Wisdom Repository.

United Nations Evaluation Group. (2016). Norms and standards for evaluation. http://www.unevaluation.org/2016-Norms-and-Standards

United States Government Accountability Office. (2011, December). GAO-12-331G government auditing standards: 2011 revision. https://www.gao.gov/assets/files.gao.gov/assets/gao-12-331g.pdf

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage.