"Best Tradition": CREATE, JCSEE and the Program Evaluation Standards

Main Article Content

Corrie Klinger
Don A. Klinger

Abstract

Background: Evaluation “is a task in the best tradition of the most abstract theoretical science as well as the most practical applied science” (Scriven, 1968, p .9). The Program Evaluation Standards of the Joint Committee on Standards for Educational Evaluation (JCSEE) operationalize the theoretical aspects of evaluation and, when used, facilitate sound evaluation methods in applied settings. Between the publications of the first and second editions of The Program Evaluation Standards, the Center for Research on Educational Accountability and Teacher Evaluation (CREATE) was funded in 1990 at Western Michigan University with federal monies of $5.2 million, and between 1990 and 1995 by the United States Department of Education, Office of Educational Research and Improvement (OERI). CREATE was established for the betterment of evaluation within the educational context (Stufflebeam, 1991; Stufflebeam & Shinkfield, 1994). CREATE’s mandate and subsequent mission furthered the work of the Program Evaluation Standards and the JCSEE by using the standards in applied settings. Keeping to Scriven’s notion of evaluation as the best tradition, the collaborative work between CREATE and JCSEE is a well-established tradition that furthers the development of theoretical aspects of evaluation and the application of the evaluation standards.


Purpose: Examine CREATE’s impact on the Program Evaluation Standards’ theoretical development and applied use.


Setting: Not applicable.


Intervention: Not applicable


Research Design: Not applicable.


Data Collection and Analysis: Systematic review of the theoretical development and applied use of the Program Evaluation Standards in the books, journal articles, monographs, special papers, meeting minutes, conference programs, and presentations associated with CREATE.


Findings: CREATE has contributed to the operationalization of the theoretical aspects of evaluation with the Program Evaluation Standards and facilitated their use in applied settings. CREATE has also furthered the work of the Personnel Evaluation Standards and the Classroom Assessment Standards (formerly the Student Evaluation Standards). Leading scholars from CREATE and the JCSEE have contributed to the standards since the 1990s. Members of CREATE have published a notable range of books, journal articles, monographs, special papers and conference presentations related to the Program Evaluation Standards. Organizational capacity and shared goals of both the JCSEE and CREATE guided the practical application and theoretical development of the Program Evaluation Standards.


Keywords: Program Evaluation Standards; Consortium for Research on Educational Assessment and Teaching Effectiveness; Joint Committee for Standards on Educational Evaluation

Downloads

Download data is not yet available.

Article Details

How to Cite
Klinger, C., & Klinger, D. A. (2023). "Best Tradition": CREATE, JCSEE and the Program Evaluation Standards. Journal of MultiDisciplinary Evaluation, 19(43), 11–33. https://doi.org/10.56645/jmde.v19i43.833
Section
Evaluation Standards Scholarship

References

Airasian, P. W. (1997). CREATE news. Journal of Personnel Evaluation in Education, 11(1), 89–92.

Airasian, P. W. (2004). Jason Millman Memorial Award & Lecture [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Colorado Springs, Colorado, United States of America.

Airasian, P. W., & Gullickson, A. (1994). Examination of teacher self-assessment. Journal of Personnel Evaluation in Education, 8(2), 195–203.

Airasian, P., & Horn, S. (1998). Consortium for Research on Educational Accountability and Teacher Evaluation. Journal of Personnel Evaluation in Education, 11(4), 381–383.

Airasian, P. W., & Horn, S. (2000a). CREATE news: Consortium for Research on Educational Accountability and Teacher Evaluation. Journal of Personnel Evaluation in Education, 14(1), 89–93.

Airasian, P. W., & Horn, S. (2000b). CREATE news: Consortium for Research on Educational Accountability and Teacher Evaluation. Journal of Personnel Evaluation in Education, 14(4), 319–327.

Bartels, L. M. (1988). Presidential primaries and the dynamics of public choice. Princeton University Press.

Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzalez, E. I., Smith, T. A., Kelly, D. L. (1998). Create news: Consortium for Research on Educational Accountability and Teacher Evaluation: Mathematics and science achievement in the middle school years: An international perspective. Journal of Personnel Evaluation in Education, 12(1), 69–73.

CREATE (2006). Opportunity to apply standards to your work: Call for Proposals: National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation. CREATE Newsletter, 9 (2), 5.

CREATE. (2006). Personnel evaluation with Dr. Barbara Howard, student evaluation with Dr. Arlen Gullickson and program evaluation with Dr. Don Yarbrough [Post-conference sessions]. Joint Committee on Standards for Educational Evaluation at the National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Dallas, Texas, United States of America.

CREATE Board of Directors. (2012). CREATE 2012–2015 Strategic Goals.

CREATE Board of Directors. (2013, September 11). CREATE 2012–2015 Strategic Goals, Version 2.0.

CREATE Board of Directors. (2014, January 16). DRAFT Consortium Document.

CREATE Board of Directors. (2015, February 6). Agenda and Minutes, CREATE Winter Board Meeting. Consortium for Research on Educational Assessment and Teaching Effectiveness (CREATE).

CREATE Business Meeting. (2015, October 17). CREATE Mission Statement (Current) and (Revised).

CREATE Bylaws. art. IV, § 3.C.

CREATE News. (1999). Journal of Personnel Evaluation in Education, 13(3), 295.

CREATE Newsletter. (2009). Keynote Speakers for the 2009 NEI. CREATE Newsletter, 12 (4), 4.

CREATE. (2010). CREATE Newsletter, 13 (2), 7.

CREATE website. (2018). Prior 2018 Website Content. Davis, D., & Muñoz, M. A. (2007).

Highlights from the 2007 National Evaluation Institute October 4-6, 2007, The Rosen Centre Hotel, Orlando, Florida. CREATE Newsletter, 11 (1), 2–3.

Dearmin, E. T. (1994). Assonance and dissonance in teacher preparation: Moving across state boundaries. Journal of Personnel Evaluation in Education, 8(2), 119–134.

Dwyer, C. A. (1994). Criteria for performance-based teacher assessments: Validity, standards, and issues. Journal of Personnel Evaluation in Education, 8(2), 135–150.

Dwyer, C.A. (2004). Keynote Address.[Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Colorado Springs, Colorado, United States of America.

Egelson, P. (2010). Joint Committee on Standards for Education Evaluation. CREATE Newsletter, 13 (3), 2.

Egelson, P. (2012). Message from the President. CREATE Newsletter, 16 (1), 1.

Egelson, P. (2014). CREATE Conference 2013 reflections – Atlanta, Georgia, October 1–12, 2013. CREATE Newsletter, 17 (1), 2.

Egelson, P., & Howard, B. (2008). Joint Committee on Standards for Educational Evaluation: Enhancing personnel evaluations through an application of the standards. CREATE Newsletter, 11 (2), 2.

Egelson, P., Lee, S., Franchini, A., Donnelly, L., Freeman, C., & McCaslin, R. (2010). Using the Program Evaluation Standards to assist in implementing alternative school program evaluation [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Williamsburg, Virginia, United States of America.

Egelson, P., Purser, R., & Fuga, C. (2018). Implementing the Program Evaluation Standards in real-time settings [Conference session]. The Consortium for Research on Educational Assessment and Teacher Effectiveness, High Point, North Carolina, United States of America.

Ellett, C. D. (1999). Special editorial announcements. Journal of Personnel Evaluation in Education, 3(3), 199–200.

Ellett, C. D. (2000). From the editor. Journal of Personnel Evaluation in Education, 14(4), 283–284.

Ellett, C. D. (2001). From the editor. Journal of Personnel Evaluation in Education, 15(1), 5–6.

Ellett, C. D., Loup, K. S., & Stronge, J. H. (1997). From the new editors. Journal of Personnel Evaluation in Education, 11(1), 5–6.

Fischetti, J. (2014a). Message from the president. CREATE Newsletter, 17 (1), 1.

Fischetti, J. (2014b). Message from the president. CREATE Newsletter, 17 (2), 1.

Gullickson, A. R. (2008). Jason Millman Memorial Award & Lecture [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Wilmington, North Carolina, United States of America.

Gullickson, A. R. (2016). A focus on standards research [Conference session]. The Consortium for Research on Educational Assessment and Teacher Effectiveness, Louisville, Kentucky, United States of America.

Gullickson, A. R., & Howard, B. (2009). The personnel evaluation standards: How to assess systems for evaluating educators (2nd ed.). Corwin Press.

Gullickson, A. R., Howard, B., & Egelson, P. (2004). The Program Evaluation Standards [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Colorado Springs, Colorado, United States of America.

Hanssen, C. (1996). Beyond CREATE: The beginnings of a new national organization for educational accountability and teacher evaluation. Journal of Personnel Evaluation in Education, 10(1), 7–10.

Horn, S. P., & Airasian, P. W. (1997). Create news: Consortium for Research on Educational Accountability and Teacher Evaluation. Journal of Personnel Evaluation in Education, 11(2), 197–200.

Horn, S., & Airasian, P. (1999). CREATE NEWS: Consortium for Research on Educational Accountability and Teacher Evaluation. Journal of Personnel Evaluation in Education, 13(1), 95–99.

Howard, B. (2007). [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Orlando, Florida, United States of America.

Howard, B. (2010a). Message from the president. CREATE Newsletter, 13 (1), 1.

Howard, B. (2010b). Message from the president. CREATE Newsletter, 13 (2), 1.

Howard, B. (2010c). Message from the president. CREATE Newsletter, 13 (3), 1.

Howard, B. (2010d). Message from the president. CREATE Newsletter, 13 (4), 1.

Howard, B., & Muñoz, M. (2012). The Classroom Assessment Standards: A guide to best practices on assessment for teachers and administrators in pre-K – 12 classrooms. CREATE Newsletter, 16(1), 6–7.

Iwanicki, E. F. (1991). About this issue. Journal of Personnel Evaluation in Education, 5(1), 5.

Iwanicki, E. F. (1992). About this issue. Journal of Personnel Evaluation in Education, 5(3), 243–244.

Joint Committee on Standards for Educational Evaluation. (1981). Standards for evaluations of educational programs, projects, and materials. McGraw-Hill.

Joint Committee on Standards for Educational Evaluation. (1988). The Personnel Evaluation Standards. Sage.

Joint Committee on Standards for Educational Evaluation. (2003). The Student Evaluation Standards. Corwin.

Kam, C. D., & Zechmeister, E. J. (2013). Name recognition and candidate support. American Journal of Political Science, 57(4), 971–986. https://doi.org/10.1111/ajps.12034

Klinger, C. R., & Klinger, D. A. (2021). Regarding evaluation. 2020 Annual Conference Proceedings of the Consortium for Research on Educational Assessment and Teacher Effectiveness, 1, 28–30. https://uploads-ssl.webflow.com/623e877dd599265fa93c3790/62b4f6c5fd674f1b61271e21_2021.CREATE..pdf

Klinger, C. R., & Klinger, D. (2021, October). Unresolved controversy: Assessment & evaluation [Conference session]. The Consortium for Research on Educational Assessment and Teaching Effectiveness. Virtual Conference.

Klinger, D. A., McDivitt, P. R., Howard, B. B., Muñoz, M. A., Rogers, W. T., & Wylie, E. C. (2015). The Classroom Assessment Standards for preK–12 teachers. Kindle Direct Press.

Krathwohl, D. R. (2009). Historical research, chapter 25. In D. R. Krathwohl (Ed.), Methods of educational and social science research: The logic of methods (pp. 605–614). Waveland Press.

Linn, R. (2004). Keynote Address.[Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Colorado Springs, Colorado, United States of America.

Marzano, R. (2004). Keynote Address. [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Colorado Springs, Colorado, United States of America.

McDivitt, P. J., Klinger, D. A., Howard, B., Muñoz, M., Rogers, W. T., & Wylie, C. (2011). Classroom Assessment Standards (formerly known as Student Evaluation Standards). CREATE Newsletter, 15(1), 4.

Muñoz, M. A. (2006). Highlights of the Consortium for Research on Educational Accountability and Teacher Evaluation (CREATE): Expanding Research and Promising Practices in Personnel, Program, and Student Evaluation [Special issue]. Journal of Personnel Evaluation in Education, 19(1/2), 1–3.

Muñoz, M. A. (2007). Forthcoming change on CREATE’s journal. CREATE Newsletter, 11 (1), 5.

Muñoz, M. (2009). Message from the president. CREATE Newsletter, 12(1), 1.

Muñoz, M. (2010). The Student Evaluation Standards: A helpful tool for implementing assessment for learning. CREATE Newsletter, 13(2), 2–3.

Muñoz, M. A., & Chang, F. C. (2007). The elusive relationship between teacher characteristics and student academic growth: A longitudinal multilevel model for change. Journal of Personnel Evaluation in Education, 20(3/4), 147–164.

Muñoz, M., & Chang, F. (2008). CREATE Newsletter, 11(2), 8.

Muñoz, M. A. & Davis, D. R. (2007). Improving teaching and learning through evaluation: A special issue on the National Evaluation Institute, Consortium for Research on Educational Accountability and Teacher Evaluation (CREATE) [Special issue]. Journal of Personnel Evaluation in Education, 20(3/4), 111–116.

Muñoz, M., & Rodosky, R. (2010). Teaching-embedded formative assessment to improve student learning: A perspective from the field. CREATE Newsletter, 13(1), 4–5.

Nevo, D. (1994). How can teachers benefit from teacher evaluation? Journal of Personnel Evaluation in Education, 8(2), 109–117.

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). Sage.

Rodosky, R. (2003). Message from the president. CREATE Newsletter, 7(1), 1.

Rodosky, R. (2005). Message from the president. CREATE Newsletter, 9(1), 1.

Rodosky, R. (2009). Jason Millman Memorial Award & Lecture [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Louisville, Kentucky, United States of America.

Rodosky, R. (2022, August 6). Robert Rodosky. LinkedIn. https://www.linkedin.com/in/robert-rodosky-75417417/

Ross, A. (2015, June 23). Longtime JCPS administrator Rodosky to retire. The Courier Journal. https://www.courier-journal.com/story/news/education/2015/06/23/longtime-jcps-administrator-rodosky-retire/29183621/

Sanders, J. R., & Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Sage.

Sanders, W. L. (2000a). Jason Millman Memorial Award & Lecture [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, San Jose, California, United States of America.

Sanders, W. L. (2000b). Value-added assessment from student achievement data: Opportunities and hurdles: CREATE National Evaluation Institute, July 21, 2000. Journal of Personnel Evaluation in Education, 14(4), 329–339.

Schwab, R. L. (1990). About this issue. Journal of Personnel Evaluation in Education, 4(1), 5–6.

Scriven, M. (1967). The methodology of evaluation. In R. W. Tyler, R. M. Gagné & M. Scriven (Eds.), Perspectives of curriculum evaluation (vol. 1, pp. 39-83). Rand McNally & Company.

Scriven, M. (1968). Evaluation as a main aim of science: Comments on Professor Bloom’s paper entitled toward a theory of testing which includes measurement-evaluation-assessment [CSE Report 10]. University of California, Los Angeles, Center for the Study of Evaluation.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Sage.

Scriven, M. (1994). Duties of the teacher. Journal of Personnel Evaluation in Education, 8(2), 151–184.

Scriven, M. (2001). Jason Millman Memorial Award & Lecture [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Wilmington, North Carolina, United States of America.

Scriven, M. (2016). Roadblocks to recognition and revolution. The American Journal of Evaluation, 37(1), 27–44. https://doi.org/10.1177/1098214015617847

Shinkfield, A. J., & Stufflebeam, D. L. (1995). Teacher evaluation: Guide to effective practice. Springer Netherlands.

Stronge, J. H. (2012). Jason Millman Memorial Award & Lecture. [Conference session]. The National Evaluation Institute of the Consortium for Research on EducationalAccountability and Teacher Evaluation, Washington, D.C., United States of America.

Stufflebeam, D. L. (1971). The Relevance of the CIPP evaluation model for educational accountability. Journal of Research and Development in Education, 5(1), 19-25.

Stufflebeam, D. L. (1991). An introduction to the Center for Research on Educational Accountability and Teacher Evaluation (CREATE). Journal of Personnel Evaluation in Education, 5(1), 85–92.

Stufflebeam, D. L. (1999). Jason Millman Memorial Award & Lecture [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Travers City, Michigan, United States of America.

Stufflebeam, D. L. (2000). The methodology of metaevaluation as reflected in metaevaluations by the Western Michigan University Evaluation Center. Journal of Personnel Evaluation in Education, 14(1), 95–125.

Stufflebeam, D. L. (2004). A note on the purposes, development, and applicability of the Joint Committee Evaluation Standards. The American Journal of Evaluation, 25(1), 99–102.

Stufflebeam, D. L. (2009). Keynote address [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Louisville, Kentucky, United States of America.

Stufflebeam, D. L., & Egelson, P. (2016). History of CREATE and Celebrating 25 Years of CREATE at This Year’s CREATE Conference.

Stufflebeam, D. L., & Madaus, G. F. (1983). The standards for evaluation of educational programs, projects, and materials. In G.F. Madaus, M. S. Scriven, & D. L. Stufflebeam (Eds.), Evaluation models: Evaluation in education and human services (vol. 6, pp. 395-404). Springer. https://doi.org/10.1007/978-94-009-6669-7_23

Stufflebeam, D. L. & Shrinkfield, A. J. (1985). Stufflebeam’s improvement-Oriented evaluation. In D. L. Stufflebeam & A. J. Shrinkfield’s (Eds.), Systematic evaluation: A self-Instructional guide to theory and practice. Evaluation in education and human services (vol 8, pp. 151-207). Springer.

Stufflebeam, D. L., & Shinkfield, A. J. (1994). About this issue. Journal of Personnel Evaluation in Education, 8(2), 103–108.

Wetherill, K. (2006). Message from the president. CREATE Newsletter, 10(1), 1.

Withers, G. (1994). Getting value from teacher evaluation. Journal of Personnel Evaluation in Education, 8(2), 185–194.

Yarbrough, D. B. (2008). National Hearing on JCSEE Standards [Conference session]. The National Evaluation Institute of the Consortium for Research on Educational Accountability and Teacher Evaluation, Wilmington, North Carolina, United States of America.

Yarbrough, D. B. (2019). 2019 Jason Millman Award & Lecture [Conference session]. The Consortium for Research on Educational Assessment and Teacher Effectiveness, High Point, North Carolina, United States of America.

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage. https://evaluationstandards.org/program/