Quality Criteria for Self-Evaluation in Higher Education
Main Article Content
Abstract
The purposes of metaevaluation go beyond the traditional functions of accountability and enhancement. It helps guide strategic organizational change and legitimizes evaluation systems. Metaevaluation results can also be used to create checklists so that the persons responsible for any evaluation can revise, monitor, and control them by themselves. In this paper, the self-evaluation stage of an evaluation process in higher education is metaevaluated. The qualitative method applied analyses to the concerns and complaints that emerged during the work sessions. The results are structured around two key elements: the input (information and self-evaluation guideline) and the throughput (the committee and the process, including the meetings and dynamics of work). The information and self-evaluation guidelines are intended to be responsive, comprehensible, and reliable. The self-evaluation committee and process should be effective, efficient, and responsible with the task. The results are discussed with regard to theoretical proposals for evaluation quality that emphasize criteria such as responsiveness and technical quality of the evaluation.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
American Evaluation Association. (1995). Guiding principles for evaluators. In W. R. Shadish, D. L. Newman, M. A. Scheirer, & C. Wye (Eds.), Developing the guiding principles. New Directions for Program Evaluation, 66, 19-26.
https://doi.org/10.1002/ev.1706
Bologna Declaration of 19 June 1999: Joint declaration of the European Ministers of Education convened in Bologna. (1999). Retrieved November 6, 2008, from http://www.ond.vlaanderen.be/hogeronderwijs/bologna/documents/MDC/BOLOGNA_DECLARATION1.pdf
Bustelo, M. (2003, October). Metaevaluation as a tool for the improvement and development of the evaluation function in public administrations. Paper presented at the 5th Biennial Conference of the European Evaluation Society. Seville, Spain.
Chelimsky, E. (1983). The definition and measurement of evaluation quality as a management tool. In R. G. St Pierre (Ed.), Management and organization of program evaluation. New Directions for Program Evaluation, 18, 113-126.
https://doi.org/10.1002/ev.1340
Chen, H. T. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
Coryn, C. L. S. (2007). The 'holy trinity' of methodological rigor: A skeptical view. Journal of MultiDisciplinary Evaluation, 4(7), 26-31.
https://doi.org/10.56645/jmde.v4i7.7
Coryn, C. L. S., Hattie, J. A., Scriven, M., & Hartmann, D. J. (2007). Models and mechanisms for evaluating government-funded research: An international comparison. American Journal of Evaluation, 28(4), 437-457.
https://doi.org/10.1177/1098214007308290
European Foundation for Quality Management. (2001). EFQM Excellence Model: Public sector and volunteer organisations. Madrid: Club Gestión de la Calidad.
European Association for Quality Assurance in Higher Education. (2005). Standards and guidelines for quality assurance in the European Higher Education Area. Retrieved November 6, 2008, from http://www.enqa.eu/files/ENQA%20Bergen%20Report.pdf
Guba, E. G., & Lincoln, Y. S. (1989). Fourth- generation evaluation. Newbury Park, CA: Sage.
Harrison, J., MacGibbon, L., & Morton, M. (2001). Regimes of trustworthiness in qualitative research: The rigors of reciprocity. Qualitative Inquiry, 7(3), 323-345.
https://doi.org/10.1177/107780040100700305
Haug, G., & Tauch, C. (2001). Towards the European Higher Education Area: Survey of main reforms from Bologna to Prague: Summary and conclusions. Retrieved November 6, 2008, from http://www.ond.vlaanderen.be/hogeronderwijs/bologna/documents/EUA_Trends_Reports/TRENDS_II-April2001.pdf
International Association of University Presidents. (2002). Towards a worldwide quality register for quality assurance and accreditation agencies. Retrieved November 6, 2008, from http://www.iaups.org/documents.php?cat_id=1
Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards. Thousand Oaks, CA: Sage.
Jeliazskova, M. (2002). Running the maze: Interpreting external review recommendations. Quality in Higher Education, 8(1), 89-96.
https://doi.org/10.1080/13538320220127489
Kells, H. R. (1983). Improving institutional performance through self-study. In K. E. Young, C. N. Chambers, H. R. Kells, & Associates (Eds.), Understanding accreditation (pp. 119-132). San Francisco: Jossey-Bass.
Kells, H. R. (1995). Building a national evaluation system for higher education: Lessons from diverse settings. Higher Education in Europe, 20(1-2), 18-26.
https://doi.org/10.1080/0379772950200104
Kristoferssen, D., Sursock, A., & Westerheijden, D. F. (1998). Manual of quality assurance: Procedures and practices. Brussels: EC/Phare/ETF.
Lawrenz, F., Keiser, N., & Lavoie, B. (2003). Evaluative site visits: A methodological review. American Journal of Evaluation, 24(3), 341-352.
https://doi.org/10.1177/109821400302400304
Leviton, L. (1994). Program theory and evaluation theory in community-based program. Evaluation Practice, 15(1), 89-92.
https://doi.org/10.1177/109821409401500111
Lincoln,Y.S.,&Guba,E.G.(1986).Butisit rigorous? Trustworthiness and authenticity in naturalistic evaluation. In D. D. Williams (Ed.), Naturalistic evaluation. New Directions for Program Evaluation, 30, 73-84.
https://doi.org/10.1002/ev.1427
Maasen, P. A. M. (1997). Quality in European higher education: Recent trends and historical roots. European Journal of Education, 32(2), 111-127.
Mintzberg, H. (1993). The rise and fall of strategic planning: Reconceiving roles for planning, plans, planners. New York: The Free Press.
Mora, J. G. (2004). A decade of quality assurance in Spanish universities. In S. Schwarz, & D. F. Westerheijden (Eds.), Accreditation and evaluation in the European higher education area (pp. 421-443). Dordrecht, The Netherlands: Kluwer.
https://doi.org/10.1007/978-1-4020-2797-0_19
Neave, G. (1988). On the cultivation of quality, efficiency and enterprise: An overview of recent trends in higher education in Western Europe 1986-1988. European Journal of Education,25(2-3),273-283.
https://doi.org/10.2307/1502961
Neave, G., & van Vught, F. (1991): Promotheus bound: The changing relationship between government and higher education in Western Europe. Oxford, NY: Pergamon.
Rebolloso, E. (1999). La evaluación de la calidad como estrategia de superviviencia and futuro de la universidad. [Evaluation of quality as a strategy for the survival and the future of the university.] Almería, Spain: Universidad de Almería.
Rebolloso, E., Fernández-Ramírez, B., & Cantón, P. (2003). Guía de evaluación de servicios universitarios. [Guide for the evaluation of university administration services.] Almería, Spain: UCUA.
Rebolloso, E., Fernández-Ramírez, B., & Cantón, P. (2008). Responsibility of educational institutions for strategic change. Journal of MultiDisciplinary Evaluation, 5(10), 5-20.
https://doi.org/10.56645/jmde.v5i10.181
Rebolloso, E., Fernández-Ramírez, B., Cantón, P., & Pozo, C. (2000). El papel de la investigación cualitativa en la evaluación de los servicios universitarios. [The role of qualitative research in the evaluation of university administration services.] Cuadernos IRC, 4, 65-82.
Rebolloso, E., Fernández-Ramírez, B., Cantón, P., & Pozo, C. (2002). Metaevaluation of a total quality management evaluation system. Psychology in Spain, 6(1), 12-25.
Rebolloso, E., Fernández-Ramírez, B., Cantón, P., & Salvador, C. M. (2005). Criterios de calidad de las evaluaciones universitarias. [Criteria of quality of university evaluations.] Paper presented at the IV Conferencia de la Sociedad Española de Evaluación. Retrieved November 6, 2008, from http://www.sociedadevaluacion.org/docs/conferencias/4thMadrid2005/com122.pdf
Rossi, P. H. (1995). Doing good and getting it right. In W. R. Shadish, D. L. Newman, M. A. Scheirer, & C. Wye (Eds.), Guiding principles for evaluators. New Directions for Program Evaluation, 66, 55-59.
https://doi.org/10.1002/ev.1711
Schwandt, T. A. (2002). Evaluation practice reconsidered. New York: Peter Lang.
Scriven, M. (2000). The logic and methodology of checklists. Retrieved October 6, 2008, from http://www.wmich.edu/evalctr/checklists/papers/logic&methodology_oct05.pdf
Shadish, W. R., Newman, D. L., Scheirer, M. A., & Wye, C. (Eds.). (1995). Developing the guiding principles. New Directions for Program Evaluation, 66.
https://doi.org/10.1002/ev.1705
Spanish Council of Universities (2002). Guía de Evaluación de la Titulación, II Plan de la Calidad de las Universidades. [Guide of evaluation of degree, II plan of quality of universities.] Madrid: Secretaría General del Consejo de Universidades.
Stronach, I. (2001). The changing face of responsive evaluation: A postmodern rejoinder. New Directions for Evaluation, 92, 59-72.
Stufflebeam, D. L. (1974). Metaevaluation. Occasional Paper Series #3. Kalamazoo: Western Michigan University Evaluation Center.
Stufflebeam, D. L. (2001a). The metaevaluation imperative. American Journal of Evaluation, 22(2), 183-209.
https://doi.org/10.1016/S1098-2140(01)00127-8
Stufflebeam, D. L. (2001b). Evaluation checklists: Practical tools for guiding and judging evaluations. American Journal of Evaluation, 22(1), 71-79.
https://doi.org/10.1016/S1098-2140(01)00119-9
Unidad para la Calidad de las Universidades Andaluzas. (2004). Guía de autoevaluación de titulaciones. [Guide for the self-evaluation of grades.] Córdoba, Spain: Author.
van Vught, F. A. (1991). Higher education quality assessment in Europe: The next step. CREaction, 96(4), 61-83.
van Vught, F. A., & Westerheijden, D. F. (1994). Towards a general model of quality Assessment in higher education. Higher Education, 28(3), 355-371.