AI for Evaluators: Opportunities and Risks

Main Article Content

Aaron Wilson Kates
Kurt Wilson


We are all familiar with how computers and smartphones have transformed work and productivity. We now stand at a new threshold with another technology that is predicted to further transform the way we work: Artificial Intelligence (AI). 

In this article, we hope to take a sober and practical look at AI from the perspective of professional evaluators. We examine how we might use it, how it might transform the nature of our tasks, what threats it might pose to our field, and what evaluators might do to protect themselves and our society from potential adverse effects of this emerging technology. 


Download data is not yet available.

Article Details

How to Cite
Kates, A. W., & Wilson, K. (2023). AI for Evaluators: Opportunities and Risks. Journal of MultiDisciplinary Evaluation, 19(45), 99–104.
Ideas to Consider
Author Biographies

Aaron Wilson Kates, Effect X

Lead Consultant

Kurt Wilson, Effect X

Owner, Effect X


Brants, T., Popat, A. C., Xu, P., Och, F. J., & Dean, J. (2007). Large language models in machine translation. Google, Inc.

Bruce, K. (2023, March 5). ChatGPT and evaluation: 3 key takeaways [Post]. LinkedIn.

Cantor, M. (2023, April 7). This economist won every bet he made on the future. Then he tested ChatGPT. The Guardian.

Castelvecchi, D. (2016). Can we open the black box of AI? Nature News, 538(7623), 20–23.

Durán, J. M., & Jongsma, K. R. (2021). Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. Journal of Medical Ethics, 47(5), 329–335.

Gabriel, I. (2020) Artificial intelligence, values, and alignment. Minds and Machines, 30, 411–437.

High-Level Expert Group on Artificial Intelligence. (2019). A definition of AI: Main capabilities and disciplines. European Commission.

Kan, M. (2023, March 2). ChatGPT is coming to an app near you: OpenAI launches API for its chatbot. PC Magazine.

Marcus, G. (2022, December 1). How come GPT can seem so brilliant one moment and so breathtakingly dumb the next? The Road to AI We Can Trust.

Microsoft. (2023, March 16). Introducing Microsoft 365 CopilotA whole new way to work. Microsoft 365 Blog.

OpenAI. (2022a, September 28). DALL·E now available without waitlist. OpenAI Blog.

OpenAI. (2022b, November 30). Introducing ChatGPT. OpenAI Blog.

Patton, M. Q. (2011). Essentials of utilization-focused evaluation. SAGE Publications.

Pethokoukis, J. (2016, June 6). What the story of ATMs and bank tellers reveals about the ‘rise of the robots’ and jobs. AEIdeas.

Scriven, M. (2013). The foundation and future of evaluation. In S. I. Donaldson (Ed.), The future of evaluation in society: A tribute to Michael Scriven (pp. 11–44). Information Age Publishing.

Simon, N. (2023, April 15). ChatGPT in evaluationAn opportunity for greater creativity? University World News.

Walker, L. (2023, March 28). Belgian man dies of suicide following exchanges with chatbot. The Brussels Times.