Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: The case of five AI content detection tools

Abstract

This paper set out to test the accuracy of five AI content tools, GPTZero, OpenAI Text Classifier, Writer.com’s AI Content Detector, Copyleaks AI Content Detector, and Giant Language model Test Room, to detect AI-generated content in the responses generated by ChatGPT, YouChat, and Chatsonic. The responses were generated from these three AI chatbots using English prompts related to applied English language studies. Then, the ChatGPT-generated responses were Google-translated into German, French, Spanish, Southern Sotho, and isiZulu, and inputted into GPTZero for it to detect the AI-generated content in them. Additionally, the ChatGPT-generated responses Google-translated into German, French and Spanish were inputted into Copyleaks AI Content Detector for it to detect the AI-generated content in them. For the ChatGPT-, YouChat-, and Chatsonic-generated responses, Copyleaks AI Content Detector emerged as the top-most performing AI content detector among the five AI content detectors. It was followed by OpenAI’s AI Text Classifier. Concerning the ChatGPT-generated responses that were Google-translated into five languages, GPTZero misidentified all of them as human-produced. For the ChatGPT-generated responses that were Google-translated into German, French and Spanish, Copyleaks AI Content Detector correctly identified three of the German-translated texts, five of the French-translated texts, and all the Spanish-translated texts as AI-generated. Thus, it is evident from this paper that all five AI content detectors seem not yet fully ready to accurately and convincingly detect AI-generated content from machine-generated texts in different contexts. This has dire consequences for AI-generated plagiarism in academic essay writing.

https://doi.org/10.37074/jalt.2023.6.2.12
PDF

Downloads

Download data is not yet available.