Are ChatGPT detectors reliable for catching plagiarism?

ChatGPT and other AI tools are a student's godsend! Providing ways to write essays and topics has never been easier. Just type in a thought and tell ChatGPT to get writing.

On the other hand, these tools can be a teacher's nightmare! Essay writing is meant for students to think critically and express their thoughts about a topic or an idea. AI tools defeat this purpose by removing the human element.

What's a teacher to do? Use a ChatGPT detector, of course!

I put 4 ChatGPT detectors to the test:

GPT Zero

Contentdetector.AI

Copyleaks

Crossplag

I created a ChaptGPT essay to test these AI detectors' abilityto recognize AI-generated content.

GPTZero

Created by Edward Tian in December 2022 while attending Princeton University, GPTZero analyzes writing patterns through syntax and sentence lengths to identify AI-generated texts.

Pros

• Can upload a document for detection

• Can copy and paste text

Cons

• May not completely detect AI-generated text

Copyleaks

Copyleaks was created for finding and detecting plagiarized content by comparing text against other sources by Yehonatan Bitton and fellow military veteran Alon Yamin, after realizing his company's website was being plagiarized.

Pros

• Has a Chrome extension

• Detects AI mixed with human text

• Detects through an LMS (learning management system)

Cons

• May take longer for the system to check for plagiarism

• Enterprise or Educational Institute are paid versions

Contentdetector.ai

This tool detects most AI written content using algorithms and technology and provides a percentage score of how artificial the content is.

Pros

• Has a word count tool

Cons

• May not completely detect AI-generated text

Crossplag

Besart Kunushevci built Crossplag for checking cross-lingual plagiarism.

Pros

• Check documents in Word, PDF, or Rich Text

Cons

• Free version only checks up to 1,000 words

After copying and pasting text from my AI-generated essay:

GPTZero

GPTZero said that it was most likely written by a human. However, when I uploaded my copied essay as a Word document, GPTZero told me that parts may have been written by an AI.

Copyleaks

Copyleaks said that AI content had been detected. Using the document uploader took a little longer, but the results were more detailed. It gave percentages, highlighted questionable material, and listed results to the side. The results were also downloadable.

Contentdetector.ai

Contentdetector.ai said that my essay was most likely written by a human. It did give a probability of AI content rating of 42.5%. It also highlighted questionable text within the copied text.

Crossplag

Crossplagsaid the text I entered was mostly written by AI and gave 100% rating for plagiarism.

These 4 ChatGPT detectors might not be a reliable source for detecting AI-generated materials.

Why not?

ChatGPT detectors were created to flag such things as repetitive language, probability of word choice, and tone. If a piece of writing is very well written, these detectors may flag it as AI-generated and vice versa. Partly written AI-generated text may also go unnoticed.