AI and Plagiarism Detection Software: A Comparative Guide

Introductory Remarks

With the increasing ubiquity of text-based generative artificial intelligence (GAI) software, powered by Large Language Models (LLMs), concerns have been raised about students’ using these tools dishonestly, to either generate text anew or revise existing text in contexts where a faculty member or instructor has prohibited this. Several software tools now on the market purport to address this problem by scanning text and identifying what percentage of that text was generated by AI. These tools vary in their pricing model (free, “freemium,” or paid only), as well as in the features they offer. Some simply check for AI-generated text, while others also scan against scholarly databases for possible plagiarism. The most complex tools also offer varying degrees of “text optimization” to improve the quality of a user’s submission or (less ethically) to render it undetectable as AI-generated.

In testing tools currently on the market against both AI-generated and human-composed text, ATS obtained results suggesting that no tool is infallible. While some performed better and more consistently than others, each had one or more vulnerabilities that could pose problems when using it to detect possible student misconduct–either by failing to detect AI-generated text with certainty, or by falsely flagging human-composed text as AI-generated. This is a good reason not to rely upon results generated by AI detection software as the sole data point in determining whether a student has committed academic misconduct; if used, detection software should be only one tool within a comprehensive examination of the evidence.

In addition, this field is far from static. While AI-detection tools are evolving, so too are text generators themselves. Thus, even if an AI-detection tool performs well now, its performance may deteriorate as text generators improve.

For all of the above reasons, IT Services does not offer a centrally supported AI-detection tool at this time. If you wish to use one of these tools, we recommend caution before proceeding, taking its affordances and vulnerabilities into account before proceeding with its use.

 

Summary of Conclusions

Each tool was tested against unedited text generated by four LLMs – ChatGPT, Microsoft Copilot, claude.ai, and PhoenixAI – as well as human-composed text. (For another study of AI detection software, see ZDNet’s ongoing examination of AI detectors, updated on April 30, 2025.)

 

For detailed results, please see: AI and Plagiarism Checkers Guide 4-18-2025 – Google Sheets and the AI software testing samples used.

 

  • GPTZero is the most consistent of the AI detectors examined. It detected all AI-generated text save that from Copilot with 100% accuracy and detected human-written text with 99% accuracy. However, it was only 63% confident when analyzing Copilot-generated text. The tool operates on a freemium model with tiered pricing plans; the free plan includes 10,000 words per month.
  • ZeroGPT is a deeply problematic tool. While it detected text generated by ChatGPT as AI-generated, it also flagged an entirely human-written paragraph as 100% AI. Its business model is also dubious: the site relies heavily on suspicious Google ads and offers to make AI text “human” so it cannot be detected by ZeroGPT’s competitors.
  • GPT2 Output Detector simply does not do what it claims to do. It failed completely to detect text generated by Copilot and PhoenixAI, and flagged text generated by claude.ai as only 5% AI-generated.
  • Originality.ai performed best of the tools examined at detecting AI-generated text. In the case of all four LLMs studied, originality.ai flagged the resulting text as AI-generated with 100% certainty. However, it also flagged human-composed text as AI-generated with 97% certainty, raising the specter of false positives that could lead to mistaken accusations of misconduct.
  • Grammarly is erratic. It detected all AI-generated submissions as AI-generated with greater than 50% certainty, save ChatGPT, but it also flagged completely human-written text as 25% AI-generated.

 

What Should I Do if I Suspect Academic Misconduct?

If you suspect academic misconduct by an undergraduate student, such as unauthorized GAI use or plagiarism, the Office of College Community Standards (OCCS) can work with you to figure out how to proceed. If you suspect misconduct by a graduate student, you should contact your area Dean of Students.

For a detailed guide to the academic integrity process, and to the role you as a faculty member or instructor can play in maintaining academic integrity, please see the page on Academic Integrity within the Canvas site Teaching in the Generative AI Landscape.