AI-generated text detection
Artificial Intelligence
Definition. AI-generated text detection is the use of software classifiers to estimate the likelihood that a passage was produced by a large language model rather than written by a person.
Explanation
Detectors are increasingly used in education and publishing, but their evidence base is mixed. A test of 14 tools, including the commercial systems Turnitin and PlagiarismCheck, concluded that the available detectors were neither accurate nor reliable, tended to classify output as human-written, and performed worse when AI text was edited, paraphrased or machine translated (Weber-Wulff et al., 2023). Detectors can also be unfair: GPT detectors frequently misclassified writing by non-native English speakers as AI-generated (Liang et al., 2023). In a medical publishing test, an AI output detector separated ChatGPT-generated abstracts from original ones with an AUROC of 0.94, while blinded human reviewers identified 68% of generated abstracts but wrongly flagged 14% of original ones (Gao et al., 2023).
A detector score is therefore probabilistic evidence, not proof of authorship, and decisions about misconduct need corroborating evidence and due process. These issues are examined in AI-Generated Text Detection and Turnitin: An Introduction (101).
Related concepts
Large language models Algorithmic bias Peer review Artificial intelligence
Read on The Health Topics
- AI-Generated Text Detection and Turnitin: An Introduction (101)
- When “AI-Detected” Does Not Mean “AI-Written”: A Reading of a New Turnitin Study
References
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digital Medicine, 6, 75. https://doi.org/10.1038/s41746-023-00819-6
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19, 26. https://doi.org/10.1007/s40979-023-00146-z
Cite this entry: Kurutkan, M. N. (2026). AI-generated text detection. In The Health Topics glossary. https://healthtopic.org/glossary/ai-generated-text-detection/
Articles that discuss this concept
2 articles on The Health Topics use this concept. The list updates as new articles are published.