Few numbers in a classroom carry as much weight today as the percentage an AI detector prints next to a student’s essay. A high score can trigger an academic integrity case; a low one can clear a paper that was, in fact, machine-made. So a fair question is whether that number actually measures what people think it measures. A recent study by Atamhenwan (2026) takes this question and, importantly, turns it into something testable. Instead of arguing about detection in the abstract, the author builds essays with a known recipe of human and machine words, then asks how closely Turnitin’s reported score tracks that recipe. The design is the study’s main virtue, and it is worth reading both for what it establishes and for where its analysis could go further. This review is written to highlight both.
Findings Worth Bringing to the Foreground
The first contribution is conceptual. Atamhenwan (2026) reframes detector accuracy as a dose-response relationship: a four-thousand-word human text is mixed with LLM-generated words in steps from five to one hundred percent, across four tools (ChatGPT, Copilot, Gemini, and Grammarly), producing eighty-one scripts with a ground truth that is known by construction. Because the true proportion of machine text is fixed in advance, the gap between truth and Turnitin’s output becomes visible and measurable. This is the right way to make the problem concrete, and it is what allows every later claim to be checked against a baseline rather than an opinion.
On the reassuring side, the study finds no evidence that Turnitin invents AI where there is none. A fully human script returned a zero AI score, and scripts that were ninety to ninety-five percent human were not flagged with a misleading number; below roughly fifteen percent machine content, Turnitin simply declined to report a score at all. For anyone worried about false accusations against honest writers, this threshold behavior is meaningful: very small machine contributions, or none, do not produce an AI flag.
The most practically useful finding is also the most uncomfortable one, and it is a finding about bias rather than about raw accuracy. The detector is systematically miscalibrated, and it miscalibrates in opposite directions at the two ends of the scale. When the true machine proportion is low (about fifteen to forty percent), Turnitin tends to report a score that is too high. When the true proportion is high (roughly seventy to one hundred percent), it reports a score that is too low. In the author’s words, the lower the actual machine content, the more inaccurately high the score; the higher the actual content, the more inaccurately low the score (Atamhenwan, 2026). The number, in other words, is directional information, not a precise measurement.
The study also shows that detection is not uniform across tools. ChatGPT-generated text was the most likely to be under-detected, read as human in far more scripts than it was over-flagged. Copilot and Grammarly behaved similarly but less severely, while Gemini was the most balanced, under- and over-detected in equal measure. The general lesson is that Turnitin, when it errs, tends to undercount machine writing rather than overcount it.
Finally, the study confirms a vulnerability that institutions cannot ignore: paraphrasing or “humanizing” tools can erase the signal. When fully machine-written text was passed through certain rewriting tools, Turnitin’s score fell sharply, in several cases all the way to zero, meaning the text was accepted as human. This single result reframes the entire enforcement question. As the author puts it, once a score appears, the issue is not whether a model was used but how and why (Atamhenwan, 2026). The honest practical takeaway for educators is to treat high scores (around sixty percent and above) as worth a serious conversation, low scores (around forty percent and below) as weak evidence, and any score as one input among several rather than a verdict.
What Needs to Be Analyzed More Carefully
The findings above are sound as description. The weaker part of the paper is the statistical machinery used to support them, and this is where a reader, or a future author building on this work, should slow down.
The core issue is that the study reaches for tools that confirm a relationship it has already built in, rather than tools that quantify the error it actually discovered. Because the design fixes a rising true proportion of machine text, a strong positive correlation with the detector’s score is almost guaranteed; the reported coefficients confirm the design more than they test a hypothesis. The interesting question is not whether truth and score move together, but by how much and in which direction they disagree. That is a measurement-agreement question, and it has well-established tools. A Bland-Altman style analysis, familiar from clinical measurement research, plots the difference between the true value and the reported value against the average of the two, and reveals at a glance whether the error grows or shrinks across the range. Given that the study’s headline finding is exactly such a range-dependent bias, this kind of agreement analysis, or a method-comparison regression such as Passing-Bablok, would have measured the phenomenon directly instead of describing it after the fact.
The regression models deserve particular caution, because they do not say what the paper claims they say. The reported pseudo R-square values of 1.000 are presented as a triumph, but a model that explains one hundred percent of the variance is a warning sign, not a success. The paper’s own appendices confirm this: the notes under the ordinal regression tables state that the data show complete separation and that the maximum likelihood estimates do not exist. A model whose estimates do not exist cannot support a conclusion. The choice of ordinal logistic regression is itself part of the problem, since the outcome is a bounded percentage from zero to one hundred and treating it as a many-category ordinal variable invites exactly this breakdown. A bounded-proportion approach, such as beta or fractional regression, would have been the appropriate family and would not have collapsed.
A third point concerns independence. The analysis treats the scripts as independent observations, but they are not: every group of four shares the same underlying human essay and the same true percentage, with only the LLM source differing. Reusing the same base text means the observations are clustered, and clustered data analyzed as if independent overstate the available information. A multilevel or mixed model, with the language model entered as a grouping factor, would respect this structure.
Two smaller but real limitations round out the picture. The detector was run once per script, so there is no measure of how stable Turnitin’s own output is; a repeated-submission check would tell readers how much of the observed variation is noise. And the section on humanizing tools rests on only sixteen scripts, a single essay per tool, which makes statements like “this tool bypasses detection completely” closer to a vivid anecdote than a finding. A larger, replicated test would let those claims carry real weight. It is also worth verifying, in future work, that the human base texts are genuinely pre-AI, and ideally testing mixed authorship as students actually produce it, interleaved and edited, rather than appended in clean blocks.
A Fair Verdict
The value of this study is that it asks the right question in a way that can be measured, and it surfaces a pattern that matters for every institution using these scores: detectors are biased estimators, generous at the low end and stingy at the high end, and easily defeated by rewriting tools. Those messages stand. What the work now needs is sharper instruments to match its sharp question, agreement statistics rather than confirmatory correlations, a model family suited to a bounded outcome, and a design that respects its own clustered structure. Read that way, the paper is less a closed answer than a well-posed starting point, and a useful caution against treating any single percentage as proof.
Reference. Atamhenwan, L. E. (2026). How are combinations of human-written words and LLM-generated words by ChatGPT, Copilot, Gemini and Grammarly detected by Turnitin? Education and Information Technologies. https://doi.org/10.1007/s10639-026-14049-2