Learn AI Create the Future — with Pranay
Main site

AI in Education

Why AI-text detectors do not work

AI-text detectors guess whether writing came from a language model by how predictable the word choices are, and that same test flags plain, simple human writing too, which is why they have a documented history of falsely accusing real students, especially non-native English speakers.

Read these first

On this page 6
  1. Why this matters
  2. How it works
  3. Where you have already seen it
  4. An honest warning
  5. Remember this
  6. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

AI-text detectors guess whether a piece of writing came from a language model. That guess is far less reliable than it is usually presented to be.

Think about a metal detector at a mall entrance beeping because of your belt buckle, not a weapon. It is a false alarm, and a real inconvenience for an innocent person. AI-text detectors produce the same kind of false alarm, on real student writing, regularly enough that this is a well-documented problem, not a rare glitch.

Why this matters

A detector flagging an essay as "likely AI-written" can trigger an academic integrity accusation. For a student, that can mean a failing grade, a disciplinary record, or worse — over a tool that is measurably unreliable.

Detectors mostly work by measuring how predictable a piece of writing is. A language model tends to choose common, expected words. Text using simpler, more common vocabulary tends to score as "AI-like" for the same reason. That same test also flags plain, plainly written human text. This includes writing by students who are not native English speakers. They tend to write in more straightforward patterns, for reasons that have nothing to do with AI.

How it works

Text sample   ->   How predictable are        ->   "likely AI"
                    the word choices?               or "likely human"
                    (common words = more
                     predictable)

A published Stanford study tested detectors on real TOEFL essays written by non-native English speakers. A large share were wrongly flagged as AI-generated. The same detectors were fooled almost completely by AI-generated essays that had only been lightly edited afterward.

Where you have already seen it

  • Turnitin's AI-detection feature, used by many schools and universities, and criticised in exactly this way by researchers and educators.
  • News stories of students wrongly accused of using AI, based on a detector's score alone, in some cases with serious academic consequences.
  • OpenAI's own AI-text classifier. The company shut it down in 2023, citing low accuracy. That is a notable admission, coming from the very company that trained the models the tool was meant to detect.

An honest warning

No publicly available AI-text detector is reliable enough to be the sole evidence in an academic integrity decision. This is not a controversial claim. It is close to a consensus position among researchers who have studied these tools directly. That is why several major universities have stopped using detector scores as grounds for accusation on their own.

If a detector is used at all, it should be one weak signal among several, reviewed by a human. Draft history and a conversation with the student matter more. A detector score should never be treated as proof by itself.

Remember this

  • AI-text detectors largely measure how predictable a text's word choices are, not whether an AI actually wrote it.
  • Simple, plain human writing — especially by non-native English speakers — gets flagged as AI at troubling rates.
  • A detector score alone should never be the basis for an academic integrity accusation.

What to learn next

Developer — Code and libraries.

Setup

No installation needed — this is plain Python string processing.

Minimal runnable code

This builds a crude version of the core idea behind real detectors — measuring how predictable a text's word choices are — to show directly why it misfires on simple human writing.

predictability_demo.py
import re

# A small set of the most common English words. Real detectors use a full
# language model's probability estimates; this crude proxy captures the same
# underlying idea -- how "predictable" the word choices in a text are.
COMMON_WORDS = {
    "the", "a", "an", "is", "are", "was", "it", "this", "that", "i", "you",
    "we", "they", "and", "or", "but", "to", "of", "in", "on", "for", "with",
    "good", "help", "helps", "many", "people", "their", "work", "fast",
    "quickly", "easy", "do", "complete", "because", "think", "idea",
}

def predictability_score(text):
    words = re.findall(r"[a-zA-Z']+", text.lower())
    common = sum(1 for w in words if w in COMMON_WORDS)
    return common / len(words)

texts = {
    "simple_non_native": "I think this is a good idea because it help many people to do their work fast and easy.",
    "ai_generated_style": "This is a good idea because it helps many people complete their work quickly and efficiently.",
    "complex_human": "Frankly, the whole scheme reeks of last-minute improvisation dressed up as strategy.",
}

for label, text in texts.items():
    score = predictability_score(text)
    verdict = "flagged as likely AI" if score > 0.55 else "reads as likely human"
    print(f"{label:20s} predictability={score:.2f}   {verdict}")
Output
simple_non_native    predictability=1.00   flagged as likely AI
ai_generated_style   predictability=0.94   flagged as likely AI
complex_human        predictability=0.15   reads as likely human

What actually happened

simple_non_native scored higher on the predictability test than the actual ai_generated_style sentence — and both got flagged as "likely AI" under this simple rule. The one sentence correctly read as human is the one with the least common, most idiomatic vocabulary.

  • This is a deliberately simplified stand-in for what real detectors measure — commonly perplexity, roughly how "surprised" a language model is by a text's word choices, and burstiness, how much sentence complexity varies within a piece of writing. Real detectors are more sophisticated than counting common words, but the underlying signal they rely on has the same fundamental blind spot demonstrated here.
  • A genuinely human sentence that happens to use simple, common, correct English scores as predictable for the same reason an AI-generated sentence does: both avoid rare words and complex structure. The detector cannot tell why a text is predictable, only that it is.
  • Nothing in this code, or in a real detector, has access to who actually typed the words. Every one of these tools is making an indirect statistical guess, not observing authorship directly.

Common mistakes

Treating a high detector score as proof. As shown above, "predictable" and "AI-written" are correlated, not identical. A predictable score is weak evidence at best, not a verdict.

Ignoring that detectors can be evaded easily. Lightly paraphrasing AI-generated text, or running it through a simple rewording tool, defeats most detectors, while doing nothing to make genuinely simple human writing pass. This asymmetry — genuine writers wrongly flagged, actual misuse easily evading detection — is the central, documented problem with these tools.

Applying one threshold to every writer. A fixed "predictability above X counts as AI" rule, as used crudely here, has no way to account for the fact that different writers, especially across language backgrounds and grade levels, naturally produce very different baseline predictability.

Try it yourself

Add a fourth text: a short, plainly written sentence from a real younger student or early-language-learner writing sample you have access to (with any identifying details removed). Score it with this same function, and see whether it also gets flagged. This small experiment reproduces, in miniature, exactly the pattern the published research on this topic found at scale.

What to learn next

Researcher — Mathematics and papers.

What detectors actually measure

Two statistics dominate practical AI-text detection:

Perplexity:  PP(W) = exp( -(1/N) * sum_{i=1}^{N} log P(w_i | w_1, ..., w_{i-1}) )
  • W — the word sequence being scored, of length N.
  • P(w_i | w_1, ..., w_{i-1}) — the probability a reference language model assigns to word w_i given everything before it.
  • Lower perplexity means the text is closer to what the reference model would have generated itself — the intuition behind flagging low-perplexity text as machine-generated.

Burstiness supplements this by measuring variance in per-sentence perplexity or syntactic complexity across a document — human writing tends to vary more sentence-to-sentence than typical LLM output, which (absent post-hoc editing) tends toward more uniform structure. GPTZero and similar tools combine both signals.

Watermarking-based detection (Kirchenbauer et al., 2023) is a distinct approach requiring cooperation from the generating model: a statistical bias is deliberately embedded in the model's token sampling at generation time, detectable later via a hypothesis test, but only for text from a model that opted to embed the watermark — it provides no signal at all for text from a model that did not.

The Stanford/Liang et al. finding

Liang et al. (2023), GPT Detectors Are Biased Against Non-Native English Writers, evaluated multiple commercial and open-source AI-text detectors against a corpus of real TOEFL essays written by non-native English speakers, alongside US 8th-grade student essays and GPT-generated text. The study reported that a majority of the non-native-authored TOEFL essays were misclassified as AI-generated by several detectors tested, while the same detectors were near the study's chance-level baseline at catching GPT-generated text that had undergone light prompt-based paraphrasing. This is a peer-reviewed, published empirical result specific to the detectors and essay corpus tested, not an anecdote, and it is the primary citation behind the fairness concerns raised throughout this lesson.

Adversarial robustness

Sadasivan et al. (2023), Can AI-Generated Text Be Reliably Detected?, provides a theoretical argument that as the total-variation distance between human and machine text distributions shrinks (i.e., as generation quality improves), the achievable detection accuracy of any detector is fundamentally bounded, independent of detector sophistication — a formal argument for why the detection problem gets strictly harder, not only practically harder, as generative models improve. Krishna et al. (2023) demonstrated this practically, showing that a purpose-built paraphrasing model (DIPPER) defeats several published detectors, including watermarking-based ones under some conditions, with only mild degradation of the paraphrased text's fluency.

OpenAI's own retirement of its classifier

OpenAI publicly retired its AI-text classifier in July 2023, citing a low rate of accuracy, in its own published announcement — a directly relevant, verifiable data point given the retiring party had privileged access to its own models' output characteristics and still assessed the tool as unreliable enough to withdraw.

Papers

  • Liang, W. et al. (2023). GPT Detectors Are Biased Against Non-Native English Writers. Patterns (Cell Press) — arxiv.org/abs/2304.02819
  • Sadasivan, V. S. et al. (2023). Can AI-Generated Text Be Reliably Detected? arxiv.org/abs/2303.11156
  • Krishna, K. et al. (2023). Paraphrasing Evades Detectors of AI-Generated Text, but Retrieval is an Effective Defense. NeurIPS — arxiv.org/abs/2303.13408
  • Kirchenbauer, J. et al. (2023). A Watermark for Large Language Models. ICML — arxiv.org/abs/2301.10226
  • Mitchell, E. et al. (2023). DetectGPT: Zero-Shot Machine-Generated Text Detection Using Probability Curvature. ICML.

Current state

No AI-text detector as of the mid-2020s achieves both high recall against adversarially paraphrased text and low false-positive rates on genuine human writing simultaneously, and the theoretical argument in Sadasivan et al. suggests this gap will not close through scaling up detector sophistication alone, as generative models keep improving. Retrieval-based defenses (comparing suspect text against a database of known model outputs, as in Krishna et al.) show more promise than purely statistical detection but require the generating platform's cooperation in logging outputs, which is not universally available. The responsible institutional position, reflected in guidance from multiple academic-integrity offices, is to treat detector output as inadmissible as sole evidence and to rely on process-based evidence (draft history, version control, oral defense of written work) instead.

What to learn next