The fundamental limits of AI-text detection
A paper that reframes AI-text detection as a statistical inference problem, proving that perfect detection requires non-overlapping human and AI writing distributions — a condition that ordinary prose cannot satisfy.
assumes you know
Basic probability (conditional probability, Bayes' theorem) · Familiarity with hypothesis testing (sensitivity, specificity, false positive rate) · Understanding of probability distributions over discrete spaces · Concept of total variation distance (or willingness to learn it)
the line
What you could say out loud at dinner.
No matter how clever the classifier, a finished text can reveal its origin with certainty only if every possible human sentence is impossible for AI and vice versa — a condition that ordinary writing never meets.
the mechanism
How it actually works. Select a passage to dive deeper.
The paper reframes AI-text detection as a problem of inverse inference: from a fixed string of words, infer the unobserved process that produced it. This is fundamentally different from Turing's imitation game, where an interrogator can ask adaptive questions to probe the respondent. A detector sees only the final transcript; it cannot request new evidence. The author reduces the problem to its simplest form: two known probability distributions over all possible texts, one for human writing (P_H) and one for AI writing (P_A), with equal prior probabilities. The optimal classifier chooses the source with higher likelihood for each observed text. Even this ideal detector makes errors wherever the two distributions overlap — that is, wherever some text has positive probability under both sources. The minimum average error is exactly half the probability overlap, which equals (1 - TV)/2 where TV is the total variation distance between the distributions. Perfect detection (zero error) requires TV = 1, meaning the两
the math
Worked from the paper's own notation.
Let X be the space of all finite texts. Two distributions P_H and P_A assign probabilities to each x in X. With equal priors, the optimal decision rule chooses human if P_H(x) > P_A(x), AI otherwise. The contribution of text x to average error is (1/2) min{P_H(x), P_A(x)}. Summing over X gives minimum average error R_best = (1/2) Σ_x min{P_H(x), P_A(x)}. Using the identity min{a,b} = (1/2)(a + b - |a-b|) and that each distribution sums to 1, the overlap Σ_x min{P_H(x), P_A(x)} = 1 - TV(P_H, P_A), where TV(P_H, P_A) = (1/2) Σ_x |P_H(x) - P_A(x)| is the total variation distance. Hence R_best = (1/2)[1 - TV(P_H, P_A)]. Zero error requires TV = 1, which in this discrete setting means disjoint supports: P_H(x) > 0 implies P_A(x) = 0 and vice versa. For posterior probability: Pr(A|x) = π_A P_A(x) / [π_A P_A(x) + π_H P_H(x)]. Certainty (Pr(A|x) = 1) requires P_H(x) = 0 and P_A(x) > 0. For a binary detector with sensitivity Pr(+|A) and specificity Pr(-|H), the positive predictive value is Pr(A|+) = π_A Pr(+|A) / [π_Awhat it overturns
What to stop believing, and what would make this wrong.
The paper overturns several widespread assumptions. First, that better algorithms can eventually achieve perfect detection — the total variation bound shows the limit belongs to the statistical experiment, not the classifier's ingenuity. Second, that a detector's percentage score (e.g., '98% AI-generated') is a calibrated posterior probability — Bayes' theorem shows it depends critically on the base rate of AI-generated text in the population, which varies wildly across contexts. Third, that 'AI-written' is a well-defined binary property — the paper demonstrates multiple plausible definitions (any AI words remain, most words from AI, first draft from AI, ideas from AI, dependence on AI) that need not agree, and shows how a binary label compresses a complex production history into a single bit. Fourth, that passive inspection of finished prose can substitute for interactive verification — the adaptive interrogation framework proves that an interrogator who can choose questions based on previous answers can, in