Skip to content
Back/Writing

AI Writing Distribution Problem

View in Graph
Updated 2026-08-26
3 min read
504 words

AI Writing Distribution Problem

What it is

The AI writing homogeneity problem is a distribution problem, not a single-output problem. A model can produce one thousand individually acceptable replies that still collapse into the same lexical, rhythmic, and structural habits. The issue is not whether any given paragraph passes a quality check, but whether the batch of model outputs matches the batch-level distribution of human writing.

Why it matters

Content teams often evaluate AI-assisted writing one article at a time. A single draft may look fluent, yet the whole corpus can drift into recognizable patterns that readers eventually find fatiguing or untrustworthy. Treating homogeneity as a distribution mismatch shifts the evaluation unit from "this draft" to "the shape of our outputs."

Key points

  • Distribution Fine-Tuning (DFT) is one technical response. Instead of scoring each reply independently, DFT compares batches of model outputs against batches of human writing and adjusts the model when their distributions diverge. Deft's DFT v1 uses n-gram token distribution distance, Maximum Mean Discrepancy (MMD), and judge-model quality (JMQ) to measure this divergence.
  • More unpredictable does not mean better. In Katie Parrott's tests, DFT v1 produced sentences that were less clichéd than typical AI writing, but also denser, harder to parse, and weaker in information hierarchy. A wider distribution can introduce noise as well as variety.
  • "More human" can conflict with "more readable." Deft's SEO article scored at an eleventh-grade reading level, while 54% of U.S. adults read at or below a sixth-grade level. A model that optimizes for human-like variance may leave its intended audience behind.
  • Strict-mode hallucinations persist. Even with detailed source briefs and strict-mode instructions, DFT v1 invented dates, dialogues, product histories, screenshots, behaviors, and emotions. Distribution-level training does not automatically solve fact-bound writing.

Evidence across sources

Source Key Claim Relevance
Every — I Tried the AI Model Built to Fix AI Writing AI writing sameness is a batch-level distribution problem; DFT compares batches of outputs against human samples Introduces the distribution-framing and Deft's implementation

Open questions

  • Can distribution-level metrics be applied usefully to Chinese or bilingual writing corpora?
  • Does optimizing for distribution variance trade off against factual control and argument structure?
  • When should a writing workflow prefer smaller, fine-tuned models over frontier models for voice consistency?

Prompts for witness

  • If Jean sampled twenty of her own AI-assisted drafts, what repeated phrases, transitions, or rhythms would show up?
  • Where in Jean's current workflow does evaluation happen per-draft rather than per-corpus?
  • What would a "human-enough but readable-enough" threshold look like for Jean's intended audiences?

Sources

Synthesized from 1 source
  • Every — I Tried the AI Model Built to Fix AI WritingPrimary source for this page.Whole pagehighbody

Evolution

1 event
  1. absorbed

    Derived from source material

    This page is currently synthesized from 1 source.

    From Every — I Tried the AI Model Built to Fix AI WritingTo AI Writing Distribution Problem
    Sources: raw/newsletters/Every/2026-08-26 I Tried the AI Model Built to Fix AI Writing.md

Linked from