AI Writing
Hallucination
A hallucination is a fluent, confident-sounding statement produced by a language model that is factually wrong — a fabricated citation, a misattributed quote, an invented statistic, or a plausible but false claim.
Definition
In language-model terminology, a hallucination is any output that is presented as factual but is not supported by reality or by the source material the model was given. Because models generate by predicting likely next tokens, they will produce smooth, well-formed prose even when no grounded answer exists. A hallucination is not a glitch; it is the same generation mechanism producing the same kind of output for a question the model cannot actually answer. The defining feature is fluent confidence, not random nonsense — which is precisely what makes hallucinations dangerous in practice.
Why it matters
Hallucinations are the single most consequential failure mode of AI-assisted writing. They damage credibility instantly when caught and silently when not. A fabricated case citation in a legal brief, an invented statistic in a marketing post, a misattributed quote in an article — each is the kind of error a careful human writer would never produce, and each is easy to ship by accident when the model's output reads convincingly. The practical implication is that any AI-assisted writing process that involves factual claims must include a verification pass. Not a vibe check; a deliberate step in which every name, number, citation, date, and definitive claim is checked against a source. Retrieval-augmented generation reduces but does not eliminate the problem. The writer remains responsible for everything the byline carries.
Examples
Fabricated citation
A model writes: 'According to a 2023 McKinsey study, 76% of remote workers report higher engagement.' No such study exists in that form. The number is plausible, the source is plausible, and the sentence is fluent — which is exactly the problem.
Misattributed quote
An article attributes 'The medium is the message' to Neil Postman. The phrase is Marshall McLuhan's. The model produced a confident-sounding but wrong attribution.
Plausible-but-false claim
A draft says 'The first commercial use of email predates the World Wide Web by two decades.' True in spirit, but the specific number is the model rounding to a confident shape. A fact-check would catch it.
Frequently asked
Why do models hallucinate?
Because they generate by probability, not by lookup. When no high-confidence completion is available, the model still produces a fluent-sounding one. There is no internal flag for 'I don't know.'
Does retrieval-augmented generation eliminate hallucinations?
It reduces them sharply for grounded queries but does not eliminate them. The model can still misread retrieved passages or smooth over gaps with invented detail.
How do I catch hallucinations before publishing?
Run a verification pass on every factual claim — names, numbers, dates, quotes, citations. Treat the AI draft as a knowledgeable first draft from a junior writer, not as a fact-checked source.
Related terms
Retrieval-Augmented Generation (RAG)
A pattern in which a model is given relevant context retrieved from your own sources before generating an answer.
AI Detection
The practice of estimating, using statistical signals, whether a piece of text was produced by a language model rather than a human.
Large Language Model (LLM)
A neural network trained on very large text corpora to predict the next token in a sequence, capable of producing fluent natural-language output.
AI Content Policy (search)
The set of guidelines that search engines use to evaluate AI-assisted content for ranking purposes.
Put the concept to work
Open the rewrite engine and apply this principle to a draft of your own.
Try a rewrite