How AI writing actually works — a clear-headed explanation

The one-sentence version

A large language model is a statistical engine trained on enormous amounts of text to predict, given everything written so far, which word is most likely to come next. Everything else — the seeming knowledge, the conversational ability, the rewrites, the summaries — is downstream of that single skill.

What 'predicting the next word' actually looks like

When you send a prompt, the model breaks your text into tokens (small units that are usually word fragments). It then computes, for every possible next token in its vocabulary, a probability score. The highest-scoring tokens are sampled — often with some controlled randomness — and the chosen token is appended to the running text. Then the whole process repeats, token by token, until the model produces a stop signal or hits a length limit. This is the entire generation loop. There is no plan being drawn up in advance, no outline being checked off, no awareness in the human sense. There is a sequence of probability distributions, sampled one at a time.

Why this produces something that feels like writing

Predicting the next word well, at scale, requires the model to have internalised an extraordinary amount of structure: grammar, common sense about the physical world, how arguments are typically constructed, how a polite request is phrased, how a recipe is formatted, how a Shakespeare sonnet sounds. The training data — a substantial fraction of the public internet, books, code, and licensed corpora — contains examples of all these things, and the model's weights encode statistical patterns across them. The result is that producing the most likely next word, conditioned on a prompt asking for a refund-policy email in a warm tone, tends to produce something that reads like a refund-policy email in a warm tone.

Why this also produces confident wrongness

Two failure modes follow directly from the mechanism. The first is hallucination — the model produces a fact that sounds right (a citation, a statistic, a person's title) because the most likely tokens, conditional on the prompt, form a confident assertion, regardless of whether the underlying claim is true. The model has no separate fact-checking process; it has the same generative loop applied to every claim. The second is bias toward the average. Because the model is sampling from a distribution over likely tokens, it tends toward statistically typical phrasing. That is why first drafts often sound generic — they are, in a precise statistical sense, the average of how the topic is discussed in the training data.

What changes when the model is bigger or newer

Larger models with more parameters compress more pattern into a more flexible representation, which makes them better at instructions, more consistent across long passages, and more capable on multi-step reasoning. Reinforcement learning from human feedback — a training step layered on top of pre-training — shapes the model to follow instructions politely, refuse certain requests, and produce more useful structure. The combination is why a 2025 model handles complex prompts well even though the underlying mechanism is the same next-token prediction that powered earlier versions. The base capability has not changed in kind; it has matured in degree.

What this means for prompts

If output quality depends heavily on what the model has been conditioned on, then prompts are not just instructions — they are conditioning. Naming the audience, the constraints, the format, and (when useful) showing one or two worked examples narrows the probability distribution toward the kind of output you actually want. Vague prompts produce average output not because the model is lazy but because 'average' is what the distribution centres on absent specific guidance. The most productive shift for a new user is to stop writing prompts the way you write search queries and start writing them the way you brief a competent collaborator.

What this means for editing

Because the model produces the statistically likely sentence, AI drafts share recognisable patterns: hedged transitions, uniform sentence length, generic noun-adjective pairs (innovative solutions, dynamic environment), and a preference for the comfortable middle distance over a sharp claim. These are not bugs to be hidden; they are the natural shape of average. The job of editing is to add the specific — the concrete example, the real number, the verb that does work, the sentence that lands shorter than the others. Done well, the result is writing that no longer reads as average because it is no longer average.

The honest answer to 'is this thinking?'

There is genuine disagreement among researchers about whether large language models perform a form of reasoning or whether they perform sophisticated pattern completion that looks like reasoning under certain conditions. For practical writing work, the useful position is operational. Treat outputs as the work of a fast, well-read collaborator with an unreliable memory and no skin in the game. Verify the facts. Add the judgement. Take the credit, and the responsibility, that come with putting your name on it.

Frequently asked