NLP Fundamentals
Context Window
The maximum amount of text — measured in tokens — that a language model can consider at once when generating its response, including both the prompt and the output.
Definition
Every language model has a fixed context window, which sets the upper bound on how much text it can attend to in a single generation. Modern production models range from a few thousand tokens to a million or more. The window covers everything the model sees: the system prompt, the conversation history, any retrieved documents, the user's current message, and the response being generated. When the conversation grows past the window, the oldest content is dropped or truncated, and the model effectively loses access to it.
Why it matters
Context window size dictates the kinds of tasks a model can perform reliably. Summarising a fifty-page document, reasoning over a long codebase, or maintaining consistency across a long conversation all require enough context to hold the relevant material in view at once. Exceeding the window does not throw an error in user-facing tools — it silently drops or summarises older content, which produces drift, repetition, or contradictions that look like model failure but are actually capacity failure. For writers, the practical effect is that long projects need to be structured. Style guides, brand voice instructions, and prior draft excerpts compete for context space alongside the current task. A clear, compact system prompt usually outperforms a sprawling one because it leaves more room for the actual work.
Examples
Conversation drift
After fifty long exchanges, the model 'forgets' an early instruction. The instruction has dropped out of the context window, not been ignored.
Long-document summarisation
A 200,000-word manuscript exceeds most context windows. Useful approaches include chapter-by-chapter summarisation with a running outline, or splitting work across multiple passes.
Token budget for prompts
A 100k-token window with a 30k-token system prompt and 20k tokens of retrieved context leaves only ~50k tokens for conversation and output — often less than it looks.
Frequently asked
Is a bigger context window always better?
Not necessarily. Large windows cost more per call, and models often weight early and late tokens more strongly than the middle — so packing a window full can degrade quality.
Do tokens and words map one-to-one?
No. English averages around 0.75 words per token. A 4,000-word document is roughly 5,300 tokens.
What happens when I exceed the context window?
Most user-facing tools silently truncate or summarise older content. The model does not warn you; the symptom is drift or forgetting.
Related terms
Tokenization
The process of breaking text into the discrete units (tokens) a language model actually processes.
System Prompt
A persistent set of instructions that frames a model's behaviour across an entire interaction or product.
Retrieval-Augmented Generation (RAG)
A pattern in which a model is given relevant context retrieved from your own sources before generating an answer.
Large Language Model (LLM)
A neural network trained on very large text corpora to predict the next token in a sequence, capable of producing fluent natural-language output.
Put the concept to work
Open the rewrite engine and apply this principle to a draft of your own.
Try a rewrite