Context window
The maximum amount of text an AI model can work with at once: your instructions, attached files, the conversation so far, and the reply it is writing. It is measured in tokens, which are chunks of words. When it is full, chat tools quietly drop or summarize the oldest parts, and more room is not automatically better.
In plain English
It’s the model’s desk. Everything it’s working with has to be on the desk at the same time: your question, the files, the chat so far. When the desk is full, the oldest papers slide off. The model answers from whatever is still on the desk.
In the real world
- A long chat that “forgets” what you told it an hour ago.
- A 90-page contract pasted in, answered from the first 40 pages.
- “Try starting a new chat” fixing a problem nobody could explain.
When to use it
- Before you paste anything longer than a few pages.
- When a chat has gone on for an hour and the answers start drifting.
- When you’re deciding which model or plan to pay for. The window is part of what you’re buying.
When not to use it
Don’t treat a big window as permission to dump everything in. As the window fills, accuracy and recall drop. Anthropic calls this context rot.1 Models also do worst on information buried in the middle of a long input.3 Pick what matters and put it first. That’s context engineering.
Good, bad, ugly
- Good: big windows mean you can hand over a whole document and get real answers.
- Bad: a long window is not a long memory. New chat, blank desk.
- Ugly: the model answers confidently from whatever survived. Noticing what’s missing is your job.
Key facts
- Measured in
- Tokens. Word pieces; how many characters make a token depends on the model and language.
- Includes
- Instructions, uploaded files, chat history, and the reply itself
- When it runs out
- Chat tools drop or summarize older turns. The API refuses the request.
Common questions
What happens when I go past the context window?
In a chat app, the oldest parts of the conversation are dropped or summarized and the model answers from what is left. In the developer version, the request is refused with a "prompt is too long" error.
Is a bigger context window always better?
No. Accuracy and recall drop as the window fills, and models do worst on details buried in the middle of a long input.
How do I work with a document that is too long?
Split it. Work one section at a time, or summarize each section first and work from the summaries.
References
- Context windowsAnthropic, Claude Platform DocsView source
- Glossary: Context window; TokensAnthropic, Claude Platform DocsView source
- Lost in the Middle: How Language Models Use Long ContextsLiu et al., arXiv · 2023-07-06View source