AI

Context Window

Context window: the maximum amount of text, counted in tokens, that a language model can take into account at once, prompt and answer included.

Long cadre rectangulaire en métal brossé, posé à l’horizontale sur un sol sombre réfléchissant, traversé par un ruban ondulant de petites tuiles de verre translucides, blanches au centre et teintées de bordeaux aux extrémités, sous un faisceau de lumière venant du haut
Fenêtre de contexte — Image générée par IA (Qwen-Image 2.1, ComfyUI)

The context window is the maximum amount of text a language model can take into account at once. It is counted in tokens, which are fragments of words, and it covers everything: the instructions, the conversation history, the documents provided and the answer being written. It's the first limit to check when an assistant or an agent starts giving off-target answers.

Technical definition

Working memory, not storage

A language model remembers nothing between two requests. With each exchange, the application sends it the entire conversation again. The context window sets the maximum size of that payload. Anything that doesn't fit simply doesn't exist for the model.

What fills the window

In a simple chat, the window fills up slowly. With an agent, it fills up fast: system instructions, tool descriptions, the contents of files read and the result of every action all pile up. The model's answer, including its internal reasoning when it produces any, also has to fit.

A setting, not just a spec

Every model has a maximum size, fixed at training time. But when you self-host a model, the window actually used is a launch parameter, often set well below that maximum by default. A model able to handle 262,144 tokens can end up running with 4,000 if nobody has touched the setting.

Current challenges

Silent failures

A saturated window doesn't always produce an error. Answers degrade, get cut off, or ignore an instruction given earlier. The symptom looks like a mediocre model, when the cause is a setting. Server logs are often the only place where the truncation shows up.

The cost of size

Enlarging the window consumes GPU memory and slows down processing. On a hosted, pay-per-use model, that shows up on the bill, since every token sent is counted. On a self-hosted model, it shows up in hardware.

Summarize rather than stack

Recent tools compact the conversation when it nears the limit: they replace the history with a summary. That only works if the tool knows the size of the window. With a self-hosted model, it's up to the user to declare it.

Frequently Asked Questions

What happens when the context window is full?

It depends on the tool. Some reject the request with a clear error. Others cut the beginning or the middle of the conversation to make room, without warning: the model then answers having lost part of its instructions. Either way, the space left for the answer shrinks, and the answer may come back truncated, or even empty.

Why does an AI agent use up so much context?

Before the first question is even asked, an agent sends the model its instructions, the description of each of its tools and sometimes the list of its skills. Every file it reads and every tool result then gets added on top. In a test documented on this site (in French), a coding agent took up nearly 25,000 tokens at startup: a 32,000-token window was saturated within a few exchanges.

Is a bigger context window always better?

No. A large window requires more memory and lengthens the processing time of every request. Models also follow very long content less reliably than short, focused content. Good practice is to size the window to the use case, and let the tool summarize the conversation when it approaches the limit.

What is the difference between a context window and a model's memory?

The context window is working memory: it only holds what is sent in the current request. The model retains nothing from one conversation to the next. What an assistant calls "memory" is a mechanism added around the model that injects notes or documents back into that window.

Resources and Documentation