What is Context Window

Definition:

A context window is the maximum amount of information an artificial intelligence model can handle in an interaction to generate a response. It is usually expressed in tokens and includes the input and generated content, according to the model’s limits.

It determines how much material the model can use at that moment, not how much knowledge it acquired during training or what information an application retains between conversations.

What takes up space in the context window

Context is not limited to the user’s latest message. A request can bring together several elements that share the available capacity:

• System and application instructions: define the behavior and conditions of the task.
• Previous conversation: messages and responses included to maintain continuity.
• Documents and other content: text, images or other supported formats provided to the model.
• Information from tools: results of searches, queries or actions performed during the work.
• Generated content: the response and, in some models, the tokens used for reasoning.

AI tokens are the units used to represent and count that information. They do not always correspond to words, so a document’s page count does not allow a precise calculation of how much context it will occupy.

Also, attaching a file to an application does not mean that the entire file is sent to the model: the system may select excerpts or extract only part of its content.

Difference between context and the response limit

The context window and maximum output length are related but distinct limits. A model may accept a long input while allowing a much shorter response.

In models that share a budget between input and output, documents, instructions and history leave less space available for generation. Checking that a file fits is therefore not enough: the required response must also be taken into account.

Context window management includes these differences in how tokens are counted. The specific limits should be checked for the model and service being used, rather than automatically carried over from one tool to another.

What happens when the limit is reached

If a request exceeds the supported capacity, the service may reject it. Some applications manage conversation growth before that point by trimming, summarizing or selecting information.

Context compaction means replacing part of the accumulated material with a shorter representation, usually a summary that preserves the instructions and data considered relevant.

This allows a task to continue, but does not necessarily preserve every detail. A qualification present in the original text may be left out of the summary.

A conversation can therefore remain visible in the interface without all its messages being fully available for every new response.

Context window and assistant memory

Context is the information available for the current response; memory is a mechanism for retaining and retrieving information between interactions.

An application can store preferences, notes or documents outside the model and retrieve them when relevant. Once included in a request, that information becomes part of its context.

Saving a conversation is not the same as retraining the model either. Stored history, memory systems and knowledge acquired during training serve different purposes.

This distinction explains why an assistant may retrieve a preference from an old conversation while not having every detail of a recent exchange available.

What a larger context window offers

Larger context windows have made it easier to work with lengthy documents, long conversations and sets of files. In transformer-based models, this development is linked to improvements in architecture, training and infrastructure.

Greater capacity allows more information to be provided without splitting it up as much, but content fitting in the window does not guarantee that the model will use it correctly. It may omit details, confuse references or struggle to relate information located far apart within a document.

Selection remains important. Including repetitive, outdated or contradictory material can make the task harder even when space is still available.

Information retrieval systems can search an external collection for relevant excerpts and include them in the request. They do not expand the model’s window: they help decide what information goes into it.