What is a context window?

Short answer: a context window is the most text a large language model can work with in a single request, counted in tokens. Everything the model reads for that request has to fit inside it: the app's instructions, your message, the earlier conversation, any files or tool results, and in most models the reply it writes. It works like short-term working memory. It is not what the model learned in training, and anything that does not fit in the window is not seen by the model for that answer.

What counts toward the context window?

More than the message you just typed. A typical request includes:

• Instructions the app or developer gives the model before your message.
• Your current message and the earlier turns of the conversation.
• Files, pasted text, images and documents you add.
• Tool definitions and tool results, such as search results or the output of a command a coding agent ran.
• The reply. In most models the answer has to fit in the same window, and reasoning models also spend tokens thinking before they answer.

This is why a coding agent can fill its window quickly. Every file it opens and every command output it reads takes space.

What is a token?

A token is the unit a model reads and writes. It can be a whole word, part of a word, a single character or a punctuation mark. For English text, OpenAI's rule of thumb is that one token is about 4 characters, or about three quarters of a word, so 100 tokens is about 75 words. Code and other languages often use more tokens for the same amount of text, and each model family splits text a little differently.

How big is a context window?

It depends on the model, and the numbers keep growing. Early large language models had windows of a few thousand tokens. Many current models handle hundreds of thousands, and some go to about a million tokens or more. Using the rough English rule of 0.75 words per token:

• 32K tokens is about 24,000 words.
• 128K tokens is about 96,000 words.
• 200K tokens is about 150,000 words.
• 1 million tokens is about 750,000 words.

Check the documentation for the exact model you use, because the limit for input and the limit for a single reply are often different.

What happens when the context window is full?

The model cannot read past the limit, so the app has to do something. Common approaches:

• Drop the oldest messages, so the model loses the start of the conversation.
• Summarize earlier turns into a shorter note and keep going. Many coding agents do this automatically, and some call it compaction.
• Pull in only the parts that matter for the current question, which is how retrieval (RAG) setups work.
• Stop with an error asking you to shorten the input.

The chat on your screen may still show every message even when the model is no longer receiving all of it. That is usually why a long chat seems to forget how it started.

Is a bigger context window always better?

Not automatically. A bigger window lets a model take in a whole codebase or a long contract at once, but long inputs cost more and take longer, and accuracy can drop as the window fills. Anthropic's documentation calls this context rot. Researchers have also found that models often use information at the start and end of a long input better than information buried in the middle, in a 2023 paper called Lost in the Middle. What you put in the window matters as much as how much room there is.

How do you work within a context window?

• Start a new chat for a new task instead of reusing one long thread.
• Paste the relevant section, not the whole document or repo.
• Ask for a short summary of decisions so far, and carry that into a fresh chat.
• Put the most important instructions at the start or the end of a long prompt.
• In coding agents, point the agent at the files that matter so it does not read everything.

Is the context window the same as memory?

No. Training data is what the model learned before release, and it does not change during your chat. The context window is the working space for one request. Memory features in some apps save notes about you between chats and add them back into the context window later, so they still have to fit inside it.

FAQ

What is a context window in simple terms? It is how much text an AI model can look at in one go, including your message, the earlier conversation, files and its own reply. It is measured in tokens.

How many words fit in a 128K context window? About 96,000 English words, using the rule of thumb that one token is about three quarters of a word. Code and other languages fit fewer words.

Does the model's answer count toward the context window? In most models, yes. The reply has to fit in the same window, so a long answer needs room too.

Why does a long chat forget what I said at the start? When the conversation outgrows the window, the app drops or summarizes older messages, so the model no longer sees them in full.

Is the context window the same as training data? No. Training data is what the model learned before release. The context window is the text it can work with for the current request.

A shirt for when it runs out

We make clothes about this exact problem. The Out of Context Tee ($29) has a small chest print that reads OUT OF CONTEXT with the last letters fading out, like a truncated reply. The Token Limit Reached Hoodie ($59) has a front print styled as a system log line. The Agent Sticker Sheet ($12) includes a context window progress bar stuck at 98% full. Product images are digital previews.