---
title: "Why context windows matter"
summary: "Models only see a finite amount of text — memory, cost, and quality all depend on how you manage it."
track: "Context"
day: 1
minutes: 16
author: "Akash Panchal"
url: https://ai-sdk-patterns.dev/learn/context/day-1/context-window
dateModified: 2026-08-29
---

# Why context windows matter

Models only see a finite amount of text — memory, cost, and quality all depend on how you manage it.

*Context · Day 1: Context windows · ~16 min. Written by [Akash Panchal](https://github.com/akashp1712).*

Canonical: https://ai-sdk-patterns.dev/learn/context/day-1/context-window

## After this topic

You will stop saying “the agent forgot.” You will say: we overflowed the array. You will know what fills the window, why tool results eat it alive, and why a 128k model is not a free pass to dump everything.

## The window is the whole world

> A “128k context” model can see on the order of a hundred thousand tokens of prompt plus completion. Your system prompt, the whole thread, every tool result, every retrieved chunk, and the reply all share that budget.

The model only sees what you send. That list has a maximum size — measured in tokens, not messages.

Day 1: the model only sees what you send. That list has a maximum size — the context window — measured in tokens, not messages. When you go over, providers truncate, error, or silently drop the beginning. In product language that is “the agent forgot”. In engineering language you overflowed the array.

Cost tracks the same number: more tokens in, more money, even if the user only said “ok”. A long system prompt is a tax on every turn. A 40kb tool result from your database is a tax on the next model step.

Context engineering is the unglamorous work of choosing what stays: recent turns verbatim, a summary of the rest, retrieved docs, tool outputs that still matter. There is no perfect policy. There is a budget and a user who notices when the ticket title from hour one is gone.

## What actually fills the window

People think of “the chat”. In an agent the tool results are often the bulk — a JSON payload from your database can dwarf the user sentence that triggered it. RAG chunks are the same: ten documents in the prompt is a choice, not a default.

Leave room for the reply. If you fill 120k of a 128k window, the model has almost no room to write. Cap what you send well below the advertised maximum.

The next topic is the simplest policy that works: keep the system prompt, keep the last N turns, drop the rest. Then you add summarization when N is not enough.

- System prompt — every turn, whether you needed it or not
- Thread — user and assistant messages you kept
- Tool results — often the largest slice in an agent
- Retrieved docs — RAG chunks you chose to include
- The completion — the reply still has to fit

## Bigger windows are not a strategy

A larger window lets you be sloppy longer. It does not make the model better at finding the one fact in the middle of 80k tokens. Attention dilutes. Cost does not.

Prefer a small, relevant prompt over a huge, complete one. That sentence is the rest of this chapter.

## Common questions

### Why context windows matter?

Models only see a finite amount of text — memory, cost, and quality all depend on how you manage it.

### What will I be able to do after this lesson?

You will stop saying “the agent forgot.” You will say: we overflowed the array. You will know what fills the window, why tool results eat it alive, and why a 128k model is not a free pass to dump everything.

### How long does this lesson take?

About 16 minutes of reading. It is a free chapter in the AI SDK Patterns TypeScript course.

### When should I not use this?

A larger window lets you be sloppy longer. It does not make the model better at finding the one fact in the middle of 80k tokens. Attention dilutes. Cost does not.