AI
AI SDK Patterns
← Course index

Summarization techniques

Compress older context instead of deleting it blindly.

Day 2: Summarization and observational memory·~16 min read

On this page

After this topic

You will know when to spend a second model call to keep a fact that a sliding window would drop — and when to extract that fact into a small store instead of hoping a paragraph of summary mentioned it.

When to summarize

Do not summarize every message. Run a cheaper model, or a dedicated step, when you cross a token threshold.

Summarization is a second model call that buys you continuity when the sliding window would drop facts.

Summarize when the thread is long but the user still needs continuity — a support ticket, a coding session, research that spans an afternoon. The summary is another model call: you spend tokens now to save tokens later, and to keep the one fact that would have fallen off the sliding window.

Summaries lie by omission. Keep recent turns verbatim so the last instruction is not a paraphrase. The hybrid is the product: a short block of “what happened earlier” plus the last N messages word-for-word.

A cheaper, faster model is enough for the summary. Save the strong model for the turn the user is looking at.

Compress, do not delete blindly. When the thread is too long, summarize the old turns into a short block. Keep recent messages verbatim so the latest instruction is not a paraphrase.

Facts, not vibes

Observational memory is a structured cousin of summarization. Instead of a paragraph that might mention the city, you extract facts (“user’s city is Berlin”, “plan is Pro”) into a small store, and inject them on later turns.

The extract step is generateText + Output.object — you already know that call. The store can be a row in Postgres or a JSON column on the conversation. The inject step is a few lines in the system prompt. None of this lives inside the model.

Use facts for durable, small, true things. Use a summary for narrative (“we tried restarting, then rolled back”). Use the sliding window for the last thing the user said. Mix them; do not pick a religion.

Extract facts, do not hope the summary kept them. A second, cheaper call pulls durable facts out of the thread — city, plan, ticket id — into a small store. You inject that store on later turns instead of replaying the whole chat.
  1. 01

    A fact is a key and a value

    Not a paragraph. city = Berlin. plan = Pro. You can show these in a sidebar. You can test them.

  2. 02

    Inject on the next turn

    Put the block in the system prompt (or a dedicated system-style message). The model only has the fact if you send it.

Trade-offs

Summarizing too often costs extra calls and can drift: the summary of a summary of a summary is how you lose the order id. Threshold, then summarize once, then keep recent verbatim.

Facts can go stale. If the user changes city, overwrite the fact. A store that never updates is a new way to “forget.”

This is the last topic in Context. Retrieval (coming next) is the same budget with documents you did not chat about — you go get them, then you decide what fits.

Common questions

What is “Summarization techniques”?
Compress older context instead of deleting it blindly.
What will I be able to do after this lesson?
You will know when to spend a second model call to keep a fact that a sliding window would drop — and when to extract that fact into a small store instead of hoping a paragraph of summary mentioned it.
How long does this lesson take?
About 16 minutes of reading. It is a free chapter in the AI SDK Patterns TypeScript course.
When should I not use this?
Summarizing too often costs extra calls and can drift: the summary of a summary of a summary is how you lose the order id. Threshold, then summarize once, then keep recent verbatim.

Written by Akash Panchal·Updated August 29, 2026