After this topic
You will have a function you can draw: keep the system prompt, keep the last N turns, drop the oldest. You will know that what you drop, the model cannot remember — and that a token budget is the same idea with a meter.
Keep what still fits
Start with a sliding window. It is one function. You can draw it. Add a token meter when N is no longer enough.
Start with a sliding window. Keep system, keep the last N user/assistant turns, drop the oldest. Users notice only when a fact they stated at the start is gone — then you add a summary of what you dropped, instead of deleting it blindly.
Token budgets are the same idea with a meter: drop or compress until you are under a number you chose (well below the model’s max, because the reply needs room too).
Do not slice the whole array. If you do, you drop the system prompt — the rules. Filter system out, slice the rest, put system back in front.
- Sliding window — keep last N messages
- Token budget — drop oldest when over a threshold you chose
- Summarization — compress old thread before dropping (next topic)
- Hybrid — summarize + keep recent verbatim
Files
- 01
Keep the system prompt
If you slice the whole array, you drop the rules. Filter system out, slice the rest, put system back in front.
- 02
KEEP is a knob
20 turns is a starting point. Measure tokens if replies get expensive or the model starts ignoring early facts.
What you drop is gone
A sliding window is honest and lossy. The ticket title from hour one will vanish. That is fine for casual chat. It is not fine for a support session that still needs the order id.
When lossiness hurts, do not raise KEEP until the window is huge. Summarize what you would have dropped. That is the next topic.
Common questions
- What is “Sliding and token-based strategies”?
- Drop oldest messages or trim by token count before each model call.
- What will I be able to do after this lesson?
- You will have a function you can draw: keep the system prompt, keep the last N turns, drop the oldest. You will know that what you drop, the model cannot remember — and that a token budget is the same idea with a meter.
- How long does this lesson take?
- About 14 minutes of reading. It is a free chapter in the AI SDK Patterns TypeScript course.
- When should I not use this?
- A sliding window is honest and lossy. The ticket title from hour one will vanish. That is fine for casual chat. It is not fine for a support session that still needs the order id.
Written by Akash Panchal·Updated August 29, 2026