Why does a short-looking conversation exceed the context budget?
#A chat application rejects a request after many tool calls, even though the last user message is tiny.
Show answer
Inspect tokenized system instructions, conversation history, tool definitions and tool results. Measure with the model-compatible tokenizer or provider usage. Reserve room for output and prune irrelevant context deliberately. Summaries should retain decisions and source links; test what information they lose.
Trade-offs
Summaries reduce context size but can omit constraints; selective retrieval adds a separate relevance decision.
Common mistake
Equating words, characters and tokens, or forgetting tool output.
Follow-up
Which information would you preserve before compressing an incident conversation?
- Instructions + history + tools
- Tokenization
- Input budget + output reserve