Long-Term Memory Token Matrix
Plan your model's context-window budget across system prompt, conversation turns, memory/RAG content, and reserved output.
Token Budget Matrix
| Component | Tokens | % of Context |
|---|
Token counts are estimates. Real tokenizer output varies by model, language, and formatting, so treat every number here as a planning baseline rather than an exact measurement.
Professional Insights & Guide
Learn how the Token Matrix budgets a context window, which formulas drive utilization, and how to read the warnings.
Core Use Case scenario
Agent builders must keep every prompt within the model's context window. The matrix sums four budgets: system prompt tokens, conversation tokens (average user tokens plus average assistant tokens times the number of turns), memory/RAG content tokens, and reserved output tokens. It then reports the total, the utilization percentage, the remaining headroom, and how much memory the window can still comfortably absorb, flagging compaction before the model quietly truncates or fails.
How the Budget Is Computed
- Conversation tokens = turns × (avg user tokens + avg assistant tokens).
- Total used = system + conversation + memory/RAG + reserved output.
- Utilization % = total used ÷ max context × 100; remaining budget = max context − total used.
- Recommended memory = 70% of max context − (system + conversation + reserved), clamped at zero, so memory never crowds out working headroom.
Troubleshooting & Edge-Case Failure Points
- Context overflow: utilization above 100% means the request cannot fit; the matrix shows how many tokens exceed the window.
- Zero-turn conversations: with no turns the conversation component vanishes, so the matrix still works for one-shot prompts.
- Memory dominance: when memory/RAG is more than half the total, retrieval is crowding out the conversation and output budget.
- Tokenizer variance: characters are not tokens, so counts from the matrix are planning baselines, not exact billing figures.
- Custom models: choosing Custom reveals a manual context length so non-listed models can be planned too.
Detailed Step-by-Step Instructions
- Pick your model from the dropdown, or choose Custom and enter a manual context length.
- Enter the system prompt tokens, the number of conversation turns, and the average user and assistant tokens per turn.
- Enter your memory/RAG content tokens and the output tokens you want to reserve for the model's reply.
- Press Compute Budget and read the utilization bar, summary cards, and component breakdown.
- Review the recommended memory allocation and compaction warnings to tune retrieval size before deployment.
Related Web Utilities (Silo Hub)
Informative Guides & Helper Articles
How to Use the LLM Context Window Budget Planner
Plans token budgets across system prompt, conversation, memory/RAG, and reserved output - so long-context agents fail predictably.
- Enter your model context window size.
- Allocate sections: system prompt, turns, retrieved memory, output reserve.
- Watch utilization warnings; the failure mode is silent truncation.
Tokens, the Budget Unit of LLMs
A token is ~4 characters or 3/4 of an English word. The arithmetic: a 128k window minus a 2k system prompt, 30k conversation, 20k retrieved docs, and 4k output reserve hits 44% before the second user question. The killer is silent truncation: most stacks drop oldest or middle turns without erroring - the agent forgets early instructions and nobody sees why. Discipline: system prompt under ~2k (it is rent, paid every call), memory is targeted retrieval not dump-everything, always reserve output tokens.
LLM Context Window Budget Planner FAQ
How many words is a token?
About 0.75 English words or ~4 characters. 1,000 tokens is roughly 750 words.
What happens when context overflows?
Usually silent truncation of oldest/middle turns. Budget explicitly and log utilization.
How big should a system prompt be?
Under ~2,000 tokens. Long system prompts cost every call and dilute instruction-following.