Optimization Tips for System Prompts & Tools

Every token in the system prompt is paid for on every request. Optimizing is therefore not cosmetic — it is a cost play and a reliability play at once: shorter prompts are cheaper and less diluted, so the model obeys them more consistently. These are the highest-leverage trims.

Cut standing context ruthlessly. Anything in the system prompt is re-sent on every call, even for requests that never touch it. Move product descriptions, glossaries, and style samples into per-request context or a retrieval step, and keep only what every request genuinely needs in the standing prompt.

Merge overlapping rules. Five sentences that all say "be accurate" can become one. Redundancy looks safe, but it consumes attention budget and can push genuinely new instructions out of the model's focus. If a rule appears twice, you have found either a candidate to delete or a sign that the rule is not landing — fix one or the other.

Tighten the tools array. A tool description that reads like a sentence of prose can usually lose half its words and call more reliably: "Returns order numbers and statuses for the user's open orders" beats a paragraph. Models compare the user's need against the description, and a crisp outcome phrase makes the match obvious.

Shrink schemas to the required minimum. Every optional parameter is a cost: it invites a value, and your handler has to tolerate whatever arrives. Remove fields you do not actually use, and set enum values when the domain allows it — constrained inputs make both the model and your code more predictable.

Put the costliest rule where the model will see it. Because early context is weighted more heavily, promote the rule that matters most to the top of the prompt and relegate boilerplate to the bottom. If the model keeps violating a rule that sits on line three, the problem is phrasing, not placement; try rephrasing before adding a second copy.

Store the prompt and the tools array as diffable artifacts. Keep them in files, not in a database blob, so a change is a diff and a diff is reviewable. A system prompt you cannot diff is a system prompt you cannot debug when behavior regresses next month.

Measure the prompt's token length and track it over time. Prompt length tends to creep upward as rules accumulate. A weekly number in your dashboard catches the creep, and the builder's generated output gives you a canonical, copyable baseline to diff against each edit.

Test against a frozen transcript set. Keep a small set of representative user requests and their expected behaviors, and re-run it after every prompt change. Because model behavior shifts across versions, the transcript protects you from mistaking a model update for a prompt improvement.

Finally, treat generated output as a starting point, not a finish line. The builder assembles a clean, complete prompt — then you iterate: ship, watch, trim, re-test. The optimization loop never really closes; it just gets faster the better your tooling and habits become.