Optimization Tips for Multi-Agent System Topologies

Multi-agent systems fail on economics before they fail on quality. Every node and edge adds model calls, tokens, and wall-clock time, so optimization is a graph problem, not a prompt problem. The levers below attack the three costs that dominate: hop count, serialization, and context size.

Collapse redundant hops. The cheapest optimization is deleting an agent that merely forwards work. If node B receives context and immediately passes it to C without transformation, B contributes latency and tokens but no value. Chain together only stages that genuinely add something, and let one agent perform multiple simple transformations rather than threading a relay of single-purpose nodes.

Parallelize anything independent. The largest latency win in any graph is converting a serial chain into a fan-out. If research, pricing, and formatting do not depend on each other, run them concurrently behind a router and join at the end. The harness's parallel edges make the intent explicit, and the exported skeleton gives you a run_parallel helper that matches the design one-to-one.

Size the hand-off, not the history. Forwarding the full context to every downstream agent multiplies token cost by the number of consumers. Instead, shape each hand-off to the smallest payload the receiver needs: a summary for a reviewer, a citation list for a writer, a single field for a formatter. This one habit frequently cuts cost by half on real pipelines.

Assign cheap models to mechanical stages. Extraction, classification, and format conversion do not need a frontier model. Wiring a gpt-4o-mini-class node where a full-size model was used before trims both latency and price per call, often with no measurable drop in output quality. Keep frontier models for synthesis and judgment where their capability actually pays.

Use conditional edges to skip work. A conditional branch that sends easy requests down a short path and hard requests down a long path is the classic free win. Most inputs never need the full pipeline. Route-to-shortcut logic cuts average latency dramatically while preserving worst-case quality for the inputs that require it.

Batch what cannot be parallelized. If a downstream agent must process many items, give it one call with a list instead of one call per item. This trades prompt size for round trips and is especially effective when the work is uniform, such as categorizing a hundred support tickets in a single structured call.

Pin cold temperatures on decision nodes. A routing or conditional agent at a low temperature produces stable, repeatable branches, which makes downstream caching possible. When decisions are deterministic, you can cache the routing outcome and skip re-evaluation on identical inputs, a saving that compounds on high-traffic systems.

Reuse results across edges. If two downstream agents both need the same expensive extraction, run the extraction once and pass the shared result to both, rather than letting each agent recompute it. Shared nodes are the graph's natural cache; the fewer times an expensive computation appears, the cheaper the whole system.

Profile before you polish. Export the topology text and count edges; each edge is a call. If latency is the pain point, look for the longest serial path and shorten it. If cost is the pain point, look for the most expensive model on the busiest edge. Optimize what the measurements indict, not what feels elegant.

Keep a lean baseline and grow by measurement. Every new agent must justify itself with a before-and-after quality or latency comparison. The discipline of re-running the same benchmark against a smaller graph is what stops topology from drifting into an expensive Rube Goldberg machine.