One capable model became the expensive default
An agent workflow can call a model many times: classify intent, select a tool, validate an output, summarize a result, or decide what happens next. The easy implementation sends every step to the same capable model. It works, but it quietly makes the most expensive path the default path.
The useful question was not “which model is best?” It was: what is the smallest amount of intelligence this step actually needs?
I stopped picturing one pipeline
I now picture an agent system as a traffic network rather than one pipeline. Each trace contains signals about the road a request took: the type of task, number of tool calls, failure pattern, token use, latency, and whether the model added reasoning that deterministic code could have supplied.
Routing should happen at the intersections. A complex reasoning step may need a strong model. A bounded classification may fit a smaller model. A stable transformation may not need a model at all.
trace -> classify work -> choose execution path
|-> deterministic function
|-> lightweight model
`-> capable reasoning modelSeparating classification from execution
The system begins by turning raw trace output into structured features. Useful categories include task shape, tool dependency, confidence, error family, expected output structure, and observed cost. Those features are grouped into behavior classes that a router can act on.
The important design choice is keeping analysis separate from execution. The classifier describes the work. The routing policy decides what to do. This makes the decision auditable and allows the policy to evolve without rebuilding the entire analysis layer.
A safe rollout also needs fallbacks. A lightweight path should be promoted to a stronger model when confidence is low, the output violates its schema, or a downstream validator rejects it. Cost optimization without a quality escape hatch is simply hidden reliability debt.
Routing was not really provider selection
At first, “model routing” sounded like provider selection. The deeper problem is broader: deciding whether a model call is necessary at all, how much capability the step needs, and how to prove that a cheaper path preserves the required quality.
The trace was also easy to mistake for observability exhaust. In reality, it can become training data for better orchestration. But that only works after the trace is converted into a consistent taxonomy rather than left as free-form text.
The open trade-off
Agent cost is an architectural property. Prompt tuning helps, but the larger lever is choosing the right execution mechanism for every step.
The reusable pattern is: observe behavior, classify the work, route conservatively, validate the result, and retain an escalation path. A model should earn its place in the loop.
What I have not settled yet is the threshold where a routing layer saves less than it costs to own. That needs workload data, not a universal answer.