Category:
AI Cost Optimization
AI ROI
Published date:
Model routing is the practice of sending each request to the cheapest model capable of handling it, rather than sending everything to the most capable model available. A router classifies the request, picks a model, and escalates to a stronger one when needed. Because model prices differ by an order of magnitude for similar tasks, routing is usually the single largest lever on AI cost.
Key Takeaways
Most production traffic does not need a frontier model. Routing exploits that gap.
Routing decisions are made by rules, by a classifier, or by trying a cheap model first and escalating.
The saving comes from price spread between models, which is wide and changes often.
The failure mode is silent quality loss, so routing needs evaluation attached from day one.
Guickly treats routing and caching as the two main optimisation levers, and both require usage visibility before they can be tuned.
How does model routing work?
Three approaches, increasingly sophisticated.
Rules. Route by task type, user tier or endpoint. Simple, predictable, easy to reason about, and crude because it ignores how hard the individual request actually is.
Classification. A small model or heuristic scores request difficulty and picks a tier. More accurate, at the cost of one extra call and a component to maintain.
Escalation. Try the cheap model, evaluate the answer, retry on a stronger model if it fails. Highest quality retention, but the retries eat some of the saving, so it needs measuring rather than assuming.
What does model routing actually save?
The saving is a function of price spread and traffic mix, so it has to be computed rather than quoted.
Prices move frequently, which is a point in itself: a routing configuration set six months ago is almost certainly no longer optimal. Claude Sonnet 5, for instance, rises from 2 and 10 dollars to 3 and 15 dollars per million input and output tokens on 1 September 2026, which changes the arithmetic for anyone routing to it by default. Routing needs reviewing whenever vendor pricing changes.
Pricing sourced on 5 August 2026 from Anthropic's published rates.
When does model routing go wrong?
When nobody is measuring quality.
Cost savings from routing appear immediately and are easy to report. Quality degradation appears gradually, in the form of slightly worse answers that nobody escalates, and it is invisible on a cost dashboard. A routing programme without an evaluation set attached will look successful and may not be.
The second failure is complexity. A router is a dependency in front of every request, so it has to be more reliable than the models behind it.
Model routing versus a single model
Single frontier model | Routed | |
|---|---|---|
Cost | Highest | Substantially lower, varies with mix |
Latency | Consistent | Lower on cheap paths, higher on escalation |
Quality | Predictable ceiling | Depends on router accuracy |
Complexity | Minimal | A component to build, tune and monitor |
Failure mode | Expensive | Quiet quality loss |
The honest read: routing is worth it once AI spend is large enough that the saving exceeds the engineering cost of maintaining a router, and only when quality is being measured.
FAQ
What is model routing? Model routing is the practice of sending each request to the cheapest model capable of handling it, rather than sending all traffic to the most capable model. A router classifies the request, selects a model, and escalates when the cheaper model is not sufficient.
How does model routing reduce AI costs? Model prices differ substantially for similar tasks, and most production requests do not need a frontier model. Routing simple requests to cheaper models captures that difference. The saving depends on the price spread and the mix of traffic.
What are the main model routing strategies? Rules based on task type or user tier, classification where a small model scores request difficulty, and escalation where a cheap model is tried first and a stronger one is used if the answer fails evaluation. Escalation retains the most quality and gives back part of the saving in retries.
What is the biggest risk with model routing? Silent quality loss. Cost savings show up immediately while degraded answers appear gradually and do not register on a cost dashboard. Routing needs an evaluation set attached from the start.
How often should routing rules be reviewed? Whenever model pricing or capability changes, which in practice means quarterly at minimum. A configuration tuned six months ago is unlikely to still be optimal, because vendor prices move.
Last updated: 5 August 2026.
![](data:image/svg+xml,<svg display="block" role="presentation" viewBox="0 0 17 17" xmlns="http://www.w3.org/2000/svg"><path d="M 3.335 0 L 13.335 0 L 13.335 1.667 L 3.335 1.667 Z M 0 3.333 L 1.667 3.333 L 1.667 13.333 L 0.001 13.333 L 0.001 3.333 Z M 15 3.333 L 16.667 3.333 L 16.667 13.333 L 15 13.333 Z M 13.333 1.667 L 15 1.667 L 15 3.333 L 13.333 3.333 Z M 1.667 1.667 L 3.333 1.667 L 3.333 3.333 L 1.668 3.333 L 1.668 1.667 Z M 3.333 16.667 L 13.333 16.667 L 13.333 15 L 3.335 15 L 3.335 16.667 Z M 13.333 15 L 15 15 L 15 13.333 L 13.333 13.333 Z M 1.667 15 L 3.333 15 L 3.333 13.333 L 1.668 13.333 L 1.668 15 Z M 7.501 3.333 L 9.168 3.333 L 9.168 9.167 L 7.501 9.167 Z M 9.168 9.167 L 10.834 9.167 L 10.834 10.833 L 9.168 10.833 Z M 10.834 10.833 L 12.501 10.833 L 12.501 12.5 L 10.834 12.5 Z" fill="rgb(43, 54, 52)" height="16.667px" id="PxKarjvpl" width="16.667px"/></svg>)