Only as much intelligence as you need

I get to look at billions of tokens every day as they pass through enterprise gateways. One pattern keeps showing up: engineers build loops and Ralph loops where the only thing that matters is whether the loop eventually finishes the job. Fan out, retry, inspect, patch, repeat. If the end product is done, the loop worked.

You can run that kind of loop with almost any model that can handle multiple turns. In practice, people keep reaching for the best of the best frontier models like Opus 4.8 or Fable 5.

That is more than overkill. It is expensive, inefficient, and, honestly, a training problem. My rule of thumb is simple: executors (or implementors) can be pretty dumb if their work is supervised. A loop is just a long running executor.

Anthropic now ships the pattern as a primitive called the advisor tool in Claude Code and the Messages API. Claude can consult a second, usually stronger model at important moments in a task: before settling on an approach, when it gets stuck on the same error, or before it says the task is done. https://code.claude.com/docs/en/advisor
A fast, cheap executor model does the mechanical work: reading files, searching, editing code. When it reaches an actual decision, it asks the stronger advisor for a second opinion, then keeps going. You get close to advisor-solo quality while most of the token generation still happens at executor-model prices. https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool

[!NOTE] The concept of the Advisor comes from the paper "Steering Black-Box LLMs with Advisor Models" by Parth Asawa, Alan Zhu, Abigail O’Neill, Matei Zaharia, Alexandros G. Dimakis and Joseph E. Gonzalez and was originally published to arXiv on 2nd October 2025 https://arxiv.org/abs/2510.02453.

Anthropic's own evals show the shape pretty clearly. Sonnet with an Opus advisor lifted SWE-bench Multilingual by 2.7 points and cut cost per task by 11.9% compared with Sonnet alone. Haiku with an Opus advisor more than doubled its BrowseComp score, from 19.7% to 41.2%, while still costing 85% less per task than Sonnet by itself. https://claude.com/blog/the-advisor-strategy

Intelligence applied only at decision points beats intelligence applied everywhere, per dollar.

why this has to be the default, not an option

Agent loops burn tokens on narration, retries, and tool output, not just on the final answer. As agentic workflows move from one developer's side project into a team's daily default, that token volume starts to compound. Paying frontier rates for the 90% of a run that is mechanical usually does not fail in one dramatic moment. It fails as a line item, quietly, every month, whether the workload is a hundred requests or a hundred million.

The price ladder already tells the story, and this is not just an Anthropic thing.
Claude Opus 4.8 runs $5/$25 per million input/output tokens, Sonnet 5 is $3/$15, Haiku 4.5 is $1/$5, and Fable 5 is $10/$50. https://platform.claude.com/docs/en/about-claude/pricing, https://www.anthropic.com/news/claude-fable-5-mythos-5 OpenAI shipped the same tiered logic on July 9 with GPT-5.6: three durable capability tiers, Sol, Terra, and Luna, priced five to six times apart from top to bottom, at $5/$30, $2.50/$15, and $1/$6 respectively. https://openai.com/index/gpt-5-6

Running everything at the top of that ladder is not a quality decision. It is a pricing decision. In many cases you're paying 5–10× more for nearly identical tokens. Teams that don't route are burning margin on autocomplete. As agentic systems scale, routing won't be a nice optimization anymore. It'll be the default architecture.

That gets even more interesting when local models enter the executor slot. I have not tried this exact pairing yet, but I would bet that Gemma 4 26B ($0.06/$0.33 per million tokenshttps://openrouter.ai/google/gemma-4-26b-a4b-it, or free on your own machine) with Fable 5 as the advisor would work well. At that point, the cost gap between the two tiers is not 5x or 10x. It is past 150x. https://www.morphllm.com/best-ai-model-for-coding

Google has not published an official SWE-bench score for Gemma 4, and the third-party numbers disagree by a wide margin, so treat that half of the chart as directional.

takeaway: not every task needs the frontier model

On SWE-bench Verified, Anthropic reports Haiku 4.5 at 73.3%, against Opus 4.5 at 80.9%. https://www.anthropic.com/claude/haiku
Seven points of accuracy for five times the price. Oof. Yes, it is only a benchmark, but it gives us a direction.

A wider comparison across models makes the same shape clearer. The Claw-SWE-Bench evaluation ran nine models through the same agent scaffold on 350 real coding tasks, tracking both accuracy and total cost.
GPT 5.5 had the highest resolved rate at 78.0%, followed by Claude Opus 4.7 at 77.1%. But accuracy ranking is not cost ranking. GPT 5.5's full run cost $1,399.10, and Opus 4.7's cost $1,082.00 for a 0.9-point lower score. DeepSeek-V4 Pro reached 71.7% for $81.30, while DeepSeek-V4 Flash reached 70.3% for $8.20. https://arxiv.org/pdf/2606.12344

Seven points separate the top model from the cheapest one in that lineup. The cost gap is over 170x.

the flow

As soon as I heard about this, I wanted to try it in Pi. My current setup works like this: GPT-5.6 Sol at xhigh reasoning effort as the advisor, Gemini 3.5 Flash as the executor. Sol only gets called in when Gemini hits a real decision or plainly does not understand something.

[!IMPORTANT] You can try and install my shitty implementation on GitHub or via npm: pi install npm:pi-advisor-flow

The pattern holds no matter which two tiers you connect: the executor grinds, the advisor decides.

Most of the work is boring. Discovery, planning, editing, checking, trying again. That's what the executor does.
The executor escalates to an advisor in three cases: before committing to a plan with real tradeoffs, when it repeats a failure, and before it declares the task done. The advisor sees the full working transcript, including tool output, not just a summary, and returns guidance.

A stronger model asked "what should I do?" without the observed state mostly gives you a more expensive generic answer. The advisor runs as a separate server-side inference pass under its own system prompt. It receives the executor's full transcript as quoted context automatically, either with no extra context from the executor or with a direct question from it, which is how I wired it up. https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool

const response = await client.beta.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  betas: ["advisor-tool-2026-03-01"],
  tools: [
    {
      type: "advisor_20260301",
      name: "advisor",
      model: "claude-fable-5",
      max_uses: 2,
    },
  ],
  messages,
});

The advisor has to be at least as capable as the executor. https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool

routing table for models

measure it on your own work, then route

The metric I care about is cost per accepted change, not cost per token.
A model is expensive when it makes work slower or less correct than a cheaper option would have. Fable is cheap when it stops a failing loop. Haiku is expensive when you hand it a judgment call it cannot see and it fails the same way three times.

Unlike classical prompt routing, you don't classify requests into "easy" and "hard" before they start. A single task spends 95% of its lifetime in a cheap executor and only borrows frontier intelligence for the handful of decisions, where it acutally matters. That's a fundamentally different way of thinking about model selection.

Give the fast model the reversible work. Give the strong model the decisions. Route between them deliberately, because at the volumes agentic engineering is heading toward, "just use the best model" is not a strategy that survives contact with the invoice.