What the paper tested

Anyone paying for more than one model API ends up asking the same thing. If a small model picks which system handles each request, how much better do the results get, and when is it safe to send work to something cheaper? Route Anything, posted to arXiv on 3 November 2026 by Pengfei Zhou, Yang You and five co-authors, tests that at scale.

The team scored 452 candidate configurations on 25 benchmarks, which came to 374,401 runs. A configuration is a model together with its harness and effort level, so one model can show up several times. The benchmarks fall into seven groups:

  • text questions and questions about images
  • image, video and speech generation
  • coding and office-style agent work
  • two-armed robot manipulation in the RoboTwin simulator

The language and agent pools include Claude Opus 5.5, GPT-6.1 Sol, GPT-6 Astra and DeepSeek V4.1 Flash. Every candidate in a pool ran on identical inputs. The data was split 70/10/20 into training, development and test sets, grouped by source so near duplicates couldn't leak between them. For robots, the 2,000 test scenes came from 10 tasks the router never saw during training.

How the routing framework works

The authors describe a router by what it gets to see, which they call its state. That can be as little as the benchmark name or as much as the whole input. Each run gets a score between 0 and 1 and costs something, measured in dollars or seconds. The router tries to maximise expected score after subtracting a price multiplied by expected cost. The operator sets that price. Set it to zero and only quality counts.

Two theory results shape everything else:

  • The information ladder. Giving a router more detail can never lower the best return it could achieve. The bottom rung is the best fixed candidate for each benchmark. The top rung is an oracle that knows, after the fact, which candidate won on each input.
  • Predict scores for every candidate. A router that learns to estimate each candidate's outcome can, in principle, reach the oracle's choice when cost is zero. A router trained only to name the winner can throw value away. The robot experiments show how much: outcome regression reached 71.6% success, while a winner-label classifier managed 42.8%. The authors trace the gap to a latency tie-break that settled 94% of the winner labels.

The main learned router is Qwen3.5-4B, fine-tuned on task text and output requirements. For robots it also reads camera frames and the robot's state. Another variant sorts inputs by capability type and sends each group to whichever candidate did best on it.

What the results show

The ceiling is modest. Across the full matrix, the hindsight oracle scored 87.98 against 82.60 for the best fixed candidate, a 5.4 point gap. The size of that gap varies a lot: 0.68 points in speech generation, 18.93 in video and 18.40 in robot manipulation. Even picking the best candidate per capability type with perfect hindsight closes only about 8% of it.

Learned routers get even less of it:

  • Over 16 benchmarks and 3,444 test inputs, the capability-tree router scored 79.98. The best fixed candidate scored 79.68 and the oracle 84.80, so the router gained just 0.30 points.
  • By group, video gained 2.12 points, code and agents 0.77 and images 0.25. Text questions lost 0.17 and speech didn't move.
  • On GenEval2 and Qwen-Image-Bench, none of 124 router configurations beat the fixed image generator by a significant margin.
  • On robots, the router reached 71.55% success against 70.45% for the best single policy. The 95% interval runs from minus 0.40 to plus 2.55, so it crosses zero. Policy time per scene fell from 14.51 to 11.05 seconds, a 24% cut, while the oracle sits at 88.85%.

The paper also shows how easily a comparison can mislead. Opus 5.5 was missing 11 of its 1,122 HLE scores. When candidates with gaps were dropped, the baseline moved from 68.30 to 65.52, and the router's result swung between a 0.40 point loss and a 2.38 point gain depending on which inputs were compared.

Cost is where routing pays off. On CodeRouterBench, a set of about 10,000 coding tasks with verified scores from eight frontier models that the same group released in June 2026 alongside Agent-as-a-Router, a fitted route was 25.5 times as efficient as always calling Claude Opus 4.6, for a 0.44 point drop. That route never calls Opus at all. On RouterBench (11 models, 36,497 prompts, with GPT-4 as the reference), the fitted route reached 7.1 times efficiency for a 0.40 point drop. In image generation, a small allowed loss cut costs by 26%. Across the four cost testbeds, efficiency rose 8.82 times on average.

Where quality floors break

A quality floor caps how much quality we'll give up against a reference model. The paper's warning is blunt: a floor fitted on one set of data rarely survives new data. A zero-loss coding route held its floor on test data in only 12.5% of 200 random splits.

The fix is certification. The authors test candidate routers in price order using valid p-values on independent inputs. They stop at the first route that fails, and if nothing passes they fall back to the reference. The certified route stayed at or above reference quality in all 200 splits, with a median efficiency of 19.4 times.

That guarantee has edges:

  • It covers the total only. The certified router still lost quality in a median 8 of 21 test cells.
  • It holds only against the reference we named.
  • It isn't free. We need the reference model's output for every certification input. The authors put that at $0.0158 per task, compared with $0.0124 per task for just running Opus.

Spending more on a single task also helps less than we might hope. When one model fails, the next model's chance of success drops by roughly half (median ratio 0.49 across 42 model pairs). For Qwen3-Max, a free compiler check gave 4.09 times dollar efficiency at a 2.4 point loss, while a list-price LLM judge came in at only 0.30 times.

Limits

  • The theory treats each router's inputs as fixed, so it doesn't cover agent loops where later steps include code an earlier model wrote. SciCode was left out for that reason, and the authors say step-level routing for long agent tasks needs more work.
  • Several studies use a single training seed, some panels are small (158 video inputs), and many intervals cross zero.
  • The paper doesn't say whether the trajectory matrix or trained routers will be released. The code link given in Agent-as-a-Router returned a 404 when we checked on 9 November 2026.

What builders can take from it

Routing is a weak way to beat our strongest model and a good way to find a cheaper route that's good enough. Most frontier APIs bill in US dollars, so for New Zealand teams whose bills are dominated by one expensive model, the cost results are the part worth testing.

  1. Run every candidate on the same sample of real inputs, and judge any router against the best single model on those exact inputs.
  2. Log each model's score and cost for every input, train the router to predict outcomes, and look for gains within each task type.
  3. Choose the reference model and the loss we'll accept before ranking routes, then certify the floor on inputs the router hasn't seen.
  4. Keep the routing step cheap. A TF-IDF router took 1.9 ms per task, about 0.06% of a Qwen3-Max call, while a grok-4.6 judge took 39 seconds and $0.0159 per task.

If you try this, we'd like to hear how it went, failed routes included, on the ASAI projects page.

Sources

Build with people working on the same things

Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.

Become a member