Model Routing

In Gen AI circles model routing is simple called query routing. The goal of model routing is to decide which model to use based on the difficulty of the user query.

LLM Routing — Intuitively and Exhaustively Explained:

LLM routing is an advanced inferencing technique which can automatically choose the right language model, out of a selection of language models, for a given prompt; improving the performance, speed, and cost in LLM-powered systems.

• Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
  • 44 citations
• AutoMix: Automatically Mixing Language Models
  • 16 citations
• FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
  • 175 citations

• IBM - An air traffic controller for LLMs
• LLM Routing — Intuitively and Exhaustively Explained

• Learned coordinators that go beyond single-model routing to build and manage a pool of models per query.
• Learning to Orchestrate Agents in Natural Language with the Conductor
  • 2026-04-26 — Sakana; 7B Conductor trained with RL writes custom instructions and spins up planner/coder/verifier pipelines
• Trinity An Evolved LLM Coordinator
  • 2026-04-25 — Sakana; <20K-param coordinator assigns Thinker/Worker/Verifier roles, optimized with evolutionary search
• Sakana Fugu One Model to Command Them All
  • 2026-06-21 — Sakana; a single foundation-model API that orchestrates a swappable pool of frontier models