LLMRouter
A Unified Library, Evaluation, and Analysis for LLM Routing
University of Illinois Urbana-Champaign · University of Maryland, College Park · Tsinghua University
Overview
No single large language model is optimal across all queries and budget constraints. LLMRouter is an open-source foundation for selecting the right model for each request, making routing methods easier to develop, compare, and deploy under a shared quality–cost objective.
It unifies routers that were previously implemented in incompatible stacks—from simple quality predictors and cost-aware cascades to graph-based, multi-turn, and personalized policies—and provides the shared infrastructure needed to evaluate them fairly.
A Unified Routing Formulation
Every router is represented as a sequential decision process. A shared formulation turns different routing ideas into comparable design choices.
Describe the state
Represent the query, user context, interaction history, and candidate-model information.
Make a routing decision
Score candidate compatibility, then dispatch, escalate, or stop under the operating budget.
Learn and evaluate
Optimize task quality and inference cost with a common protocol across routing settings.
xRouteBench
LLMRouter constructs routing supervision by running a candidate pool across benchmarks, scoring each response with its task metric, and recording token-level cost. Every router then faces the same queries, models, metrics, and quality–cost protocol.
The resulting benchmark spans generic LLM tasks, memory-augmented reasoning, image and video understanding, time-series, and personalized routing. This makes it possible to compare both performance and cost instead of optimizing one in isolation.
Tracks: Generic LLM tasks · Memory · Vision & video · Time-series · Personalization
Key Findings
Learned routing improves over fixed-model baselines. The empirical study finds a 14.6% relative improvement over the strongest fixed-model baseline.
No router dominates every deployment. Rankings change across tasks and reverse as cost constraints become tighter.
User context changes the right answer. Personalized routing gains from preference and interaction history when it is available.
BibTeX
@misc{feng2026llmrouter,
title = {LLMRouter: A Unified Library, Evaluation, and Analysis for LLM Routing},
author = {Tao Feng and Haozhen Zhang and Zijie Lei and Haodong Yue and Chongshan Lin and Jiaxuan You},
year = {2026}
}