Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 39 additions & 20 deletions content/docs/(guide)/overview/what-is-bitrouter.mdx
Original file line number Diff line number Diff line change
@@ -1,39 +1,58 @@
---
title: What is BitRouter?
description: BitRouter is an open-source, adaptive LLM gateway that wraps your agent loop in a second loop — act, observe, evaluate, learn — so routing tightens with every run, with zero harness changes.
description: The context-aware LLM router that continuously improves your agent workflows — send bitrouter/auto instead of a model name, and let a policy you own tighten against your own production traffic.
---

We started BitRouter out of frustration. We were running agents in production, and every step of every loop — every model call, every tool call, every sub-agent — billed at frontier prices by default. The routers we pointed our loops at were static: they forwarded each call to a fixed route, and that route was exactly as good on run one thousand as it was on run one. That felt backwards to us. A router sits in the middle of everything your agent does — given the right signals, it should be the part of your stack that learns.
**BitRouter is a context-aware LLM router that continuously improves your agent workflows.** It takes model selection off your plate without taking it out of your hands: send `bitrouter/auto`, and every call in the loop is matched to a model against the objective you set — from a routing policy that lives in your repo, that you read, review, and can override line by line.

So we built it. BitRouter is an **open-source, adaptive LLM gateway and router** for production agent loops: a single local binary that gives any agent one endpoint for its model calls, tools, and sub-agents, and routes each one to the best path that still reaches the goal — with **zero harness changes**. Point your runtime at it and every step of every loop stops billing at frontier prices by default.
A frontier release lands every few weeks, and each one moves the cost, latency, and accuracy frontier somewhere different. Nobody re-benchmarks a whole agent that often, so the choice has been to pin one frontier model and overpay on every trivial step, or hand-tune a table that's stale a month later. BitRouter is the third option: a router that keeps re-testing the frontier for you, scored against *your* workload instead of a public leaderboard.

It runs anywhere your agent runs, with no dependencies to install, and operates as a permissionless network where any provider can register and any agent can connect. The **Core** is **open-source under Apache 2.0 and self-hostable for free** — bring your own keys or run a local model and you owe us nothing. **Cloud** is an optional hosted layer that adds managed providers, agentic payments, and account-wide policies on top. You can [install either mode](/docs/overview/quickstart) in under a minute, and browse the full [models & pricing](/docs/overview/supported-models) catalog.
On Terminal-Bench 2.1, routing an agent through BitRouter cuts total run cost by roughly **30%**.

## The idea: a second loop
## One model id replaces every model decision

Our bet is simple: routing is a learning problem. BitRouter wraps your agentic loop in a **second loop**. Each loop gets its own [policy spec](/docs/overview/quickstart#adaptive-routing) — a config file that declares how its calls, tools, and agents should route — and against that spec BitRouter runs a continuous **act → observe → evaluate → learn** cycle. Every step is a component it already ships:
Two things change — the base URL, and the model you ask for:

- **Act — the router.** Each model, tool, and agent call is rewritten to a chosen route: policy-table routing, cross-protocol translation, multi-account failover. See [Provider selection](/docs/models-and-routing/provider-selection).
- **Observe — telemetry.** Every hop is attributed with cost, tokens, latency, and outcome, exported over OTLP to any backend you run. See [OpenTelemetry](/docs/evals-and-tracing/opentelemetry).
- **Evaluate — the adequacy signal.** Each served request is scored against the route that served it — did the cheaper path still reach the goal? — and every request is cost-metered. Run-level, objective-scored evals are the next milestone. See [Evaluation](/docs/evals-and-tracing/evaluation).
- **Learn — the policy engine.** The observed signal folds back into the policy spec: proven downgrades materialize into the table, failed ones escalate back. The next turn of the loop acts on the improved spec. See [Adaptive routing](/docs/overview/quickstart#adaptive-routing).
```bash
curl https://api.bitrouter.ai/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "bitrouter/auto", "messages": [{"role": "user", "content": "..."}]}'
```

You choose what the loop optimizes for, and it improves the longer it runs in production. We call this recursive self-improvement applied to infrastructure: the router gets better at routing your loop every time the loop runs.
Explicit model ids stay passthrough — `openai/gpt-5` still means `openai/gpt-5`, so you can always pin a call the router shouldn't touch. The `vendor/auto` shape is deliberately the one other gateways already use: a config pointing at some other `.../auto` only needs its vendor segment changed.

## Not just models
What differs is what happens after the request lands. A classifier-based router makes an opaque per-prompt guess you can't inspect. BitRouter projects each request onto **where it sits in the agent's trajectory and how much risk it carries**, then resolves that projection through a signed policy file you own — ordinary `edit` and `test` work to economy, read-only `review` and long-context execution to balanced, anything guarded or unrecognized to strong. Risk is scored apart from the work itself: eight prior assistant action turns raises expected redo risk, and an observed execution failure stays guarded through the next two execution observations, so a model-induced loop can't quietly keep buying cheap tokens.

Here's something we noticed early: an agentic loop consumes three things, and most routers govern only one. We built BitRouter to make all three routable, observable, and governed:
Because `bitrouter/auto` resolves against a policy, it needs one bound before first use — an unbound install reports the missing binding rather than guessing a route.

- **Models** — route LLM calls across providers, accounts, and wire protocols (OpenAI, Anthropic, Google). See [Models](/docs/overview/supported-models#how-model-ids-work).
- **Capabilities** — an **MCP gateway** and an **AgentSkills gateway**: tools and skills become governed, routable resources, not hardcoded endpoints. See [Tools](/docs/usage/mcp-gateway).
- **Agents** — an **ACP gateway**: sub-agents are first-class routable primitives, so a task can go to the sub-agent that best fits the objective — just as a call routes to the best-fit model. See [Agents](/docs/models-and-routing/acp-gateway).
## Your production traffic is the training signal

Optimizing a loop isn't just model selection — it's the model, the tool, *and* the sub-agent that best serve the loop's objective.
Other routers are tuned once, offline, on somebody else's benchmark. Observability tools show you your own numbers, then leave you the homework of re-tuning the config. BitRouter closes the loop: it trials cheaper routes on live traffic, scores what came back, and updates the policy from the result — **act → observe → evaluate → learn**:

## Where we are today
- **Act — the router.** Each model, tool, and agent call is rewritten to a chosen route: policy-table routing, cross-protocol translation, multi-account failover.
- **Observe — telemetry.** Every hop is attributed with cost, tokens, latency, and outcome.
- **Evaluate — the reward.** Each served request is classified by outcome against the route that served it, deterministically and with no LLM judge in the path.
- **Learn — the policy engine.** Downgrades that keep succeeding qualify for the cheap tier; ones that fail pin back up.

**Today BitRouter optimizes for cost.** That's the objective that ships: trivial calls stop billing at frontier prices, and the loop keeps tightening as evidence accumulates. Latency and accuracy objectives are next on the roadmap — the loop is multi-objective by design, and we'd rather ship one objective that works than three that half-work.
The evidence rule is asymmetric on purpose: one hard failure escalates immediately, while a cheaper route has to succeed repeatedly before it earns the traffic. It optimizes for cost today; latency and accuracy are next. What it learns lands in a lock file next to your config, and the router never publishes it without you — you read the diff and commit it, so the improved policy ends up in Git where a policy belongs.

## It optimizes the workflow, not the request

Every other gateway's unit of optimization is the request: pick a model, return a response, done. BitRouter's unit is the workflow — the whole run, judged on whether it did the job you wanted.

Out of the box `bitrouter/auto` routes on general evidence. Optimizing replaces that with evidence from **your** workflow: you name the command that exercises your agent and write a success contract describing a good run, and BitRouter runs that command twice — once as-is, once with exactly one routing change — then compares cost and quality. One variable, measured on your code, against your definition of good. It is not "swap in a cheaper model."

Nothing moves until you say so: review reports the deltas, publishing is a separate explicit action, and a bad call rolls back by digest. The intent, the resolved identities, and the success contract are three files you own and commit — and the result serves under the same `bitrouter/auto`, so nothing in your application changes.

Scope isn't only models, either. A workflow's cost is its model calls plus its tool definitions plus its sub-agents, so all three are routable and governed: an [MCP gateway](/docs/usage/mcp-gateway) and an AgentSkills gateway turn tools and skills into routable resources instead of hardcoded endpoints, and an [ACP gateway](/docs/models-and-routing/acp-gateway) makes sub-agents first-class routable primitives.

The Core is Apache 2.0 and free to self-host; Cloud adds managed providers and account-wide policies on top. [Quickstart](/docs/overview/quickstart) has the split.

## Next steps

BitRouter is a drop-in proxy for any runtime that supports a custom OpenAI or Anthropic base URL. The [Quickstart](/docs/overview/quickstart) gets you routing in under a minute; [Models & Routing](/docs/models-and-routing/provider-selection) covers models, routing, and [tool calling](/docs/models-and-routing/tool-calling/server-tools) in depth, and [Usage](/docs/usage/mcp-gateway) the gateways and the CLI. Per-runtime recipes (Claude Code, OpenClaw, Codex, and more) live in [Integrations](/docs/integrations), and end-to-end walkthroughs in [Guides](/docs/guides/cloud-api).
<Cards>
<Card title="Quickstart" href="/docs/overview/quickstart" description="Self-host or Cloud, routing in under a minute." />
<Card title="Start optimizing" href="/docs/overview/quickstart#start-optimizing" description="Run the loop against your own workflow and publish the result." />
<Card title="Integrations" href="/docs/integrations" description="Recipes for Claude Code, Codex, OpenClaw, and more." />
<Card title="BitRouter vs OpenRouter" href="/docs/overview/bitrouter-vs-openrouter" description="An honest side-by-side with the cloud catalog." />
</Cards>
Loading