FAQ

Frequently asked questions

Straight answers to the questions we get most. If yours isn't here, ask on Discord.

Which models does Busbar support?

All of them, and here's why that isn't a boast. Models are built by providers: Anthropic, DeepSeek, Z.ai, Alibaba, Moonshot, xAI. Each provider speaks a language for reaching its models: a wire protocol, and there are only about six in the world (OpenAI-compatible, Anthropic, Gemini, Bedrock, Cohere, Responses). Busbar speaks all six. So it supports every provider, and therefore every model, out of the box. There is no model list to be on, because a model name in Busbar is a config value you define, pointed at whichever provider serves it.

In practice: GPT and o-series (OpenAI), Gemini (Google), Claude (Anthropic or Bedrock), Qwen (DashScope), DeepSeek, GLM (Z.ai), Mistral, Grok (xAI), Kimi (Moonshot), Command (Cohere), and Llama on Groq, Together, Fireworks, and friends, via the 113 verified providers in the shipped catalog. Community fine-tunes ride the hosts that serve them: DeepInfra, Novita, Featherless, and the other aggregators. And when a new model ships, there is nothing to wait for: it works the moment its provider serves it, because the model name is your config value, not our release note. If you want a hosted aggregator's entire menu, OpenRouter itself is a catalog entry, so anything it serves is one lane away, with Busbar's failover and budgets on top. The full maker-to-language lookup lives on the providers page, or skip the reading: pick your models in the config builder and copy the YAML.

Provider X isn't in your catalog.

The catalog is a convenience, not a boundary. Any provider that speaks one of the six protocols, including any OpenAI-compatible endpoint and local runtimes like Ollama or vLLM, is a few lines in your own providers.yaml: a name, a base_url, the env var holding its key. Adding a provider walks through it.

The genuine gap is a provider whose wire format is none of the six. That needs new translator code, and it's the one case where you'd wait on us.

Is Busbar a gateway or a control plane?

A control plane, and the difference matters. A gateway forwards LLM traffic. A control plane decides: whose fault a failure was, whether the request is inside its budget and rate limit, which pool member should serve it, and whether to fail over mid-flight. Busbar is an AI control plane: it makes those decisions in your infrastructure, before the request is forwarded, at a cost of about 38 microseconds. And it covers more than LLMs: chat, embeddings, images, audio, moderations, and rerank, across every provider.

One test separates the two words from marketing: a control plane that lives in your data path had better cost nothing to pass through, and every one of its decisions had better work air-gapped. Busbar passes both. It is a single Rust binary, complete on its own: everything it does here works air-gapped, and that is permanent. Nothing in it phones home to us.

Why not just use LiteLLM?

If you want an in-process Python router library with a big preconfigured catalog, LiteLLM is the right call, and we say so. On coverage itself the counts mislead: LiteLLM lists providers, Busbar speaks protocols, and any provider on one of the six, including every OpenAI-compatible endpoint, is a minute of YAML in your own providers.yaml. If this layer is production infrastructure carrying all of your AI traffic, the differences start to matter: no privileged protocol, a fault-attributed circuit breaker, and a runtime footprint measured in microseconds and megabytes. The full comparison, with their numbers cited to their docs, is at Busbar vs LiteLLM.

Why not just use OpenRouter?

Different side of a different line. OpenRouter is hosted: every prompt goes through their servers, and that's fine for many teams and disqualifying for others. Busbar runs in your infrastructure; the only external traffic is the provider calls you configure. The rest is at Busbar vs OpenRouter.

We're locked to one provider for compliance reasons. Why add a control plane now?

Because Busbar is valuable before the second provider exists. Say your data involves PHI and your BAA covers AWS, so everything runs through Bedrock. Point your Bedrock SDK's endpoint_url at Busbar today and change nothing else: same-protocol traffic passes through byte-for-byte, and since Busbar runs in your infrastructure, no new entity enters the data path. Your BAAs remain with your providers.

That bare passthrough is already a safer posture than calling the provider directly, before you enable a single feature. Your provider key moves out of every app deployment and into one process, so a leaked app credential is no longer a leaked provider account, and rotating the real key takes one restart. max_concurrent caps a runaway loop at your ceiling instead of your bill. Every request becomes visible in /metrics and /stats where direct SDK calls are invisible. And the request path is hardened: SSRF-guarded upstreams, constant-time token checks, body caps, secrets never logged. We run Busbar exactly this way ourselves: one provider, straight passthrough, no features enabled.

Turn on governance when you want more: per-team virtual keys with budget caps and rate limits, spend tracked per key and queryable, and a per-request audit webhook. Caps are enforced before a request is forwarded, atomically, so a runaway loop stops at the limit instead of showing up on the invoice. All of it on your one provider, today.

Then the constraint moves, as constraints do: you sign a direct BAA with Anthropic, or a cheaper model fits a workload, or a provider has a bad month. Adding that backend is a lane in config.yaml. Your application code, written against the Bedrock SDK, never changes and never learns which model answered. Without the control plane already in place, that same day starts with rewriting every LLM integration you own. The endpoint swap is a one-line change; the future it buys is free.

What happens if a provider fails mid-stream?

Before the first byte reaches your client, Busbar reroutes to the next lane in the pool, even on a streaming request. That boundary covers most real failures: connection errors, timeouts, rate limits, and 5xx responses all arrive before any content does.

After the first byte, no gateway or control plane can save the request, and you should distrust any that claims otherwise: a stream is a stateful continuation, a byte sent cannot be unsent, and a different provider cannot resume another's generation. What Busbar does instead is record the fault against the lane and emit a clean, native-format error in your protocol, so your client library fails fast instead of hanging. The full boundary is documented at In-flight failover.

Do I need Redis or a database?

No. Breaker state, rate windows, and routing state live in the process. Governance, if you enable it, uses an embedded SQLite file. There is nothing to provision, and "deploy" means copying one ~9 MB binary and a YAML file.

What does the Apache-2.0 license require from me?

Almost nothing, and that's the point. Busbar is licensed under the Apache License 2.0: a permissive license with no copyleft. Use it commercially, run it as an internal or customer-facing service, modify it privately, embed it in your own products. You are never required to publish your changes, your configs, or the applications that call it.

What it does ask: keep the license and copyright notices with the code if you redistribute it, and note significant changes if you distribute a modified version. In return you also get an express patent grant from every contributor. If your legal team wants it in writing, that's the license itself, and this paragraph isn't legal advice.