Busbar, in numbers

There’s a good conversation happening right now about what an AI control plane should cost you in memory, latency, and operational weight. Here are my answers, measured on the shipped binary rather than a roadmap.

Why a compiled control plane, and why now?

Because a control plane sits in the hot path of every request, and an interpreted, garbage-collected process pays for that on every call. It pays in startup weight, in memory that climbs with concurrency, and in GC pauses that show up as tail latency exactly when you’re busiest. A compiled, no-GC data plane removes all three. The industry agrees, which is why gateways are moving to Rust. I already did it: Busbar shipped as a single Rust binary in v1.0, and it’s in production today. There’s no migration to wait for.

How much memory does Busbar use?

About ~5.6 MB resident at idle. It stays there under load, because a single static binary with no interpreter and no garbage collector has no runtime heap to grow, and nothing that pauses to sweep it. Thousands of concurrent requests don’t move it. (Measured on v1.3.2, idle RSS, Apple Silicon, up from ~4.5 MB on v1.1.0, as the always-on Admin API and hook plane landed in 1.3.)

For reference, a widely-used gateway’s own published benchmark reports peak memory around 359 MB for its Python build and about 32 MB for its new Rust core. Those are different harnesses on different hardware, so read it as orders of magnitude rather than a stopwatch. The shape is the point: less to carry means less to go wrong at 3am.

How much does Busbar cost per request?

Two different numbers, and I keep them apart on purpose. Added latency is the end-to-end, wall-clock cost a caller actually sees (through-Busbar minus direct-to-provider). Per-request processing time is just the CPU Busbar itself burns, the microseconds it spends parsing, translating, and serializing, not the network and not the model. That processing time is a slice of the added latency, not the added latency itself, so I never call it “latency.”

Busbar measures the processing slice on its own clock and can report it in-band on every response via an opt-in Server-Timing: busbar;dur= header:

• 21 µs (0.021 ms) of CPU for a small call, p50 (busbar 1.5.2, c7g.8xlarge processing-time run)
• 84 µs (0.084 ms) of CPU for a full 12k-token cross-protocol translation, Anthropic in and a different protocol out, both directions (busbar 1.5.1, c7g.8xlarge, measured 2026-08-03)

And the tail stays tight: p99 sits at 33 µs for the same small call, well within a single-digit multiple of the median, because a no-GC request path has no pause to spike it. Full methodology and per-protocol numbers.

How much throughput?

Saturating two pinned cores, Busbar sustained 21,626 req/s (~10,813 per core) (busbar 1.5.1, c7g.8xlarge, measured 2026-08-03), and the Tokio runtime scales across cores from there.

Are these benchmarks reproducible?

Yes. The overhead harness (a mock upstream, Busbar, and a load client that times each request in microseconds) is checked in under bench/, and Busbar can report its own added latency in-band on every response via the opt-in Server-Timing header. You don’t have to trust my number. You can read Busbar’s overhead on your own traffic, in production, per request. That’s the number that can’t be cherry-picked.

How does Busbar compare?

Lower overhead. An order of magnitude less memory. Mine and a widely-used gateway’s, each from its own published benchmark.

• Overhead: Busbar adds about 21 µs (p50, busbar 1.5.2). The fastest-moving alternative reports about 50 µs for its Rust core (LiteLLM’s published rust-core overhead, cited 2026-07-09).
• Memory: Busbar sits around ~5.6 MB resident (idle RSS, v1.3.2, an older build; the current figure is ~7.79 MiB on busbar 1.5.0). That same alternative reports about 32 MB for its Rust build and about 359 MB for its Python one. Busbar is 6 to 64 times lighter.

Fair caveat: these are each side’s own numbers on its own hardware, so treat the overhead figures as same-class-or-lower rather than a controlled stopwatch. Memory barely moves with CPU, so that gap is real and architectural, not a hardware artifact.

And both sets of numbers are meant to be reproducible. Mine is: the harness is checked in under bench/, and Busbar can report its own added latency in-band on every response (an opt-in header), so you can verify it on your own traffic instead of taking my word for it.

I won’t call Busbar “the fastest control plane in existence,” because I haven’t benchmarked every compiled control plane on one machine. What the numbers on the table say today is clear: lower overhead, far less memory, and it shipped.

Does it speak my API?

Natively, in both directions. Six wire protocols (OpenAI, OpenAI Responses, Anthropic, Gemini, Amazon Bedrock, Cohere) through a superset intermediate representation. Busbar doesn’t flatten everything to one vendor’s shape, so native features survive the hop: Anthropic thinking blocks, structured-output schemas, Bedrock tool use. Point a Bedrock SDK at Busbar and reach an Anthropic backend, losslessly, streaming included. This is the one thing a Rust rewrite of an OpenAI-normalized gateway still won’t give you.

And when a provider fails?

Busbar fails over inside the request, before your client sees a byte, even mid-stream, across protocol families. A circuit breaker on every provider connection classifies each failure (provider outage, your bad request, context-length, auth or billing) and treats each differently, instead of retrying into a wall.

What do you actually deploy?

One file. A single static Rust binary, about 10.89 MB on disk (busbar 1.5.3, current release), with nothing interpreted and nothing garbage-collected in the request path. Linux and macOS on Intel and ARM, Windows on Intel. Your keys never leave your network. No v2, no migration, nothing to wait for.

Where to start

Busbar is open source at getbusbar.com. If you run multi-provider LLM traffic in production, I’d love to talk. I’m taking on design partners.