Comparison

Busbar vs Bifrost

Bifrost is a real, working Go gateway, and we tested it fairly. Busbar sustains 67,059 req/s to Bifrost's 4,791 under a realistic 20 ms model delay (~14x), adds 83 µs p99 vs Bifrost's 1029 µs, and peaks at 22.6 MiB vs Bifrost's 0.9 GB. Where Bifrost is good, we say so; where its own published numbers didn't reproduce, we show our work.

Head-to-head

	Busbar	Bifrost
Added latency, p99 gateway minus direct	83 µs in-band on every response	claimed 11 µs measured 1029 µs ~12x slower see below
Sustained RPS @20ms model delay	67,059 req/s	4,791 req/s ~14x less
Max proxy RPS instant mock, raw forwarding	67,059 req/s	5,535 req/s ~12x less
Peak memory sustained load	22.6 MiB idle: 7.4 MiB	0.9 GB idle: 216.9 MiB

Why a gap this wide between two compiled proxies? The two systems don't do the same amount of work per request. Most of an LLM proxy's cost is (de)serializing JSON bodies, and Go's encoding/json is reflection-based and allocation-heavy, so every request pays GC pressure a Rust proxy doesn't. Busbar's edge is bigger still on same-protocol traffic, where it passes the body through byte-for-byte with no re-encoding. And every figure here uses unique bodies on both sides, so both gateways actually proxy (fire identical requests and a gateway can answer from cache without proxying, which is not a real proxy number).

Memory

Busbar idles at 7.4 MiB and peaks at 22.6 MiB; Bifrost idles at 216.9 MiB and peaks at 0.9 GB, a ~40x peak difference on the same box and load. Bifrost's pooled buffers (Go GC) fill under sustained load and don't release. Busbar runs on jemalloc with a background purge thread: buffers are freed when requests complete, and RSS falls to idle when the load stops.

The shape matters more than any single peak. Over onthebench's five-minute soak (300 s under load, then the load removed), Busbar's RSS holds flat: it settles at a ~22 MiB steady state with essentially no growth, then recovers toward its 7.4 MiB idle when the load stops. Bifrost's climbs the whole way: it never reaches a steady state, sitting at ~822 MiB and still rising at ~59 MiB/min at the end of the run. That is the neutral harness measuring, on the same box and the same recipe as every other gateway, what an unbounded memory profile looks like next to a bounded one.

Their "11 µs": and what the neutral harness measured

Bifrost publishes a "11 µs overhead." We quote it because hiding it would be the dishonest move, but we could not reproduce it, and neither can you: their own benchmark tool never emits it and their /metrics can't resolve it. Their docs say it "excludes JSON marshalling and the HTTP call," but for a network proxy that IS the work, and you can't measure a request's real cost without it from outside the process.

On the neutral m7g field box, added latency p99 (gateway minus direct, concurrency 1, instant mock) is 1029 µs for Bifrost vs 83 µs for Busbar, ~12x Busbar on the same axis, and about 94× the 11 µs Bifrost publishes, measured the same way for every gateway on the board.

We reached out to the Bifrost team asking how the 11 µs is instrumented and whether it's reproducible from their tooling. We will update if we hear anything.

Busbar's own compute (per-request processing time, NOT latency) is reported in-band on every response: Server-Timing: busbar;dur, ~33 µs p99 on the 16-core c7g scaling run, a different box than the m7g head-to-head above (how end-to-end added latency and busbar;dur relate).

Where Bifrost is genuinely good

It scales cleanly at low-to-moderate load. 100% success across the throughput sweep, linear scaling, low latency when it isn't saturated or filling its pools. It is a working gateway. It just isn't "The Fastest LLM Gateway" its own banner claims: Busbar sustains ~14x its throughput, adds ~12x less latency, and holds ~40x less peak memory.
Bundled governance/telemetry out of the box. Some of its per-request cost buys features Busbar keeps opt-in. If you want that always-on, it's there.

How this page is measured. Unless a figure says otherwise, every number here is from the neutral OSS harness (GetBusbar/benchmarking), an AWS m7g.4xlarge (Graviton3, 4-core pin), run the same way as /performance, on Busbar v1.5.5, measured 2026-08-26. Figures tagged "their published" are that project's own numbers; a core-scaling sweep, where one appears, is a separate test, on its own Busbar build, and is labelled as such. And you don't have to take our framing of the results: onthebench.ai publishes the field neutrally, every AI gateway measured identically, side by side, and anyone can re-run it.

← Performance overview · Busbar vs LiteLLM · Reproduce it