Comparison

Busbar vs Bifrost

Bifrost is a real, working Go gateway, and we tested it fairly: the identical benchmark on both, same machine, same method, the same 2-to-16 core sweep, each driven with unique request traffic so every request is real proxy work. Both scale linearly with cores. But Busbar sustains about 9× the throughput per core on identical real-proxy traffic, and under sustained large-payload load Bifrost's memory runs away into an OOM while Busbar stays bounded. Where Bifrost is good, we say so; where its own published numbers didn't reproduce, we show our work.

The benchmark we ran (both setups identical)

One c7g.8xlarge (32 vCPU Graviton3) (Graviton has no hyperthreading, so one vCPU is one real core — honest per-core scaling). The gateway under test is pinned to N cores; a dedicated load generator and a fast mock upstream get their own cores, so the gateway is never starved and the mock (~250k req/s ceiling) is never the bottleneck. Critically, every request carries a unique body — identical bodies let a gateway cache and answer without proxying, which inflates throughput past what the upstream can even serve. We swept both gateways 2 → 16 cores and recorded req/s at 100% success.

Throughput scales linearly — for both

req/s at 100% success vs cores, same box, real proxy traffic. Both lines are straight — Busbar at ~9750+ req/s per core, Bifrost at ~1,147. Hover any point for its numbers.

Head-to-head at 16 cores

	Busbar	Bifrost
Throughput (16 cores, 100% success)	156,223 req/s	18,346 req/s ~9× less
Per core	~9,750 req/s	~1,147 req/s ~9× less
Added latency, p99 its own overhead	~33 µs flat, in-band on every response	claimed 11 µs measured ~1.4 ms ~42× slower see below
Memory under load 150 KB bodies, sustained	~1.2 GB plateaus, falls to idle	~19 GB ~15× more, never released
Scales with cores?	Yes — linearly	Yes — linearly

Both scale linearly on one box, so this isn't "Bifrost can't scale." It's that it doesn't scale nearly as well. On identical 16 cores Busbar carries 156,223 req/s to Bifrost's 18,346, about 9× the throughput. Matching Busbar takes ~9× the cores (or 9× the machines). And under real large-payload load the memory gap is far wider still, which is the next section.

Why a ~9× gap between two compiled proxies? A gap this size isn't Bifrost being broken — it's the two systems not doing the same amount of work per request. Most of the cost in an LLM proxy is (de)serializing JSON bodies, and Go's encoding/json is reflection-based and allocation-heavy, so every request pays GC pressure that a Rust proxy doesn't. Busbar's edge is bigger still on same-protocol traffic, where it passes the body through byte-for-byte with no re-encoding at all. The other half of the story is the direction of the comparison: Bifrost's own headline numbers come from a setup where identical requests can be served from cache without proxying (see the caveat at the bottom of this page), whereas every figure here uses unique bodies on both sides, so both gateways are actually proxying. Same real work, measured the same way — the gap is efficiency per request, not a trick.

Memory: a bounded plateau vs a sawtooth that never comes down

Bifrost publishes a 3.34 GB peak-memory figure from its own benchmark. A short, small-payload run never reproduces it (the pools don't fill, and a snapshot right after boot reads ~150 MB), so we filled the pools the way their big-payload benchmark does: unique 150 KB request bodies, 1,500 concurrent, held for minutes, sampling resident memory throughout. Same box, same mock, Bifrost on its own documented pool config (initial_pool_size 15000, buffer_size 20000) against Busbar 1.4.0.

Resident memory under sustained 150 KB load, same box. Bifrost's pooled buffers jump to a ~19 GB sawtooth (Go GC), well above its own published 3.34 GB (dashed). The load stops at 300s: Busbar returns to ~252 MB within ~30s; Bifrost holds its pools. (Bifrost's line is measured to 222s, then held flat, since its footprint doesn't shrink.)

Two different behaviours. Busbar's memory is a bounded working set: it holds a few copies of each in-flight request body, so peak tracks concurrency × payload and a completed request frees its buffers. The same 150 KB payload at concurrency 200 uses only ~140 MB. Busbar 1.4.0 runs on jemalloc with a background purge thread, so when the load stops the pages go back to the OS and RSS falls to ~252 MB.

Bifrost's memory is its pools. They fill under load, sawtooth on Go's GC at ~19 GB (about 15× Busbar, and already ~5× their own published 3.34 GB), and they don't come back when the load stops. It also scales with payload: at 400 KB bodies it held ~43 GB, and at 300 KB with 2,500 concurrent it climbed past 45 GB and, still rising, OOM-killed the 61 GB box out from under itself.

One caveat, stated plainly, because it's the fair question to ask: we tested Bifrost on its documented pool config (initial_pool_size 15000, buffer_size 20000) — the same settings behind its own published 3.34 GB figure — not on a memory-minimized tuning. Bifrost does expose pool-sizing knobs, and smaller values would cap the ceiling lower; that's a real lever a production deployment could pull. But two things don't change with it: the config Bifrost itself ships and benchmarks already balloons well past its own published number, and the underlying behaviour is a retained pool on Go's GC — turning the knob down trades burst headroom for a lower cap, it doesn't make the memory a bounded, self-releasing working set the way Busbar's is. The full harness and raw CSVs are in bench/memory/ — retune it yourself and the shape holds.

Their "11 µs" — and what we actually measured

Bifrost publishes an "11 µs overhead." We quote it because it's so far from anything an outside test produces that hiding it would be the dishonest move — but we could not reproduce it, and neither can you: their own benchmark tool never emits it and their /metrics can't resolve it. Their docs say it "excludes JSON marshalling and the HTTP call" — but for a network proxy that IS the work, and you can't measure a request's real cost from outside the process. So we measured Bifrost the only way anyone outside their code can: end-to-end, same t3.xlarge box, same mock, 1,000 RPS (unsaturated, so p99 ≈ per-request cost). Bifrost's p99 is 2.19 ms against Busbar's 0.73 ms on the identical path — so Bifrost adds ~1.4 ms, roughly 132× its published 11 µs. We may be missing an internal instrument we can't see, but on the wire the number a user actually pays is milliseconds, not microseconds. We reached out to the Bifrost team asking how the 11 µs is instrumented and whether it's reproducible from their tooling. We will update if we hear anything. Busbar's own added time, by contrast, is in-band and checkable on every response: Server-Timing: busbar;dur, ~33 µs p99, flat under any load.

A note on the two Busbar numbers. This section quotes Busbar's 0.73 ms end-to-end p99, but the headline figure elsewhere on the site is ~33 µs. They measure different things and both are real: the 0.73 ms is the whole round trip on this box — client → gateway → mock upstream → back, network hops and the mock's own service time included — and Busbar's ~33 µs is the slice of that which Busbar itself adds. We use the end-to-end number here because it's the only apples-to-apples way to compare against Bifrost from outside its process; the in-band Server-Timing figure is what you'd read on your own traffic to isolate Busbar's cost.

Where Bifrost is genuinely good

It scales cleanly at low-to-moderate load. 100% success across the throughput sweep, linear scaling, low latency when it isn't saturated or filling its pools. It is a working gateway. It just isn't "The Fastest LLM Gateway" its own banner claims: on identical hardware, Busbar does ~9× its throughput, holds ~15× less memory under sustained load, adds ~42× less latency, and doesn't OOM.
It scales vertically. Add cores, it adds throughput in a straight line, same as Busbar. The difference is efficiency per core, not whether it scales.
Bundled governance/telemetry out of the box. Some of its per-request cost buys features Busbar keeps opt-in. If you want that always-on, it's there.

Methodology

One c7g.8xlarge (32 vCPU Graviton3), unique request bodies (real proxy, no cache); gateway pinned to N cores; DEDICATED 12-core load gen + 4-core mock so the gateway is never starved. Busbar 1.4.1 and Bifrost 1.6.4. Raw per-core CSVs and the load generator are archived with the site under bench/; the full method (and every gotcha) is in the benchmark runbook. Re-run it and you'll get the same shape. Measured 2026-07-19.

← Performance overview · Busbar vs LiteLLM · Reproduce it