Scales to 156k+ req/s on 16 cores

One static binary. Linear with cores. Microseconds of overhead, single-digit-MB idle.

Throughput 9,750+ rps per core, linear · 156,223 on 16 cores

Added latency 33 µs busbar;dur p99, one request (v1.4.1)

Memory 8.3 MB resident, idle · one static binary (v1.4.1)

Add cores, add capacity — linearly

Busbar's request path is CPU-bound on the work that matters (parse, translate, serialize) and nothing else — no GC, no database in the hot path, no per-request lock. So throughput tracks the cores you give it, in a straight line. Measured on one c7g.8xlarge (32 vCPU Graviton3), driving unique requests through a fast upstream (so every request is real proxy work, not a cache hit):

req/s (scales linearly) added latency (flat)

Throughput (green) climbs in a straight line — about 9,764 req/s per core — while Busbar's own added latency (busbar;dur, amber) stays flat at ~33 µs no matter the load. 2 cores 19k → 16 cores 156k, 100% success, on one c7g.8xlarge box.

And it stays flat and fast until it runs out of CPU — not until it falls over. At 16 cores busbar delivers every request you offer, at sub-millisecond added latency (~33 µs p99), right up to the full 156k/s; past that it's simply out of CPU budget (add cores and the ceiling moves up), never unstable. Busbar reports its own added time in-band on every response (Server-Timing: busbar;dur), so you can read the 33 µs on your own traffic instead of trusting us.

How that stacks up

We stand on our own numbers — but here's the short version against the two alternatives people ask about most. Two separate comparisons to Busbar, not a ranking of Bifrost against LiteLLM. Figures we measured are tagged we measured; each project's own published figures their published.

vs Bifrost we measured throughput · memory · same box, real proxy ~9× the throughput Same box, same fast upstream, unique traffic (real proxy work — no cache). On 16 cores Busbar sustains 156,223 req/s where Bifrost does 18,346 — about 9× — at a fraction of the memory. Both scale with cores; Busbar sustains ~9× the throughput per core on identical real-proxy traffic. The full head-to-head → vs LiteLLM their published p99 overhead · memory · their published specs ~394× lower p99 overhead LiteLLM's published p99 overhead is 13 ms — Busbar's is 33 µs, at ~43× less memory and ~134× the throughput per vCPU. We take their number at their word. The full comparison →

Two kinds of number, labelled as such: the Bifrost figures we measured ourselves on identical hardware (same box, same fast upstream, unique traffic so it's real proxy work — not a cache hit); the LiteLLM figures are their own published numbers on their own hardware, taken at their word. Every Busbar number reads from one version-stamped source (facts.ts) and is re-measured each release — the full method and raw data are linked below.