Scales to 209k+ req/s on 16 cores
One static binary, linear with cores. Microseconds of added latency, single-digit-MB idle.
Add cores, add capacity, linearly
Busbar's request path is CPU-bound (parse, translate, serialize), no GC and no database in the hot path, so throughput scales linearly with cores.
busbar;dur: CPU, not latency) stays flat at
~33 µs, reported in-band on every response (v1.5.1).
This is a single-box linearity demo: the load generator and mock run on the same 32-core box as Busbar (8 cores each), so this per-core rate is conservative, depressed by co-located contention rather than a clean ceiling. The isolated per-core ceiling is a separate experiment: ~16,765 req/s per core, Busbar alone on a 4-core pin with no co-tenant load (onthebench (v1.5.5)).
How that stacks up
We stand on our own numbers, but here's the short version against the two alternatives people ask about most. Figures we measured are tagged we measured; each project's own published figures their published.
Throughput runs 67,059 req/s to their 48,354. Memory isn't close: 7.4 MB idle against ~253 MB (onthebench (v1.5.5)).
And their Rust build is still beta (December 2026), covering one Anthropic
/v1/messages endpoint; the rest runs through the Python sidecar. Busbar ships
six-protocol translation today, any SDK to any backend, governance in the core.
The full comparison → vs Bifrost we measured throughput · memory · same box, real proxy ~14× the throughput
Same box, same fast upstream, unique traffic (real proxy work, no cache), both
gateways measured identically by onthebench (v1.5.5). Busbar sustains 67,059
req/s at a realistic 20 ms model delay where Bifrost does 4,791,
about 14×, at a fraction of the memory.
The full head-to-head → vs LiteLLM (Python) their published p99 overhead · memory · their published specs ~394× lower p99 overhead
LiteLLM's published p99 overhead is 13 ms. Busbar's is
33 µs of CPU (busbar;dur p99, per-request processing, 16-core c7g scaling run (v1.5.1)), at ~48× less memory and
~179× the throughput per vCPU. We take their number at their word.
The full comparison → Our benchmark: one clean box per gateway we measured
Every gateway on its own fresh, identical box: one dedicated machine each, so no run bleeds into the next. Live on the neutral field at onthebench.ai.
| Gateway | Added latency (end-to-end) p99 | Throughput (RPS) | Memory idle / peak |
|---|---|---|---|
| BusbarBusbar | 83 µs | 67,059 | 7.422 / 23 MiB |
| LiteLLM · Rust | 106 µs | 48,354 | 252.621 / 258 MiB |
| AISIX (api7) | 170 µs | 34,945 | 99.969 / 133 MiB |
| Higress | 234 µs | 26,098 | 876.25 / 883 MiB |
| agentgateway | 235 µs | 24,672 | 24.844 / 47 MiB |
| Helicone | 294 µs | 14,941 | 42.898 / 57 MiB |
| APISIX | 343 µs | 29,172 | 189.602 / 227 MiB |
| Kong | 389 µs | 21,520 | 403.473 / 621 MiB |
| Bifrost | 1,029 µs | 4,791 | 216.934 / 906 MiB |
| GoModel | 2,309 µs | 1,738 | 50.797 / 86 MiB |
| Portkey | 3,720 µs | 855 | 123.848 / 249 MiB |
| LiteLLM · Python | 8,143 µs | 163 | 744.813 / 779 MiB |
Added latency = gateway p99 − direct-to-mock p99 (c1). Throughput = sustained RPS at a realistic 20 ms model delay. Memory = idle / peak RSS under load; did not serve = couldn't hold the endpoint. Measured 2026-08-26.
Green is Busbar's real work, timed
on its own clock (busbar;dur), no wire in it. Grey is kernel + TCP time every
proxy pays; co-locate Busbar with your app and it's sub-millisecond, and against an ~800 ms
model call all of it vanishes.
we measured is our number on identical hardware; their published is that project's own figure, taken at their word.