Speed, measured and published

We do not claim to be fast: we measure it every hour, against our own proxy and against the provider directly, and publish both numbers. The gap between them is what going through us costs.

Last measured: Nov 21, 2026, 3:16 PM

Last 7 days

ModelFirst token (p50 / p95)Tokens per second (p50)What we add
auto
500 / 2748 ms through our API
299 / 978 ms straight to the provider
26.7201 ms
deepseek-v4
613 / 4692 ms through our API
590 / 1806 ms straight to the provider
10.923 ms
gemma-4-26b
274 / 381 ms through our API
260 / 465 ms straight to the provider
0.014 ms
glm-5.2
666 / 6889 ms through our API
449 / 3831 ms straight to the provider
2.6217 ms
gpt-oss-120b
299 / 448 ms through our API
375 / 1006 ms straight to the provider
24.3-76 ms
gpt-oss-20b
435 / 612 ms through our API
359 / 472 ms straight to the provider
14.976 ms
llama-3.3-70b
284 / 947 ms through our API
269 / 536 ms straight to the provider
66.015 ms
mistral-medium
263 / 492 ms through our API
297 / 736 ms straight to the provider
68.7-34 ms
mistral-small
293 / 11319 ms through our API
241 / 10404 ms straight to the provider
95.552 ms
qwen3-235b
305 / 1342 ms through our API
304 / 627 ms straight to the provider
64.21 ms
qwen3.5-397b
503 / 1135 ms through our API
467 / 898 ms straight to the provider
0.036 ms
qwen3.5-9b
405 / 543 ms through our API
353 / 8694 ms straight to the provider
0.052 ms
qwen3.6-27b
781 / 12707 ms through our API
336 / 5565 ms straight to the provider
0.0445 ms
qwen3.6-35b
312 / 452 ms through our API
298 / 526 ms straight to the provider
0.014 ms
qwen3.8-27b
344 / 777 ms through our API
407 / 5659 ms straight to the provider
8.8-63 ms

Last 30 days

ModelFirst token (p50 / p95)Tokens per second (p50)What we add
auto
424 / 1634 ms through our API
268 / 937 ms straight to the provider
28.7201 ms
deepseek-v4
461 / 2853 ms through our API
493 / 2941 ms straight to the provider
8.323 ms
gemma-4-26b
229 / 379 ms through our API
212 / 461 ms straight to the provider
0.014 ms
glm-5.2
457 / 2468 ms through our API
256 / 1145 ms straight to the provider
0.0217 ms
gpt-oss-120b
304 / 796 ms through our API
290 / 974 ms straight to the provider
26.9-76 ms
gpt-oss-20b
356 / 4217 ms through our API
271 / 1944 ms straight to the provider
6.376 ms
llama-3.3-70b
263 / 894 ms through our API
253 / 749 ms straight to the provider
77.015 ms
mistral-medium
271 / 674 ms through our API
250 / 825 ms straight to the provider
67.8-34 ms
mistral-small
258 / 7633 ms through our API
216 / 7549 ms straight to the provider
103.652 ms
qwen3-235b
306 / 1262 ms through our API
285 / 829 ms straight to the provider
57.21 ms
qwen3.5-397b
474 / 1020 ms through our API
433 / 843 ms straight to the provider
0.036 ms
qwen3.5-9b
298 / 14182 ms through our API
270 / 19121 ms straight to the provider
0.052 ms
qwen3.6-27b
423 / 26491 ms through our API
343 / 19766 ms straight to the provider
0.0445 ms
qwen3.6-35b
291 / 526 ms through our API
275 / 667 ms straight to the provider
0.014 ms
qwen3.8-27b
355 / 5582 ms through our API
327 / 7296 ms straight to the provider
7.9-63 ms

How auto decides

87 requests named auto in the last 30 days. This is where they went and why, over every account at once — no single customer is visible in these numbers.

Sent to the small model
0%
Small-model answers passing checks
—
Default-model answers passing checks
—
Shadow agreement with the default model
—
Shadowed calls
0

Pass rates come from the automatic checks on aliases that point at auto: 0 checked answers in the window. A dash means too few to say.

The shadow figures come from keys whose owners switched shadow sampling on: one call in a hundred is also sent to the small model and the two answers compared, at the owner's expense and within their budget. Auto trusts a task to the small model on the shadow alone when at least 100 shadowed calls agree 95 % of the time, and keeps the default model when 100 or more agree under 80 %, whatever the checks say.

Why auto routed each request, with the number of calls per reason
ReasonWhat it meansCalls
router_offThe router is switched off: everything goes to the default model.0
profileA task profile: your alias's task type has a model that proved itself on it.0
toolsThe request declared tools, which the small model handles less reliably.0
structuredJSON output was required.0
long_answerA long answer was asked for (max_tokens above the small model's comfort).0
long_promptThe prompt itself was long.87
codeThe prompt looks like code.0
shortNone of the above: a short, plain request, which is what the small model is for.0

Every answer that came through auto carries the reason in its X-Auto-Reason header.

Method

Every hour we send 3 requests per model, with the same prompt of about 800 input tokens and at most 600 output tokens, at temperature 0.7 so no answer can come from the cache. Time to first token is measured at the first chunk carrying text, not at the first byte. The same requests are repeated straight against the provider, from the same region.

The probe's source is in the repository: core/app/bench/probe.py