Speed, measured and published
We do not claim to be fast: we measure it every hour, against our own proxy and against the provider directly, and publish both numbers. The gap between them is what going through us costs.
Last measured: Nov 21, 2026, 3:16 PM
Last 7 days
| Model | First token (p50 / p95) | Tokens per second (p50) | What we add |
|---|---|---|---|
| auto | 500 / 2748 ms through our API 299 / 978 ms straight to the provider | 26.7 | 201 ms |
| deepseek-v4 | 613 / 4692 ms through our API 590 / 1806 ms straight to the provider | 10.9 | 23 ms |
| gemma-4-26b | 274 / 381 ms through our API 260 / 465 ms straight to the provider | 0.0 | 14 ms |
| glm-5.2 | 666 / 6889 ms through our API 449 / 3831 ms straight to the provider | 2.6 | 217 ms |
| gpt-oss-120b | 299 / 448 ms through our API 375 / 1006 ms straight to the provider | 24.3 | -76 ms |
| gpt-oss-20b | 435 / 612 ms through our API 359 / 472 ms straight to the provider | 14.9 | 76 ms |
| llama-3.3-70b | 284 / 947 ms through our API 269 / 536 ms straight to the provider | 66.0 | 15 ms |
| mistral-medium | 263 / 492 ms through our API 297 / 736 ms straight to the provider | 68.7 | -34 ms |
| mistral-small | 293 / 11319 ms through our API 241 / 10404 ms straight to the provider | 95.5 | 52 ms |
| qwen3-235b | 305 / 1342 ms through our API 304 / 627 ms straight to the provider | 64.2 | 1 ms |
| qwen3.5-397b | 503 / 1135 ms through our API 467 / 898 ms straight to the provider | 0.0 | 36 ms |
| qwen3.5-9b | 405 / 543 ms through our API 353 / 8694 ms straight to the provider | 0.0 | 52 ms |
| qwen3.6-27b | 781 / 12707 ms through our API 336 / 5565 ms straight to the provider | 0.0 | 445 ms |
| qwen3.6-35b | 312 / 452 ms through our API 298 / 526 ms straight to the provider | 0.0 | 14 ms |
| qwen3.8-27b | 344 / 777 ms through our API 407 / 5659 ms straight to the provider | 8.8 | -63 ms |
Last 30 days
| Model | First token (p50 / p95) | Tokens per second (p50) | What we add |
|---|---|---|---|
| auto | 424 / 1634 ms through our API 268 / 937 ms straight to the provider | 28.7 | 201 ms |
| deepseek-v4 | 461 / 2853 ms through our API 493 / 2941 ms straight to the provider | 8.3 | 23 ms |
| gemma-4-26b | 229 / 379 ms through our API 212 / 461 ms straight to the provider | 0.0 | 14 ms |
| glm-5.2 | 457 / 2468 ms through our API 256 / 1145 ms straight to the provider | 0.0 | 217 ms |
| gpt-oss-120b | 304 / 796 ms through our API 290 / 974 ms straight to the provider | 26.9 | -76 ms |
| gpt-oss-20b | 356 / 4217 ms through our API 271 / 1944 ms straight to the provider | 6.3 | 76 ms |
| llama-3.3-70b | 263 / 894 ms through our API 253 / 749 ms straight to the provider | 77.0 | 15 ms |
| mistral-medium | 271 / 674 ms through our API 250 / 825 ms straight to the provider | 67.8 | -34 ms |
| mistral-small | 258 / 7633 ms through our API 216 / 7549 ms straight to the provider | 103.6 | 52 ms |
| qwen3-235b | 306 / 1262 ms through our API 285 / 829 ms straight to the provider | 57.2 | 1 ms |
| qwen3.5-397b | 474 / 1020 ms through our API 433 / 843 ms straight to the provider | 0.0 | 36 ms |
| qwen3.5-9b | 298 / 14182 ms through our API 270 / 19121 ms straight to the provider | 0.0 | 52 ms |
| qwen3.6-27b | 423 / 26491 ms through our API 343 / 19766 ms straight to the provider | 0.0 | 445 ms |
| qwen3.6-35b | 291 / 526 ms through our API 275 / 667 ms straight to the provider | 0.0 | 14 ms |
| qwen3.8-27b | 355 / 5582 ms through our API 327 / 7296 ms straight to the provider | 7.9 | -63 ms |
How auto decides
87 requests named auto in the last 30 days. This is where they went and why, over every account at once — no single customer is visible in these numbers.
- Sent to the small model
- 0%
- Small-model answers passing checks
- —
- Default-model answers passing checks
- —
- Shadow agreement with the default model
- —
- Shadowed calls
- 0
Pass rates come from the automatic checks on aliases that point at auto: 0 checked answers in the window. A dash means too few to say.
The shadow figures come from keys whose owners switched shadow sampling on: one call in a hundred is also sent to the small model and the two answers compared, at the owner's expense and within their budget. Auto trusts a task to the small model on the shadow alone when at least 100 shadowed calls agree 95 % of the time, and keeps the default model when 100 or more agree under 80 %, whatever the checks say.
| Reason | What it means | Calls |
|---|---|---|
| router_off | The router is switched off: everything goes to the default model. | 0 |
| profile | A task profile: your alias's task type has a model that proved itself on it. | 0 |
| tools | The request declared tools, which the small model handles less reliably. | 0 |
| structured | JSON output was required. | 0 |
| long_answer | A long answer was asked for (max_tokens above the small model's comfort). | 0 |
| long_prompt | The prompt itself was long. | 87 |
| code | The prompt looks like code. | 0 |
| short | None of the above: a short, plain request, which is what the small model is for. | 0 |
Every answer that came through auto carries the reason in its X-Auto-Reason header.
Method
Every hour we send 3 requests per model, with the same prompt of about 800 input tokens and at most 600 output tokens, at temperature 0.7 so no answer can come from the cache. Time to first token is measured at the first chunk carrying text, not at the first byte. The same requests are repeated straight against the provider, from the same region.
The probe's source is in the repository: core/app/bench/probe.py