Time to first token, p50 vs p95: 28 LLM APIs in one week of production traffic
Data: week of 2026-11-02 to 2026-11-08 (UTC), from the weekly benchmarks page of the byesu API gateway. The numbers describe requests that went through that gateway, on its routes, with its customers' prompts. They are not a measurement of each vendor's API called directly.
Short answer
Median time to first token (TTFT) was under 2 seconds for every Claude model and between 3.6 and 7.8 seconds for most GPT and Gemini models. The bigger difference is in the tail. For the Claude models the 95th percentile was about 2x the median. For the GPT, Grok, Kimi and GLM models it was 4x to 12x the median, which means one streaming request in twenty waited 18 to 49 seconds for its first token. If you size timeouts or judge a model by the median, you miss this.
Why the median is not enough
A chat interface feels as slow as its slow requests. If the median TTFT is 4 seconds and the p95 is 45, a user who sends twenty messages will, on average, hit one that sits blank for most of a minute. Averages hide this too: a mean of 12 seconds can be produced by many 4-second waits and a few very long ones.
The numbers
Streaming requests only. TTFT is measured at the gateway, from receiving the request to the first output token, and includes thinking time for reasoning models.
| Model | TTFT p50 | TTFT p95 | p95 / p50 |
|---|---|---|---|
claude-opus-4-7 | 1.1 s | 2.1 s | 1.8 |
claude-sonnet-4-6 | 1.4 s | 2.1 s | 1.5 |
claude-opus-4-8 | 1.5 s | 3.1 s | 2.1 |
claude-sonnet-5 | 1.5 s | 3.2 s | 2.2 |
claude-opus-4-6 | 1.6 s | 3.4 s | 2.1 |
claude-opus-5 | 1.7 s | 3.4 s | 2.0 |
claude-fable-5-1 | 1.7 s | 3.5 s | 2.1 |
claude-haiku-5-5 | 1.7 s | 3.9 s | 2.3 |
claude-opus-5-5 | 1.9 s | 4.5 s | 2.4 |
claude-sonnet-5-5 | 1.9 s | 4.4 s | 2.3 |
claude-haiku-4-5 | 2.0 s | 3.9 s | 2.0 |
grok-4.5 | 2.5 s | 18.7 s | 7.4 |
grok-4.7 | 3.3 s | 22.9 s | 7.0 |
gemini-3-flash | 3.6 s | 12.2 s | 3.4 |
deepseek-v4.1-flash | 3.6 s | 8.0 s | 2.2 |
gemini-3.6-flash | 3.6 s | 5.4 s | 1.5 |
gpt-5.6-sol | 3.7 s | 18.7 s | 5.1 |
gpt-5.6-luna | 4.1 s | 49.1 s | 12.0 |
gpt-6-astra | 4.3 s | 19.5 s | 4.5 |
gpt-5.6-terra | 4.3 s | 27.7 s | 6.4 |
gpt-6.1-sol | 4.6 s | 21.9 s | 4.8 |
gpt-6-sol | 5.4 s | 44.5 s | 8.2 |
kimi-k3 | 5.5 s | 27.2 s | 5.0 |
gemini-3.8-flash-high | 5.5 s | 19.2 s | 3.5 |
glm-5.3-flash | 5.9 s | 38.1 s | 6.5 |
gemini-3.8-flash | 7.0 s | 33.3 s | 4.7 |
gpt-6-luna | 7.8 s | 46.1 s | 5.9 |
gemini-4-argon | 12.1 s | 71.4 s | 5.9 |
By vendor, taking the median across each vendor's models in the table:
| Vendor | Models | Median of p50 | Median of p95 |
|---|---|---|---|
| Anthropic | 11 | 1.7 s | 3.4 s |
| xAI | 2 | 2.9 s | 20.8 s |
| DeepSeek | 1 | 3.6 s | 8.0 s |
| OpenAI | 7 | 4.3 s | 27.7 s |
| 5 | 5.5 s | 19.2 s | |
| Moonshot AI | 1 | 5.5 s | 27.2 s |
| Zhipu AI | 1 | 5.9 s | 38.1 s |
What stands out
Two shapes of distribution. The Claude models and two of the Gemini Flash models have a tight tail: p95 is 1.5 to 2.4 times the median. Most of the others have a long tail, with p95 at 4 to 12 times the median.
Much of the long tail is thinking time. For reasoning models the clock runs while the model thinks, before any visible token. A model that thinks for 40 seconds on a hard prompt and 2 seconds on an easy one will show exactly this pattern, and that is a property of the workload as much as of the API. It still matters to whoever is watching an empty screen.
The median ranks models differently than the tail does. gpt-5.6-luna has a better median than gpt-6.1-sol (4.1 s against 4.6 s) and a p95 more than twice as long (49.1 s against 21.9 s). grok-4.5 has the fastest median outside the Claude family and a p95 of 18.7 seconds.
What to do with this
- Set timeouts from the tail. A first-token timeout of three times the median would have cut off more than 5% of successful requests on most non-Claude models in this table.
- Show progress during thinking. If your model streams reasoning summaries or progress events, surface them. A spinner for 30 seconds reads as broken.
- Measure your own traffic. These are someone else's prompts. Your p95 depends on your prompt length, your
max_tokens, your reasoning settings and your region. A small open-source script for this is described in How to measure LLM API latency yourself.
Method and limits
- Source: per-request logs of the byesu gateway for the week, successful requests only. Percentiles come from a 1-in-10 sample; a model needs at least 100 sampled streaming requests to appear, which is why 28 of the 33 text models on the page are listed here.
- Not a controlled benchmark. Prompts differ by model. A model people use for long agent runs is given harder prompts than one used for autocomplete.
- Gateway in the path. Time includes the gateway's own routing. A request that was retried on another route counts once, with the full wait.
- One week. Tails move from week to week. The benchmarks page shows the last four weeks and offers the data as CSV and JSON.