Skip to content
Keln

Reliability & SLOs

Keln measures every request you send against the two speed targets it sets for every provider. Your usage dashboard shows the latency your own requests got. This page lists the targets, what Keln measures, and what you see when a request fails.

What Keln measures

Keln reports the latency your own requests received:

  • TTFT (time to first token), hourly averages and maxima, per model, in the usage dashboard
  • Success rate, non-5xx fraction of your requests
  • Saturation behavior, when the pool is saturated you get a fast, retryable 503

Service objectives

Keln sets two speed targets for every provider, measured continuously on live traffic:

Objective Target
TTFT, streaming requests First token within 5 seconds, on standard-size prompts. Very large prompts are excluded because prefill time grows with prompt size.
Decode speed 50 tokens per second sustained

Routing works to meet these targets on every request.

What failure looks like

Condition What you see
Chosen route slower than its predicted TTFT usually nothing, Keln starts the same request on a second route
Node dies mid-stream usually nothing, the stream resumes from another node
No capacity for the model fast 503; retry with backoff
Every capable route tried and none served 502, safe to retry
A slow-but-working request not an error: Keln retries the attempt on faster routes (details)

The mechanics behind these numbers, with diagrams: Routing & failover ↗