Reliability & SLOs

Reliability on Keln has two parts: the machinery that keeps requests fast (routing & failover), and delivered numbers, measured on your traffic and shown in your dashboard.

What we measure

Keln reports the latency your own requests received:

• TTFT (time to first token), hourly averages and maxima, per model, in the usage dashboard
• Success rate, non-5xx fraction of your requests
• Saturation behavior, when the pool is saturated you get a fast, retryable 503

Service objectives

During the current phase, published SLOs are targets under measurement, displayed next to delivered numbers, tightened as the fleet grows:

Objective	Target
TTFT, streaming requests	p95 within the per-model target shown on its rate-card entry
Availability	99.9% monthly

Automatic service credits against these SLOs ship when targets graduate from "under measurement" to committed, changes are announced in advance.

Published speed-target performance

The model catalog shows a rolling 7-day figure: the percentage of served requests that met each speed target, first token within 5 seconds and decode at 50 tokens/second or faster. Methodology:

• Measured by Keln on every real served request, across the entire pool, not synthetic probes.
• The first-token figure covers standard-size prompts (very large prompts are excluded from the target, as in routing, because prefill time grows with prompt size).
• The figure is withheld below a minimum sample size.

Routing works to meet these targets on every request, and delivered performance against them is published.

What failure looks like

Condition	What you see
Chosen route late against its predicted first-token window	usually nothing, a hedge races a second route
Node dies mid-stream	usually nothing, the stream resumes from another node
No capacity for the model	fast 503; retry with backoff
Every capable route tried and none served	502, safe to retry
A slow-but-working request	not an error: the attempt is hedged and re-raced (details)