Skip to content
Keln

How Keln works

What happens between your request and your first token.

The request path

  1. You call the Keln API, an OpenAI-compatible endpoint. Your API key authenticates the request. The key belongs to a workspace, personal or team, and usage bills to that workspace.
  2. Keln normalizes the request. It checks each parameter against what the model is confirmed to support on Keln, and rejects prompts that exceed the context window before any tokens are billed.
  3. Keln picks a route. It keeps a pool of serving capacity for each model and scores it continuously on live health signals (queue depth, memory pressure, measured speed) and on delivered latency, throughput, and error rate. For your request it filters to capacity with headroom for your model, then picks the route predicted fastest, using predicted TTFT for your exact prompt size. If your recent requests share a prompt prefix, Keln prefers the capacity that already has it cached.
  4. Keln streams tokens back in one canonical shape, reasoning trace included.

Reliability machinery (always on)

  • Predictive hedging. Keln predicts a TTFT for each route at your prompt size. If a route's first token takes meaningfully longer than predicted, Keln starts the same request on a second route and cancels the slower attempt. You are billed once.
  • Mid-stream failover. If a node fails or stalls while streaming, Keln re-prompts another node with the tokens generated so far and resumes the stream. You receive one continuous response.
  • No give-up timer. Keln does not time out a request that is still being served. It retries slow attempts on faster routes, and the wait it allows for a first token grows with prompt size.

Details and diagrams: Routing & failover

Quality machinery (always on)

  • Continuous verification. Keln verifies that every node serves the model and quality level it claims, by comparing its outputs against trusted reference capacity, using synthetic probes. Nodes that fail are excluded from routing automatically. How verification works
  • White-labeled by design. Every model has one price and one SLO. Keln does not expose the serving provider, and you cannot select one.

Billing

Keln bills the engine-reported token usage on each request: prompt tokens, completion tokens, and cached prompt tokens at the per-model cached rate. Reasoning tokens bill at the output rate, itemized as usage.completion_tokens_details.reasoning_tokens. The usage dashboard shows every charge per request, and your prepaid balance is on the Billing page.

Where to go next