Skip to content

How it works

What happens between your request and your first token.

The request path

  1. You call the Keln API, an OpenAI-compatible endpoint. Your API key authenticates and selects your workspace (personal or team) for billing.
  2. Your request is normalized. Parameters are checked against what the target model verifiably supports; oversized prompts are rejected before billing.
  3. The router picks a route. Keln keeps a pool of serving capacity for each model and scores all of it continuously, on live health signals (queue depth, memory pressure, measured speed) and on what each route actually delivers under real traffic: latency, throughput, and error behavior. The router filters to capacity with real headroom for your model right now, then picks the route predicted fastest for your request, using predicted time-to-first-token for your exact prompt size, not averages. If your recent requests share a prompt prefix, routing prefers the capacity that already has it cached.
  4. Tokens stream back, in one canonical shape, reasoning traces included.

Reliability machinery (always on)

  • Predictive hedging, every route is held to its own predicted first-token window for your prompt size; when the wait meaningfully exceeds what that route should have delivered, a second route races the first. A hedged request bills once, the racing attempt is part of the platform service, not something you pay for.
  • Mid-stream failover, if a node fails or stalls while streaming, Keln re-prompts another node with the text generated so far and resumes your stream. You see one clean stream.
  • No give-up timer, Keln does not time out a request that is still being served. Slow attempts are hedged and re-raced, and first-token patience scales with prompt size.

Details and diagrams: Routing & failover

Quality machinery (always on)

  • Continuous verification. Keln verifies that every node serves the model and quality level it claims, by comparing its outputs against trusted reference capacity, using synthetic probes. Nodes that fail are excluded from routing automatically. How verification works
  • Supply-blind by design. Requests are served under one price and one SLA per model; the serving provider is not exposed or selectable.

Billing

Billing counts the engine-reported token usage on each request, prompt tokens, completion tokens, and cached prompt tokens, which bill at the per-model cached rate. Reasoning tokens are generated output: they bill at the output rate, and the split is itemized as usage.completion_tokens_details.reasoning_tokens. Every charge is visible per-request in your usage dashboard, and your prepaid balance lives on Billing.