Skip to content

Streaming

Set "stream": true and responses arrive as server-sent events in OpenAI chunk format, byte-shape compatible with every OpenAI SDK's streaming mode.

The stream

data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"}}]}
data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"Let me think","reasoning":"Let me think"}}]}
data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"The answer"}}]}
data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" is 391."}}]}
data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
  • delta.content, answer tokens.
  • delta.reasoning_content, thinking tokens, when the model reasons (one canonical field on every model); mirrored in delta.reasoning for OpenRouter/Vercel-style readers.
  • delta.tool_calls, streamed tool-call fragments, standard OpenAI shape.
  • The stream always terminates with data: [DONE].

Usage in the final chunk

{ "stream_options": { "include_usage": true } }

adds a final chunk carrying the complete usage block , prompt, completion, and cached token counts for the whole response, plus cost (this response's exact spend in USD) and, on reasoning responses, completion_tokens_details.reasoning_tokens.

OpenAI-shaped, not a byte-for-byte copy

Chunks follow the OpenAI chunk schema and every OpenAI SDK reads them unmodified, but Keln is a relay, not a pass-through pipe. Events carrying a reasoning trace are re-serialized (the trace is mirrored into both reasoning_content and reasoning), and the final usage frame and [DONE] are emitted by Keln itself. Frames Keln synthesizes, the final usage frame, a terminal error frame, carry the stream id chatcmpl-keln, which can differ from the id on the content chunks. Read the id per frame rather than assuming one id across the whole stream.

Mid-stream failures

If serving capacity fails mid-generation, the stream resumes on another node; the final usage chunk covers the whole response. If every recovery attempt fails after the stream started (the HTTP status is committed to 200 by then), the stream ends with one terminal chunk carrying the error, then [DONE]:

data: {"id":"…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"error"}],"error":{"message":"…","type":"keln_error","code":"stream_interrupted"}}

Retrying is safe; only delivered tokens bill. A stream never ends without either a normal finish_reason or this chunk.

Non-streaming

Omit stream (or set false) and the same request returns one JSON body. Streaming and non-streaming cost the same, there are no per-request fees either way.