# Streaming

> Server-sent events, the final usage chunk, keep-alives and mid-stream errors.

Set `"stream": true` to receive the response as [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events) while the model generates it. The response has `Content-Type: text/event-stream`.

## With the OpenAI SDKs

```python
stream = client.chat.completions.create(
    model="google/gemini-3.8-flash",
    messages=[{"role": "user", "content": "Explain TCP slow start."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\ncost:", chunk.usage.cost)
```

```js
const stream = await client.chat.completions.create({
  model: "google/gemini-3.8-flash",
  messages: [{ role: "user", content: "Explain TCP slow start." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log("\ncost:", chunk.usage.cost);
}
```

Check that `choices` isn’t empty before reading it: the final usage chunk has no choices.

## Event format

Each event is a line starting with `data:` followed by a JSON chat completion chunk, then a blank line. The stream ends with `data: [DONE]`.

```text
data: {"id":"gen-01M67FGKKVQ8ZK2M3N5P7R9S1V","object":"chat.completion.chunk","created":1793400000,"model":"google/gemini-3.8-flash","provider":"Google AI Studio","choices":[{"index":0,"delta":{"role":"assistant","content":"TCP"},"finish_reason":null}]}

data: {"id":"gen-01M67FGKKVQ8ZK2M3N5P7R9S1V","object":"chat.completion.chunk","created":1793400000,"model":"google/gemini-3.8-flash","provider":"Google AI Studio","choices":[{"index":0,"delta":{"content":" slow start"},"finish_reason":null}]}

data: {"id":"gen-01M67FGKKVQ8ZK2M3N5P7R9S1V","object":"chat.completion.chunk","created":1793400000,"model":"google/gemini-3.8-flash","provider":"Google AI Studio","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"gen-01M67FGKKVQ8ZK2M3N5P7R9S1V","object":"chat.completion.chunk","created":1793400000,"model":"google/gemini-3.8-flash","provider":"Google AI Studio","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":240,"total_tokens":252,"prompt_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"reasoning_tokens":0},"cost":0.000909,"is_byok":false}}

data: [DONE]
```

Every chunk carries the same `id` (the generation id, also sent in the `x-clonvo-generation-id` response header), the canonical `model` id that is answering, and **`provider`**, the display name of the provider serving the request.

### Deltas

| Field | Description |
| --- | --- |
| `delta.role` | `assistant`, in the first chunk. |
| `delta.content` | The next piece of text. |
| `delta.reasoning` | **Extension.** The next piece of reasoning text, for models that stream it. |
| `delta.tool_calls` | Tool call fragments. Concatenate `function.arguments` by `index`. See [Tool calling](https://clonvo.com/docs/tool-calling.md#streaming-tool-calls). |
| `finish_reason` | `null` until the last content chunk. |

### The usage chunk

Every stream ends with one chunk that has an empty `choices` array and a `usage` object, sent just before `data: [DONE]`. It has the same fields as `usage` in a [non-streaming response](https://clonvo.com/docs/chat-completions.md#response), including `cost` (US dollars at list price) and `is_byok`. It’s always sent, whether or not you set `stream_options.include_usage`, and also after an error.

## Keep-alive comments

While a model is thinking or a provider is slow, Clonvo sends an SSE comment whenever the stream has been quiet for about 15 seconds:

```text
: keep-alive
```

Lines starting with `:` are comments in the SSE format. The OpenAI SDKs and standard SSE parsers ignore them; if you parse the stream yourself, skip them.

Clonvo holds the response for up to 3 seconds while it waits for the provider’s first content, so a failing provider can still be replaced by a [fallback](https://clonvo.com/docs/fallbacks.md) before anything is sent. The response headers can arrive up to 3 seconds later than they would without this check.

## Errors

If something goes wrong **before** the stream starts (invalid request, no credits, unknown model, the provider rejects the request), you get a normal JSON error response with an HTTP error status. See [Errors](https://clonvo.com/docs/errors.md).

If the provider fails **after** the stream has started, the HTTP status is already `200`. Clonvo then sends a chunk with an `error` object and `finish_reason: "error"`, then the usage chunk, then `data: [DONE]`:

```text
data: {"id":"gen-01M67FGKKVQ8ZK2M3N5P7R9S1V","object":"chat.completion.chunk","created":1793400000,"model":"google/gemini-3.8-flash","provider":"Google AI Studio","error":{"message":"The provider’s stream ended unexpectedly. Try again.","type":"server_error","code":"upstream_stream_error","status":502},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}

data: {"id":"gen-01M67FGKKVQ8ZK2M3N5P7R9S1V","object":"chat.completion.chunk","created":1793400000,"model":"google/gemini-3.8-flash","provider":"Google AI Studio","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":96,"total_tokens":108,"prompt_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"reasoning_tokens":0},"cost":0.000369,"is_byok":false}}

data: [DONE]
```

| `error` field | Description |
| --- | --- |
| `message` | What went wrong. Provider errors start with the provider’s name. |
| `type` | Error category, such as `server_error`, `rate_limit_error` or `timeout_error`. |
| `code` | A specific code, such as `upstream_stream_error`, `upstream_error`, `upstream_overloaded` or `upstream_timeout`. |
| `status` | The HTTP status the error would have had before streaming started. |

Treat `finish_reason: "error"` as a failed generation. You’re charged for the tokens produced before the failure, as reported in the usage chunk. If the provider reports no output tokens for a failed stream, or fewer than were streamed, Clonvo estimates them from the streamed text (about four characters per token).

## Fallbacks and streaming

[Fallback models](https://clonvo.com/docs/fallbacks.md) are only tried before the first chunk is sent. Once streaming has started, Clonvo can’t switch models without sending you a second response.

## Cancelling

Close the connection to stop generation (for example `stream.close()` in Python or `controller.abort()` in Node.js). Clonvo cancels the request to the provider and records the generation as cancelled, with the tokens counted up to that point, estimated the same way as for a failed stream. You’re charged for those tokens.