> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scitix.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming

Streaming lets a model return incremental content while it is generating, instead of waiting until the full response is complete. Your application can display output as it arrives, which reduces the perceived wait time for long responses.

Streaming is commonly used in chat interfaces, long-form generation, code generation, and agent progress displays. It does not necessarily shorten the total time needed to generate the full answer, but it lets users see the model start responding earlier.

This capability is available through an OpenAI-compatible calling pattern. Set `stream=True` in the request, then read the returned content chunk by chunk on the client side.

## Use Cases

* **Chat UI**: Display the model response as it is generated to reduce perceived waiting.
* **Long-form generation**: Stream reports, summaries, plans, or code while they are being generated.
* **Code and reasoning tasks**: Let users see output progress earlier and decide whether to stop or adjust the request.
* **Agent interactions**: Show intermediate output or task progress in multi-step workflows.

## Supported Models

Streaming support may vary by model. Check the model details on the [Models](https://console.scitix.ai/model-inference/models) page for the latest support status.

## How It Works

After Streaming is enabled, a request usually follows this flow:

1. The client sends a request with `stream=True`.
2. After the model starts generating, the server continuously returns incremental content.
3. The client reads each chunk in order and extracts the incremental fields it needs to display or save.
4. When the client receives the finish reason, it stops reading and completes the response.

<Note>
  If the connection is interrupted or times out, the client needs to decide whether to retry, notify the user, or keep the partial output already received. Streaming only changes how the response is returned; it does not change the model's generation logic. Your application still needs to handle authentication, timeouts, disconnects, user cancellation, content assembly, and final result storage.
</Note>

## Key Parameters

* `stream`: Enables streamed responses. When set to `True`, the client must consume the response as a stream.
* `max_completion_tokens`: Controls the maximum generation length. Long responses still need a reasonable output limit to keep cost and latency predictable.

## Response Content

A streamed response consists of multiple chunks. The fields in each chunk may vary by model and by capability combination, so the client should handle fields based on whether they are present.

The `object` value of a streamed chunk is usually `chat.completion.chunk`. The `delta` object may include the following fields:

* `role`: The message role. It usually appears near the beginning of the streamed response.
* `content`: The text increment generated by the model. This is usually what you concatenate into the final response shown to the user.
* `reasoning_content`: Reasoning content or intermediate thinking generated by the model. Whether it is returned, and how much is returned, depends on the specific model and request result.

<Note>
  Do not assume every chunk contains `content`. Some chunks may contain only `role`, `reasoning_content`, a finish reason, or usage information. The client should skip empty increments or non-display fields that it does not need.
</Note>

`finish_reason` indicates why generation ended. It usually appears in the last chunk that contains `choices`.

`usage` may be returned at the end of the stream and can be used to inspect token usage for the request. The returned usage fields may differ by model. Use the actual response as the source of truth.

## Example

The example reads the API Key from an environment variable so the key is not written into code.

```bash theme={null}
export API_KEY="YOUR_API_KEY"
```

The following example enables Streaming and prints incremental content as soon as it is received.

This code performs the following steps:

1. Sets `stream=True` to enable streamed responses.
2. Iterates over each chunk in the response.
3. Reads `delta.content` first and appends it to the final display text.
4. Optionally collects `delta.reasoning_content` separately. Do not mix it into the final answer by default.

```python theme={null}
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["API_KEY"],
    base_url="https://api.scitix.ai/model-api",
)

stream = client.chat.completions.create(
    model="glm-5.2",
    messages=[
        {
            "role": "user",
            "content": "Explain what streaming output is in three short paragraphs.",
        }
    ],
    stream=True,
    max_completion_tokens=1024,
)

final_answer = []
reasoning_trace = []

for chunk in stream:
    if not chunk.choices:
        continue

    delta = chunk.choices[0].delta

    content = getattr(delta, "content", None)
    if content:
        final_answer.append(content)
        print(content, end="", flush=True)

    reasoning_content = getattr(delta, "reasoning_content", None)
    if reasoning_content:
        reasoning_trace.append(reasoning_content)

print()
```

The example only displays `delta.content` by default. If your product needs to show or store reasoning output, handle `reasoning_trace` according to your business requirements. If you do not need it, ignore `reasoning_content`.

The exact chunk structure may change with the API version, model capability, and request parameters. Use the API guides and actual responses as the source of truth.

## Best Practices

* **Separate final answers from reasoning content**: Use `delta.content` to assemble the final answer shown to users. If `reasoning_content` is returned, handle it separately based on your product requirements.
* **Display incrementally but save the full result**: The frontend can render text progressively, while the backend should still save the complete response for auditing, retries, and downstream processing.
* **Handle non-content chunks**: Some chunks may not contain text content. The client should skip fields it does not need instead of treating them as errors.
* **Handle user cancellation**: In chat or agent scenarios, users may stop generation. The application should stop reading the stream and update the conversation state correctly.
* **Set timeout and retry policies**: Streaming connections may stay open for a long time. Configure reasonable client timeouts and define how disconnects are handled.
* **Do not parse incomplete structures too early**: If Streaming is used together with Structured Output or JSON output, wait until the full response has been received before parsing the JSON.
* **Control output length**: For long responses, set a reasonable `max_completion_tokens` value to avoid excessive cost or long-running connections.

## Limitations

* Streaming requires the client to continuously read the response stream. If the client, proxy, or gateway does not support long-lived connections, it may not receive the full output reliably.
* Streaming does not guarantee that the model finishes generation faster. It only returns partial content earlier.
* If the request is interrupted midway, the received content may be incomplete. The application needs to handle partial results, retries, and user-facing messages.
* When used together with Function Calling, Structured Output, reasoning content, or other capabilities, the exact response structure and handling approach may differ. Use the API guides, model details, and actual responses as the source of truth.
* Token usage, finish reasons, and chunk fields may vary by model. Avoid relying on unconfirmed internal fields in business logic.