Long Context

Long Context lets a model receive more context in a single request, such as long documents, codebase snippets, conversation history, retrieval results, or multiple source materials. Compared with a short prompt, long context can reduce the amount of pre-trimming needed and lets the model read, analyze, and generate based on more complete information. Long context is not long-term memory. The model can only refer to content within the current request’s context window. History, files, or external knowledge outside the request still need to be passed in again by the application, injected after retrieval, or managed through other product capabilities.

Use Cases

Long-document question answering: Provide contracts, reports, papers, or product documents as context and ask questions about the material.
Document summarization and comparison: Summarize long materials, extract outlines, compare differences, or identify risks.
Codebase understanding: Provide multiple files, call chains, or error logs so the model can analyze logic, locate issues, or suggest changes.
Multi-turn conversation continuity: Keep necessary history in the conversation so the model understands earlier constraints and user preferences.
RAG result synthesis: Put multiple retrieved snippets into the context and ask the model to generate an answer based on evidence.

Supported Models

Context window size varies by model. Before using long context, check the model details on the Models page to confirm the context length, maximum output length, pricing, and capability support.

Context window size, maximum output length, and pricing are changeable information. For production use, rely on the model details on the Models page rather than only on example models in the docs.

How It Works

The context in a model request usually consists of:

Role, rules, and output requirements in the system message.
The current question, task instructions, and source materials in the user message.
Historical assistant and user messages.
Documents, retrieval results, code, logs, or structured data injected by the application.
The output budget needed for the model’s response.

You can think of the context window as the model’s working space for the current request. Both input and output consume token budget. When processing long materials, reserve enough output space for the final answer. Historical messages in multi-turn messages are also part of the current request context. Keeping more history can help the model understand previous context, but it also consumes the context window. If the history is too long, keep only the necessary turns or summarize the history before passing it in. A typical workflow is:

Choose a model with enough context length.
Clean and organize the materials you want to pass in, removing irrelevant content.
Make the task, material boundaries, and output format clear in the prompt.
Set a reasonable max_completion_tokens value to reserve output space.
Send the request and decide whether to split materials, add retrieval, or adjust the prompt based on the result.

Relationship with RAG and Prompt Cache

Long Context, RAG, and Prompt Cache solve different problems:

Capability	Main purpose	Suitable for
Long Context	Put more context into a single request	Materials are manageable in size and need to be understood together
RAG	Retrieve relevant snippets first, then pass them into the request	Large knowledge bases, open-ended questions, or cases where irrelevant context should be reduced
Prompt Cache	Reuse repeated context to reduce repeated processing cost or latency	Multiple requests share the same long prefix or fixed system prompt

This page only explains how to use Long Context. Whether Prompt Cache / Context Cache is available, how to enable it, and how it is billed should be checked in the corresponding capability doc or API guide.

Key Parameters

messages: Carries system prompts, user questions, conversation history, and long materials. Long materials should have clear boundaries so the model does not confuse source text with user instructions.
model: Selects the model. Context window size and maximum output length vary by model. Use the Models page as the source of truth.
max_completion_tokens: Controls the maximum output length for the response. For long inputs, do not spend the entire context budget on input; reserve space for output. If the response reaches this limit, it may end with finish_reason: "length" and the content may be truncated.
stream: For long responses, you can set this to True so the client displays output as it arrives.

For models that generate reasoning content, the reasoning process may also consume output budget. If max_completion_tokens is too small, the final answer may be truncated, or content may even be empty.

Use the API guides as the source of truth for the complete request parameters and response fields. This capability page only explains the basic usage pattern and application-side responsibilities for Long Context.

Usage Recommendations

Check whether long context is actually needed: If the question only depends on a few snippets, pass in the most relevant content instead of the entire material.
Preserve material structure: Add titles and separators for documents, sections, code files, or retrieval results to help the model identify boundaries.
Remove irrelevant content: Longer context usually increases cost and latency. Too much irrelevant content can also interfere with the model’s judgment.
Clarify the task and evidence scope: Tell the model which materials to rely on, whether it may use general knowledge, and what output format to use.
Reserve output space: Long-input scenarios still need a reasonable max_completion_tokens value so the answer is not truncated.
Reserve budget for reasoning and final answers: If the model generates reasoning content, account for both the reasoning process and the final answer in the output budget.
Use Streaming for long responses: If the expected output is long, enable stream=True so users can see the result earlier.
Set longer client timeouts: Very long inputs can increase processing time, and latency may vary significantly by model. In production, set timeouts based on the model, input length, and network conditions.
Use retrieval for large knowledge bases: If the material is much larger than a single request can handle, or if the question only needs local evidence, use Embedding / RAG to find relevant snippets before asking the model to generate the answer.

Limitations

Context window size, maximum output length, and billing behavior vary by model. Use the Models page as the source of truth.
Long context increases input token count and usually increases cost and response latency.
A longer context does not always produce a better result. Too much irrelevant, repeated, or conflicting information may reduce answer quality.
If the input and expected output exceed model limits, reduce the input, split the task, or choose a model with a larger context window.
The model does not automatically remember content from previous requests unless it is passed in again. To continue context, the application must explicitly include necessary history or summaries.
Successfully processing a long input in one request does not represent the model’s or platform’s maximum context limit. Use the model details and actual API responses for maximum context length, maximum output length, and over-limit behavior.
Long-context requests may return usage fields such as usage.prompt_tokens and usage.completion_tokens. Some models may also return details such as reasoning tokens or cached tokens. Use the API guides and usage page as the source of truth for fields and billing.
When combining Long Context with Structured Output, Function Calling, Streaming, RAG, or caching capabilities, confirm the support scope and parameter requirements for each capability.

Example

The example reads the API Key from an environment variable so the key is not written into code.

export API_KEY="YOUR_API_KEY"

This code performs the following steps:

Puts the long material into the user message.
Uses clear delimiters to mark material boundaries.
Asks the model to answer only based on the provided material.
Sets max_completion_tokens to reserve output budget.

The model in the example is only used to demonstrate the calling pattern. For production use, choose a model on the Models page that supports your target context length, and confirm its pricing, maximum output length, and capability limits.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["API_KEY"],
    base_url="https://api.scitix.ai/model-api",
)

document = """
Put the long document, code snippets, meeting notes, or retrieval results here.
In production, your application can read and assemble this content from files,
databases, or retrieval systems.
"""

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[
        {
            "role": "system",
            "content": "You are a precise document analysis assistant. Answer only based on the material provided by the user.",
        },
        {
            "role": "user",
            "content": f"""
Read the following material and provide:
1. three key conclusions;
2. questions that need further confirmation;
3. actionable next steps.

<document>
{document}
</document>
""",
        },
    ],
    max_completion_tokens=1024,
)

print(response.choices[0].message.content)

Get Started

Models

Usage

API Guides

Tool Integrations

Use Cases

Supported Models

How It Works

Relationship with RAG and Prompt Cache

Key Parameters

Usage Recommendations

Limitations

Example

​Use Cases

​Supported Models

​How It Works

​Relationship with RAG and Prompt Cache

​Key Parameters

​Usage Recommendations

​Limitations

​Example

Use Cases

Supported Models

How It Works

Relationship with RAG and Prompt Cache

Key Parameters

Usage Recommendations

Limitations

Example