Skip to main content
Rate limits control the request and token throughput of model APIs. When a call returns HTTP 429, check the error type and retry hints before sending another request.

Understand rate limits

API keys support RPM and TPM limits. These limits serve different purposes from spending quotas: Monitor RPM and TPM separately. A small number of requests can still consume many tokens when contexts or outputs are long. An available spending balance does not prevent rate limiting. Concurrency is not RPM. Concurrency is the number of requests in progress at the same time; RPM measures how quickly requests are sent. Even sequential calls can trigger a rate limit if each new request starts as soon as the previous one finishes.

View and configure limits

Open API Keys in the console and click Edit for the key. Check or update RPM Limit and TPM Limit. For key creation, spending quotas, and other key management tasks, see Manage API keys. Leaving an RPM or TPM field empty, or setting it to 0, means that no custom limit is set for that field.
Removing a custom key limit does not remove every service limit or guarantee that requests will never return 429. Use the actual response to identify the cause, rather than relying only on the key configuration.

Identify a rate-limit response

Check the HTTP status first, followed by error.type and, when present, error.code. Keep the original error type and code. Use them together with the HTTP status and message; not every failure is a rate-limit error. HTTP 402 indicates a billing rejection, so check your balance or spending quota. For HTTP 401, 400, or 422, fix authentication or the request first. See Error handling for the full classification.

Response example

The following example shows an RPM rejection for a non-streaming Chat Completions request. The limit of 1 is illustrative, not the platform’s default RPM. Request identifiers have been replaced with placeholders. Selected response headers:
Response body:
Responses may also include the x-ratelimit-* request-limit headers shown above. Keep their original values for troubleshooting, but do not treat them as a guarantee that the next request will succeed or substitute x-ratelimit-reset-requests for an explicit retry hint. Fields and headers vary by error. Clients must handle missing optional fields rather than requiring every protocol or every 429 response to match this example.

Inspect a single request

This curl example disables automatic retries so that you can inspect one request’s HTTP status, headers, and body. Set API_KEY to an existing key first. Do not put keys in source code or logs.
Replace model with an available model ID as needed. This example is for inspecting a response, not for repeatedly sending requests to trigger a limit. A successful request can incur charges. HTTP 200 alone does not establish that generation finished normally: also check the content and the protocol’s completion status.

Wait and retry within bounds

A retry hint does not guarantee recovery. The next request can still return 429 after you wait. Set a retry budget, and do not rotate API keys to bypass limits.
Handle an explicit rate-limit error in this order:
  1. Record the error. Keep the HTTP status, error type, error code, request ID, and retry hints.
  2. Check the retry budget. Set a maximum number of attempts, a cumulative wait limit, and a timeout for the entire operation. Count the initial request as an attempt.
  3. Calculate the wait. Wait at least the number of seconds in error.retry_after_sec, when present. Also read a parseable Retry-After header. If neither provides a usable hint, apply your application’s exponential backoff and jitter instead of retrying immediately.
  4. Wait, then try again. Read the new response’s hints rather than reusing the previous wait value. Further 429 responses remain subject to the same budget.
  5. Stop at the limit. Let the application handle the failure, for example by asking the user to try later or pausing a batch. Contact support if failures persist.
If two parseable retry hints disagree, the client can conservatively use the longer wait or stop automatic retries to investigate. This is a client policy, not a server-defined precedence rule. If the wait exceeds the remaining budget, stop rather than shortening the wait and retrying early. The following is pseudocode, not executable code. A maximum of three attempts and 120 seconds of cumulative waiting is only an example client budget, not a platform limit or recovery promise. Set the overall operation timeout separately to suit your application.
Check whether your SDK, HTTP client, or application already retries requests. Avoid stacking retries across layers. A network timeout or disconnected connection does not prove that a request was never processed; let the application decide whether resending is appropriate instead of applying the 429 loop automatically.

Retry boundaries for streaming

For a request that explicitly returns 429 before any output begins, use the strategy above to decide whether to retry. Do not automatically replay a request after receiving partial output: doing so can duplicate content or incur additional charges. For streaming calls, check in-stream errors and completion events as well as the initial HTTP status. Follow the event format for the selected protocol; the non-streaming JSON example is not a universal format for streaming errors.

Control traffic and observe changes

Backoff handles failed requests; it does not replace traffic control during normal operation.
  • Smooth request traffic. Queue batch work instead of sending a whole batch at once.
  • Limit concurrency. Bound in-flight requests and separately control the rate at which requests start. Lower concurrency alone may not prevent RPM limiting.
  • Manage token usage. Remove unnecessary repeated context and set an output budget that fits the task. Do not treat billable tokens or a response’s usage as the remaining live TPM allowance.
  • Prevent retry amplification. Apply the same client-side traffic controls to initial requests and retries.
Use Metrics to observe call trends, average RPM, and average TPM. Use Usage Detail for individual requests. Aggregate metrics describe the selected time range. They do not show a live remaining allowance or guarantee acceptance of the next request. Check client logs, actual errors, and key settings together. See View metrics and View usage.

Investigate persistent rate limiting

Check the key in use, its RPM/TPM settings, request timing, and automatic retries in your client and SDK. If 429 continues after the indicated waits, stop repeatedly trying and provide support with:
  • Request time and time zone.
  • Model ID, endpoint path, and whether the request used streaming.
  • HTTP status, error.type, error.code when present, and error message.
  • Request ID and traceId when present.
  • Current and previous retry hints, actual wait durations, and attempt count.
  • Original x-ratelimit-* headers when present.
  • The key’s RPM/TPM configuration and client traffic pattern.
Do not share the API key or the full Authorization header. Redact business data in request bodies, logs, and screenshots.

FAQ

0 removes the custom limit for that field, not the other limit or every service limit. For example, if TPM is 0, still check RPM. Identify the limiting factor from the actual error rather than the setting alone.
A retry hint does not guarantee acceptance. Read the latest error and wait value, and check request timing and stacked retries. Stop automatic retries when the budget is exhausted or failures persist, then contact support with the request identifiers.
Do not infer a reset at a clock-minute boundary from “per minute” or an example value of 60. Read the actual retry hint instead of fixing the recovery time at 60 seconds, and handle failures that persist after waiting.
The console’s average RPM is aggregated over the selected time range, not the state at each request’s start. Read the error to distinguish request-rate limits from token-rate limits, and inspect finer-grained client timestamps rather than comparing averages alone.
Do not treat a response’s usage.total_tokens as a rate-limit counter or a live remaining TPM allowance. Usage describes that request’s consumption. Cache, reasoning, and other tokens follow protocol-specific counting rules; billing usage alone does not establish the TPM counting rule.
Spending quotas, account balance, and rate limits are separate. For 402, investigate billing. For 429, investigate the actual limit and request traffic. Raising a spending quota does not raise throughput, and adding or rotating keys must not be used to bypass limits.
Rate-limit rejections with 429 are not billed; see Token usage and billing. Billing and rate-limit counting are separate questions. An unbilled rejection is not a reason to retry repeatedly without waiting. Follow retry hints and your retry budget.