Guides

Errors and retries

Every status the inference gateway returns, the error envelope, which ones are safe to retry, and which are billed.

The envelope

Failures come back in an OpenAI-shaped envelope with a machine-readable type and a short message:

{"error": {"message": "invalid api key", "type": "auth_error"}}

Branch on type and the HTTP status, not on the message: the message is written for a log line and may be reworded; the slug is stable.

Statuses

Status	`type`	Meaning	Billed	Retry
`401`	`auth_error`	Key missing, wrong or revoked; or `"API Key Expired"` for a key past its expiry.	No	No. Fix the key.
`402`		Budget exhausted: the organisation's own budget or this key's credit cap. The body does not say which; the dashboard does.	No	After a top-up.
`403`	`model_scope_error`	The key's `allowed_models` does not include this route.	No	No. Use an allowed route or a different key.
`404`		Unknown route id. Checked before authentication, so a typo in `model` looks like a missing endpoint.	No	No. Check the catalogue.
`429`		Rate limited: past this key's own requests-per-minute limit (60 unless it was set higher).	No	Yes, with backoff.
`5xx`	`proxy_error`	The key was accepted and the request routed, then the gateway's hop to the model server failed. An i2-side problem, not yours.	No	Yes, with backoff; if it persists, the route may be down.

Only 2xx responses are billed. A request that fails at any gate above is never sent upstream and never charged; a proxy_error after routing is not charged either.

Retrying

429 and 5xx are safe to retry. Use exponential backoff with jitter (for example 0.5 s, 1 s, 2 s, then give up) and cap concurrent requests per key; the limit is per minute, so spreading calls out matters more than waiting long. How many requests a window actually lost to 429 and to server-side timeouts is reported by GET /api/admin/usage-throttling, and on the dashboard's Request logs page.
401, 402, 403 and 404 will not fix themselves. Surface them: the fix is a configuration change, a top-up or a code change.
Chat completions are not idempotent. A retried request that succeeds is a second, billed generation. Do not retry a request whose response you may have already received partially; for streams, treat a client_disconnect on your side as done, not as retryable.

Streams that stop early

A stream can end without a finish_reason: "stop" chunk. The request log records why:

Termination reason	What happened
`success`	Completed normally.
`client_disconnect`	Your side closed the connection.
`proxy_guillotine`	The gateway's server-side timeout cut the request off.
`upstream_stream_aborted`	The model server dropped the stream mid-way.

Tokens relayed before the stop are billed; nothing after it is.

Seeing errors in the dashboard

The Request logs page filters by status class (4xx, 5xx) and by termination reason, and each row shows the upstream error message when there was one. That is the fastest way to tell "my key is wrong" from "the route is down" without reading raw responses.

Control plane errors

The control plane API uses a different convention ({"error": "…"} or DRF-style {"detail": …}), documented in Error responses.

Something unclear or wrong on this page? Tell us. Machine-readable copies: llms.txt.