Retry, Fallback & Reroute

Automatic retry, provider fallback, and unconditional model reroute for resilient LLM routing.

Directing each request to the right LLM provider at the right time, with automatic fallback when something goes wrong. Per-request retry and provider fallback is one of two routing techniques in the Edgee Agent Gateway; Plan-cap continuity is the other.

Two configuration lists shape routing on every scope, and they answer different questions:

	Fallback	Reroute
Engages	Only when a provider fails	On every request
Requested model	Tried first	Never contacted
Configured as	fallbacks	reroutes
When all entries fail	Error from the last attempt	Error

A third mechanism, Routing Strategies, replaces both with a budget-driven policy: rules that change which model serves a request as spend accumulates. A scope with a strategy assigned ignores its reroute list, and its error handling moves to the strategy’s own fallback model.

When a provider request fails, Edgee automatically retries and falls back to the next available provider — transparently, without any changes to your code.

How it works

Every request goes through an ordered list of providers. Edgee tries each one in sequence, retrying transient failures before moving on. If all providers are exhausted without success, the error from the last attempt is returned to the caller.

Primary provider  ──► (retry once on transient error) ──► success
        │ (failure)
        ▼
Fallback provider 1 ──► success
        │ (failure)
        ▼
Fallback provider 2 ──► success
        │ (failure)
        ▼
Return error

Provider ordering

Fallback order is determined automatically by each provider’s success rate, computed from recent request history. Providers with higher success rates are tried first. When multiple providers have the same score, they are shuffled randomly for load distribution.

If you use BYOK keys, only your own provider keys are eligible — Edgee’s shared providers are not used as fallbacks. If no BYOK key is available for a model, shared providers are used instead.

Retry behavior

Edgee distinguishes three categories of error:

Category	Errors	Behavior
Retry then fallback	Rate limit (429), Service unavailable (5xx)	Retry the same provider once, then fall back
Immediate fallback	Timeout (408, 504), Credential not found, Stream parse error	Skip retry, move to next provider immediately
Terminal	Invalid token (401), Configuration error	Return error immediately — no retry, no fallback

The primary provider gets up to 2 attempts (1 initial + 1 retry). Fallback providers get 1 attempt each.

There is no backoff delay between attempts.

Streaming

For streaming responses, retries are only possible before any chunks have been sent to the client. Once the first chunk is delivered, the connection is committed and errors propagate directly — the request cannot be retried or rerouted mid-stream.

Response headers

Every non-streaming response carries the provider that served it:

X-Edgee-Provider: anthropic

When a fallback was used, the response additionally carries:

X-Edgee-Fallback-Used: 1

This lets you detect in your application or logs that the primary provider was bypassed.

Neither header is set on streaming responses: HTTP headers are sent before the provider future executes, so the winning provider is not known yet. For streams, read the provider from the usage record instead.

Reroute

Reroute substitutes the model before dispatch. Fallback is error-driven — it engages only when a provider fails. Reroute is unconditional: the requested model is replaced on every request, whether or not anything is wrong.

Reroute is a substitution, not an extension of the chain. When a reroute list is set, its entries are tried in order and the originally requested model is never contacted — not even when every reroute entry has failed. In that case the request returns an error.

Request for anthropic/claude-opus-5
        │
        ▼
Reroute entry 1 ──► success
        │ (failure)
        ▼
Reroute entry 2 ──► success
        │ (failure)
        ▼
Return error          (anthropic/claude-opus-5 is never called)

Typical uses: pinning a whole squad to a cheaper model, standardizing an organization on one model regardless of what each agent requests, or moving traffic off a provider during an incident without touching any developer’s config.

Where to configure

Fallback and reroute are available only on paid AI Gateway seats. ☁️

Scope	Where in the console	Applies to
Organization	Settings → AI Gateway → Reroute models	Every request in the org
Squad	Team → Squads → squad → Settings	Every member of the squad
API key	Team → member card → agent settings icon	That key only

Settings resolve org > squad > key: the highest scope that sets a value wins, and a scope left on Inherit falls through to the next level down. This is the same three-tier resolution used by Squads.

Self-hosted endpoints as targets

A reroute or fallback entry can point at your own OpenAI-compatible endpoint instead of a catalog model, using the form:

custom:<provider_key_id>:<model_name>

<provider_key_id> is the ID of a registered custom endpoint provider key. Requests to a custom: entry are sent straight to that endpoint’s base URL as an OpenAI-compatible chat completion, bypassing the model catalog. Malformed entries are rejected when you save them rather than silently dropped at request time.

Observability

All failed attempts — retries and fallbacks — are recorded in the observability dashboard as separate events with zero token cost. This gives you full visibility into provider health and which fallback paths are being exercised.

Log rows carry the routing decision that produced them:

Flag	Meaning
is_fallback	Served by a fallback entry after the primary provider failed
is_reroute	Served by a reroute entry instead of the requested model
is_plan_fallback	Served by Edgee-covered plan-cap continuity

These flags are available on the log rows in the console and in the log export.