> ## Documentation Index
> Fetch the complete documentation index at: https://www.edgee.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Retry, Fallback & Reroute

> Automatic retry, provider fallback, and unconditional model reroute for resilient LLM routing.

Directing each request to the right LLM provider at the right time, with automatic fallback when something goes wrong. Per-request retry and provider fallback is one of two routing techniques in the Edgee Agent Gateway; [Plan-cap continuity](/docs/introduction/why-edgee#route) is the other.

Two configuration lists shape routing on every scope, and they answer different questions:

|                       | **Fallback**                | **Reroute**      |
| --------------------- | --------------------------- | ---------------- |
| Engages               | Only when a provider fails  | On every request |
| Requested model       | Tried first                 | Never contacted  |
| Configured as         | `fallbacks`                 | `reroutes`       |
| When all entries fail | Error from the last attempt | Error            |

A third mechanism, [Routing Strategies](/docs/features/routing-strategies), replaces both with a budget-driven policy: rules that change which model serves a request as spend accumulates. A scope with a strategy assigned ignores its reroute list, and its error handling moves to the strategy's own fallback model.

When a provider request fails, Edgee automatically retries and falls back to the next available provider — transparently, without any changes to your code.

## How it works

Every request goes through an ordered list of providers. Edgee tries each one in sequence, retrying transient failures before moving on. If all providers are exhausted without success, the error from the last attempt is returned to the caller.

```
Primary provider  ──► (retry once on transient error) ──► success
        │ (failure)
        ▼
Fallback provider 1 ──► success
        │ (failure)
        ▼
Fallback provider 2 ──► success
        │ (failure)
        ▼
Return error
```

## Provider ordering

Fallback order is determined automatically by each provider's **success rate**, computed from recent request history. Providers with higher success rates are tried first. When multiple providers have the same score, they are shuffled randomly for load distribution.

If you use [BYOK](/docs/features/byok) keys, only your own provider keys are eligible — Edgee's shared providers are not used as fallbacks. If no BYOK key is available for a model, shared providers are used instead.

## Retry behavior

Edgee distinguishes three categories of error:

| Category                | Errors                                                       | Behavior                                         |
| ----------------------- | ------------------------------------------------------------ | ------------------------------------------------ |
| **Retry then fallback** | Rate limit (429), Service unavailable (5xx)                  | Retry the same provider once, then fall back     |
| **Immediate fallback**  | Timeout (408, 504), Credential not found, Stream parse error | Skip retry, move to next provider immediately    |
| **Terminal**            | Invalid token (401), Configuration error                     | Return error immediately — no retry, no fallback |

The **primary provider** gets up to 2 attempts (1 initial + 1 retry). **Fallback providers** get 1 attempt each.

There is no backoff delay between attempts.

## Streaming

For streaming responses, retries are only possible **before any chunks have been sent to the client**. Once the first chunk is delivered, the connection is committed and errors propagate directly — the request cannot be retried or rerouted mid-stream.

## Response headers

Every non-streaming response carries the provider that served it:

```
X-Edgee-Provider: anthropic
```

When a fallback was used, the response additionally carries:

```
X-Edgee-Fallback-Used: 1
```

This lets you detect in your application or logs that the primary provider was bypassed.

<Note>
  Neither header is set on **streaming** responses: HTTP headers are sent before the provider
  future executes, so the winning provider is not known yet. For streams, read the provider
  from the usage record instead.
</Note>

## Reroute

Reroute substitutes the model **before** dispatch. Fallback is error-driven — it engages only when a provider fails. Reroute is unconditional: the requested model is replaced on every request, whether or not anything is wrong.

<Warning>
  Reroute is a substitution, not an extension of the chain. When a reroute list is set, its entries
  are tried in order and **the originally requested model is never contacted** — not even when every
  reroute entry has failed. In that case the request returns an error.
</Warning>

```
Request for anthropic/claude-opus-5
        │
        ▼
Reroute entry 1 ──► success
        │ (failure)
        ▼
Reroute entry 2 ──► success
        │ (failure)
        ▼
Return error          (anthropic/claude-opus-5 is never called)
```

Typical uses: pinning a whole squad to a cheaper model, standardizing an organization on one model regardless of what each agent requests, or moving traffic off a provider during an incident without touching any developer's config.

### Where to configure

<Note>
  Fallback and reroute are available only on paid AI Gateway seats. ☁️
</Note>

| Scope            | Where in the console                       | Applies to                |
| ---------------- | ------------------------------------------ | ------------------------- |
| **Organization** | Settings → AI Gateway → **Reroute models** | Every request in the org  |
| **Squad**        | Team → Squads → squad → Settings           | Every member of the squad |
| **API key**      | Team → member card → agent settings icon   | That key only             |

Settings resolve **org > squad > key**: the highest scope that sets a value wins, and a scope left on **Inherit** falls through to the next level down. This is the same three-tier resolution used by [Squads](/docs/features/squads#how-settings-resolve).

### Self-hosted endpoints as targets

A reroute or fallback entry can point at your own OpenAI-compatible endpoint instead of a catalog model, using the form:

```
custom:<provider_key_id>:<model_name>
```

`<provider_key_id>` is the ID of a registered custom endpoint provider key. Requests to a `custom:` entry are sent straight to that endpoint's base URL as an OpenAI-compatible chat completion, bypassing the model catalog. Malformed entries are rejected when you save them rather than silently dropped at request time.

## Observability

All failed attempts — retries and fallbacks — are recorded in the [observability dashboard](/docs/features/observability) as separate events with zero token cost. This gives you full visibility into provider health and which fallback paths are being exercised.

Log rows carry the routing decision that produced them:

| Flag               | Meaning                                                                      |
| ------------------ | ---------------------------------------------------------------------------- |
| `is_fallback`      | Served by a fallback entry after the primary provider failed                 |
| `is_reroute`       | Served by a reroute entry instead of the requested model                     |
| `is_plan_fallback` | Served by Edgee-covered [plan-cap continuity](/docs/introduction/why-edgee#route) |

These flags are available on the log rows in the console and in the [log export](/docs/api-reference/console-api-export-logs).