When OpenAI goes down, your application should not. Automatic provider switching keeps LLM requests flowing.
LLM providers have outages. OpenAI has had multiple multi-hour incidents. Anthropic, Google, and others have had their own. If your application depends on a single provider, a provider outage becomes your outage.
LLM failover solves this by automatically routing requests to a backup provider when the primary fails. Your application does not need to know which provider is handling the request.
An LLM proxy maintains a list of providers that can handle a given model class. When a request to the primary provider fails (timeout, 5xx error, rate limit), the proxy retries the request against the next provider in the chain.
The retry happens transparently. Your application gets a response as if the primary provider handled it. The proxy logs which provider actually served the request, so you can track failover events.
More sophisticated implementations use health checks to detect degraded providers before they start failing requests, and route around them proactively.
Failover works best with model aliasing. Your application requests default-model. The proxy maps that to gpt-4o on OpenAI, with claude-sonnet-4-5-20250929 on Anthropic as the fallback. If OpenAI is down, the proxy routes to Anthropic automatically.
Without model aliasing, failover requires the backup provider to support the same model name, which limits your options to providers that offer OpenAI-compatible models.
Stockyard includes automatic failover routing as a built-in middleware module. Configure multiple providers, set a fallback chain, and the proxy handles the rest. Failed requests are retried against the next provider with the original request parameters.
Combined with model aliasing and proxy-only mode, you can set up a resilient multi-provider routing layer in under 5 minutes. Install Stockyard.
Try Stockyard. One binary, 16 providers, under 60 seconds.
Get Started