Your Model Provider Ships the Worst SLO in Your Stack

Go and read your model provider’s service tier documentation. Not the pricing page, the tier documentation. On Anthropic’s, the standard tier is described like this:

The standard tier is the default service tier for all API requests. The API prioritizes these requests alongside all other requests with best-effort availability.

Best-effort availability. That is the documented commitment for the tier almost every workload runs on. There was one tier that named a number, and the same page says this about it:

Priority Tier capacity commitments are no longer available for purchase.

The number it named was 99.5%, and the page is careful about the verb: Priority Tier “targets” 99.5% uptime. Not guarantees, not commits, and there is no published credit schedule attached to missing it.

So the strongest availability commitment you could buy from this provider, before it stopped being sold, was a target of 99.5%. Hold that next to the least impressive SLA in ordinary infrastructure. Amazon’s EC2 SLA commits to 99.99% monthly uptime at the region level and 99.5% for a single instance, and both come with a published credit schedule that pays out 10%, 30% or 100% of the bill depending on how badly the number was missed.

The floor of commodity compute, one instance with no redundancy at all, is the ceiling of what a frontier lab would sell you. And the compute vendor writes it into a contract with money attached, while the model vendor writes it as a target on a docs page.

99.5% is a much weaker promise than it sounds#

Monthly uptime percentages compress badly, so it is worth doing the arithmetic once. A thirty day month holds 43,200 minutes. Half a percent of that is 216 minutes, so a 99.5% target permits three hours and thirty-six minutes of downtime every month before anyone has technically missed anything.

Now put a real incident against that budget. On 30 July, Anthropic’s status page recorded elevated errors across many models, affecting claude.ai, the API, Claude Code and Claude Cowork. First update at 05:57 UTC, errors back to baseline at 09:58 UTC. Just over four hours.

One incident, and the entire monthly budget of the best tier they used to sell is gone with change to spare.

The shape of that incident is the part worth studying, because it is the part that breaks retry logic. The updates were not one drop and one recovery. All models except Opus 5 recovered at 06:26. Opus 5 came back at 07:33 and Sonnet 5 got worse. Sonnet 5 recovered at 08:06 and Fable 5 started erroring. Then at 08:33: “we are back to errors across all models.”

Three all-clears that were not all-clears. Any client with a naive backoff walked into the next wave, and any operator watching the status page was told twice that it was over before it was.

The incident cadence is easy to check for yourself, because the status page publishes an Atom feed. The 25 most recent entries on it span 26 July to 19 August, which is 25 days, landing on 16 separate days. Not all of those are outages, and plenty are single-model degradations rather than a full stop. That is the point. The normal operating condition of this dependency is that something is degraded most days.

I want to be fair about why my evidence is one-sided here. I am picking on the provider whose tier language and incident history are the easiest to verify precisely, which is a consequence of them publishing both in detail. OpenAI’s SLA and credit terms sit behind a bot wall I could not read, so I am not going to characterise them. Transparency is not the failing being described. The failing is what the words say when you actually read them.

The escalation path is the other half of an SLO#

An availability number is only half of what makes a dependency dependable. The other half is what happens when you are the one who is down.

In late July a team posted a Tell HN reporting that their paid Claude Team subscription had been unavailable for more than a week. Bills paid, subscription active, no access. Their escalation path was an support bot named Fin, and as one commenter put it, “Fin became judge, jury, and executioner and closed my attempts to raise the issue with an automated response.” The thread resolved when the outage mysteriously fixed itself after about a week, possibly because the complaint went public.

Forty-five points and twenty-two comments is a small thread. I am not citing it as a statistic. I am citing it because it is a specific documented case of a paying customer with no human escalation path, which is a property you can design around once you accept it is true.

Think about what you would do if your database vendor did this. You would not shrug. You would either buy a support contract that produces a human, or you would architect so that vendor’s bad week is not your bad week. Model providers get a third option that nobody else gets, which is that teams simply absorb it.

The fix is not a router#

Here is where most posts on this topic go wrong, and where I would have gone wrong two years ago. The instinct is to put an abstraction layer in front of the provider and route around failure. That instinct has now been tested in public and it mostly failed.

Manifest built exactly that: a router that classified each request into complexity tiers and dispatched it to the appropriate model across Anthropic, DeepSeek, OpenAI and Mistral. They shipped it in March 2026, ran it across 7,000 cloud users, and deprecated it in June. Their central finding is that the classification problem is not solvable at the point you need to solve it:

The prompt alone does not contain the whole task; it is just the trigger.

Their example is the one that convinced me. “Evaluate the tests for this repo” is a trivial request against a static HTML page and an enormous one against the Linux kernel, and nothing in the prompt tells you which you are holding. The complexity reveals itself through tool calls and retrieval, which is to say after you have already committed to a model.

Two more of their findings matter for the reliability version of this argument. Cache reads run 75% to 90% cheaper than uncached input, so a router that moves traffic between providers is throwing away prefix caching, and the savings it destroys can exceed the savings it captures. And switching models inside a session degrades output in ways users notice, because they have calibrated on one model’s behaviour.

The HN thread ran to 132 points and 87 comments and mostly landed on a distinction I think is right: generic routers are bad, orchestrators are fine. One capable model decomposing work and handing specific steps to specific models is a different design from a classifier guessing which model a request deserves.

What actually works is a chain, decided per workload, ahead of time#

The design that survives contact is duller than a router. You pick an ordered list of providers for each workload, ahead of time, based on what that workload actually needs, and you fall through it on failure. No request-time cleverness.

worldmonitor, an AGPL geopolitical intelligence dashboard, is a good open example because you can read the whole thing. Its provider chain is a plain constant:

const PROVIDER_CHAIN = [
  'ollama',
  'openrouter',
  'openrouter-free',
  'openrouter-free-backup',
  'groq',
  'generic',
] as const;

Six deep, local model first, two free tiers held in reserve as backups. Nothing decides that order at runtime.

The health gate in front of it contains the detail I found genuinely instructive, because it names a distinction that “is the provider up” hides:

// Two independent questions live here, and they are deliberately kept apart:
//   1. Is the provider ORIGIN reachable?  -> isProviderAvailable(), async, probes.
//   2. Is this MODEL usable on it?        -> isModelUsable(), sync, no network.
// The origin probe cannot answer (2): it GETs the bare origin, so a provider
// that is up but does not serve the configured model ID still looks healthy.

A provider can be reachable, green on its own status page, and completely unable to serve the model you asked for. Those are separate failures and they need separate detection. The same file quarantines a model after two provider-side rejections inside a ten minute window, and is explicit about why two rather than one: an HTTP 400 might be a malformed request, and reading that as a dead model would pull a working model out of the chain.

The real work is careful failure classification, and none of it is routing intelligence.

The exit has to be rehearsed#

The strongest evidence I found this month is a team that actually ran the exit rather than writing it into a doc. Unblocked moved their agent loops from Anthropic to GLM over five weeks and published what it cost them.

Their audit found over 80% of spend came from frontier usage in the main agent loops, almost entirely Opus. GLM 5.2 listed at under a third of frontier rates, and against Opus specifically the per-token saving looked closer to 95%. The actual saving on code review was 68%.

The gap between 95% and 68% is the whole lesson, and they state the mechanism cleanly:

The arithmetic is multiplicative, and only one of the two terms is printed on the rate card: effective cost = token price × tokens required per task

The cheaper model needed more tokens to do the same job, because it made more tool calls, made more mistakes, and ingested larger tool results. On their search flow, average tokens per call went from about 18,500 to about 24,700.

Then there is the integration surface, which is where I would have underestimated the work by the largest margin. Every provider involved was nominally OpenAI-compatible. In practice, Fireworks returned reasoning in a reasoning_content field while Together used reasoning. Some providers ignored the cache key field entirely and keyed off user instead, so multi-turn sessions missed cache on every single request. One provider treated any effort value other than the literal string high as maximum effort, silently maxing the bill. Baseten rejected response_format: json_schema outright, and zero of 336 structured calls succeeded over seven days before anyone noticed. Their conclusion:

There is no shared behavior at the edges.

That is a fortnight of work you discover only by doing it, and it is exactly the work that a documented fallback plan lets you believe you have already done. Until you have sent production traffic down the second path, the fallback is a hypothesis.

Their end state is the design I am arguing for, arrived at empirically: adaptive routing across several providers with circuit breakers, Claude retained as the fallback rather than discarded, and workloads split by requirement. Code review ended at 68% lower unit cost with GLM serving 86% of calls. The search agent ended at 43% lower cost and 43% of calls. Customers with compliance requirements stayed on Claude entirely.

Where this loses#

Three places, and the first one is the one that nearly changed my mind.

The fallback layer is a system, and it fails too. worldmonitor’s own source is where I found the sharpest evidence against my position. Two documented incidents, both caused by the reliability machinery rather than by any provider. In one, a completion timeout was clamped to 15 seconds, below the fastest completion the primary provider had ever produced, so the primary could never succeed and every run wrote a SEED_ERROR. In the other, a worker picked up the timeout policy without the matching provider routing policy, so requests were free-routed to backends four to seven times slower than the timeout allowed, and, in the comment’s own words, “every market_implications run failed.” Both are silent failures inside code whose entire purpose was to prevent silent failure. If you add a chain and do not instrument which link served each call, you have added failure modes and bought nothing.

For a single conversational product, consistency can beat availability. This is Manifest’s argument and I think it wins on its own ground. If users have calibrated on one model’s behaviour, serving them a different one during an incident may be worse than serving an honest error and a retry button. The chain argument is strongest for batch, pipeline and agent workloads where nobody is watching a specific response, and weakest for a chat product with a personality.

The neutral fallback layer is consolidating. The obvious way to get multi-provider access cheaply is a gateway, and in the same month I am arguing for one, Stripe reportedly agreed to acquire OpenRouter for more than $7B. Routing your model traffic through a payments company is a different risk profile than routing it through a neutral utility, and 473 points and 292 comments of argument on that thread suggests I am not the only one recalculating. Buying the abstraction means inheriting whatever the abstraction becomes.

The takeaway#

Read the tier documentation for the provider you depend on, and find the sentence that says what they actually owe you. For a great many teams that sentence will say best-effort, which is a weaker commitment than they have from any other vendor in the stack, including the one selling them virtual machines by the hour.

Then do the cheap version of the work. Decide, per workload, what should happen when the provider returns errors for four hours: fail closed, degrade to a smaller model, or queue and wait. Write the chain down as a constant. Instrument which provider served each call, so that falling back is a thing you can see rather than a thing you assume. And send real production traffic down the second path at some point before you need it, because everything Unblocked found out about incompatible reasoning fields and silently broken caching was invisible until they did.

The provider is not going to fix this for you. They have told you so, in the documentation, in a sentence most people have never read.

I build and run this kind of infrastructure, and I am available for platform and reliability work. More about what I do.