The model worked with my key and with nobody else's
Days before submitting a mobile app to a store, I tested it with a freshly created provider key rather than my own, and every generation failed with a 404.
The app generates quiz questions from a written lesson, and at that point questions could only be created that way — there was no hand-authoring path left. So a parent installing the app, creating their own provider key as the setup flow instructs, could generate nothing, and therefore publish nothing. The product was unusable for every new user and worked perfectly for me.
The model identifier was pinned as a constant. That constant was correct. The model still existed. It was still returned by the provider’s own list-models endpoint. And it returned 404 for the new key, with a message saying it is no longer available to new users.
Availability is a per-account fact, and the model list does not know whose account is asking. Mine was an older key with the older entitlement. Every test I had ever run proved a property of my account rather than a property of the platform.
The list endpoint is a catalogue, not a capability statement
That is the sentence I would put above the desk of anyone integrating a model provider.
A list-models response tells you what the vendor has. It does not reliably tell you what your credential may call, whether the model accepts the request shape you are about to send, or whether it does the thing your feature needs. It reads like an inventory of your options and it is closer to a product catalogue.
The same project hit all three of those failures, on three different providers, within a fortnight.
Entitlement. The 404 above. Listed, real, and not callable by a new key.
Modality. One provider’s integration discovered its model automatically by taking the first entry the list endpoint returned. That entry had, at some point, become a speech synthesis model. It is a perfectly good model. It returns 400 to every text-completion call, because that is not what it does. The provider had been silently dead for an unknown length of time, and the failure looked like a generic bad request rather than like a configuration mistake.
Behaviour. On another provider, discovery could land on a reasoning model, which then spends its entire output token budget on internal reasoning and returns nothing usable. Nothing errors. You get a successful response containing no answer, which is the most expensive way to fail.
Two of the five configured providers turned out to be dead at the same time — one on the speech model, one on billing — which is the sort of thing you only find out when you go and look, because a failover chain hides exactly this.
Discovery is the wrong instinct
Auto-discovery gets written because pinning feels brittle. Models get retired, and a pinned constant means a code change when that happens. Discovery sounds like the resilient option.
It is the opposite. Discovery hands the choice of what your product does to a list whose ordering, contents and semantics are entirely under someone else’s control and change without notice. Selecting the first entry is a lottery run once per deployment, and the prize is that your feature either works or 400s for months.
What replaced it is small and dull:
Pin the model, with an explicit ordered fallback. A short list, most specific first, ending in a rolling alias that the provider maintains. Nothing else is ever selected.
Advance only on the specific signal that means gone. The fallback moves to the next entry on a 404 and on nothing else. A rate limit is not “this model does not exist” and must not silently change which model your product uses. A chain that advances on any error will eventually reconfigure your product because of a transient network problem.
Cache the resolved answer. Once a model has answered, stop re-deciding.
Five regression tests cover this, two of which were confirmed to fail with the fallback removed.
The other half of the same problem
Adding a fifth provider, on a different API family from the other four, produced the mirror-image lesson: even with the model correctly chosen, the request shape is not portable.
That family’s API differed in five ways from the shape the existing code sent, and each difference is a 400 rather than a graceful degradation. Two of them were genuinely surprising.
The output token limit governs the model’s internal reasoning and its visible reply together, so a limit sized for the answer produces an empty answer once the model decides to think. That arithmetic is the reason the existing configuration disables reasoning entirely on the routes that need a bounded reply.
And the obvious control — the parameter that trades cost against thinking effort, the first knob anyone reaches for — is unsupported on the specific model chosen, and 400s every call rather than being ignored.
An unsupported parameter that returns an error is better than one that is silently dropped, and you should assume you will meet both. The only way to know which you have is to send it and read the response, once, deliberately, before writing the code that depends on it.
What I would tell someone starting
Test with a credential created today. Not your credential, not the team’s shared one — a new one, created the way your users create theirs, on a new account if the platform allows it. Every integration test you have run so far has been an assertion about your own account’s history.
Pin the model. Write the fallback list explicitly. Advance on one status code, not on failure in general.
And record, somewhere a person will read, which model each provider is actually resolving to right now. The speech-model failure went undetected because nothing in the system ever stated the answer to “which model are we calling?” It was derivable at runtime and never derived, and a fact that is technically available and never surfaced is, operationally, a fact nobody has.