How is the model chosen?
On a graded set of the organisation’s own cases. A published leaderboard measures tasks belonging to someone else, on data the model may already have seen. The graded set is a few dozen real cases from the business with reviewed expected outcomes, and it answers the only question that matters: which model is correct more often on this work.
Run every candidate against it, score them the same way, and record the result with the date. That record is what makes a later switch a measurement rather than an argument.
What couples a system to one model
Very little of it is the model itself. Switching cost accumulates in the things built around a particular model’s behaviour.
| Coupling | What it is | Cost to break |
|---|---|---|
| Prompts tuned to one model | Phrasing that works because of this model’s quirks | Days, and a full re-grade |
| Provider-specific tool calling | Function schemas in one vendor’s shape | Days, contained if it sits behind the gateway |
| Embeddings from one provider | The whole index built in one vector space | A full re-index, and it cannot be partial |
| Output parsed by exact format | Downstream code depending on a response shape | Weeks, and it surfaces as production defects |
The third is the expensive one and the least visible. Embeddings from different providers occupy different vector spaces, so a change of embedding model means re-embedding every passage in the corpus. For a large corpus that is a scheduled job with a cost attached, and it has to complete before retrieval quality returns to where it was.
“Switching cost is not paid to the model. It is paid to everything built around one model’s behaviour.”
on where portability is actually won or lostHow is the cost measured rather than guessed?
By running the switch on a small scale before it is needed. Point the graded set at a second model through the gateway, score it, and record what broke. The result is a number: how many cases regressed, which parts of the system needed changing, and how long re-embedding would take.
Organisations that have never done this describe switching as impossible, because the cost is unknown and unknown costs are treated as infinite. Running it once turns it into a line item.
Portability is a build decision
A system built for portability routes every model call through one gateway, keeps prompts in configuration rather than in code, treats the response as data to be validated rather than a format to be parsed, and records which model answered each request. None of that is expensive at build time. All of it is expensive to retrofit.
Providers deprecate models on published dates, and behaviour moves underneath a version string without one. Portability is what keeps either from becoming an emergency.
What do we do about it?
The model is chosen by its score on the graded cases from the discovery session, and the system is built so that model can be replaced. We track deprecation dates at least ninety days ahead, and re-run the graded set against the current and the candidate model before a switch is proposed.
Article Published 29 August 2026 · By Agile Labs
Agile Labs is a Singapore software engineering company. Since 2016 we have built, taken over, secured and maintained software and AI systems.
Sources
- Stanford CRFM, Holistic Evaluation of Language Models, on benchmark contamination and task fit.
- OpenAI and Anthropic model deprecation policies, published retirement dates.
- LiteLLM and Envoy AI Gateway documentation on model routing and virtual keys.
