AI Model Migration Checklist

The checklist we use before every model retirement. 5 sections, about 30 checks, free.

What's inside

Codebase & config audit: copy-paste search commands for Python, TypeScript, YAML, JSON, .env and Terraform, plus CI, gateways and prompt registries.
Direct API vs Bedrock / Vertex / Azure retirement date diff: the same model retires on different days per platform.
Replacement model parity matrix: context window, tool calling, structured outputs, reasoning parameters, streaming, limits, price, region.
Eval & shadow-traffic cutover steps: golden set, shadow traffic, canary percentages, one-line rollback.
Post-shutdown 404 guardrails: alerts, fallback model, CI block on retired IDs.

From AI Deprecations (https://aideprecations.com) - free, sourced AI model shutdown calendar. Confirm every date on the provider's official page before acting.

1. Codebase and config audit

Find every place a model ID is hard-coded, including config, CI, infra and prompts stored as data.

Search source code (Python / TypeScript / JavaScript / Go):

rg -n --hidden -g '!node_modules' -e 'claude-sonnet-4-5' -e '"gpt-5(-mini|-nano|-pro)?(-20[0-9-]+)?"' -e '"o3(-pro)?(-20[0-9-]+)?"' .

Search config and infra (YAML / JSON / TOML / .env / Terraform / Helm):

rg -n --hidden -g '*.{yml,yaml,json,toml,tf,tfvars,env}' -g '.env*' -e 'model' -e 'modelId' -e 'model_id' .

Search CI and deploy files: .github/workflows/, .gitlab-ci.yml, Dockerfiles, notebooks, eval configs.
Check runtime config stores: feature flags, database rows, secrets managers, prompt registries, LLM gateway/router configs.
List every dated snapshot AND alias you call (aliases can move; dated snapshots get retired).
Check fine-tuned models: OpenAI says fine-tuned versions follow the same shutdown schedule as their base models.

2. Direct API vs Bedrock / Vertex / Azure: diff the retirement dates

The same model can retire on different dates on each platform.

For each model ID, note which platform you call it through (direct provider API, AWS Bedrock, Google Vertex AI, Azure OpenAI, a gateway).
Look up the date for THAT platform (https://aideprecations.com/providers/ links each official page).
Do not assume a Bedrock or Vertex date matches the direct API date, or vice versa.
Put each shutdown date on a shared calendar with an owner.

3. Replacement model parity matrix

Fill one row per (old model -> replacement) pair before switching.

Check	Old model	Replacement	Notes
Context window / max output tokens
Tool / function calling format
Structured outputs / JSON schema mode
Reasoning / effort parameters
Streaming behavior
Vision / file / audio inputs
Rate limits and quotas on your tier
Price per input / output token
Region / data residency availability

Read the replacement's official docs for removed or renamed parameters.

4. Evals and shadow traffic cutover

Freeze a golden test set from real traffic (with PII removed).
Run old vs replacement on it: accuracy, refusals, JSON validity, tool-call validity, latency, cost.
Shadow traffic: send a copy of production requests to the replacement, compare outputs, do not serve them.
Canary: 1% -> 10% -> 50% -> 100%, with a one-line rollback (model ID in config, not code).
Finish the cutover at least a week before the shutdown date.

5. Post-shutdown 404 guardrails

Alert on model-not-found / 404 / deprecated-model errors from every provider SDK.
Add a fallback model in config so a retired ID degrades instead of failing.
Add a CI check that fails the build when a retired model ID appears in the repo.
Subscribe to shutdown alerts so the next retirement is on your calendar: https://aideprecations.com/#subscribe

Upcoming: Claude Sonnet 4.5 shuts down Nov 30 · GPT-5 and o3 shut down Dec 11