AIM-04-004-P · Primer — Model Registry & Versioning

The idea in one sentence

A model registry gives every trained model artifact a stable, semantically-versioned identity — MAJOR.MINOR.PATCH — so promotion, rollback, and lineage tracking are deterministic instead of tribal knowledge.

Why it exists

Without a registry, teams overwrite a single "prod_model.pkl" file in place: retrain, save, deploy, repeat. Nobody can say what exact model served traffic last Tuesday, so when a regression ships there is no known-good artifact to roll back to — only a vague memory of "the version before this one." The failure this prevents is real: a team retrains a fraud model, accuracy looks fine offline, but a preprocessing change silently breaks a downstream feature contract in production. With no versioned, immutable artifact and no recorded metadata (training data hash, metrics, schema), the only fix is a frantic re-derivation of what used to work. A registry (MLflow Model Registry, or Kubeflow's model metadata store, or ClearML's versioned registry) makes every model version an addressable, immutable object with a stage (Staging/Production/Archived) — rollback becomes "point the serving endpoint at version 2.1.0" instead of "does anyone remember the old code."

How it works

A model registry stores each trained artifact under a name plus a semantic version MAJOR.MINOR.PATCH, together with metadata: training data hash, hyperparameters, evaluation metrics, and a lifecycle stage. The three components of the version answer three different questions. MAJOR bumps mean "the contract changed" — the input schema, output schema, or feature set changed in a way that breaks existing callers, so consumers must update their code to use the new version. MINOR bumps mean "it got better, compatibly" — same inputs and outputs, improved performance (e.g., retrained on more data), so callers can adopt it with zero code changes. PATCH bumps mean "a bug got fixed" — no behavioral contract change, just a correction (a timezone bug in a feature computation, a serialization fix).

Worked example: fraud-model is registered as version 1.0.0 — logistic regression over 25 features, AUC 0.87. The team adds 5 new features to improve recall; because the input schema now has 30 features instead of 25, every caller building the feature vector must change, so this is a breaking change: bump to 2.0.0. Later the team retrains 2.0.0 on twice as much historical data with the same 30 features; AUC rises from 0.87 to 0.91, callers need change nothing — that's a compatible improvement: bump to 2.1.0. A week later an engineer finds the feature pipeline was computing "days since last transaction" in UTC instead of the user's local timezone, off by up to a day near midnight; fixing it doesn't change the schema or meaningfully move AUC — that's a patch: bump to 2.1.1. Semantic versions compare like tuples, \((\text{MAJOR}, \text{MINOR}, \text{PATCH})\), so 2.1.1 > 2.1.0 > 2.0.0 > 1.0.0 is unambiguous ordering for any tool deciding what to serve.

The payoff shows up during an incident: if 2.1.1's "harmless" patch turns out to have introduced a regression (say the timezone fix changed which transactions count as "recent" and shifted the score distribution), ops does not retrain anything — they point the serving endpoint back at the immutable artifact registered as 2.1.0, and traffic is served by exactly the bits that were validated before, in seconds.

See it — Bumping a version resets every lower-order component to zero — schema changes zero minor and patch; compatible improvements zero patch.

flowchart TD
  A["New trained<br/>model artifact"] --> B{"Input/output<br/>schema changed?"}
  B -- "Yes" --> C["Bump MAJOR<br/>1.4.2 to 2.0.0"]
  B -- "No" --> D{"Metric improved,<br/>still compatible?"}
  D -- "Yes" --> E["Bump MINOR<br/>1.4.2 to 1.5.0"]
  D -- "No" --> F["Bump PATCH<br/>1.4.2 to 1.4.3"]
  C --> G["Register new<br/>version + metadata"]
  E --> G
  F --> G
  G --> H["Rollback = point<br/>at old version, no retrain"]

The one gotcha

Never overwrite an artifact in place or serve "latest" — always register an immutable, versioned copy, or rollback becomes archaeology instead of a config change.

The one-line takeaway

Version every model like an API contract.

Where you'll meet it

MLflow Model Registry (numeric versions + stages None/Staging/Production/Archived), Kubeflow (pipeline-tracked model versions + metadata store), ClearML (model package versions + approval status), Aim (versioned lineage).

Named concepts: semver