Feature and Data Pipelines in Production
Versioning and retiring features
Feature versioning tracks exactly which formula produced which stored value, so changing how a feature is computed never silently breaks a model still relying on the old one.
- 7 min read
- 3 reading levels
- Published
Read these first
On this page 7
One lesson, three depths. Pick the one that fits you today — you can switch any time.
Beginner — No maths. Plain English.
The short answer
Feature versioning tracks exactly which formula produced which stored value. A change never silently breaks a model still relying on the old one.
The analogy you have already lived
When a country changes its currency notes, old notes stay valid for a transition period. Both the old and new notes exist side by side for a while, and everyone knows exactly which is which. Nobody quietly swaps one for the other without telling you.
Changing how a feature is calculated is the same kind of event. It needs a transition, not a silent swap.
Why it exists
A feature's formula sometimes has a bug, or a better version becomes available. Someone fixes it.
Say the fix overwrites the old formula in place instead. Every model trained on the old values now sees new values it has never seen before. This happens without retraining, without warning. Its predictions can shift for no reason anyone told it about.
How it works
purchase_count_30d v1 (Jan 2026): counts orders where status != 'cancelled'
-- bug: counted refunds as purchases
purchase_count_30d v2 (Jun 2026): counts orders where status = 'completed'
-- the fix
both versions exist in the registry.
old models keep reading v1 until THEY are retrained and moved to v2 on purpose.The registry is a small, boring table: feature name, version number, what changed, when it started, when (if ever) it was retired.
A real example you have seen
A ride-hailing app changing how "surge multiplier" is calculated. If it changed instantly for every driver mid-shift with no notice, earnings would look inconsistent and support tickets would flood in. A real rollout is staged and dated.
Remember this
- Changing a feature's formula creates a new version — it does not silently replace the old one.
- Old models keep reading the version they were trained on, until someone deliberately moves them.
- A small registry — name, version, formula, dates — makes "which version made this number" answerable months later.
What to learn next
- Re-embedding and reindexing — the specific, expensive version of this problem when the "feature" is a vector from a model.
- Data versioning — the same discipline applied to raw datasets rather than derived features.
- MLflow — tracking which model was trained against which version of everything.
Developer — Code and libraries.
Setup
No installation needed — this uses only Python's standard library sqlite3.
A minimal feature registry
import sqlite3
conn = sqlite3.connect(":memory:")
conn.execute("""
CREATE TABLE feature_registry (
name TEXT,
version INTEGER,
formula TEXT,
created_at TEXT,
deprecated_at TEXT
)
""")
def register(name, version, formula, created_at, deprecated_at=None):
conn.execute(
"INSERT INTO feature_registry VALUES (?,?,?,?,?)",
(name, version, formula, created_at, deprecated_at),
)
# v1 shipped in January. A bug was found: it counted refunds as purchases.
register("purchase_count_30d", 1, "COUNT(orders WHERE status != 'cancelled')",
"2026-01-10", deprecated_at="2026-06-01")
# v2 fixes it -- a new version, not an edit to v1's row, so any model still
# reading v1 keeps getting the exact values it was trained on.
register("purchase_count_30d", 2, "COUNT(orders WHERE status = 'completed')",
"2026-06-01", deprecated_at=None)
conn.commit()
def active_version(name, as_of):
row = conn.execute(
"SELECT version, formula FROM feature_registry "
"WHERE name = ? AND created_at <= ? "
"AND (deprecated_at IS NULL OR deprecated_at > ?) "
"ORDER BY version DESC LIMIT 1",
(name, as_of, as_of),
).fetchone()
return row
print("active on 2026-03-01:", active_version("purchase_count_30d", "2026-03-01"))
print("active on 2026-08-01:", active_version("purchase_count_30d", "2026-08-01"))
all_versions = conn.execute(
"SELECT version, formula, created_at, deprecated_at FROM feature_registry "
"WHERE name = ? ORDER BY version", ("purchase_count_30d",)
).fetchall()
print("\nfull history:")
for v in all_versions:
print(" ", v)active on 2026-03-01: (1, "COUNT(orders WHERE status != 'cancelled')") active on 2026-08-01: (2, "COUNT(orders WHERE status = 'completed')") full history: (1, "COUNT(orders WHERE status != 'cancelled')", '2026-01-10', '2026-06-01') (2, "COUNT(orders WHERE status = 'completed')", '2026-06-01', None)
Line-by-line walkthrough
register inserts a new row for a new version — it never edits an existing row's formula. That single habit is what makes old predictions explainable later: the row that produced them still exists, unchanged, forever.
active_version answers "what was true on this date", the same point-in-time question the very first lesson in this section asked about raw event data — here applied to the feature's own definition.
Common mistakes
Editing the formula in place. This is the single habit this whole lesson argues against. It feels harmless for a small fix and is exactly how a model's behaviour changes without anyone deciding it should.
No deprecated_at, only a "latest" flag. Without a real timestamp, you cannot answer "which version was active when this old prediction was made" — only "which version is active now".
Deleting a version once it is deprecated. Deprecated is not deleted. Keep the row; a past prediction's explanation depends on it existing.
Bumping the version number for a change that does not affect values. A rename or a comment update is not a new version. Reserve version bumps for changes that could shift a stored number.
Try it yourself
Add a v3 that changes the window from 30 days to 14 days, starting today. Query active_version for a date before and after today, and confirm the registry answers correctly both times.
What to learn next
- Re-embedding and reindexing — the specific, expensive version of this problem when the "feature" is a vector from a model.
- Data versioning — the same discipline applied to raw datasets rather than derived features.
- MLflow — tracking which model was trained against which version of everything.
Researcher — Mathematics and papers.
Versioning as a contract, not a log
A feature registry is more useful as an enforced contract than as a passive audit log. The distinction: a log records what happened; a contract prevents what should not happen — for example, refusing to serve a model against a feature version that has been deprecated for longer than the model's own last-retrained date, which would silently mean the model has never seen the feature it is currently reading.
Semantic versioning does not transfer cleanly
Software's major.minor.patch convention assumes callers can reason about compatibility from the version number alone. Feature versions cannot offer that guarantee in general: even a "bug fix" changes every downstream statistic the feature feeds, including model weights trained on the old values. The safer default is to treat every value-affecting change as a breaking change, and let the registry's created_at/deprecated_at window — not a semver-style number — be the actual compatibility signal.
Coupling to model lineage
A feature version is only useful if it is joinable against model lineage: which model was trained on which feature version, at which timestamp. This is the same problem MLflow and a model registry solve for model artifacts — a mature feature platform stores a foreign key from every training run to the exact feature-registry row it read, not only the feature's name.
$$\text{lineage}(m) = {(f_i, v_i) : f_i \text{ read at version } v_i \text{ during training of } m}$$
Without this join, "why did the model's behaviour change" degrades into manual archaeology across two separate systems' logs.
Deprecation as a staged rollout
Retiring a feature version safely mirrors the deployment strategies in model serving's researcher section: a canary period where both versions run and are compared, before the old version's compute is actually torn down. The cost of keeping a deprecated version computing for a transition window is small compared to the cost of an unannounced break to a model still depending on it.
Papers and systems
- Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS 2014 — names "undeclared consumers" as a specific technical-debt pattern this registry design defends against, where a feature is silently relied upon by systems its owner does not know about.
- Vartak et al., ModelDB: A System for Machine Learning Model Management, HILDA 2016 — early academic treatment of the lineage-tracking problem feature-model coupling extends.
- Feast and Tecton's feature-registry documentation both implement versioned feature definitions as first-class, queryable objects.
What to learn next
- Re-embedding and reindexing — the specific, expensive version of this problem when the "feature" is a vector from a model.
- Data versioning — the same discipline applied to raw datasets rather than derived features.
- MLflow — tracking which model was trained against which version of everything.