A Metrics Dictionary Is a Product, Not a Document
“Retention is up” can be true and still useless if the team hearing it and the team writing it mean different things.
“Retention is up.” Every word in that sentence can be true, and the sentence can still be useless — if the team hearing it means one thing and the team that wrote it means another.
That’s what a metrics dictionary is actually for. It’s not a document. It’s a contract about what a word means — with an owner, a version, and a changelog, like any contract worth honouring.
Most teams I’ve seen write that contract once, during onboarding or a data-quality push, and never open it again. It drifts. The dashboards built on top of it keep reporting numbers with confidence.
“Retention” gets redefined by a new analyst who wasn’t in the room. “Active user” picks up a new session-length threshold nobody announced. Revenue changes tax treatment and “net” stops meaning what the meeting assumes. None of it shows up as a bug.
Where the drift surfaces
The problem shows up later: when someone compares this quarter with the last, reads an experiment against an older baseline, or feeds the same metric into an LTV model. Both numbers can be technically correct and no longer measure the same thing.
So when a definition changes, ask two questions: who approved it, and which dashboards, reports, and models just changed meaning underneath you? If the answer isn’t part of the change itself, the dashboard will keep working perfectly.
The decision built on it may not.