What is a guardrail metric in an A/B test?
A metric the experiment is not allowed to damage beyond an agreed threshold while it optimises its primary metric. For a monetization test, typical guardrails are retention, payer conversion and session depth.
Why decide guardrails before the test starts?
Revenue moves quickly; retention, payer behaviour and paid-cohort payback move slowly. If the acceptable trade-off is decided after the results arrive, the team judges it knowing which metric improved — which biases the call.
What should a stop or rollback rule look like?
A specific, observable condition and a date: “roll back if D7 retention in the test group falls more than 1.5 points below control at the D14 read, or payer conversion falls more than 5% relative.” Vague rules get renegotiated mid-test.
What is an expiry condition?
The change in evidence or context that would reopen the decision after rollout — for example, a later retention decline or a shift in traffic mix. It stops temporary evidence becoming a permanent rule.