MONETIZATION UTILITY · SHEET 08

Whale-Skewed A/B Test
Significance Grader.

Standard statistical calculators assume normally distributed metrics. For apps and games where spend is heavily skewed by a few high-paying "whales," use this calculator to estimate true statistical runtime requirements.

This is for teams deciding when it is safe to call an A/B test on a monetization metric. Use it before ending a test early or greenlighting a launch off a metric like ARPU or IAP conversion, where a handful of high-spending users can produce a false winner if the sample size assumption does not account for spend concentration.

FILE · AB_TEST_08

Test Inputs

01 · Baseline IAP Conversion %
02 · Average Order Value (AOV) $
03 · Whale Skew (Pareto Shape) α=
04 · Minimum Detectable Lift (MDE) %
05 · Total Daily Traffic (DAU)

A/B Test Grader Outcomes

True Sample Size Required
0
Required sample size *per variation* adjusting for spend variance
Standard Calc Duration
0 Days
Underpowered runtime computed by naive calculators
Mean ARPU
$0.00
Baseline average revenue per active user
SAMPLE SIZE REQUIREMENT: THE UNDERPOWERED DANGER ZONE
SAMPLE SIZE TRAJECTORY: EFFECT OF WHALE SKEW CONCENTRATION
02Statistical Principles

The mathematics of skewed experiments.

A/B testing is a tool for capital protection. Running experiments on whale-skewed ARPU metrics without adjusting your variance metrics guarantees false-positive data.

VARIANCE EXPLOSION

The Whale-Variance Trap

In standard t-tests, sample size scales with variance. Because a tiny fraction of players (whales) can spend $100+ while others spend $0, the variance is orders of magnitude larger than the mean. Standard calculators miss this entirely.

Statistical Friction: Whale Outliers
FALSE POSITIVES

Underpowered Noise Chase

If you run a test underpowered (e.g. stopping at 10,000 users when you need 80,000), a single random whale transaction in variation B will spike the mean ARPU. You will declare a "winner" that was purely random noise.

Statistical Friction: Early Halting
PARETO SHAPE

Pareto Alpha Mechanics

The Pareto Alpha parameter (α) defines how concentrated spend is. An alpha of 1.15 represents extreme whale concentration (common in deep-Gacha RPGs), requiring huge sample volumes. An alpha of 2.2 represents flat spend.

Statistical Friction: Pareto Shape
LIFT VELOCITY

The MDE Leverage Factor

Minimum Detectable Effect (MDE) is the scale of lift you care about. Detecting a minor 2% change requires a massive sample base. Scaling your target MDE to 10% reduces sample sizes by 25x, enabling faster runtimes.

Statistical Friction: MDE Scaling
DECISION-GRADE EXPERIMENTATION & ANALYTICS AUDIT

Establish clean, math-validated testing frameworks.

Product teams regularly burn millions of dollars shipping features that failed A/B significance parameters due to whale dilution noise. We audit analytics architectures, design variance-adjusted sample engines, map cohort testing schedules, and configure clean telemetry metrics.

Book a Testing Framework Audit
FAQ

Common questions.

What does this calculator do?

It estimates the true sample size and test runtime needed to detect a statistically significant lift in whale-skewed monetization metrics, like ARPU, ARPDAU, or IAP conversion, where a small share of high-spending users inflates variance far beyond what standard A/B test calculators assume.

What inputs do I need?

Baseline IAP Conversion Rate, Average Order Value (AOV), Whale Skew (Pareto Shape alpha), Minimum Detectable Lift (MDE), Total Daily Traffic (DAU), Statistical Power, and Significance Level.

How is statistical significance calculated for skewed monetization metrics?

The calculator uses a standard two-sample z-test sample-size formula, but replaces the naive assumption that spend variance is roughly twice the mean (used by most standard calculators) with a variance estimate driven by your Pareto Shape parameter (alpha), which models how concentrated spend is among whales. A lower alpha means more extreme spend concentration and a larger required sample. It computes required sample size and runtime under both the standard assumption and the whale-adjusted assumption side by side, so you can see the gap.