Should You Even Bother?

A reality check for your testing roadmap

Stakeholder-Safe Mode
%
% relative lift
visitors/day

The Verdict

Samples per variant
Total samples needed
Test duration

How this works (if you care)

This calculator uses the standard formula for determining sample size in a two-proportion z-test:

n = (Zα/2 + Zβ)² × [p₁(1-p₁) + p₂(1-p₂)] / (p₁ - p₂)²

Where:

  • p₁ is your baseline conversion rate
  • p₂ is the expected improved rate (baseline × MDE)
  • Zα/2 is the z-score for your significance level
  • Zβ is the z-score for your statistical power

For multiple variants, we apply a Bonferroni correction to maintain the overall significance level. The duration is calculated by dividing the total required samples by your daily traffic and rounding up to whole days.