9 min read
A/B/n testing: how to compare three or more variants without a false winner
A/B/n testing multiplies the chances of a false winner. How the Bonferroni and Holm corrections work, with a four-variant example and the traffic cost.
Articles about Multiple testing
RSS feedArticles for product managers and marketers about Multiple testing: plain explanations, examples and common mistakes.
9 min read
Peeking in A/B testing: why "already significant" is not a result
Peeking in A/B testing inflates false positives. In 100,000 simulated A/A tests, daily checks turned a 5% error rate into 27.6%. What to do instead.