Reading Results
Avoid false wins.
Start With the Primary Metric
When the test ends, look first at the one primary metric you pre-registered. That single number decides the winner.
Resist the urge to hunt through secondary metrics for any positive signal. Cherry-picking a stray win is how teams convince themselves a flat test succeeded.
Read the Results Table
A clean results table shows visitors, conversions, conversion rate, relative lift, and confidence for each variant.
Scan it top to bottom. Confirm sample sizes are balanced and the confidence level meets your threshold before you celebrate any lift.
Variant | Visitors | Conv | Rate | Conf
A (ctrl)| 15,200 | 760 | 5.00% | --
B | 15,050 | 880 | 5.85% | 97%
Lift: +17% relative, 97% confidenceAll lessons in this course
- What to Test
- Building a Hypothesis
- Statistical Significance
- Reading Results