p-value and significance
What a p-value really says, what it doesn't, and statistical versus practical significance.
Lesson 8 of 24~28 min of learningIncludes ~20 min for questions and tasks
Contents1 of 47 steps
Alex Tutor
You
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 1
Which is the closest correct definition of a p-value?
Alex Tutor
You
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 2
Which of these are common misinterpretations of a p-value that you should avoid saying out loud? Select all that apply.
You
Alex Tutor
Alex Tutor
Question 3
Getting a p-value above 0.05 (failing to reject H0) proves that the null hypothesis is true.
Alex Tutor
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 4
For a two-tailed z-test, a z-score of about 1.96 corresponds to what classic p-value threshold?
Units: (e.g. 0.05)
You
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 5
Match each term to its definition.
Tap an answer, then tap the row it belongs to. You can also drag.
- p-value
- Significance level (α)
- Statistical significance
- Practical significance
Answers left to place
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 6
CatChow tested a tiny checkout copy tweak on 500,000 customers per group: control converts at 5.0%, variant at 5.5% — statistically significant at p < 0.001. What is the relative lift, (variant - control) / control × 100?
Units: % (e.g. 12.5)
You
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 7Short answer · AI-checked task
In your own words, why is treating p = 0.05 as a hard pass/fail line (0.049 'works', 0.051 'doesn't') a bit misleading, even though it's a useful convention?
Write your answer and get a score with feedback from our AI reviewer.
Log in to get AI feedbackYou
Alex Tutor
You
Alex Tutor
Question 8Case · AI-checked task
Review the report below before it reaches the leadership deck.
A colleague writes: 'We tested a new loyalty-points banner on 800,000 customers per group. Control redeemed points at 2.00%, variant at 2.04%. p = 0.0004, highly significant! Recommend rolling this out company-wide immediately, no further discussion needed.' Write a short response: is the statistical claim correct? What would you want the leadership deck to also say?
Write your answer and get a score with feedback from our AI reviewer.
Log in to get AI feedbackAlex Tutor
That’s the lesson. You answered every task — nicely done.