Repeated measures

Alex Tutor

We're tracking monthly orders for the same 5 loyalty subscribers at three checkpoints: signup week, week 4, and week 8.

You

So it's like the before/after test, just with three time points instead of two?

Alex Tutor

Exactly the same spirit, extended. Measuring the same subjects more than twice makes it a repeated-measures design.

You

Could I just run three paired t-tests: Week0–Week4, Week4–Week8, Week0–Week8?

Alex Tutor

You could, but you'd hit the same multiple-comparisons problem as running three t-tests instead of a one-way ANOVA.

Repeated-measures ANOVA does for many time points what the paired t-test did for two: one test, one honest error rate.

Alex Tutor

Question 1

Why not just run three separate paired t-tests (Week0–Week4, Week4–Week8, Week0–Week8) instead of one repeated-measures ANOVA?

Choose one answerPaired t-tests cannot be run more than once on the same subjectsEach extra paired t-test adds its own chance of a false positive, inflating the overall error rate above 5%, the same problem three independent t-tests haveRepeated-measures ANOVA is always more powerful no matter how many time points there are, so it's just a shortcutPaired t-tests require a control group, which this design doesn't have

Alex Tutor

Monthly orders per subscriber: U1 2, 6, 10; U2 6, 7, 11; U3 7, 11, 12; U4 8, 13, 15; U5 12, 13, 17 (Week 0, Week 4, Week 8).

Alex Tutor

Monthly orders per subscriber over timeEach subscriber starts at a different level, but all five rise by roughly the same amount from checkpoint to checkpoint.

Chart data

Checkpoint	U1	U2	U3	U4	U5
Week 0	2	6	7	8	12
Week 4	6	7	11	13	13
Week 8	10	11	12	15	17

Every subscriber's baseline differs by design (6, 8, 10, 12, 14 on average), yet each one climbs by a similar amount over time.

Repeated-measures ANOVA splits total variability into three pieces, not two: Subjects, Time, and Error, instead of one-way ANOVA's Between/Within.

Alex Tutor

SS_{total} = SS_{subjects} + SS_{time} + SS_{error}

Total sum of squares splits into subject variability, the time effect, and whatever's left over.

You

So it's one-way ANOVA, plus an extra 'Subjects' slice carved out of the error?

Alex Tutor

Question 2

A repeated-measures ANOVA splits total variability into three parts: between-subject variability, the time effect, and a leftover error term, instead of one-way ANOVA's two-part Between/Within split.

TrueFalse

Alex Tutor

Grand mean is 10. Subject means are 6, 8, 10, 12, 14. Time means (Week 0, 4, 8) are 7, 10, 13.

Summing squared deviations from 10 across all 15 numbers gives SS_total = 220.

From the subject means: SS_subjects = 120. From the time means: SS_time = 90.

Alex Tutor

Question 3

For this repeated-measures design, SS_total = 220, SS_subjects = 120, and SS_time = 90. Compute SS_error (the residual).

Your answer

Units: sum of squares, e.g. 12

Degrees of freedom: subjects 5 − 1 = 4, time 3 − 1 = 2, error (5 − 1)(3 − 1) = 8. MS_time = 90 / 2 = 45. MS_error = 10 / 8 = 1.25.

Compare that error's 8 degrees of freedom with the 15 − 3 = 12 you'd get by mistakenly treating all 15 numbers as one independent-groups design by week alone.

You

More degrees of freedom sounds like it should be better, though?

Alex Tutor

More df doesn't help if the error term itself is inflated by all the between-subject spread we just pulled out into its own slice.

Alex Tutor

Question 4

With SS_time = 90 (df = 2) and SS_error = 10 (df = 8), compute the F-statistic for the time effect.

Your answer

Units: F-value, e.g. 12.5

Alex Tutor

Ignoring who's who, error SS would be SS_total − SS_time = 220 − 90 = 130 over 12 df: MS_error ≈ 10.83, giving F ≈ 4.15.

Properly paired by subject, the exact same data gives F = 36, an enormous gain in power from identical 15 numbers.

You

Same data, wildly different F. That's a little unsettling.

Alex Tutor

It's the same edge the paired t-test had over the independent-samples t-test, just extended to three or more time points.

Alex Tutor

Question 5

Match each term to what it describes.

Tap an answer, then tap the row it belongs to. You can also drag.

Repeated-measures ANOVA
Friedman test
Cohort
Degrees of freedom (df)

Answers left to place

Alex Tutor

Repeated-measures ANOVA assumes roughly normal, similarly spread differences between time points.

When that's shaky, or the sample is tiny, the Friedman test is the fallback: it ranks each subject's own measurements and compares rank sums across time points.

You

Ranks again? Feels like this whole course is just "when in doubt, rank it."

Alex Tutor

Not far off. Ranks trade a little power for a lot of safety against outliers and skewed data.

Alex Tutor

Question 6

Which statements about repeated-measures designs are true? Select all that apply.

Choose all that applyA repeated-measures design generally needs fewer subjects than an independent-groups design to detect the same effectThe error term's degrees of freedom, (n − 1)(k − 1), are smaller than the N − k an independent-groups design would useA repeated-measures design and an independent-groups design always give the same F-statistic on the same raw numbersThe Friedman test is the non-parametric alternative to repeated-measures ANOVAA repeated-measures design requires a completely different set of subjects at every time point

Alex Tutor

Same subjects, several checkpoints: pull the subject variability into its own slice, and the real time effect stands out far more clearly.

Even with F = 36, still check the actual size of the change per subscriber, not just the F-statistic, before calling it a business win.

You

Got it: same subjects, one 'Subjects' slice removed, then judge the trend on its own merits.

Alex Tutor

Two tasks left: explain that F jump in your own words, then a case with a tiny, lopsided repeated-measures sample.

Alex Tutor

Question 7Short answer · AI-checked task

In 2–4 sentences, explain why the same 15 measurements gave F ≈ 4.15 when treated as an independent-groups design by week, but F = 36 as a proper repeated-measures analysis.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Now the case: NPS tracked at three checkpoints for the same 5 power users after a new feature.

Four users trend nicely upward, but one user's score crashes at the last checkpoint after an unrelated billing error.

You

Ready.

Alex Tutor

Go for it.

Alex Tutor

Question 8Case · AI-checked task

What test would you use, and how would you handle the outlier?

CatChow tracks NPS at three checkpoints (Week 0, Week 4, Week 8) for the same 5 power users after adding a new feature. Scores: U1 20, 30, 40; U2 10, 15, 25; U3 0, 10, 20; U4 −10, 0, 10; U5 9, 20, −20. U5's score crashes at Week 8 the same week they hit an unrelated billing error. A colleague suggests running a standard repeated-measures ANOVA on the raw scores anyway.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Alex Tutor

That’s the lesson. You answered every task — nicely done.

Next lesson: Correlation is not causation