Paired tests: before and after

Alex Tutor

CatChow launched a loyalty program last month. Marketing wants to know: did it actually raise monthly spend, or did spend just drift up anyway?

You

Easy: compare average spend last month vs this month with a t-test, like we did for our A/B groups.

Alex Tutor

Careful: that independent-samples t-test compares two averages built from whichever customers happened to order in each month, not the same people.

Here we tracked the exact same 8 customers, before and after. That's a paired (related) samples design, and it deserves its own test.

Alex Tutor

You

Why does it matter that it's the same people?

Alex Tutor

Because each customer becomes their own baseline. A paired test only looks at each person's own change, removing the noise from customers who simply spend differently from one another.

Alex Tutor

Question 1

Why is a paired t-test more appropriate than an independent-samples t-test for comparing the same 8 customers' spend before and after the loyalty program?

Choose one answerPaired tests always give smaller p-values, so they're a safer default choiceIt removes each customer's own baseline spend from the comparison, leaving less noise than treating before and after as two independent groupsAn independent-samples t-test cannot be computed with only 8 peoplePaired tests don't require any assumption about how the data are distributed

Alex Tutor

Monthly spend before → after, USD, for our 8 loyalty customers: U1 42→47, U2 38→40, U3 51→59, U4 29→30, U5 60→66, U6 33→37, U7 47→50, U8 55→62.

Alex Tutor

Change in monthly spend per customerEvery customer's difference (after minus before) is positive, though the size of the change varies quite a bit.

Chart data

Customer	After − before
U1	5
U2	2
U3	8
U4	1
U5	6

Show 3 more rows

U6	4
U7	3
U8	7

Look at the differences: every single customer spent more after the program, some just a little, some a lot.

You

So it obviously worked?

Alex Tutor

Looks promising, but "obviously" isn't a number. We still need to test whether the average difference is bigger than plain noise.

Alex Tutor

Question 2

A paired t-test can compare "before" measurements from one group of customers with "after" measurements from a completely different group of customers.

TrueFalse

Alex Tutor

An independent-samples t-test splits its degrees of freedom across two groups: n1 + n2 − 2. Paired data collapses to one group of differences instead.

Each customer becomes one number: their own after-minus-before difference. Then it's a simple question: is the mean difference zero?

Alex Tutor

t = \dfrac{\bar{d}}{s_d / \sqrt{n}}

The paired t-test: mean difference divided by the standard error of the difference (SD of differences over the square root of n pairs).

Our 8 differences, in dollars: 5, 2, 8, 1, 6, 4, 3, 7.

Their sum is 36, so the mean difference is 36 / 8 = 4.5 dollars.

Alex Tutor

Question 3

The loyalty program's before/after differences (after − before) for the 8 customers are: 5, 2, 8, 1, 6, 4, 3, 7 dollars. Compute the mean difference.

Your answer

Units: USD, e.g. 4.5

Now the spread: squared deviations from 4.5 sum to 42. Dividing by n − 1 = 7 gives a variance of 6, so the standard deviation of the differences is about 2.45.

The standard error divides that by the square root of n: 2.45 / √8 ≈ 0.87 dollars.

You

So how big does the t-statistic need to be to matter?

Alex Tutor

For df = 7, a two-tailed 5% test has a critical value of about 2.365. Anything beyond that, in absolute value, is surprising if the program truly changed nothing.

Alex Tutor

Question 4

Using the same 8 differences (5, 2, 8, 1, 6, 4, 3, 7), the sample standard deviation of the differences is about 2.449. Compute the standard error of the mean difference.

Your answer

Units: USD, e.g. 0.87

Alex Tutor

t = 4.5 / 0.87 ≈ 5.20, with df = 7. That's more than double the critical value of 2.365.

We reject H0: the mean before/after difference is very unlikely to be zero. This 4.5 dollar rise looks real, not noise.

You

So the loyalty program caused it?

Alex Tutor

One caveat: with no control group, a seasonal trend could also explain part of it. The paired test proves the change is real, not that this one cause fully explains it.

Alex Tutor

Question 5

Match each term to what it describes.

Tap an answer, then tap the row it belongs to. You can also drag.

Paired (related) samples
Paired t-test
Sign test
Wilcoxon signed-rank test

Answers left to place

Alex Tutor

The paired t-test assumes the differences are roughly normal. With badly skewed differences or a tiny sample, that assumption gets shaky.

The sign test is the simplest fallback: it only counts how many differences are positive versus negative, ignoring their size entirely.

The Wilcoxon signed-rank test keeps a bit more information: it ranks the absolute differences and compares the rank sums of the positive and negative shifts.

You

So Wilcoxon is usually the better pick over the sign test?

Alex Tutor

Question 6

Which statements about paired designs and paired tests are true? Select all that apply.

Choose all that applyA paired t-test uses n − 1 degrees of freedom, where n is the number of pairsPairing generally increases power to detect a real effect compared with treating the data as two independent groupsThe sign test uses the exact size of each difference, not just its directionThe Wilcoxon signed-rank test is a safer choice than the paired t-test when the differences are heavily skewed or have big outliersA paired design needs roughly twice as many people as an independent-samples design for the same statistical power

Alex Tutor

Usually yes: Wilcoxon uses more of the data, so it has more power, while the sign test is the simplest, most robust fallback when even ranks feel shaky.

Whichever test you pick, always check the raw differences and the sample size first. Eight customers is enough for a clean lesson, not always for a confident launch decision.

You

Noted: same people, one column of differences, then pick the test that matches how those differences look.

Alex Tutor

Two tasks left: put pairing's advantage into your own words, then a trickier case with a tiny, lopsided sample.

Alex Tutor

Question 7Short answer · AI-checked task

In 2–4 sentences, explain why pairing before/after measurements on the same 8 customers gives more statistical power than comparing two independent groups of 8 different customers each.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Now for the case: a redesigned help centre, 6 users, satisfaction rated 1 to 10 before and after.

No clean t = 5.20 this time. One score jumps suspiciously far, and the sample is tiny.

You

Bring it on.

Alex Tutor

That's the spirit.

Alex Tutor

Question 8Case · AI-checked task

What test would you use, and how would you handle the outlier?

CatChow asked 6 support-chat users to rate satisfaction (1–10) before and after a help-centre redesign. Before: 5, 6, 4, 7, 5, 1. After: 6, 7, 6, 7, 6, 10. The last user's score jumped from 1 to 10 the same week they also received an unrelated free-gift email. With only 6 users and one dominating difference, a colleague suggests running a paired t-test on the raw scores anyway.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Alex Tutor

That’s the lesson. You answered every task — nicely done.

Next lesson: Repeated measures