Mann–Whitney: when data isn't normal

Alex Tutor

New scenario. We redesigned the product page and measured how long visitors spend on it: 6 sessions on the old page, 6 on the new one.

You

Great, t-test time! I already have the means.

Alex Tutor

Not so fast. Look at the raw numbers first: old page is 38, 42, 47, 49, 55, 61 seconds. New page is 58, 60, 65, 68, 72... and 130.

You

130 seconds? That one visitor must have gone to make coffee and left the tab open.

Alex Tutor

Exactly the kind of thing that wrecks a mean — remember outliers from lesson one? That single session would drag the new page's average way up, making the 't' from a t-test unreliable here.

You

So what do we do, just delete that session and pretend it didn't happen?

Alex Tutor

Tempting, but there's a cleaner fix: a test that only cares about order, not raw magnitude, so one extreme value can't dominate it. Time to meet the Mann-Whitney U test.

Quick check before we dive in.

Alex Tutor

Question 1

Which situation is the clearest reason to prefer Mann-Whitney over a classic t-test?

Choose one answerOne group's data has an extreme outlier that would badly distort its meanBoth groups have thousands of roughly normal, outlier-free observationsYou're comparing the same customers' spend before and after a promotionYou want to compare conversion rates (proportions) between two groups

Alex Tutor

Step one: throw all 12 sessions into one pile, sort them smallest to largest, and give each one a rank — 1 for the shortest, 12 for the longest.

Alex Tutor

All 12 sessions, sorted and rankedBar height is session duration; position is its rank (1 = shortest). Old-page sessions cluster at low ranks, new-page sessions at high ranks — except one outlier that doesn't change the overall pattern.

Chart data

Rank	Old page	New page
1	38
2	42
3	47
4	49
5	55

Show 7 more rows

6		58
7		60
8	61
9		65
10		68
11		72
12		130

You

The new page mostly sits at the high ranks — except one old-page session at rank 8, sneaking in above two new-page ones.

Alex Tutor

Good eye. Real data is rarely perfectly clean. Step two: add up the ranks that belong to each group. That total is called the rank sum.

Old page got ranks 1, 2, 3, 4, 5, 8 — a rank sum of 23. New page got the other six ranks, summing to 55. Check: 23 + 55 = 78, and 1 through 12 also sums to 78. Good, nothing lost.

Time to turn those rank sums into the actual test statistic.

Alex Tutor

Question 2

All 12 sessions are ranked 1 (shortest) to 12 (longest). The old page's ranks are 1, 2, 3, 4, 5 and 8. What is its rank sum R₁?

Your answer

Units: rank points (e.g. 20)

Alex Tutor

U_1 = R_1 - \frac{n_1(n_1+1)}{2}, \quad U_2 = R_2 - \frac{n_2(n_2+1)}{2}, \quad U = \min(U_1, U_2)

Each group's U subtracts out the rank sum it would have if it only ranked against itself. The smaller of the two is the Mann-Whitney U statistic.

You

That n(n+1)/2 term looks familiar from the rank-sum check we just did.

Alex Tutor

Good catch, it's the same formula — the sum of ranks 1 through n. Subtracting it from a group's actual rank sum tells you how much that group 'over-performed' compared to ranking purely against itself.

You

So a bigger U for a group means its values tended to rank higher overall?

Alex Tutor

Right — U₁ and U₂ each describe one group's own showing. It's only once we take the smaller of the two that we get the actual test statistic. You already have R₁ = 23 for the old page, with n₁ = 6. Compute U₁.

Alex Tutor

Question 3

Old page: R₁ = 23, n₁ = 6. What is U₁ = R₁ - n₁(n₁+1)/2?

Your answer

Units: U (e.g. 5)

You

U₁ = 2 — tiny. Does that already tell us the old page barely 'wins' any pairwise comparison against the new one?

Alex Tutor

Exactly that intuition — U literally counts, roughly, how many times a value from one group beats a value from the other. Now the mirror image: same recipe for the new page.

You

R₂ = 55, n₂ = 6?

Alex Tutor

Exactly. Go for it.

Alex Tutor

Question 4

New page: R₂ = 55, n₂ = 6. What is U₂ = R₂ - n₂(n₂+1)/2?

Your answer

Units: U (e.g. 5)

So U = min(2, 34) = 2. Here's the twist: for Mann-Whitney, small U is what signals a difference — the smaller it is, the less the two groups' ranks overlap.

You

That's the opposite of the t-test, where a bigger number was the exciting one.

Alex Tutor

Worth repeating because it trips people up constantly: small U, not big U, is the exciting result here. Keep that backwards rule somewhere visible.

You

Our U is 2, and the critical value is 5. 2 is less than 5, so... the page redesign really did change things?

Alex Tutor

Question 5

With U = min(2, 34) = 2 and a two-tailed critical value of 5 (n₁ = n₂ = 6, α = 0.05), we reject the null hypothesis that the two pages have the same distribution of session length.

TrueFalse

Alex Tutor

Nicely done getting through the mechanics. Before we talk about when this test earns its keep versus a regular t-test, let's lock in the vocabulary.

You

Fine by me, I'd rather not mix up 'rank' and 'rank sum' in a meeting.

Alex Tutor

Wise — half of statistical embarrassment is just mixing up two similarly-named things in front of an audience.

Match these ideas, then.

Alex Tutor

Question 6

Match each term to its definition.

Tap an answer, then tap the row it belongs to. You can also drag.

Rank
Rank sum (R)
U statistic
Non-parametric test

Answers left to place

Mann-Whitney is a non-parametric test: it makes no assumption about the shape of the data, unlike the t-test, which is parametric and leans on approximate normality.

You

So should we just always use Mann-Whitney and skip worrying about normality entirely?

Alex Tutor

Tempting, but no — when data genuinely is roughly normal with no nasty outliers, the t-test is more powerful: better at detecting a real effect from the same amount of data. Save the rank-based test for when skewness or outliers are actually a problem.

One more scenario where ranks are your only option, not just the safer one.

Alex Tutor

Question 7

Which of these are good reasons to reach for Mann-Whitney instead of a t-test? Select all that apply.

Choose all that applyThe metric is heavily right-skewed, with a long tail of unusually high valuesYou only know each customer's satisfaction rank (1st, 2nd, 3rd...), not an exact scoreBoth groups have tens of thousands of clean, roughly normal observationsYou're testing whether the share of users who churned differs between two plans

You

One thing still bugs me: why not just remove the 130-second session and run a normal t-test on what's left?

Alex Tutor

You could, and sometimes people do document that choice explicitly. But deleting inconvenient data is a slippery habit, and a real 130-second engaged session might be exactly the kind of user the redesign was meant to create.

You

So ranking is really a way of saying 'I trust that this happened, I just don't trust the exact number' — is that fair?

Alex Tutor

That's a great way to put it. It's a deliberate trade: you give up some precision (exact seconds) to gain robustness (immunity to that one wild number).

In your own words: why does ranking the data, rather than deleting or averaging it, let us keep that outlier honestly in the analysis?

Alex Tutor

Question 8Short answer · AI-checked task

In your own words, why does ranking the data (rather than deleting the 130-second session or leaving it to distort the mean) let us handle that outlier honestly?

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Solid. Last one — a colleague's Mann-Whitney write-up needs a second pair of eyes before it reaches their manager.

You

Let me guess, they mixed up the decision rule?

Alex Tutor

No spoilers. Read it yourself, the same way you'd want someone to check your work.

You

Fair. Judging someone else's math is somehow more fun than judging my own, anyway.

Alex Tutor

Question 9Case · AI-checked task

Review a colleague's Mann-Whitney write-up below.

A colleague compared time-in-app for two onboarding flows: n₁ = n₂ = 8 per group. They computed U = 25, looked up a two-tailed critical value of 13 for n₁ = n₂ = 8 at α = 0.05, and wrote: 'U = 25 is bigger than the critical value of 13, so the difference is significant — new onboarding wins.' Is this conclusion correct? Explain your reasoning.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Alex Tutor

That’s the lesson. You answered every task — nicely done.

Next lesson: p-value and significance