Confidence intervals

You

Last week's average order value was $50.00. So our AOV is $50, right? Case closed?

Alex Tutor

Almost. $50.00 is the average of that sample. A different week's sample would give a slightly different number, even if nothing really changed.

You

So the real AOV could be anything?

Alex Tutor

Not anything — but a range of plausible values around $50.00. That range is a confidence interval (CI): instead of one number, we report an interval likely to contain the true average.

Alex Tutor

Question 1

A 95% confidence interval means there is a 95% probability that the true average order value lies inside this specific interval.

TrueFalse

Alex Tutor

Building one is mechanical. First: the standard error (SE) — how much the sample mean would bounce around from sample to sample.

Alex Tutor

SE = \frac{s}{\sqrt{n}}

Standard error of the mean: sample standard deviation s divided by the square root of the sample size n.

You

So a bigger sample gives a smaller SE?

Alex Tutor

Exactly — SE shrinks with

\sqrt{n}

, not n itself. Quadrupling your sample only halves the SE.

Take last week's 225 orders: sample SD $15.00, so SE = $15.00 / √225 = $15.00 / 15 = $1.00.

Alex Tutor

Question 2

This month CatChow sampled n = 400 orders with sample standard deviation s = $20.00. What is the standard error of the mean, in dollars?

Your answer

Units: USD (e.g. 1.25)

Now the full interval: point estimate ± (critical value × SE). For a 95% CI, the critical value is about 1.96 (from the normal distribution).

Alex Tutor

\bar{x} \pm z_{\alpha/2} \cdot SE

95% CI for a mean, using the normal critical value z = 1.96.

Back to last week: $50.00 ± 1.96 × $1.00 = $50.00 ± $1.96, so the interval runs from $48.04 to $51.96.

You

So the margin of error there is just the $1.96 part?

Alex Tutor

Right — the margin of error is critical value × SE. Report it alongside the point estimate; it's the honest picture leadership actually needs.

Alex Tutor

Question 3

Using a critical value of 1.96 and SE = $2.50, what is the margin of error, in dollars?

Your answer

Units: USD (e.g. 1.96)

You

Can we just always use 99% confidence to be extra sure?

Alex Tutor

You can, but the interval gets wider. Higher confidence means a bigger critical value, which means a bigger margin of error.

Alex Tutor

Margin of error grows with confidence level (SE = \$1.00)

Chart data

Confidence level	Margin of error (USD)
90%	1.645
95%	1.96
99%	2.576

At 99% confidence that same $1.00 SE gives a ±$2.58 margin instead of ±$1.96 — a wider, less precise interval.

You

So if I want both high confidence AND a narrow interval...

Alex Tutor

...you need a bigger sample. SE shrinks with

\sqrt{n}

, so a narrower interval at the same confidence level means collecting more data, not wishing harder.

Alex Tutor

Question 4

To make a 95% CI narrower without lowering the confidence level, what should Sofia do?

Choose one answerCollect a larger random sampleCollect a smaller sampleReport the standard deviation insteadRound the average to fewer decimal places

Let's make sure that sticks — which levers actually narrow a CI, and which just look like they should?

You

Some things obviously narrow it — bigger n. But higher variability confuses me.

Alex Tutor

More variability in the data (bigger s) makes SE bigger, which widens the CI — the opposite of what you want.

You

And a non-random, cherry-picked sample?

Alex Tutor

That doesn't just fail to narrow the CI — it makes the whole interval meaningless, because the formula assumes a random sample.

Alex Tutor

Question 5

Which of these narrow a 95% CI, holding everything else fixed? Select all that apply.

Choose all that applyIncreasing the sample sizeLowering the confidence level to 90%Increasing the data's standard deviationSwitching to a smaller, hand-picked (non-random) sample

Alex Tutor

Everything so far used a mean (dollars). CatChow also tracks proportions — like conversion rate. The idea is the same, the SE formula changes slightly.

Alex Tutor

SE_p = \sqrt{\frac{p(1-p)}{n}}

Standard error of a sample proportion p, from a sample of size n.

Same shape as before: point estimate ± critical value × SE. We'll build a full example — including the significance test — in the next two lessons.

You

Okay, I think I've got the mechanics. Can we practice the order of steps?

Alex Tutor

Sure — walking through the steps out of order is a good way to see if it's actually stuck, not just memorised.

Alex Tutor

Question 6

Put the steps of building a confidence interval in the right order.

Tap an item, then tap where it should go. You can also drag or use the arrows.

Alex Tutor

One more thing before you go present this. Leadership sometimes only wants the single number — the CI feels like unnecessary hedging to them.

You

Isn't a single number just... simpler?

Alex Tutor

Simpler, yes. Honest, no. $50.00 with no context invites overconfidence: someone might compare it to $49.50 next month and call that a 'drop.'

You

Even though $49.50 is well inside this month's interval.

Alex Tutor

Exactly. Your turn — put that into words for someone who's never heard of a CI.

Alex Tutor

Question 7Short answer · AI-checked task

Sofia needs to explain to a marketing manager, in 2-3 sentences, why CatChow should report the confidence interval for AOV (e.g. $48.04-$51.96), not just the $50.00 average. Write her explanation.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Nice. One more - a slightly trickier situation to reason through.

You

Bring it on.

Alex Tutor

CatChow's finance lead saw this month's AOV interval, $48.04-$51.96, and last month's, $47.50-$50.10, and asked if AOV is now definitely higher.

You

Overlapping intervals... that's the trick, isn't it?

Alex Tutor

Question 8Case · AI-checked task

Has CatChow's AOV really increased?

CatChow's finance lead compares two months' AOV confidence intervals: this month $48.04-$51.96 (mean $50.00, n = 225, SE = $1.00), last month $47.50-$50.10 (mean $48.80, SE ≈ $0.66). She wants a one-paragraph answer: has AOV really increased? Write your recommendation, and mention what further check (if any) you'd want before calling this a real change.

Write your answer and get a score with feedback from our AI reviewer.

Log in to get AI feedback

Good instinct - flagging the overlap without over-claiming either way. That's the core skill this module builds on.

You

Next up: how big a sample do we actually need before we even start?

Alex Tutor

Exactly the right question. That's power and sample size - next lesson.

Alex Tutor

That’s the lesson. You answered every task — nicely done.

Next lesson: Power and sample size