16 Sample Size Calculations (Enrichment)

Optional

This is an optional enrichment session. We’re going to dig into a sample size problem from the practice midterm that turned out to be trickier than intended. The goal is to really understand what’s going on, not to add new material you need to know for the exam.

The Problem

A distributor receives deliveries of a million bottles from a brewer. They can’t inspect all of them, so they sample some and reject the delivery if the proportion of bad bottles in the sample exceeds 2%.

The brewer claims that no more than 1% of their bottles are bad. You’re asked to recommend a sample size so that, if the brewer is right, no more than 5% of deliveries get rejected.

Which sample size should you recommend: 100, 400, or 1600?

The Simple Approach

Here’s how I meant for you to solve this.

The delivery gets rejected when the sample proportion \(\hat\theta\) exceeds 0.02. We want to choose \(n\) so that this happens no more than 5% of the time. That means we want 95% of draws from \(\hat\theta\)’s sampling distribution to be at most 0.02.

We’ll make three simplifying assumptions:

We’ll ensure the middle 95% of draws are below 0.02. This is overkill—we really only need the smallest 95% to be below 0.02—but it makes the math easier.
We’ll assume the worst about the brewer: \(\theta = 0.01\) exactly.
We’ll assume the sampling distribution is approximately normal.

Under these assumptions, the middle 95% of draws lie in \[ \theta \pm 1.96 \sqrt{\frac{\theta(1-\theta)}{n}}. \] We want the upper bound to be at most 0.02: \[ 0.01 + 1.96 \sqrt{\frac{0.01 \times 0.99}{n}} \le 0.02. \] Rearranging: \[ 1.96 \sqrt{\frac{0.01 \times 0.99}{n}} \le 0.01 \] \[ \sqrt{n} \ge 1.96 \frac{\sqrt{0.01 \times 0.99}}{0.01} \approx 2 \times \frac{0.1}{0.01} = 20. \] So \(n \ge 400\). The answer is 400.

What Made It Hard

The problem is that all three of our simplifying assumptions are, well, simplifications. Let’s see what happens when we drop them.

Dropping Assumption 1: Middle 95% vs Smallest 95%

When we focus on the smallest 95% instead of the middle 95%, we’re asking for the 95th percentile of the sampling distribution to be at most 0.02. For a normal distribution, the 95th percentile is at \(\theta + 1.64\sigma\) instead of \(\theta + 1.96\sigma\).

This changes our answer by a factor of \((1.64/1.96)^2 \approx 0.70\). So instead of needing \(n = 400\), we’d need \(n \approx 280\). That’s a meaningful difference, but not enough to change our answer from 400 to 100.

Dropping Assumption 2: Assuming the Worst

This assumption is actually fine. The upper bound \(\theta + 1.96\sqrt{\theta(1-\theta)/n}\) increases as \(\theta\) increases (at least for small \(\theta\)). So if we ensure it’s below 0.02 for \(\theta = 0.01\), it’s automatically below 0.02 for all smaller \(\theta\).

Dropping Assumption 3: Normal Approximation

This is where things get interesting. Let’s look at the actual sampling distribution at each sample size.

The actual sampling distribution (gray bars) is hypergeometric, not normal (red curve). At \(n=100\), the normal approximation is pretty bad. At \(n=400\), it’s okay. At \(n=1600\), it’s good.

The dashed blue line marks our threshold of 0.02. We want 95% of the distribution to be to the left of it.

A Lucky Coincidence

Here’s the punchline: our answer of \(n=400\) actually works, even though our assumptions were wrong.

Why? The errors in our approximations happened to cancel out:

We used the 97.5th percentile (middle 95%) instead of the 95th percentile (smallest 95%). This made our bound too high.
We used the normal approximation, which has a lower 95th percentile than the actual hypergeometric distribution at these sample sizes. This made our bound too low.

These two errors approximately cancel, and \(n=400\) turns out to give us exactly what we want.

The purple line is what we actually care about: the 95th percentile of the true sampling distribution. The red line is what we calculated (97.5th percentile of normal approximation). They cross 0.02 at almost exactly the same place.

The Sawtooth Pattern

If you look closely at the purple line, you’ll notice it has a sawtooth pattern. Why?

The number of bad bottles in the sample is discrete—it’s 0, 1, 2, 3, etc. The 95th percentile of this count jumps up at certain sample sizes. But when we convert to a proportion (dividing by \(n\)), we’re dividing by an increasing number. So between jumps, the 95th percentile of the proportion decreases, creating the downward slopes. Then it jumps up when the count’s 95th percentile increases.

The 95th percentile of the NUMBER of bad bottles (not proportion)

This is a nice example of how discrete distributions can behave in unexpected ways.

The Takeaway

For the exam, the simple approach is what you need: assume normal, use the middle 95%, solve for \(n\). That’s the calculation you should be able to do.

But it’s worth knowing that real sample size calculations can be more subtle. The normal approximation isn’t always great (especially for rare events like 1% defect rates), and there’s often more than one reasonable way to set up the problem. When the stakes are high, you’d want to check your answer against the actual sampling distribution, as we did here.