Basic Inferences

Chapter 6 Basic Inferences

The purpose of this chapter is to introduce two basic but powerful tools of inferential statistics, the confidence interval and the hypothesis test (also called test of significance), in the simplest case of looking for the population mean of a quantitative RV.

🔗

This simple case of these tools is based, for both of them, on a beautiful and amazing theorem called the Central Limit Theorem, which is therefore the subject of the first section of the chapter. The following sections then build the ideas and formulæ first for confidence intervals and then for hypothesis tests.

🔗

Throughout this chapter, we assume that we are working with some (large) population on which there is defined a quantitative RV \(X\text{.}\) The population mean \(\mu_X\) is, of course, a fixed number, out in the world, unchanging but also probably unknown, simply because to compute it we would have to have access to the values of \(X\) for the entire population.

🔗

Strangely, we assume in this chapter that while we do not know \(\mu_X\text{,}\) we do know the population standard deviation \(\sigma_X\), of \(X\text{.}\) This is actually quite a silly assumption — how could we know \(\sigma_X\) if we didn’t already know \(\mu_X\text{?}\) But we make this assumption because if makes this first version of confidence intervals and hypothesis tests particularly simple. (Later chapters in this Part will remove this silly assumption.)

🔗

Finally, we always assume in this chapter that the samples we use are simple random samples, since by now we know that those are the best kind.

🔗

Section 6.1 The Central Limit Theorem

Taking the average [mean] of a sample of quantitative data is actually a very nice process: the arithmetic is simple, and the average often has the nice property of being closer to the center of the data than the values themselves being combined or averaged. This is because while a random sample may have randomly picked a few particularly large (or particularly small) values from the data, it probably also picked some other small (or large) values, so that the mean will be in the middle. It turns out that these general observations of how nice a sample mean can be explained and formalized in a very important Theorem:

🔗

The Central Limit Theorem [CLT].

Suppose we have a large population on which is defined a quantitative random variable \(X\) whose population mean is \(\mu_X\) and whose population standard deviation is \(\sigma_X\text{.}\) Fix a whole number \(n\ge30\text{.}\) As we take repeated, independent SRSs of size \(n\text{,}\) the distribution of the sample means \(\overline{x}\) of these SRSs is approximately \(N(\mu_X, \sigma_X/\sqrt{n})\). That is, the distribution of \(\overline{x}\) is approximately Normal with mean \(\mu_X\) and standard deviation \(\sigma_X/\sqrt{n}\text{.}\)

🔗

Furthermore, as \(n\) gets bigger, the Normal approximation gets better.

🔗

Note that the CLT has several nice pieces. First, it tells us that the middle of the histogram of sample means, as we get repeated independent samples, is the same as the mean of the original population — the mean of the sample means is the population mean. We might write this as \(\mu_{\overline{x}}=\mu_X\text{.}\)

🔗

Second, the CLT tells us precisely how much less variation there is in the sample means because of the process noted above whereby averages are closer to the middle of some data than are the data values themselves. The formula is \(\sigma_{\overline{x}}=\sigma_X/\sqrt{n}\text{.}\)

🔗

Finally and most amazingly, the CLT actually tells us exactly what is the shape of the distribution for \(\overline{x}\) — and it turns out to be that complicated formula we gave Definition 4.62. This is completely unexpected, but somehow the universe knows that formula for the Normal distribution density function and makes it appear when we construct the histogram of sample means.

🔗

Here is an example of how we use the CLT:

🔗

Example 6.1.

We have said elsewhere that adult American males’ heights in inches are distributed like \(N(69, 2.8)\text{.}\) Supposing this is true, let us figure out what is the probability that 52 randomly chosen adult American men, lying down in a row with each one’s feet touching the next one’s head, stretch the length of a football field. [Why 52? Well, an American football team may have up to 53 people on its active roster, and one of them has to remain standing to supervise everyone else’s formation lying on the field....]

🔗

First of all, notice that a football field is 100 yards long, which is 300 feet or 3600 inches. If every single one of our randomly chosen men was exactly the average height for adult men, that would a total of \(52*69=3588\) inches, so they would not stretch the whole length. But there is variation of the heights, so maybe it will happen sometimes....

🔗

So imagine we have chosen 52 random adult American men. Measure each of their heights, and call those numbers \(x_1, x_2, \dots, x_{52}\text{.}\) What we are trying to figure out is whether \(\sum x_i \ge 3600\text{.}\) More precisely, we want to know

\begin{equation*} P\left(\sum x_i \ge 3600\right)\ . \end{equation*}

Nothing in that looks familiar, but remember that the 52 adult men were chosen randomly. The best way to choose some number, call it \(n=52\text{,}\) of individuals from a population is to choose an SRS of size \(n\text{.}\)

🔗

Let’s also assume that we did that here. Now, having an SRS, we know from the CLT that the sample mean \(\overline{x}\) is \(N(69, 2.8/\sqrt{52})\) or, doing the arithmetic, \(N(69, .38829)\text{.}\)

🔗

But the question we are considering here doesn’t mention \(\overline{x}\), you cry! Well, it almost does: \(\overline{x}\) is the sample mean given by

\begin{equation*} \overline{x} = \frac{\sum x_i}{n} = \frac{\sum x_i}{52} \ . \end{equation*}

What that means is that the inequality

\begin{equation*} \sum x_i \ge 3600 \end{equation*}

amounts to exactly the same thing, by dividing both sides by 52, as the inequality

\begin{equation*} \frac{\sum x_i}{52} \ge \frac{3600}{52} \end{equation*}

or, in other words,

\begin{equation*} \overline{x} \ge 69.23077\ . \end{equation*}

Since these inequalities all amount to the same thing, they have the same probabilities, so

\begin{equation*} P\left(\sum x_i \ge 3600\right) = P\left(\overline{x} \ge 69.23077\right)\ . \end{equation*}

But remember \(\overline{x}\) was \(N(69, .38829)\text{,}\) so we can calculate this probability with LibreOffice Calc or Microsoft Excel as

\begin{align*} P\left(\overline{x} \ge 69.23077\right) \amp = 1 - P\left(\overline{x} \lt 69.23077\right)\\ \amp = \text{NORM.DIST}(69.23077, 69, .38829, 1)\\ \amp = .72385 \end{align*}

where here we first use the probability rule for complements to turn around the inequality into the direction that NORM.DIST calculates.

🔗

Thus the chance that 52 randomly chosen adult men, lying in one long column, are as long as a football field, is 72.385%.

🔗

Section 6.2 Basic Confidence Intervals

As elsewhere in this chapter, we assume that we are working with some (large) population on which there is defined a quantitative RV \(X\text{.}\) The population mean \(\mu_X\) is unknown, and we want to estimate it.

🔗

We continue also with our strange assumption that while we do not know \(\mu_X\text{,}\) we do know the population standard deviation \(\sigma_X\), of \(X\text{.}\)

🔗

Our strategy to estimate \(\mu_X\) is to take an SRS of size \(n\text{,}\) compute the sample mean \(\overline{x}\) of \(X\text{,}\) and then to guess that \(\mu_X\approx\overline{x}\text{.}\) But this leaves us wondering how good an approximation \(\overline{x}\) is of \(\mu_X\text{.}\)

🔗

The strategy we take for this is to figure how close \(\mu_X\) must be to \(\overline{x}\) — or \(\overline{x}\) to \(\mu_X\text{,}\) it’s the same thing, and in fact to be precise enough to say what is the probability that \(\mu_X\) is a certain distance from \(\overline{x}\text{.}\) That is, if we choose a target probability, call it \(L\text{,}\) we want to make an interval of real numbers centered on \(\overline{x}\) with the probability of \(\mu_X\) being in that interval being \(L\text{.}\)

🔗

Actually, that is not really a sensible thing to ask for: probability, remember, is the fraction of times something happens in repeated experiments. But we are not repeatedly choosing \(\mu_X\) and seeing if it is in that interval. Just the opposite, in fact: \(\mu_X\) is fixed (although unknown to us), and every time we pick a new SRS — that’s the repeated experiment, choosing new SRSs! — we can compute a new interval and hope that that new interval might contain \(\mu_X\text{.}\) The probability \(L\) will correspond to what fraction of those newly computed intervals which contain the (fixed, but unknown) \(\mu_X\text{.}\)

🔗

How to do even this?

🔗

Well, the Central Limit Theorem tells us that the distribution of \(\overline{x}\) as we take repeated SRSs — exactly the repeatable experiment we are imagining doing — is approximately Normal with mean \(\mu_X\) and standard deviation \(\sigma_X/\sqrt{n}\text{.}\) By the 68-95-99.7 Rule:

68% of the time we take samples, the resulting \(\overline{x}\) will be within \(\sigma_X/\sqrt{n}\) units on the number line of \(\mu_X\text{.}\) Equivalently (since the distance from A to B is the same as the distance from B to A!), 68% of the time we take samples, \(\mu_X\) will be within \(\sigma_X/\sqrt{n}\) of \(\overline{x}\text{.}\) In other words, 68% of the time we take samples, \(\mu_X\) will happen to lie in the interval from \(\overline{x}-\sigma_X/\sqrt{n}\) to \(\overline{x}+\sigma_X/\sqrt{n}\text{.}\)
🔗

🔗
Likewise, 95% of the time we take samples, the resulting \(\overline{x}\) will be within \(2\sigma_X/\sqrt{n}\) units on the number line of \(\mu_X\text{.}\) Equivalently (since the distance from A to B is still the same as the distance from B to A!), 95% of the time we take samples, \(\mu_X\) will be within \(2\sigma_X/\sqrt{n}\) of \(\overline{x}\text{.}\) In other words, 95% of the time we take samples, \(\mu_X\) will happen to lie in the interval from \(\overline{x}-2\sigma_X/\sqrt{n}\) to \(\overline{x}+2\sigma_X/\sqrt{n}\text{.}\)
🔗

🔗
Likewise, 99.7% of the time we take samples, the resulting \(\overline{x}\) will be within \(3\sigma_X/\sqrt{n}\) units on the number line of \(\mu_X\text{.}\) Equivalently (since the distance from A to B is still the same as the distance from B to A!), 99.7% of the time we take samples, \(\mu_X\) will be within \(3\sigma_X/\sqrt{n}\) of \(\overline{x}\text{.}\) In other words, 99.7% of the time we take samples, \(\mu_X\) will happen to lie in the interval from \(\overline{x}-3\sigma_X/\sqrt{n}\) to \(\overline{x}+3\sigma_X/\sqrt{n}\text{.}\)
🔗

🔗

🔗

Notice the general shape here is that the interval goes from \(\overline{x}-z_L^*\sigma_X/\sqrt{n}\) to \(\overline{x}+z_L^*\sigma_X/\sqrt{n}\), where this number \(z_L^*\) has a name:

🔗

Definition 6.2. Critical value \(z_L^*\) with probability \(L\).

The critical value \(z_L^*\) with probability \(L\) for the Normal distribution is the number such that the Normal distribution \(N(\mu_X, \sigma_X)\) has probability \(L\) between \(\mu_X-z_L^*\sigma_X\) and \(\mu_X+z_L^*\sigma_X\text{.}\)

🔗

Note the probability \(L\) in this definition is usually called the confidence level.

🔗

If you think about it, the 68-95-99.7 Rule is exactly telling us that \(z_L^*=1\) if \(L=.68\text{,}\) \(z_L^*=2\) if \(L=.95\text{,}\) and \(z_L^*=3\) if \(L=.997\text{.}\) It’s actually convenient to make a table of similar values, which can be calculated on a computer from the formula for the Normal distribution.

🔗

Critical values table.

Here is a useful table of critical values for a range of possible confidence levels:

🔗

Note that, oddly, the \(z_L^*\) here for \(L=.95\) is not \(2\text{,}\) but rather \(1.96\text{!}\) This is actually more accurate value to use, which you may choose to use, or you may continue to use \(z_L^*=2\) when \(L=.95\text{,}\) if you like, out of fidelity to the 68-95-99.7 Rule.

🔗

Now, using these accurate critical values we can define an interval which tells us where we expect the value of \(\mu_X\) to lie.

🔗

Definition 6.4. Confidence interval for \(\mu_X\) with confidence level \(L\).

For a probability value \(L\text{,}\) called the confidence level, the interval of real numbers from \(\overline{x}-z_L^*\sigma_X/\sqrt{n}\) to \(\overline{x}+z_L^*\sigma_X/\sqrt{n}\) is called the confidence interval for \(\mu_X\) with confidence level \(L\).

🔗

The meaning of confidence here is quite precise (and a little strange):

🔗

Fact.

Any particular confidence interval with confidence level \(L\) might or might not actually contain the sought-after parameter \(\mu_X\). Rather, what it means to have confidence level \(L\) is that if we take repeated, independent SRSs and compute the confidence interval again for each new \(\overline{x}\) from the new SRSs, then a fraction of size \(L\) of these new intervals will contain \(\mu_X\text{.}\)

🔗

Note that any particular confidence interval might or might not contain \(\mu_X\) not because \(\mu_X\) is moving around, but rather the SRSs are different each time, so the \(\overline{x}\) is (potentially) different, and hence the interval is moving around. The \(\mu_X\) is fixed (but unknown), while the confidence intervals move.

🔗

Sometimes the piece we add and subtract from the \(\overline{x}\) to make a confidence interval is given a name of its own:

🔗

Definition 6.5. Margin of error.

When we write a confidence interval for the population mean \(\mu_X\) of some quantitative variable \(X\) in the form \(\overline{x}-E\) to \(\overline{x}+E\text{,}\) where \(E=z_L^*\sigma_X/\sqrt{n}\text{,}\) we call \(E\) the margin of error [or, sometimes, the sampling error] of the confidence interval.

🔗

Note that if a confidence interval is given without a stated confidence level, particularly in the popular press, we should assume that the implied level was .95.

🔗

Subsection 6.2.1 Cautions

The thing that most often goes wrong when using confidence intervals is that the sample data used to compute the sample mean \(\overline{x}\) and then the endpoints \(\overline{x}\pm E\) of the interval is not from a good SRS. It is hard to get SRSs, so this is not unexpected. But we nevertheless frequently assume that some sample is an SRS, so that we can use it to make a confidence interval, even of that assumption is not really justified.

🔗

Another thing that can happen to make confidence intervals less accurate is to choose too small a sample size \(n\text{.}\) We have seen that our approach to confidence intervals relies upon the CLT, therefore it typically needs samples of size at least 30.

🔗

Example 6.6.

A survey of 463 first-year students at Euphoria State University [ESU] found that the [sample] average of how long they reported studying per week was 15.3 hours. Suppose somehow we know that the population standard deviation of hours of study per week at ESU is 8.5. Then we can find a confidence interval at the 99% confidence level for the mean study per week of all first-year students by calculating the margin of error to be

\begin{equation*} E=z_L^*\sigma_X/\sqrt{n} = 2.576\cdot8.5/\sqrt{463} = 1.01759 \end{equation*}

and then noting that the confidence interval goes from

\begin{equation*} \overline{x}-E = 15.3 - 1.01759 = 14.28241 \end{equation*}

\begin{equation*} \overline{x}+E = 15.3 + 1.01759 = 16.31759\,. \end{equation*}

🔗

Note that for this calculation to be doing what we want it to do, we must assume that the 463 first-year students were an SRS out of the entire population of first-year students at ESU.

🔗

Note also that what it means that we have 99% confidence in this interval from 14.28241 to 16.31759 (or \([14.28241, 16.31759]\) in interval notation) is not, in fact, that we any confidence at all in those particular numbers. Rather, we have confidence in the method, in the sense that if we imagine independently taking many future SRSs of size 463 and recalculating new confidence intervals, then 99% of these future intervals will contain the one, fixed, unknown \(\mu_X\text{.}\)

🔗

Section 6.3 Basic Hypothesis Testing

Let’s start with a motivating example, described somewhat more casually than the rest of the work we usually do, but whose logic is exactly that of the scientific standard for hypothesis testing.

🔗

Example 6.7.

Suppose someone has a coin which they claim is a fair coin (including, in the informal notion of a fair coin, that successive flips are independent of each other). You care about this fairness perhaps because you will use the coin in a betting game.

🔗

How can you know if the coin really is fair?

🔗

Obviously, your best approach is to start flipping the coin and see what comes up. If the first flip shows heads [H], you wouldn’t draw any particular conclusion. If the second was also an H, again, so what? If the third was still H, you’re starting to think there’s a run going. If you got all the way to ten Hs in a row, you would be very suspicious, and if the run went to 100 Hs, you would demand that some other coin (or person doing the flipping) be used.

🔗

Somewhere between two and 100 Hs in a row, you would go from bland acceptance of fairness to nearly complete conviction that this coin is not fair — why? After all, the person flipping the coin and asserting its fairness could say, correctly, that it is possible for a fair coin to come up H any number of times in a row. Sure, you would reply, but it is very unlikely: that is, given that the coin is fair, the conditional probability that those long runs without Ts would occur is very small.

🔗

Which in turn also explains how you would draw the line, between two and 100 Hs in a row, for when you thought the the improbability of that particular run of straight Hs was past the level you would be willing to accept. Other observers might draw the line elsewhere, in fact, so there would not be an absolutely sure conclusion to the question of whether the coin was fair or not.

🔗

It might seem that in the above example we only get a probabilistic answer to a yes/no question (is the coin fair or not?) simply because the thing we are asking about is, by nature, a random process: we cannot predict how any particular flip of the coin will come out, but the long-term behavior is what we are asking about; no surprise, then, that the answer will involve likelihood. But perhaps other scientific hypotheses will have more decisive answers, which do not invoke probability.

🔗

Unfortunately, this will not be the case, because we have seen above that it is wise to introduce probability into an experimental situation, even if it was not there originally, in order to avoid bias. Modern theories of science (such as quantum mechanics, and also, although in a different way, epidemiology, thermodynamics, genetics, and many other sciences) also have some amount of randomness built into their very foundations, so we should expect probability to arise in just about every kind of data.

🔗

Let’s get a little more formal and careful about what we need to do with hypothesis testing.

🔗

Subsection 6.3.1 The Formal Steps of Hypothesis Testing

State what is the population under study.
🔗

🔗
State what is the variable of interest for this population. For us in this section, that will always be a quantitative variable \(X\text{.}\)
🔗

🔗
State which is the resulting population parameter of interest. For us in this section, that will always be the population mean \(\mu_X\) of \(X\text{.}\)
🔗

🔗
State two hypotheses about the value of this parameter. One, called the null hypothesis and written \(H_0\), will be a statement that the parameter of interest has a particular value, so

\begin{equation*} H_0: \mu_X = \mu_0 \end{equation*}

where \(\mu_0\) is some specific number. The other is the interesting alternative we are considering for the value of that parameter, and is thus called the alternative hypothesis, written \(H_a\). The alternative hypothesis can have one of three forms:

\begin{align*} H_a: \mu_X \amp\lt \mu_0\ ,\\ \text{or}\quad H_a: \mu_X \amp> \mu_0\ ,\text{ or}\\ H_a: \mu_X \amp\neq \mu_0\ , \end{align*}

where \(\mu_0\) is the same specific number as in \(H_0\text{.}\)

🔗

🔗
Gather data from an SRS and compute the sample statistic which is best related to the parameter of interest. For us in this section, that will always be the sample mean \(\overline{X}\)
🔗

🔗
Compute the following conditional probability

\begin{equation*} p=P\left( \begin{matrix} \text{getting values of the statistic which are as extreme,}\\ \text{or more extreme, as the ones you did get} \end{matrix}\ \middle|\ H_0\right)\ . \end{equation*}

This is called the \(p\)-value of the test.

🔗

🔗
If the \(p\)-value is sufficiently small — typically, \(p\lt .05\) or even \(p\lt .01\) — announce
🔗

“We reject \(H_0\text{,}\) with \(p=\langle\text{number here}\rangle\text{.}\)”
🔗

🔗

Otherwise, announce
🔗

“We fail to reject \(H_0\text{,}\) with \(p=\langle\text{number here}\rangle\text{.}\)”
🔗

🔗

🔗
Translate the result just announced into the language of the original question. As you do this, you can say “There is strong statistical evidence that ...” if the \(p\)-value is very small, while you should merely say something like “There is evidence that...” if the \(p\)-value is small but not particularly so.
🔗

🔗

Note that the hypotheses \(H_0\) and \(H_a\) are statements, not numbers. So don’t write something like \(H_0=\mu_X=17\text{;}\) you might use

\begin{equation*} H_0=\text{"}\mu_X=17\text{"} \end{equation*}

\begin{equation*} H_0: \mu_X=17 \end{equation*}

(we always use the latter in this book).

🔗

Subsection 6.3.2 How Small is Small Enough, for p-values?

Remember how the \(p\)-value is defined:

\begin{equation*} p=P\left( \begin{matrix} \text{getting values of the statistic which are as extreme,}\\ \text{or more extreme, as the ones you did get} \end{matrix}\ \middle|\ H_0\right)\ . \end{equation*}

In other words, if the null hypothesis is true, maybe the behavior we saw with the sample data would sometimes happen, but if the probability is very small, it starts to seem that, under the assumption \(H_0\) is true, the sample behavior was a crazy fluke. If the fluke is crazy enough, we might want simply to say that since the sample behavior actually happened, it makes us doubt that \(H_0\) is true at all.

🔗

For example, if \(p=.5\text{,}\) that means that under the assumption \(H_0\) is true, we would see behavior like that of the sample about every other time we take an SRS and compute the sample statistic. Not much of a surprise.

🔗

If the \(p=.25\text{,}\) that would still be behavior we would expect to see in about one out of every four SRSs, when the \(H_0\) is true.

🔗

When \(p\) gets down to \(.1\text{,}\) that is still behavior we expect to see about one time in ten, when \(H_0\) is true. That’s rare, but we wouldn’t want to bet anything important on it.

🔗

Across science, in legal matters, and definitely for medical studies, we start to reject \(H_0\) when \(p\lt .05\text{.}\) After all, if \(p\lt .05\) and \(H_0\) is true, then we would expect to see results as extreme as the ones we saw in fewer than one SRS out of 20.

🔗

There is some terminology for these various cut-offs.

🔗

Definition 6.8. Statistically significant at level \(\alpha\).

When we are doing a hypothesis test and get a \(p\)-value which satisfies \(p\lt \alpha\text{,}\) for some real number \(\alpha\text{,}\) we say the data are statistically significant at level \(\alpha\). Here the value \(\alpha\) is called the significance level of the test, as in the phrase “We reject \(H_0\) at significance level \(\alpha\text{,}\)” which we would say if \(p\lt \alpha\text{.}\)

🔗

Example 6.9.

If we did a hypothesis test and got a \(p\)-value of \(p=.06\text{,}\) we would say about it that the result was statistically significant at the \(\alpha=.1\) level, but not statistically significant at the \(\alpha=.05\) level. In other words, we would say “We reject the null hypothesis at the \(\alpha=.1\) level,” but also “We fail to reject the null hypothesis at the \(\alpha=.05\) level,”.

🔗

Fact.

The courts in the United States, as well as the majority of standard scientific and medical tests which do a formal hypothesis test, use the significance level of \(\alpha=.05\text{.}\)

🔗

In this chapter, when not otherwise specified, we will use that value of \(\alpha=.05\) as a default significance level.

🔗

Example 6.10.

We have said repeatedly in this book that the heights of American males are distributed like \(N(69, 2.8)\text{.}\) Last semester, a statistics student named Mohammad Wong said he thought that had to be wrong, and decide to do a study of the question. MW is a bit shorter than 69 inches, so his conjecture was that the mean height must be less, also. He measured the heights of all of the men in his statistics class, and was surprised to find that the average of those 16 men’s heights was 68 inches (he’s only 67 inches tall, and he thought he was typical, at least for his class

When an experimenter tends to look for information which supports their prior ideas, it’s called confirmation bias — MW may have been experiencing a bit of this bias when he mistakenly thought he was average in height for his class.

). Does this support his conjecture or not?

🔗

Let’s do the formal hypothesis test.

🔗

The population that makes sense for this study would be all adult American men today — MW isn’t sure if the claim of American men’s heights having a population mean of 69 inches was always wrong, he is just convinced that it is wrong today.

🔗

The quantitative variable of interest on that population is their height, which we’ll call \(X\text{.}\)

🔗

The parameter of interest is the population mean \(\mu_X\).

🔗

The two hypotheses then are

\begin{align*} H_0: \mu_X \amp= 69\quad \text{and}\\ H_a: \mu_X \amp\lt 69\ , \end{align*}

where the basic idea in the null hypothesis is that the claim in this book of men’s heights having mean 69 is true, while the new idea which MW hopes to find evidence for, encoded in alternative hypothesis, is that the true mean of today’s men’s heights is less than 69 inches (like him).

🔗

MW now has to make two bad assumptions: the first is that the 16 students in his class are an SRS drawn from the population of interest; the second, that the population standard deviation of the heights of individuals in his population of interest is the same as the population standard deviation of the group of all adult American males asserted elsewhere in this book, 2.8. These are definitely bad assumptions — particularly that MW’s male classmates are an SRS of the population of today’s adult American males — but he has to make them nevertheless in order to get somewhere.

🔗

The sample mean height \(\overline{X}\) for MW’s SRS of size \(n=16\) is \(\overline{X}=68\text{.}\)

🔗

MW can now calculate the \(p\)-value of this test, using the Central Limit Theorem. According to the CLT, the distribution of \(\overline{X}\) is \(N(69, 2.8/\sqrt{16})\text{.}\) Therefore the \(p\)-value is

\begin{equation*} p=P\left( \begin{matrix} \text{MW would get values of }\overline{X}\text{ which are as}\\ \text{extreme, or more extreme, as the ones he did get} \end{matrix}\ \middle|\ H_0\right) = P(\overline{X}\lt 69)\ . \end{equation*}

Which, by what we just observed the CLT tells us, is computable by

\begin{equation*} \text{normalcdf}(-9999, 68, 69, 2.8/\sqrt{16}) \end{equation*}

on a calculator, or

\begin{equation*} \text{NORM.DIST(68, 69, 2.8/SQRT(16), 1)} \end{equation*}

in a spreadsheet, either of which gives a value around .07656.

🔗

This means that if MW uses the 5% significance level, as we often do, the result is not statistically significant. Only at the much cruder 10% significance level would MW say that he rejects the null hypothesis.

🔗

In other words, he might conclude his project by saying

🔗

“My research collected data about my conjecture which was statistically insignificant at the 5% significance level but the data, significant at the weaker 10% level, did indicate that the average height of American men is less than the 69 inches we were told it is (\(p=.07656\)).”
🔗

🔗

People who talk to MW about his study should have additional concerns about his assumptions of having an SRS and of the value of the population standard deviation

🔗

Subsection 6.3.3 Calculations for Hypothesis Testing of Population Means

We put together the ideas in Subsection 6.3.1 above and the conclusions of the Central Limit Theorem to summarize what computations are necessary to perform:

🔗

Fact.

Suppose we are doing a formal hypothesis test with variable \(X\) and parameter of interest the population mean \(\mu_X\). Suppose that somehow we know the population standard deviation \(\sigma_X\) of \(X\text{.}\) Suppose the null hypothesis is

\begin{equation*} H_0: \mu_X = \mu_0 \end{equation*}

where \(\mu_0\) is a specific number. Suppose also that we have an SRS of size \(n\) which yielded the sample mean \(\overline{X}\text{.}\) Then exactly one of the following three situations will apply:

If the alternative hypothesis is \(H_a:\mu_X\lt \mu_0\) then the \(p\)-value of the test can be calculated in any of the following ways
1. the area to the left of \(\overline{X}\) under the graph of a \(N(\mu_0, \sigma_X/\sqrt{n})\) distribution,
  🔗
  
  🔗
2. normalcdf\((-9999, \overline{X}, \mu_0, \sigma_X/\sqrt{n})\) on a calculator, or
  🔗
  
  🔗
3. NORM.DIST(x̄, μ₀, σ_X/SQRT(n), 1) on a spreadsheet.
  🔗
  
  🔗
🔗
🔗
If the alternative hypothesis is \(H_a:\mu_X>\mu_0\) then the \(p\)-value of the test can be calculated in any of the following ways
1. the area to the right of \(\overline{X}\) under the graph of a \(N(\mu_0, \sigma_X/\sqrt{n})\) distribution,
  🔗
  
  🔗
2. normalcdf\((\overline{X}, 9999, \mu_0, \sigma_X/\sqrt{n})\) on a calculator, or
  🔗
  
  🔗
3. 1-NORM.DIST(x̄, μ₀, σ_X/SQRT(n), 1) on a spreadsheet.
  🔗
  
  🔗
🔗
🔗
If the alternative hypothesis is \(H_a:\mu_X\neq \mu_0\) then the \(p\)-value of the test can be found by using the approach in exactly one of the following three situations:
1. If \(\overline{X}\lt \mu_0\) then \(p\) is calculated by any of the following three ways:
  1. two times the area to the left of \(\overline{X}\) under the graph of a \(N(\mu_0, \sigma_X/\sqrt{n})\) distribution,
    🔗
    
    🔗
  2. 2 * normalcdf\((-9999, \overline{X}, \mu_0, \sigma_X/\sqrt{n})\) on a calculator, or
    🔗
    
    🔗
  3. 2 * NORM.DIST(x̄, μ₀, σ_X/SQRT(n), 1) on a spreadsheet.
    🔗
    
    🔗
  🔗
  🔗
2. If \(\overline{X}>\mu_0\) then \(p\) is calculated by any of the following three ways:
  1. two times the area to the right of \(\overline{X}\) under the graph of a \(N(\mu_0, \sigma_X/\sqrt{n})\) distribution,
    🔗
    
    🔗
  2. 2 * normalcdf\((\overline{X}, 9999, \mu_0, \sigma_X/\sqrt{n})\) on a calculator, or
    🔗
    
    🔗
  3. 2 * (1-NORM.DIST(x̄, μ₀, σ_X/SQRT(n), 1)) on a spreadsheet.
    🔗
    
    🔗
  🔗
  🔗
3. If \(\overline{X}=\mu_0\) then \(p=1\text{.}\)
  🔗
  
  🔗
🔗
🔗

🔗

Note the reason that there is that multiplication by two if the alternative hypothesis is \(H_a:\mu_X\neq \mu_0\) is that there are two directions — the distribution has two tails — in which the values can be more extreme than \(\overline{X}\). For this reason we have the following terminology:

🔗

Definition 6.11. One-tailed and two-tailed tests.

If we are doing a hypothesis test and the alternative hypothesis is \(H_a:\mu_X>\mu_0\) or \(H_a:\mu_X\lt \mu_0\) then this is called a one-tailed test. If, instead, the alternative hypothesis is \(H_a:\mu_X\neq\mu_0\) then this is called a two-tailed test.

🔗

Example 6.12.

Let’s do one very straightforward example of a hypothesis test:

🔗

A cosmetics company fills its best-selling 8-ounce jars of facial cream by an automatic dispensing machine. The machine is set to dispense a mean of 8.1 ounces per jar. Uncontrollable factors in the process can shift the mean away from 8.1 and cause either underfill or overfill, both of which are undesirable. In such a case the dispensing machine is stopped and recalibrated. Regardless of the mean amount dispensed, the standard deviation of the amount dispensed always has value .22 ounce. A quality control engineer randomly selects 30 jars from the assembly line each day to check the amounts filled. One day, the sample mean is \(\overline{X}=8.2\) ounces. Let us see if there is sufficient evidence in this sample to indicate, at the 1% level of significance, that the machine should be recalibrated.

🔗

The population under study is all of the jars of facial cream on the day of the 8.2 ounce sample.

🔗

The variable of interest is the weight \(X\) of the jar in ounces.

🔗

The population parameter of interest is the population mean \(\mu_X\) of \(X\text{.}\)

🔗

The two hypotheses then are

\begin{align*} H_0: \mu_X \amp= 8.1\quad \text{and}\\ H_a: \mu_X \amp\neq 8.1\ . \end{align*}

🔗

The sample mean is \(\overline{X}=8.2\text{,}\) and the sample — which we must assume to be an SRS — is of size \(n=30\text{.}\)

🔗

Using the case in Fact Assemblage where the alternative hypothesis is \(H_a:\mu_X\neq \mu_0\) and the sub-case where \(\overline{X}>\mu_0\text{,}\) we compute the \(p\)-value by

\begin{equation*} \text{2\,*\,(1-NORM.DIST(}8.2, 8.1, .22/\text{SQRT(30)}, 1\text{))} \end{equation*}

on a spreadsheet, which yields \(p=.01278\text{.}\)

🔗

Since \(p\) is not less than \(.01\text{,}\) we fail to reject \(H_0\) at the \(\alpha=.01\) level of significance.

🔗

The quality control engineer should therefore say to company management

🔗

“Today’s sample, though off weight, was not statistically significant at the stringent level of significance of \(\alpha=.01\) that we have chosen to use in these tests, that the jar-filling machine is in need of recalibration today (\(p=.01278\)).”
🔗

🔗

🔗

Subsection 6.3.4 Cautions

As we have seen before, the requirement that the sample we are using in our hypothesis test is a valid SRS is quite important. But it is also quite hard to get such a good sample, so this is often something that can be a real problem in practice, and something which we must assume is true with often very little real reason.

🔗

It should be apparent from the above Facts and Examples that most of the work in doing a hypothesis test, after careful initial set-up, comes in computing the \(p\)-value.

🔗

Be careful of the phrase statistically significant. It does not mean that the effect is large! There can be a very small effect, the \(\overline{X}\) might be very close to \(\mu_0\) and yet we might reject the null hypothesis if the population standard deviation \(\sigma_X\) were sufficiently small, or even if the sample size \(n\) were large enough that \(\sigma_X/\sqrt{n}\) became very small. Thus, oddly enough, a statistically significant result, one where the conclusion of the hypothesis test was statistically quite certain, might not be significant in the sense of mattering very much. With enough precision, we can be very sure of small effects.

🔗

Note that the meaning of the \(p\)-value is explained above in its definition as a conditional probability. So \(p\) does not compute the probability that the null hypothesis \(H_0\) is true, or any such simple thing. In contrast, the Bayesian approach to probability, which we chose not to use in the book, in favor of the frequentist approach, does have a kind of hypothesis test which includes something like the direct probability that \(H_0\) is true. But we did not follow the Bayesian approach here because in many other ways it is more confusing.

🔗

In particular, one consequence of the real meaning of the \(p\)-value as we use it in this book is that sometimes we will reject a true null hypothesis \(H_0\) just out of bad luck. In fact, if \(p\) is just slightly less than \(.05\text{,}\) we would reject \(H_0\) at the \(\alpha=.05\) significance level even though, in slightly less than one case in 20 (meaning 1 SRS out of 20 chosen independently), we would do this rejection even though \(H_0\) was true.

🔗

We have a name for this situation.

🔗

Definition 6.13. Type I error.

When we reject a true null hypothesis \(H_0\) this is called a type I error. Such an error is usually (but not always: it depends upon how the population, variable, parameter, and hypotheses were set up) a false positive, meaning that something exciting and new (or scary and dangerous) was found even though it is not really present in the population.

🔗

Example 6.14.

Let us look back at the cosmetic company with a jar-filling machine from Example Example 6.12. We don’t know what the median of the SRS data was, but it wouldn’t be surprising if the data were symmetric and therefore the median would be the same as the sample mean \(\overline{X}=8.2\text{.}\) That means that there were at least 15 jars with 8.2 ounces of cream in them, even though the jars are all labelled “8oz.” The company is giving way at least \(.2\times15=3\) ounces of the very valuable cream — in fact, probably much more, since that was simply the overfilling in that one sample.

🔗

So our intrepid quality assurance engineer might well propose to management to increase the significance level \(\alpha\) of the testing regime in the factory. It is true that with a larger \(\alpha\text{,}\) it will be easier for simple randomness to result in type I errors, but unless the recalibration process takes a very long time (and so results in fewer jars being filled that day), the cost-benefit analysis probably leans towards fixing the machine slightly too often, rather than waiting until the evidence is extremely strong it must be done.

🔗

Section 6.4 Exercises

Checkpoint 6.15.

You buy seeds of one particular species to plant in your garden, and the information on the seed packet tells you that, based on years of experience with that species, the mean number of days to germination is 22, with standard deviation 2.3 days.

🔗

What is the population and variable in that information? What parameter(s) and/or statistic(s) are they asserting have particular values? Do you think they can really know the actual parameter(s) and/or statistic’s(s’) value(s)? Explain.

🔗

You plant those seeds on a particular day. What is the probability that the first plant closest to your house will germinate within half a day of the reported mean number of days to germination — that is, it will germinate between 21.5 and 22.5 after planting?

🔗

You are interested in the whole garden, where you planted 160 seeds, as well. What is the probability that the average days to germination of all the plants in your garden is between 21.5 and 22.5 days? How do you know you can use the Central Limit Theorem to answer this problem — what must you assume about those 160 seeds from the seed packet in order for the CLT to apply?

🔗

Checkpoint 6.16.

You decide to expand your garden and buy a packet of different seeds. But the printing on the seed packet is smudged so you can see that the standard deviation for the germination time of that species of plant is 3.4 days, but you cannot see what the mean germination time is.

🔗

So you plant 100 of these new seeds and note how long each of them takes to germinate: the average for those 100 plants is 17 days.

🔗

What is a 90% confidence interval for the population mean of the germination times of plants of this species? Show and explain all of your work. What assumption must you make about those 100 seeds from the packet in order for your work to be valid?

🔗

What does it mean that the interval you gave had 90% confidence? Answer by talking about what would happen if you bought many packets of those kinds of seeds and planted 100 seeds in each of a bunch of gardens around your community.

🔗

Checkpoint 6.17.

An SRS of size 120 is taken from the student population at the very large Euphoria State University [ESU], and their GPAs are computed. The sample mean GPA is 2.71. Somehow, we also know that the population standard deviation of GPAs at ESU is .51. Give a confidence interval at the 90% confidence level for the mean GPA of all students at ESU.

🔗

You show the confidence interval you just computed to a fellow student who is not taking statistics. They ask, “Does that mean that 90% of students at ESU have a GPA which is between \(a\) and \(b\text{?}\)” where \(a\) and \(b\) are the lower and upper ends of the interval you computed. Answer this question, explaining why if the answer is yes and both why not and what is a better way of explaining this 90% confidence interval, if the answer is no.

🔗

Checkpoint 6.18.

The recommended daily calorie intake for teenage girls is 2200 calories per day. A nutritionist at Euphoria State University believes the average daily caloric intake of girls in her state to be lower because of the advertising which uses underweight models targeted at teenagers. Our nutritionist finds that the average of daily calorie intake for a random sample of size \(n=36\) of teenage girls is 2150.

🔗

Carefully set up and perform the hypothesis test for this situation and these data. You may need to know that our nutritionist has been doing studies for years and has found that the standard deviation of calorie intake per day in teenage girls is about 200 calories.

🔗

Do you have confidence the nutritionist’s conclusions? What does she need to be careful of, or to assume, in order to get the best possible results?

🔗

Checkpoint 6.19.

The medication most commonly used today for post-operative pain relief after minor surgery takes an average of 3.5 minutes to ease patients’ pain, with a standard deviation of 2.1 minutes. A new drug is being tested which will hopefully bring relief to patients more quickly. For the test, 50 patients were randomly chosen in one hospital after minor surgeries. They were given the new medication and how long until their pain was relieved was timed: the average in this group was 3.1 minutes. Does this data provide statistically significant evidence, at the 5% significance level, that the new drug acts more quickly than the old?

🔗

Clearly show and explain all your set-up and work, of course!

🔗

Checkpoint 6.20.

The average household size in a certain region several years ago was 3.14 persons, while the standard deviation was .82 persons. A sociologist wishes to test, at the 5% level of significance, whether the mean household size is different now. Perform the test using new information collected by the sociologist: in a random sample of 75 households this past year, the average size was 2.98 persons.

🔗

Checkpoint 6.21.

A medical laboratory claims that the mean turn-around time for performance of a battery of tests on blood samples is 1.88 business days. The manager of a large medical practice believes that the actual mean is larger. A random sample of 45 blood samples had a mean of 2.09 days. Somehow, we know that the population standard deviation of turn-around times is 0.13 day. Carefully set up and perform the relevant test at the 10% level of significance. Explain everything, of course.

🔗

Prev Top Next