Definition 5.1.
A [population] parameter is a number which is computed by knowing the values of a variable for every individual in the population.
Chapter 5 Bringing Home the Data
In this chapter, we start to get very practical on the matter of tracking down good data in the wild and bringing it home. This is actually a very large and important subject — there are entire courses and books on Experimental Design, Survey Methodology, and Research Methods specialized for a range of particular disciplines (medicine, psychology, sociology, criminology, manufacturing reliability, etc.) — so in this book we will only give a broad introduction to some of the basic issues and approaches.
The first component of this introduction will give several of the important definitions for experimental design in the most direct, simplest context: collecting sample data in an attempt to understand a single number about an entire population. As we have mentioned before, usually a population is too large or simply inaccessible and so to determine an important feature of a population of interest, a researcher must use the accessible, affordable data of a sample. If this approach is to work, the sample must be chosen carefully, so as to avoid the dreaded bias. The basic structure of such studies, the meaning of bias, and some of the methods to select bias-minimizing samples, are the subject of the first section of this chapter.
It is more complicated to collect data which will give evidence for causality, for a causal relationship between two variables under study. But we are often interested in such relationships — which drug is a more effective treatment for some illness, what advertisement will induce more people to buy a particular product, or what public policy leads to the strongest economy. In order to investigate causal relationships, it is necessary not merely to observe, but to do an actual experiment; for causal questions about human subjects, the gold standard is a randomized, placebo-controlled, double-blind experiment, sometimes called simply a randomized, controlled trial [RCT], which we describe in the second section.
There is something in the randomized, controlled experiment which makes many people nervous: those in the control group are not getting what the experimenter likely thinks is the best treatment. So, even though society as a whole may benefit from the knowledge we get through RCTs, it almost seems as if some test subjects are being mistreated. While the scientific research community has come to terms with this apparent injustice, there are definitely experiments which could go too far and cross an important ethical lines. In fact, history has shown that a number of experiments have actually been done which we now consider to be clearly unethical. It is therefore important to state clearly some ethical guidelines which future investigations can follow in order to be confident to avoid mistreatment of test subjects. One particular set of such guidelines for ethical experimentation on human subjects is the topic of the third and last section of this chapter.
Section 5.1 Studies of a Population Parameter
Suppose we are studying some population, and in particular a variable defined on that population. We are typically interested in finding out the following kind of characteristic of our population:
Example 5.2.
If \(X\) is a quantitative variable on some population, the population mean \(\mu_X\) of \(X\) is a population parameter — to compute this mean, you need to add together the values of \(X\) for all of individuals in the population. Likewise, the population standard deviation \(\sigma_X\) of \(X\) is another parameter.
For example, we asserted in Example Example 4.70 that the heights of adult American men are \(N(69, 2.8)\text{.}\) Both the \(69\) and \(2.8\) are population parameters here.
Example 5.3.
If, instead, \(X\) were a categorical variable on some population, then the relative frequency (also called the population proportion) of some value \(A\) of \(X\) — the fraction of the population that has that value — is another population parameter. After all, to compute this fraction, you have to look at every single individual in the population, all \(N\) of them, say, and see how many of them, say \(N_A\text{,}\) make the \(X\) take the value \(A\text{,}\) then compute the relative frequency \(N_A/N\text{.}\)
Sometimes one doesn’t have to look at the specific individuals and compute that fraction \(n_A/N\) to find a population proportion. For example, in Example Example 4.70, we found that \(14.1988\)% of adult American men are taller than \(6\) feet, assuming, as stated above, that adult American men’s heights are distributed like \(N(69, 2.8)\) — using, notice, those parameters \(\mu_X\) and \(\sigma_X\) of the height distribution, for which the entire population must have been examined. What this means is that the relative frequency of the value “yes” for the categorical variable “is this person taller than \(6\) feet?” is \(.141988\text{.}\) This relative frequency is also a parameter of the same population of adult American males.
Parameters must be thought of as fixed numbers, out there in the world, which have a single, specific value. However, they are very hard for researchers to get their hands on, since to compute a parameter, the variable values for the entire population must be measured. So while the parameter is a single, fixed value, usually that value is unknown.
What can (and does change) is a value coming from a sample.
Definition 5.4.
A [sample] statistic is a number which is computed by knowing the values of a variable for the individuals from only a sample.
Example 5.5.
Clearly, if we have a population and quantitative variable \(X\text{,}\) then any time we choose a sample out of that population, we get a sample mean and sample standard deviation \(S_x\text{,}\) both of which are statistics.
Similarly, if we instead have a categorical variable \(Y\) on some population, we take a sample of size \(n\) out of the population and count how many individuals in the sample — say \(n_A\) — have some value \(A\) for their value of \(Y\text{,}\) then the \(n_A/n\) is a statistic (which is also called the sample proportion and frequently denoted \(\widehat{p}\)).
Two different researchers will choose different samples and so will almost certainly have different values for the statistics they compute, even if they are using the same formula for their statistic and are looking at the same population. Likewise, one researcher taking repeated samples from the same population will probably get different values each time for the statistics they compute. So we should think of a statistic as an easy, accessible number, changing with each sample we take, that is merely an estimate of the thing we want, the parameter, which is one, fixed number out in the world, but hidden from out knowledge.
So while getting sample statistics is practical, we need to be careful that they are good estimates of the corresponding parameters. Here are some ways to get better estimates of this kind:
-
Pick a larger sample. This seems quite obvious, because the larger is the sample, the closer it is to being the whole population and so the better its approximating statistics will estimate the parameters of interest. But in fact, things are not really quite so simple. In many very practical situations, it would be completely infeasible to collect sample data on a sample which was anything more than a miniscule part of the population of interest. For example, a national news organization might want to survey the American population, but it would be entirely prohibitive to get more than a few thousand sample data values, out of a population of hundreds of millions — so, on the order of tenths of a percent.Fortunately, there is a general theorem which tells us that, in the long run, one particular statistic is a good estimator of one particular parameter:
The Law of Large Numbers.
Let \(X\) be a quantitative variable on some population. Then as the sizes of samples (each made up of individuals chosen randomly and independently from the population) get bigger and bigger, the corresponding sample means \(\overline{x}\) get closer and closer to the population mean \(\mu_X\text{.}\) -
Pick a better statistic. It makes sense to use the sample mean as a statistic to estimate the population mean and the sample proportion to estimate the population proportion. But it is less clear where the somewhat odd formula for the sample standard deviation came from — remember, it differs from the population standard deviation by having an \(n-1\) in the denominator instead of an \(n\text{.}\) The reason, whose proof is too technical to be included here, is that the formula we gave for \(S_X\) is a better estimator for \(\sigma_X\) than would have be the version which simply had the same \(n\) in the denominator.In a larger sense, “picking a better statistic” is about getting higher quality estimates from your sample. Certainly using a statistic with a clever formula is one way to do that. Another is to make sure that your data is of the highest quality possible. For example, if you are surveying people for their opinions, the way you ask a question can have enormous consequences in how your subjects answer: “Do you support a woman’s right to control her own body and her reproduction?” and “Do you want to protect the lives of unborn children?” are two heavy-handed approaches to asking a question about abortion. Collectively, the impacts of how a question is asked are called wording effects, and are an important topic social scientists must understand well.
-
Pick a better sample. Sample quality is, in many ways, the most important and hardest issue in this kind of statistical study. What we want, of course, is a sample for which the statistic(s) we can compute give good approximations for the parameters in which we are interested. There is a name for this kind of sample, and one technique which is best able to create these good samples: randomness.
Definition 5.6.
A sample is said to be representative of its population if the values of its sample means and sample proportions for all variables relevant to the subject of the research project are good approximations of the corresponding population means and proportions.
It follows almost by definition that a representative sample is a good one to use in the process of, as we have described above, using a sample statistic as an estimate of a population parameter in which you are interested. The question is, of course, how to get a representative sample.
The answer is that it is extremely hard to build a procedure for choosing samples which guarantees representative samples, but there is a method — using randomness — which at least can reduce as much as possible one specific kind of problem samples might have.
Definition 5.7.
Any process in a statistical study which tends to produce results which are systematically different from the true values of the population parameters under investigation is called biased. Such a systematic deviation from correct values is called bias.
The key word in this definition is systematically: a process which has a lot of variation might be annoying to use, it might require the researcher to collect a huge amount of data to average together, for example, in order for the estimate to settle down on something near the true value — but it might nevertheless not be biased. A biased process might have less variation, might seem to get close to some particular value very quickly, with little data, but would never give the correct answer, because of the systematic deviation it contained.
The hard part of finding bias is to figure out what might be causing that systematic deviation in the results. When presented with a sampling method for which we wish to think about sources of possible bias, we have to get creative.
Example 5.8.
In a democracy, the opinion of citizens about how good a job their elected officials are doing seems like an interesting measure of the health of that democracy. At the time of this writing, approximately two months after the inauguration of the \(45^{th}\) president of the United States, the widely respected Gallup polling organization reports[1] that \(56\)% of the population approve of the job the president is doing and \(40\)% disapprove. [Presumably, \(4\)% were neutral or had no opinion.]
According to the site from which these numbers are taken,
“Gallup tracks daily the percentage of Americans who approve or disapprove of the job Donald Trump is doing as president. Daily results are based on telephone interviews with approximately 1,500 national adults....”
Presumably, Gallup used the sample proportion as an estimator computed with the responses from their sample of \(1500\) adults. So it was a good statistic for the job, and the sample size is quite respectable, even if not a very large fraction of the entire adult American population, which is presumably the target population of this study. Gallup has the reputation for being a quite neutral and careful organization, so we can also hope that the way they worded their questions did not introduce any bias.
A source of bias that does perhaps cause some concern here is that phrase “telephone interviews.” It is impossible to do telephone interviews with people who don’t have telephones, so there is one part of the population they will miss completely. Presumably, also, Gallup knew that if they called during normal working days and hours, they would not get working people at home or even on cell phones. So perhaps they called also, or only, in the evenings and on weekends — but this approach would tend systematically to miss people who had to work very long and/or late hours.
So we might worry that a strategy of telephone interviews only would be biased against those who work the longest hours, and those people might tend to have similar political views. In the end, that would result in a systematic error in this sampling method.
Another potential source of bias is that even when a person is able to answer their phone, it is their choice to do so: there is little reward in taking the time to answer an opinion survey, and it is easy simply not to answer or to hang up. It is likely, then, that only those who have quite strong feelings, either positive or negative, or some other strong personal or emotional reason to take the time, will have provided complete responses to this telephone survey. This is potentially distorting, even if we cannot be sure that the effects are systematically in one direction or the other.
[Of course, Gallup pollsters have an enormous amount of experience and have presumably thought the above issues through completely and figure out how to work around it — but we have no particular reason to be completely confident in their results other than our faith in their reputation, without more details about what work-arounds they used. In science, doubt is always appropriate.]
One of the issues we just mentioned about the Gallup polling of presidential approval ratings has its own name:
Definition 5.9.
A sample selection method that involves any substantial choice of whether to participate or not suffers from what is called voluntary sample bias.
Voluntary sample bias is incredibly common, and yet is such a strong source of bias that it should be taken as a reason to disregard completely the supposed results of any study that it affects. Volunteers tend to have strong feelings that drive them to participate, which can have entirely unpredictable but systematic distorting influence on the data they provide. Web-based opinion surveys, numbers of thumbs-up or -down or of positive or negative comments on a social media post, percentages of people who call in to vote for or against some public statement, etc., etc. — such widely used polling methods produce nonsensical results which will be instantly rejected by anyone with even a modest statistical knowledge. Don’t fall for them!
We did promise above one technique which can robustly combat bias: randomness. Since bias is based on a systematic distortion of data, any method which completely breaks all systematic processes in, for example, sample selection, will avoid bias. The strongest such sampling method is as follows.
Definition 5.10.
A simple random sample [SRS] is a sample of size \(n\text{,}\) say, chosen from a population by a method which produces all samples of size \(n\) from that population with equal probability.
It is oddly difficult to tell if a particular sample is an SRS. Given just a sample, in fact, there is no way to tell — one must ask to see the procedure that had been followed to make that sample and then check to see if that procedure would produce any subset of the population, of the same size as the sample, with equal probability. Often, it is easier to see that a sampling method does not make SRSs, by finding some subsets of the population which have the correct size but which the sampling method would never choose, meaning that they have probability zero of being chosen. That would mean some subsets of the correct size would have zero probability and others would have a positive probability, meaning that not all subsets of that size would have the same probability of being chosen.
Note also that in an SRS it is not that every individual has the same probability of being chosen, it must be that every group of individuals of the size of the desired sample has the same probability of being chosen. These are not the same thing!
Example 5.11.
Suppose that on Noah’s Ark, the animals decide they will form an advisory council consisting of an SRS of 100 animals, to help Noah and his family run a tight ship. So a chimpanzee (because it has good hands) puts many small pieces of paper in a basket, one for each type of animal on the Ark, with the animal’s name written on the paper. Then the chimpanzee shakes the basket well and picks fifty names from the basket. Both members of the breeding pair of that named type of animal are then put on the advisory council. Is this an SRS from the entire population of animals on the Ark?
First of all, each animal name has a chance of \(50/N\text{,}\) where \(N\) is the total number of types of animals on the Ark, of being chosen. Then both the male and female of that type of animal are put on the council. In other words, every individual animal has the same probability — \(50/N\) — of being on the council. And yet there are certainly collections of \(100\) animals from the Ark which do not consist of \(50\) breeding pairs: for example, take 50 female birds and 50 female mammals; that collection of \(100\) animals has no breeding pairs at all.
Therefore this is a selection method which picks each individual for the sample with equal probability, but not each collection of \(100\) animals with the same probability. So it is not an SRS.
With a computer, it is fairly quick and easy to generate an SRS:
Fact.
Suppose we have a population of size \(N\) out of which we want to pick an SRS of size \(n\text{,}\) where \(n<N\text{.}\) Here is one way to do so: assign every individual in the population a unique ID number, with say \(d\) digits (maybe student IDs, Social Security numbers, new numbers from \(1\) to \(N\) chosen in any way you like — randomness not needed here, there is plenty of randomness in the next step). Have a computer generate completely random \(d\)-digit number, one after the other. Each time, pick the individual from the population with that ID number as a new member of the sample. If the next random number generated by the computer is a repeat of one seen before, or if it is a \(d\)-digit number that doesn’t happen to be any individual’s ID number, then simply skip to the next random number from the computer. Keep going until you have \(n\) individuals in your sample.
The sample created in this way will be an SRS.
Section 5.2 Studies of Causality
If we want to draw conclusions about causality, observations are insufficient. This is because simply seeing B always follow A out in the world does not tell us that A causes B. For example, maybe they are both caused by Z, which we didn’t notice had always happened before those A and B, and A is simply a bit faster than B, so it seems always to proceed, even to cause, B. If, on the other hand, we go out in the world and do A and then always see B, we would have more convincing evidence that A causes B.
Therefore, we distinguish two types of statistical studies
Definition 5.12.
An observational study is any statistical study in which the researchers merely look at (measure, talk to, etc.) the individuals in which they are interested. If, instead, the researchers also change something in the environment of their test subjects before (and possibly after and during) taking their measurements, then the study is an experiment.
Example 5.13.
A simple survey of, for example, opinions of voters about political candidates, is an observational study. If, as is sometimes done, the subject is told something like “let me read you a statement about these candidates and then ask you your opinion again” [this is an example of something called push-polling], then the study has become an experiment.
Note that to be considered an experiment, it is not necessary that the study use principles of good experimental design, such as those described in this chapter, merely that the researchers do something to their subjects.
Example 5.14.
If I slap my brother, notice him yelp with pain, and triumphantly turn to you and say “See, slapping hurts!” then I’ve done an experiment, simply because I did something, even if it is a stupid experiment [tiny non-random sample, no comparison, etc., etc.].
If I watch you slap someone, who cries out with pain, and then I make the same triumphant announcement, then I’ve only done an observational study, since the action taken was not by me, the “researcher.”
When we do an experiment, we typically impose our intentional change on a number of test subjects. In this case, no matter the subject of inquiry, we steal a word from the medical community:
Definition 5.15.
The thing we do to the test subjects in an experiment is called the treatment.
Subsection 5.2.1 Control Groups
If we are doing an experiment to try to understand something in the world, we should not simply do the interesting new treatment to all of our subjects and see what happens. In a certain sense, if we did that, we would simply be changing the whole world (at least the world of all of our test subjects) and then doing an observational study, which, as we have said, can provide only weak evidence of causality. To really do an experiment, we must compare two treatments.
Therefore any real experiment involves at least two groups.
Definition 5.16.
In an experiment, the collection of test subjects which gets the new, interesting treatment is called the experimental group, while the remaining subjects, who get some other treatment such as simply the past common practice, are collectively called the control group.
When we have to put test subjects into one of these two groups, it is very important to use a selection method which has no bias. The only way to be sure of this is [as discussed before] to use a random assignment of subjects to the experimental or control group.
Subsection 5.2.2 Human-Subject Experiments: The Placebo Effect
Humans are particularly hard to study, because their awareness of their environments can have surprising effects on what they do and even what happens, physically, to their bodies. This is not because people fake the results: there can be real changes in patients’ bodies even when you give them a medicine which is not physiologically effective, and real changes in their performance on tests or in athletic events when you merely convince them that they will do better, etc.
Definition 5.17.
A beneficial consequence of some treatment which should not directly [e.g., physiologically] cause an improvement is called the Placebo Effect. Such a “fake” treatment, which looks real but has no actual physiological effect, is called a placebo.
Note that even though the Placebo Effect is based on giving subjects a “fake” treatment, the effect itself is not fake. It is due to a complex mind-body connection which really does change the concrete, objectively measurable situation of the test subjects.
In the early days of research into the Placebo Effect, the pill that doctors would give as a placebo would look like other pills, but would be made just of sugar (glucose), which (in those quite small quantities) has essentially no physiological consequences and so is a sort of neutral dummy pill. We still often call medical placebos sugar pills even though now they are often made of some even more neutral material, like the starch binder which is used as a matrix containing the active ingredient in regular pills — but without any active ingredient.
Since the Placebo Effect is a real phenomenon with actual, measurable consequences, when making an experimental design and choosing the new treatment and the treatment for the control group, it is important to give the control group something. If they get nothing, they do not have the beneficial consequences of the Placebo Effect, so they will not have as good measurements as the experimental group, even if the experimental treatment had no actual useful effect. So we have to equalize for both groups the benefit provided by the Placebo Effect, and give them both an treatment which looks about the same (compare pills to pills, injections to injections, operations to operations, three-hour study sessions in one format to three-hour sessions in another format, etc.) to the subjects.
Definition 5.18.
An experiment in which there is a treatment group and a control group, which control group is given a convincing placebo, is said to be placebo-controlled.
Subsection 5.2.3 Blinding
We need one last fundamental tool in experimental design, that of keeping subjects and experimenters ignorant of which subject is getting which treatment, experimental or control. If the test subjects are aware of into which group they have been put, that mind-body connection which causes the Placebo Effect may cause a systematic difference in their outcomes: this would be the very definition of bias. So we don’t tell the patients, and make sure that their control treatment looks just like the real experimental one.
It also could be a problem if the experimenter knew who was getting which treatment. Perhaps if the experimenter knew a subject was only getting the placebo, they would be more compassionate or, alternatively, more dismissive. In either case, the systematically different atmosphere for that group of subjects would again be a possible cause of bias.
Of course, when we say that the experimenter doesn’t know which treatment a particular patient is getting, we mean that they do not know that at the time of the treatment. Records must be kept somewhere, and at the end of the experiment, the data is divided between control and experimental groups to see which was effective.
Definition 5.19.
When one party is kept ignorant of the treatment being administered in an experiment, we say that the information has been blinded. If neither subjects nor experimenters know who gets which treatment until the end of the experiment (when both must be told, one out of fairness, and one to learn something from the data that was collected), we say that the experiment was double-blind.
Subsection 5.2.4 Combining it all: RCTs
This, then is the gold standard for experimental design: to get reliable, unbiased experimental data which can provide evidence of causality, the design must be as follows:
Definition 5.20.
An experiment which is
-
randomized
-
placebo-controlled.
-
double-blind
is called, for short, a randomized, controlled trial [RCT] (where the “placebo-” and “double-blind” are assumed even if not stated).
Subsection 5.2.5 Confounded Lurking Variables
A couple of last terms in this subject are quite poetic but also very important.
Definition 5.21.
A lurking variable is a variable which the experimenter did not put into their investigation.
So a lurking variable is exactly the thing experimenters most fear: something they didn’t think of, which might or might not affect the study they are doing.
Next is a situation which also could cause problems for learning from experiments.
Definition 5.22.
Two variables are confounded when we cannot statistically distinguish their effects on the results of our experiments.
When we are studying something by collecting data and doing statistics, confounded variables are a big problem, because we do not know which of them is the real cause of the phenomenon we are investigating: they are statistically indistinguishable.
The combination of the two above terms is the worst thing for a research project: what if there is a lurking variable (one you didn’t think to investigate) which is confounded with the variable you did study? This would be bad, because then your conclusions would apply equally well (since the variables are statistically identical in their consequences) to that thing you didn’t think of ... so your results could well be completely misunderstanding cause and effect.
The problem of confounding with lurking variables is particularly bad with observational studies. In an experiment, you can intentionally choose your subjects very randomly, which means that any lurking variables should be randomly distributed with respect to any lurking variables — but controlled with respect to the variables you are studying — so if the study finds a causal relationship in your study variables, in cannot be confounded with a lurking variable.
Example 5.23.
Suppose you want to investigate whether fancy new athletic shoes make runners faster. If you just do an observational study, you might find that those athletes with the new shoes do run faster. But a lurking variable here could be how rich the athletes are, and perhaps if you looked at rich and poor athletes they would have the same relationship to slow and fast times as the new- vs old-shoe wearing athletes. Essentially, the variable what kind of shoe is the athlete wearing (categorical with the two values new and old) is being confounded with the lurking variable how wealthy is the athlete. So the conclusion about causality fancy new shoes make them run faster might be false, and instead the real truth might be wealthy athletes, who have lots of support, good coaches, good nutrition, and time to devote to their sport, run faster.
If, instead, we did an experiment, we would not have this problem. We would select athletes at random — so some would be wealthy and some not — and give half of them (the experimental group) the fancy new shoes and the other half (the control group) the old type. If the type of shoe was the real cause of fast running, we would see that in our experimental outcome. If really it is the lurking variable of the athlete’s wealth which matters, then we would see neither group would do better than the other, since they both have a mixture of wealthy and poor athletes. If the type of shoe really is the cause of fast running, then we would see a difference between the two groups, even though there were rich and poor athletes in both groups, since only one group had the fancy new shoes.
In short, experiments are better at giving evidence for causality than observational studies in large part because an experiment which finds a causal relationship between two variables cannot be confounding the causal variable under study with a lurking variable.
Section 5.3 Experimental Ethics
Experiments with human subjects are technically hard to do, as we have just seen, because of things like the Placebo Effect. Even beyond these difficulties, they are hard because human subjects just don’t do what we tell them, and seem to want to express their free will and autonomy.
In fact, history has many (far too many) examples of experiments done on human subjects which did not respect their humanity and autonomy — see, for example, the Wikipedia page on unethical human experimentation[9].
The ethical principles for human subject research which we give below are largely based on the idea of respecting the humanity and autonomy of the test subjects, since the lack of that respect seems to be the crucial failure of many of the generally acknowledged unethical experiments in history. Therefore the below principles should always be taken as from the point of view of the test subjects, or as if they were designed to create systems which protect those subjects. In particular, a utilitarian calculus of the greatest good for the greatest number might be appealing to some, but modern philosophers of experimental ethics generally do not allow the researchers to make that decision themselves. If, for example, some subjects were willing and chose to experience some negative consequences from being in a study, that might be alright, but it is never to be left up to the researcher.
Subsection 5.3.1 "Do No Harm"
The Hippocratic Oath, a version of which is thought in popular culture to be sworn by all modern doctors, is actually not used much at all today in its original form. This is actually not that strange, since it sounds quite odd and archaic to modern ears — it begins
1
It dates from the \(5^{th}\) century BCE, and is attributed to Hippocrates of Kos[7].
I swear by Apollo the physician, and Asclepius, and Hygieia and Panacea and all the gods and goddesses as my witnesses that...
It also has the odd requirements that physicians not use a knife, and will remain celibate, etc.
One feature, often thought to be part of the Oath, does not exactly appear in the traditional text but is probably considered the most important promise: First, do no harm [sometimes seen in the Latin version, primum nil nocere]. This principle is often thought of as constraining doctors and other care-givers, which is why, for example, the American Medical Association forbids doctors from participation in executions, even when they are legal in certain jurisdictions in the United States.
It does seem like good general idea, in any case, that those who have power and authority over others should, at the very least, not harm them. In the case of human subject experimentation, this is thought of as meaning that researchers must never knowingly harm their patients, and must in fact let the patients decide what they consider harm to be.
Subsection 5.3.2 Informed Consent
Continuing with the idea of letting subjects decide what harms they are willing to experience or risk, one of the most important ethical principles for human subject research is that test subjects must be asked for informed consent. What this means is that they must be informed of all of the possible consequences, positive and (most importantly) negative, of participation in the study, and then given the right to decide if they want to participate. The information part does not have to tell every detail of the experimental design, but it must give every possible consequence that the researchers can imagine.
It is important when thinking about informed consent to make sure that the subjects really have the ability to exercise fully free will in their decision to give consent. If, for example, participation in the experiment is the only way to get some good (health care, monetary compensation in a poor neighborhood, a good grade in a class, advancement in their job, etc.) which they really need or want, the situation itself may deprive them of their ability freely to say no — and therefore yes, freely.
Subsection 5.3.3 Confidentiality
The Hippocratic Oath does also require healers to protect the privacy of their patients. Continuing with the theme of protecting the autonomy of test subjects, then, it is considered to be entirely the choice of subject when and how much information about their participation in the experiment will be made public.
The kinds of information protected here run from, of course, the subjects’ performance in the experimental activities, all the way to the simple fact of participation itself. Therefore, ethical experimenters must make it possible for subject to sign up for and then do all parts of the experiment without anyone outside the research team knowing this fact, should the subject want this kind of privacy.
As a practical matter, something must be revealed about the experimental outcomes in order for the scientific community to be able to learn something from that experiment. Typically this public information will consist of measures like sample means and other data which are aggregated from many test subjects’ results. Therefore, even if it were know what the mean was and that a person participated in the study, the public would not be able to figure out what that person’s particular result was.
If the researchers want to give more precise information about one particular test subject’s experiences, or about the experiences of a small enough number of subjects that individual results could be disaggregated from what was published, then the subjects’ identities must be hidden, or anonymized. This is done by removing from scientific reports all personally identifiable information [PII] such as name, social security or other ID number, address, phone number, email address, etc.
Subsection 5.3.4 External Oversight [IRB]
One last way to protect test subjects and their autonomy which is required in ethical human subject experimentation is to give some other, disinterested, external group as much power and information as the researchers themselves. In the US, this is done by requiring all human subject experimentation to get approval from a group of trained and independent observers, called the Institutional Review Board [IRB] before the start of the experiment. The IRB is given a complete description of all details of the experimental design and then chooses whether or not to give its approval. In cases when the experiment continues for a long period of time (such as more than one year), progress reports must be given to the IRB and its re-approval sought.
Note that the way this IRB requirement is enforced in the US is by requiring approval by a recognized IRB for experimentation by any organization which wants ever to receive US Federal Government monies, in the form of research grants, government contracts, or even student support in schools. IRBs tend to be very strict about following rules, and if the ever see a violation at some such organization, that organization will quickly get excluded from federal funds for a very long time. As a consequence, all universities, NGOs, and research institutes in the US, and even many private organizations or companies, are very careful about proper use of IRBs.
Section 5.4 Exercises
Checkpoint 5.24.
In practice, wording effects are often an extremely strong influence on the answers people give when surveyed. So... Suppose you were doing a survey of American voters opinions of the president. Think of a way of asking a question which would tend to maximize the number of people who said they approved of the job he is doing. Then think of another way of asking a question which would tend to minimize that number [who say they approve of his job performance].
Checkpoint 5.25.
Think of a survey question you could ask in a survey of the general population of Americans in response to which many [most?] people would lie. State what would be the issue you would be investigating with this survey question, as a clearly defined, formal variable and parameter on the population of all Americans. Also tell exactly what would be the wording of the question you think would get lying responses.
Now think of a way to do an observational study which would get more accurate values for this variable and for the parameter of interest. Explain in detail.
Checkpoint 5.26.
Many parents believe that their small children get a bit hyperactive when they eat or drink sweets (candies, sugary sodas, etc.), and so do not let their kids have such things before nap time, for example. A pediatrician at Euphoria State University Teaching Hospital [ESUTH] thinks instead that it is the parents’ expectations about the effects of sugar which cause their children to become hyperactive, and not the sugar at all.
Describe a randomized, placebo-controlled, double-blind experiment which would collect data about this ESUTH pediatrician’s hypothesis. Make sure you are clear about both which part of your experimental procedure addresses each of those important components of good experimental design.
Checkpoint 5.27.
Is the experiment you described in the previous exercise an ethical one? What must the ESUTH pediatrician do before, during, and after the experiment to make sure it is ethical? Make sure you discuss (at least) the checklist of ethical guidelines from this chapter and how each point applies to this particular experiment.
