Part II Good Data
It is something of an aphorism among statisticians that
The plural of anecdote is not data.
It is hard to be certain of the true origins of this phrase. The political scientist Raymond Wolfinger is sometimes given credit[3]— for a version without the “not,” actually. Sometime later, then, it became widespread with the “not.”
The distinction being emphasized here is between the information we might get from a personal experience or a friend’s funny story — an anecdote — and the cold, hard, objective information on which we want to base our scientific investigations of the world — data.
In this Part, our goal is to discuss aspects of getting good data. It may seem counter-intuitive, but the first step in that direction is to develop some of the foundations of probability theory, the mathematical study of systems which are non-deterministic — random — but in a consistent way. The reason for this is that the easiest and most reliable way to ensure objectivity in data, to suppress personal choices which may result in biased information from which we cannot draw universal, scientific conclusions, is to collect your data randomly. Randomness is a tool which the scientist introduces intentionally and carefully, as barrier against bias, in the collection of high quality data. But this strategy only works if we can understand how to extract precise information even in the presence of randomness — hence the importance of studying probability theory.
After a chapter on probability, we move on to a discussion of some fundamentals of experimental design — starting, not surprisingly, with randomization, but finishing with the gold standard for experiments (on humans, at least): randomized, placebo-controlled, double-blind experiments [RCTs]. Experiments whose subjects are not humans share some, but not all, of these design goals.
It turns out that, historically, a number of experiments with human subjects have had very questionable moral foundations, so it is very important to stop, as we do in the last chapter of this Part, to build an outline of experimental ethics.
