Section 4.4 Random Variables
In the previous section we dealt with the variability of the population. Next we consider the variability of a random variable. As an example, consider taking a sample of size \(n=1\) from the population (a single person) and measuring his/her height.
The object
pop.1$height is a sequence with 100,000 entries. Think of it as a population. We will apply the function “sample” to this sequence:
sample(pop.1$height,1)
## [1] 177
The first entry to the function is the given sequence of heights. When we set the second argument to 1 then the function selects one of the entries of the sequence at random, with each entry having the same likelihood of being selected. Specifically, in this example an entry that contains the value 162 was selected. Let us run the function again:
sample(pop.1$height,1)
## [1] 168
In this instance an entry with a different value was selected. Try to run the command several times yourself and see what you get. Would you necessarily obtain a different value in each run?
Now let us enter the same command without pressing the return key:
sample(pop.1$height,1)
Can you tell, before pressing the key, what value will you get?
The answer to this question is of course No. There are 100,000 entries with a total of 94 distinct values. In principle, any of the values may be selected and there is no way of telling in advance which of the values will turn out as an outcome.
A random variable is the future outcome of a measurement, before the measurement is taken. It does not have a specific value, but rather a collection of potential values with a distribution over these values. After the measurement is taken and the specific value is revealed then the random variable ceases to be a random variable! Instead, it becomes data.
Although one is not able to say what the outcome of a random variable will turn out to be. Still, one may identify patterns in this potential outcome. For example, knowing that the distribution of heights in the population ranges between 117 and 217 centimeter one may say in advance that the outcome of the measurement must also be in that interval. Moreover, since there is a total of 3,476 subjects with height equal to 168 centimeter and since the likelihood of each subject to be selected is equal then the likelihood of selecting a subject of this height is 3,476/100,000 = 0.03476. In the context of random variables we call this likelihood probability. In the same vain, the frequency of subjects with hight 192 centimeter is 488, and therefore the probability of measuring such a height is 0.00488. The frequency of subjects with height 200 centimeter or above is 393, hence the probability of obtaining a measurement in the range between 200 and 217 centimeter is 0.00393.
Subsection 4.4.1 Sample Space and Events
Let us turn to the formal definition of a random variable: A random variable refer to numerical values, typically the outcome of an observation, a measurement, or a function thereof.
A random variable is characterized via the collection of potential values it may obtain, known as the sample space and the likelihood of obtaining each of the values in the sample space (namely, the probability of the value). In the given example, the sample space contains the 94 integer values that are marked in Figure 4.3.1. The probability of each value is the height of the bar above the value, divided by the total frequency of 100,000 (namely, the relative frequency in the population).
We will denote random variables with capital Latin letters such as \(X\text{,}\) \(Y\text{,}\) and \(Z\text{.}\) Values they may obtain will be marked by small Latin letters such as \(x\text{,}\) \(y\text{,}\) \(z\text{.}\) For the probability of values we will use the letter “\(\Prob\)”. Hence, if we denote by \(X\) the measurement of height of a random individual that is sampled from the given population then:
\begin{equation*}
\Prob(X = 168) = 0.03476
\end{equation*}
and
\begin{equation*}
\Prob(X \geq 200) = 0.00393\;.
\end{equation*}
Consider, as yet another example, the probability that the height of a random person sampled from the population differs from 170 centimeter by no more than 10 centimeters. (In other words, that the height is between 160 and 180 centimeters.) Denote by \(X\) the height of that random person. We are interested in the probability \(\Prob(|X -170| \leq 10)\).
1
The expression \(\{|X -170| \leq 10\}\) reads as “the absolute value of the difference between \(X\) and 170 is no more that 10”. In other words, \(\{-10 \leq X - 170 \leq 10\}\text{,}\) which is equivalent to the statement that \(\{160 \leq X \leq 180\}\text{.}\) It follows that \(\Prob(|X-170|\leq 10) = \Prob(160\leq X \leq 180)\text{.}\)
The random person can be any of the subjects of the population with equal probability. Thus, the sequence of the heights of the 100,000 subjects represents the distribution of the random variable \(X\text{:}\)
pop.1 <- read.csv(file="_data/pop1.csv")
X <- pop.1$height
Notice that the object “
X” is a sequence of lenght 100,000 that stores all the heights of the population. The probability we seek is the relative frequency in this sequency of values between 160 and 180. First we compute the probability and then explain the method of computation:
mean(abs(X-170) <= 10)
## [1] 0.64541
We get that the height of a person randomly sampled from the population is between 160 and 180 centimeters with probability 0.64541.
Let us produce a small example that will help us explain the computation of the probability. We start by forming a sequence with 10 numbers:
Y <- c(6.3, 6.9, 6.6, 3.4, 5.5, 4.3, 6.5, 4.7, 6.1, 5.3)
The goal is to compute the proportion of numbers that are in the range \([4,6]\) (or, equivalently, \(\{|Y-5| \leq 1\}\)).
The function “
abs” computes the absolute number of its input argument. When the function is applied to the sequence “Y-5” it produces a sequence of the same length with the distances between the components of “Y” and the number 5:
abs(Y-5)
## [1] 1.3 1.9 1.6 1.6 0.5 0.7 1.5 0.3 1.1 0.3
Compare the resulting output to the original sequence. The first value in the input sequence is 6.3. Its distance from 5 is indeed 1.3. The fourth value in the input sequence is 3.4. The difference 3.4 - 5 is equal to -1.6, and when the absolute value is taken we get a distance of 1.6.
The function “
<=” expects an argument to the right and an argument to the left. It compares each component to the left with the parallel component to the right and returns a logical value, “TRUE” or “FALSE”, depending on whether the relation that is tested holds or not:
abs(Y - 5) <= 1
## [1] FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE TRUE
Observe the in this example the function “
<=” produced 10 logical values, one for each of the elements of the sequence to the left of it. The first input in the sequence “Y” is 6.3, which is more than one unit away from 5. Hence, the first output of the logical expression is “FALSE”. On the other hand, the last input in the sequence “Y” is 5.3, which is within the range. Therefore, the last output of the logical expression is “TRUE”.
Next, we compute the proportion of “
TRUE” values in the sequence:
mean(abs(Y - 5) <= 1)
## 0.4
When a sequence with logical values is entered into the function “
mean” then the function replaces the TRUE’s by 1 and the FALSE’s by 0. The average produces then the relative frequency of TRUE’s in the sequence as required. Specifically, in this example there are 4 TRUE’s and 6 FALSE’s. Consequently, the output of the final expression is \(4/10 = 0.4\text{.}\)
The computation of the probability that the sampled height falls within 10 centimeter of 170 is based on the same code. The only differences are that the input sequence “
Y” is replaced by the sequence of population heights “X” as input. the number “5” is replaced by the number “170” and the number “1” is replaced by the number “10”. In both cases the result of the computation is the relative proportion of the times that the values of the input sequence fall within a given range of the indicated number.
The probability function of a random variable is defined for any value that the random variable may obtain and produces the distribution of the random variable. The probability function may emerge as a relative frequency as in the given example or it may be a result of theoretical modeling. Examples of theoretical random variables are presented mainly in the next two chapters.
Consider an example of a random variable. The sample space and the probability function specify the distribution of the random variable. For example, assume it is known that a random variable \(X\) may obtain the values 0, 1, 2, or 3. Moreover, imagine that it is known that \(\Prob(X=1) = 0.25\text{,}\) \(\Prob(X=2) = 0.15\text{,}\) and \(\Prob(X=3)= 0.10\text{.}\) What is \(\Prob(X=0)\text{,}\) the probability that \(X\) is equal to 0?
The sample space, the collection of possible values that the random variable may obtain is the collection \(\{0,1,2,3\}\text{.}\) Observe that the sum over the positive values is:
\begin{equation*}
\Prob(X > 0) = \Prob(X=1) + \Prob(X=2) + \Prob(X=3) = 0.25 + 0.15 + 0.10 = 0.50\;.
\end{equation*}
It follows, since the sum of probabilities over the entire sample space is equal to 1, that \(\Prob(X=0) = 1- 0.5 = 0.5\text{.}\)
| Value | Probability | Cum. Prob |
|---|---|---|
| 0 | 0.5 | 0.5 |
| 1 | 0.25 | 0.75 |
| 2 | 0.15 | 0.9 |
| 3 | 0.1 | 1 |
Table 4.4.1 summarizes the distribution of the random variable \(X\text{.}\) Observe the similarity between the probability function and the notion of relative frequency that was discussed in Chapter 2. Both quantities describe distribution. Both are non-negative and sum to 1. Likewise, notice that one may define the cumulative probability the same way cumulative relative frequency is defined: Ordering the values of the random variable from smallest to largest, the cumulative probability at a given value is the sum of probabilities for values less or equal to the given value.
Knowledge of the probabilities of a random variable (or the cumulative probabilities) enables the computation of other probabilities that are associated with the random variable. For example, considering the random variable \(X\) of Table 4.4.1, we may calculate the probability of \(X\) falling in the interval \([0.5, 2.3]\text{.}\) Observe that the given range contains two values from the sample space, 1 and 2, therefore:
\begin{equation*}
\Prob(0.5 \leq X \leq 2.3) = \Prob(X=1) + \Prob(X = 2) = 0.25 + 0.15 = 0.40\;.
\end{equation*}
Likewise, we may produce the probability of \(X\) obtaining an odd value:
\begin{equation*}
\Prob(X = \mbox{odd}) = \Prob(X=1) + \Prob(X=3) = 0.25 + 0.10 = 0.35\;.
\end{equation*}
Observe that both \(\{0.5 \leq X \leq 2.3\}\) and \(\{X = \mbox{odd}\}\) refer to subsets of values of the sample space. Such subsets are denoted events. In both examples the probability of the event was computed by the summation of the probabilities associated with values that belong to the event.
Subsection 4.4.2 Expectation and Standard Deviation
We may characterize the center of the distribution of a random variable and the spread of the distribution in ways similar to those used for the characterization of the distribution of data and the distribution of a population.
The expectation marks the center of the distribution of a random variable. It is equivalent to the data average \(\bar x\) and the population average \(\mu\text{,}\) which was used in order to mark the location of the distribution of the data and the population, respectively.
Recall from Chapter 3 that the average of the data can be computed as the weighted average of the values that are present in the data, with weights given by the relative frequency. Specifically, we saw for the data
\begin{equation*}
1,\; 1,\; 1,\; 2,\; 2,\; 3,\; 4,\; 4,\; 4,\; 4,\; 4
\end{equation*}
that
\begin{equation*}
\frac{1 + 1 + 1 + 2 + 2 + 3 + 4 + 4 + 4 + 4 + 4}{11} = 1\times \frac{3}{11} + 2 \times \frac{2}{11} + 3 \times \frac{1}{11} + 4 \times \frac{5}{11}\;,
\end{equation*}
producing the value of \(\bar x =2.727\) in both representations. Using a formula, the equality between the two ways of computing the mean is given in terms of the equation:
\begin{equation*}
\bar x = \frac{\sum_{i=1}^n x_i}{n} = \sum_x \big(x \times (f_x/n)\big)\;.
\end{equation*}
In the first representation of the arithmetic mean, the average is computed by the summation of all data points and dividing the sum by the sample size. In the second representation, that uses a weighted sum, the sum extends over all the unique values that appear in the data. For each unique value the value is multiplied by the relative frequency of the value in the data. These multiplications are summed up to produce the mean.
The expectation of a random variable is computed in the spirit of the second formulation. The expectation of a random variable is marked with the letter “\(\Expec\)” and is define via the equation:
\begin{equation*}
\Expec(X) = \sum_x \big(x \times \Prob(x)\big)\;.
\end{equation*}
In this definition all the unique values of the sample space are considered. For each value a product of the value and the probability of the value is taken. The expectation is obtained by the summation of all these products. In this definition the probability \(\Prob(x)\) replaces the relative frequency \(f_x/n\) but otherwise, the definition of the expectation and the second formulation of the mean are identical to each other.
Consider the random variable \(X\) with distribution that is described in Table 4.4.1. In order to obtain its expectation we multiply each value in the sample space by the probability of the value. Summation of the products produces the expectation (see Table 4.4.2):
\begin{equation*}
\Expec(X) = 0 \times 0.5 + 1 \times 0.25 + 2 \times 0.15 + 3\times 0.10 = 0.85\;.
\end{equation*}
| Value | Probability | Value × Probability |
|---|---|---|
| 0 | 0.5 | 0 |
| 1 | 0.25 | 0.25 |
| 2 | 0.15 | 0.3 |
| 3 | 0.1 | 0.3 |
Expectation = sum(Value×Probability) = 0.85
In the example of height we get that the expectation is equal to 170.035 centimeter. Notice that this expectation is equal to \(\mu\text{,}\) the mean of the population. This is no accident. The expectation of a potential measurement of a randomly selected subject from a population is equal to the average of the measurement across all subjects.
2
The mean of the population can be computed with the expression “
mean(pop.1$height)”
The sample variance (\(s^2\)) is obtained as the sum of the squared deviations from the average, divided by the sample size (\(n\)) minus 1:
\begin{equation*}
s^2 = \frac{\sum_{i=1}^n (x_i - \bar x)^2}{n-1}\;.
\end{equation*}
A second formulation for the computation of the same quantity is via the use of relative frequencies. The formula for the sample variance takes the form
\begin{equation*}
s^2 = \frac{n}{n-1}\sum_x \big((x - \bar x)^2\times (f_x/n)\big)\;.
\end{equation*}
In this formulation one considers each of the unique value that are present in the data. For each value the deviation between the value and the average is computed. These deviations are then squared and multiplied by the relative frequency. The products are summed up. Finally, the sum is multiplied by the ratio between the sample size \(n\) and \(n-1\) in order to correct for the fact that in the sample variance the sum of squared deviations is divided by the sample size minus 1 and not by the sample size.
In a similar way, the variance of a random variable may be defined via the probability of the values that make the sample space. For each such value one computes the deviation from the expectation. This deviation is then squared and multiplied by the probability of the value. The multiplications are summed up in order to produce the variance:
\begin{equation*}
\Var(X) = \sum_x\big( (x-\Expec(X))^2 \times \Prob(x)\big)\;.
\end{equation*}
Notice that the formula for the computation of the variance of a random variable is very similar to the second formulation for the computation of the sample variance. Essentially, the mean of the data is replaced by the expectation of the random variable and the relative frequency of a value is replaced by the probability of the value. Another difference is that the correction factor is not used for the variance of a random variable.
| \(x\) | \(P(X=x)\) | \(x - E[X]\) | \((x - E[X])^2\) | \(P(X=x)*(x - E[X])^2\) |
|---|---|---|---|---|
| 0 | 0.5 | -0.85 | 0.7225 | 0.36125 |
| 1 | 0.25 | 0.15 | 0.0225 | 0.005625 |
| 2 | 0.15 | 1.15 | 1.3225 | 0.198375 |
| 3 | 0.1 | 2.15 | 4.6225 | 0.46225 |
As an example consider the variance of the random variable \(X\text{.}\) The computation of the variance of this random variable is carried out in Table 4.4.3. The sample space, the values that the random variable may obtain, are given in the first column and the probabilities of the values are given in the second column. In the third column the deviation of the value from the expectation \(\Expec(X) = 0.85\) is computed for each value. The 4th column contains the square of these deviations and the 5th and last column involves the product of the square deviations and the probabilities. The variance is obtained by summing up the products in the last column. In the given example:
\begin{align*}
\Var(X) = \amp (0-0.85)^2 \times 0.5 + (1-0.85)^2 \times 0.25\\
\amp + (2-0.85)^2\times 0.15 + (3-0.85)^2\times 0.10= 1.0275\;.
\end{align*}
The standard deviation of a random variable is the square root of the variance. The standard deviation of \(X\) is \(\sqrt{\Var(X)} = \sqrt{1.0275} = 1.013657\text{.}\)
In the example that involves the height of a subject selected from the population at random we obtain that the variance is \(126.1576\text{,}\) equal to the population variance, and the standard deviation is 11.23199, the square root of the variance.
Other characterization of the distribution that were computed for data, such as the median, the quartiles, etc., may also be defined for random variables.
