Skip to main content

Think Stats Exploratory Data Analysis in Python

Allen B. Downey

Exercises 3.7 Exercises

For the exercises in this chapter, we’ll use the NSFG respondent file, which contains one row for each respondent. Instructions for downloading the data are in the notebook for this chapter.
🔗
Listing 3.7.1. Python Code
🔗
download("https://github.com/AllenDowney/ThinkStats/raw/v3/data/2002FemResp.dct")
download("https://github.com/AllenDowney/ThinkStats/raw/v3/data/2002FemResp.dat.gz")
The nsfg.py module provides a function that reads the respondent file and returns a DataFrame.
🔗
Listing 3.7.2. Python Code
🔗
$ from nsfg import read_fem_resp

resp = read_fem_resp()
resp.shape
(7643, 3092)
This DataFrame contains 7643 rows and 3092 columns.
🔗

1. Exercise 3.1.

Select the column numbabes, which records the "number of babies born alive" to each respondent. Make a FreqTab object and display the frequencies of the values in this column. Check that they are consistent with the frequencies in the code book. Are there any special values that should be replaced with NaN?
🔗
Then make a Pmf object and plot it as a bar graph. Is the distribution symmetric, skewed to the left, or skewed to the right?
🔗
Listing 3.7.3. Python Code
🔗
pmf.bar()
decorate(xlabel="Number of babies", ylabel="PMF")
🔗

2. Exercise 3.2.

In the same way that the mean identifies a central point in a distribution, and variance quantifies its spread, there is another statistic, called skewness, that indicates whether a distribution is skewed to the left or right.
🔗
Given a sample, we can compute the skewness by computing the sum of the cubed deviations and dividing by the standard deviation cubed. For example, here’s how we compute the skewness of numbabes.
🔗
Listing 3.7.4. Python Code
🔗
numbabes = resp["numbabes"].replace(97, np.nan)
Listing 3.7.5. Python Code
🔗
$ deviations = numbabes - numbabes.mean()
skewness = np.mean(deviations**3) / numbabes.std(ddof=0) ** 3
skewness
np.float64(1.7018914266755958)
A positive value indicates that a distribution is skewed to the right, and a negative value indicates that it is skewed to the left.
🔗
If you are given a Pmf, rather than a sequence of values, you can compute skewness like this:
🔗
  1. Compute the deviation of each quantity in the Pmf from the mean.
    🔗
    🔗
  2. Cube the deviations, multiply by the probabilities in the Pmf, and add up the products.
    🔗
    🔗
  3. Divide the sum by the standard deviation cubed.
    🔗
    🔗
Write a function called pmf_skewness that takes a Pmf object and returns its skewness. Use your function and the Pmf of numbabes to compute skewness, and confirm you get the same result computed above.
🔗
🔗

3. Exercise 3.3.

Something like the class size paradox appears if you survey children and ask how many children are in their family. Families with many children are more likely to appear in your sample, and families with no children have no chance to be in the sample at all.
🔗
From resp, select numkdhh, which records the number of children under 18 in each respondent’s household. Make a Pmf of the values in this column.
🔗
Use the bias function to compute the distribution we would see if we surveyed the children and asked them how many children under 18 (including themselves) are in their household.
🔗
Plot the actual and biased distributions, and compute their means.
🔗
🔗
🔗
PrevTopNext