Weighted Bootstrap

Section 13.4 Weighted Bootstrap

The NSFG dataset includes a column called finalwgt that contains each respondent’s sampling weight, which is the number of people in the population they represent. We can use these weights during the resampling process to correct for stratified sampling. The following function takes a DataFrame and the name of the column that contains the sampling weights. It resamples the rows of the DataFrame, taking the sampling weights into account, and returns a new DataFrame.

🔗

Listing 13.4.1. Python Code

🔗

def resample_rows_weighted(df, column="finalwgt"):
    n = len(df)
    weights = df[column]
    return df.sample(n, weights=weights, replace=True)

The current dataset includes respondents from several iterations of the survey, called cycles, so in order to resample, we have to group the respondents by cycle, resample each group, and then put the groups back together. That’s what the following function does.

🔗

Listing 13.4.2. Python Code

🔗

def resample_cycles(resp):
    grouped = resp.groupby("cycle")
    samples = [resample_rows_weighted(group) for _, group in grouped]
    return pd.concat(samples)

To get started, we’ll resample the data once.

🔗

Listing 13.4.3. Python Code

🔗

# Seed the random number generator so we get the same results every time
np.random.seed(1)

Listing 13.4.4. Python Code

🔗

sample = resample_cycles(resp)

Later we’ll resample the data several times, so we can see how much variation there is due to random sampling.

🔗

The following figure shows the results with resampling, compared to the results from the previous section without resampling, shown as dotted lines.

🔗

Listing 13.4.5. Python Code

🔗

for label, surv in surv_map.items():
    surv.plot(ls=":", color="gray", alpha=0.6)

survs_resampled = make_survival_map(sample, cohorts)

for label, surv in survs_resampled.items():
    surv.plot(label=label)

decorate(xlabel="Age", ylabel="Prob never married", ylim=ylim)

Listing 13.4.6. Python Code

🔗

plt.rc("axes", prop_cycle=plt.rcParamsDefault["axes.prop_cycle"])

Listing 13.4.8. Python Code

🔗

plt.rc("axes", prop_cycle=plt.rcParamsDefault["axes.prop_cycle"])

The difference, with and without resampling, is substantial, which shows that we need to correct for stratified sampling to get accurate results.

🔗

Now let’s get to the second problem, dealing with incomplete data.

🔗

Prev Top Next