Chapter 3 Statistical Inference - Case Study Satisfaction with Government

Subsubsection 3.3.3.1 Only one sample

Subsubsection 3.3.3.2 A Primer on Null Hypothesis Testing

• The Central Limit Theorem also gives us a rationale for quantifying uncertainty, because if repeated sample estimates can be represented as a normal distribution, then we can use the normal distribution as a frame of reference for how credible the results from our one sample are.
  🔗
• To do so, we make specific assumptions about the distribution we test our sample estimate against (e.g., a distribution that assumes no difference in means between two groups).
  🔗
• 1. We develop a research hypothesis or a so-called alternative hypothesis (e.g., women are more satisfied with the government than men).
     🔗
  2. We use a random sample and estimate the difference in means of government satisfaction between men and women.
     🔗
  3. We test our result against a (theoretical) distribution under which we assume the null hypothesis (-> there is NO difference in government satisfaction between men and women) to be true (i.e., the test distribution).
     🔗
  4. If the estimate from our sample is located near the center of the distribution that assumes the null hypothesis, this would be not much evidence for the alternative hypothesis; however: If the estimate is at the margins of that distribution, we would conclude that there is evidence that favor the alternative hypothesis, which states that in the population, women are more satisfied with the government than men.
     🔗
  🔗
• The prerequisite for this testing procedure is that we standardize our sample estimate to make it comparable to the test distribution (For example, this can be achieved by: \(z = \frac{estimate}{{standard error}}\)).
  🔗

Subsubsection 3.3.3.3 Standard error

• To determine the variation around a population parameter, we could use the standard deviation of the sampling distribution.
  🔗
• However, since in the applied case we do not have many samples, but only one, we need to estimate this variation from the one sample.
  🔗
• This estimated variation of an estimate is called the standard error, and the formula for calculating it differs depending on the estimator (usually we normalize a measure of variance or standard deviation using a measure that involves the number of observations; for example, the standard error of mean the formula is given as: \(\bar x = \frac{s}{\sqrt{n}}\)).
  🔗

Subsubsection 3.4.2.1 Mean

data_ess <- read_dta("data/ESS9_DE.dta", encoding = "latin1") 

hist(data_ess$stfgov, breaks = "FD")

summary(data_ess$stfgov)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##    0.00    3.00    4.00    4.28    6.00   10.00      66

# We store the mean, the standard deviation and the number of observations as objects, because this way we can refer to them later on
n <- 2292
xbar <- mean(data_ess$stfgov, na.rm = TRUE)
s <- sd(data_ess$stfgov, na.rm = TRUE)

# Set confidence level with 1-alpha (alpha = our willingness to be wrong in repeated samples)
conf.level <- 0.95

# Calculating the critical z-value for a two-sided test 
z <- qnorm((1 + conf.level) / 2)

# Calculate the confidence interval
lower.ci <- xbar - z * s / sqrt(n)
upper.ci <- xbar + z * s / sqrt(n)

# Print confidence intervals
cat("The", conf.level*100,"% confidence interval for the population mean is (",round(lower.ci, 2), ",", round(upper.ci, 2),").\n")

## The 95 % confidence interval for the population mean is ( 4.19 , 4.37 ).

t.test(data_ess$stfgov, conf.level = 0.95)

## 
##  One Sample t-test
## 
## data:  data_ess$stfgov
## t = 92.783, df = 2291, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  4.189216 4.370120
## sample estimates:
## mean of x 
##  4.279668

df <- data.frame(xbar, lower.ci, upper.ci)

ggplot(df, aes(x = 1, y = xbar)) +
  theme(axis.text.y = element_blank(),
        axis.ticks.x = element_blank()) +
  geom_point(size = 3, shape = 21, fill = "white", colour = "black") +
  geom_errorbar(aes(ymin = lower.ci, ymax = upper.ci), width = 0.2) +
  coord_flip() +
  labs(x = "Value", y = "Mean with 95% CI") +
  scale_x_continuous(breaks = seq(0, 10, by = 1), limits = c(1, 1))

Subsubsection 3.4.2.2 Proportion

# Recode of variable
data_ess <- data_ess %>%  
  mutate(stfgov_di =
           case_when(stfgov <= 5 ~ 0,
                     stfgov > 5 ~ 1))

# Proportion via table command
prop.table(table(data_ess$stfgov_di))

##        0        1 
## 0.693281 0.306719

# Correspondence with the mean of the summary command
summary(data_ess$stfgov_di)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.0000  0.0000  0.0000  0.3067  1.0000  1.0000      66

# Set confidence level with 1-alpha
conf.level <- 0.95

# Calculating the critical z-value for a two-sided test
z <- qnorm((1 + conf.level) / 2)

# Calculation of the confidence interval
lower.ci <- 0.3067 - z * sqrt((0.3067*(1-0.3067))/n)
upper.ci <- 0.3067 + z * sqrt((0.3067*(1-0.3067))/n)

# Print the confidence intervals
cat("The", conf.level*100, "% confidence interval for the proportion value is (", round(lower.ci, 2), ",", round(upper.ci, 2), ").\n")

## The 95 % confidence interval for the proportion value is ( 0.29 , 0.33 ).

# Test using t.test
t.test(data_ess$stfgov_di, conf.level = 0.95)

## 
##  One Sample t-test
## 
## data:  data_ess$stfgov_di
## t = 31.837, df = 2291, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  0.2878265 0.3256115
## sample estimates:
## mean of x 
##  0.306719

Subsubsection 3.4.2.3 Correlation coefficient

cor <- cor.test(data_ess$hinctnta, data_ess$stfgov)
cor

## 
##  Pearson's product-moment correlation
## 
## data:  data_ess$hinctnta and data_ess$stfgov
## t = 1.805, df = 2047, p-value = 0.07122
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.003445968  0.083023781
## sample estimates:
##        cor 
## 0.03986354

ciplot <- ggplot(data=data_ess, aes(x = hinctnta , y = stfgov)) + 
  geom_smooth(method = lm, se = TRUE, level = 0.95) +
  xlab("Income (Deciles)") +
  ylab("Satisfaction with the government")
ciplot

Subsubsection 3.5.2.1 General remarks on P-values

• The smaller the p-value, the more "statistically significant" the result (or the more reason we have to reject the null hypothesis, which, in turn, provides evidence in favor of the alternative hypothesis).
  🔗
• Contingent upon the probability of error, if p is smaller than \(\alpha\text{,}\) then the result is statistically significant (usually < 0.05 or 5%).
  🔗
• If the test statistic associated with the estimate is less than or greater than the critical z- or t-values, respectively, then p < 0.05.
  🔗
• • If the confidence interval includes the null hypothesis value (usually 0), this is consistent with a high p-value, indicating weak evidence against the null hypothesis.
    🔗
  • If the confidence interval does not include 0, it is consistent with a low p-value, suggesting strong evidence against the null hypothesis.
    🔗
  • For example: A mean difference of 0.5 with a 95% confidence interval of [0.35; 0.65] is statistically significant at the \(\alpha\)=0.05 level (reject \(H_0\)). However, a mean difference of -0.1 with a 95% confidence interval of [-0.25; 0.05] is statistically not significant at the \(\alpha\)=0.05 level (confidence intervals include 0, fail to reject \(H_0\)).
    🔗
  🔗
• To calculate the exact p-value, we rely on tables or statistical software.
  🔗
• A p-value says nothing about the probability that \(H_0\) or \(H_A\) is true; remember \(P\)(estimate | \(H_0\))!
  🔗
• When we formulate one-sided hypotheses but use a two-sided test (which is the norm in social science research), we are especially careful and do not want to make a 1st kind error (\(H_0\) is true, but we reject it); we can divide the p-value by 2 and end up with the p-value we would have gotten if we tested one-sided.
  🔗

Subsubsection 3.5.2.2 Test implementation

mean(data_ess$stfgov[data_ess$hinctnta_di==1], na.rm = TRUE)-
  mean(data_ess$stfgov[data_ess$hinctnta_di==0], na.rm = TRUE)

## [1] 0.1543566

t.test(data_ess$stfgov[data_ess$hinctnta_di==1], data_ess$stfgov[data_ess$hinctnta_di==0])

##  Welch Two Sample t-test
## 
## data:  data_ess$stfgov[data_ess$hinctnta_di == 1] and data_ess$stfgov[data_ess$hinctnta_di == 0]
## t = 1.5882, df = 2046.8, p-value = 0.1124
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.03624397  0.34495717
## sample estimates:
## mean of x mean of y 
##  4.354545  4.200189