Skip to main content

Section 5.2 Test-Retest and Internal Consistency Methods

Two commonly used indicators of a scale’s reliability are test-retest reliability and internal consistency. Test-retest reliability involves administering a scale to the same individuals on two separate occasions and calculating the correlation between the scores obtained each time. A high test-retest correlation indicates greater reliability of the scale.
On the other hand, internal consistency assesses the extent to which the items covering the scale all measure the same underlying attribute. There are various methods to measure internal consistency, with Cronbach’s alpha coefficient being the most widely used statistic which is also available in R.

Subsection 5.2.1 Cronbach’s Alpha Coefficient

Cronbach’s alpha coefficient is one of the most commonly used indicators of internal consistency. Ideally, the Cronbach alpha coefficient of a scale should be above 0.7 (DeVellis, 2012). However, Cronbach’s alpha values are influenced by the number of items in the scale. For short scales, such as those with fewer than ten items, it is common to observe relatively low Cronbach values (e.g., below 0.7). In such cases, reporting the scale’s mean inter-item correlation may be appropriate. An optimal range for the inter-item correlation is between 0.2 and 0.5 (Pallant, 2016).

Subsection 5.2.2 Importing the Data in Stata Format

First, you will download the revised data (rNCVS2016.dta) from the shared Google Drive folder containing the rNCVS2016.dta data. I removed numerous cases and variables that are less pertinent to the focus of this chapter. Next, we need to import this data into R. The data is in Stata format, indicated by the .dta extension. You can read the data from the haven package discussed in a previous chapter.
library(haven)
rNCVS2016 <- read_dta("rNCVS2016.dta")
view(rNCVS2016)

Subsection 5.2.3 Guardianship

We would like to examine the internal consistency of capable guardianship against identity theft in this dataset. Capable guardianship, the presence of a measure that deters criminal activity, involves a potential target’s ability to protect themselves. There are seven items considered to create the capable guardianship scale. Respondents were asked if they had taken seven specific self-protection measures in the past 12 months, including:
  • Checking their credit report
  • Changing passwords on financial accounts
  • Purchasing credit monitoring services or identity theft insurance
  • Destroying documents containing personally identifying information
  • Reviewing banking or credit card statements for unfamiliar charges
  • Using a security software program on their computer to protect against credit card loss or theft
  • Purchasing identity theft protection from a company offering protection services
These items were labeled as Guardian1, Guardian2, Guardian3, Guardian4, Guardian5, Guardian6, and Guardian7. The response options for each item ranged from 0 (no) to 1 (yes).

Subsection 5.2.4 Psych Package

The psych package in R is a comprehensive toolbox for psychometric and psychological research. It offers various functions for conducting various types of analyses, including factor analysis, reliability analysis, and item response theory.
install.packages("psych")
library(psych)

Subsection 5.2.5 Alpha Function

Now let’s compute Cronbach’s alpha.
# Select the columns corresponding to the Guardian variables and store them
# in a separate dataframe
guardian_data <- rNCVS2016 %>%
  select(starts_with("Guardian")) # Select columns starting with "Guardian"
# Calculate Cronbach's alpha
alpha_result <- alpha(guardian_data, check.keys = TRUE)
# Print the results
print(alpha_result)
I used the tidyverse package to select from the dataset rNCVS2016 the columns whose internal consistency I wanted to examine. Then, the alpha function from the psych package was used to calculate the Cronbach’s alpha for the selected columns of the dataset. The parameter check.keys = TRUE was included to ensure that the function checks for missing data and calculates alpha accordingly.
The syntax print(alpha_result) printed the results of the Cronbach’s alpha analysis to the console. The output includes various statistics related to internal consistency reliability, such as the Cronbach’s alpha value, item statistics, and frequencies.

Subsection 5.2.6 Reporting the Results Regarding the Internal Consistency

Here is how you report the results regarding internal consistency.
While the reliability of the capable guardianship scale (Cronbach \(\alpha = .67\)) was slightly lower than the widely accepted standard (\(\alpha = .70\text{,}\) DeVellis, 2012), this alpha is sensitive to the small number of items in the scale. Briggs and Cheek (1986) suggested that as the mean inter-item correlation (.22) for the items is above .2, this scale can be regarded as reliable, indicating that the seven items are reliable sources to capture the levels of guardianship.