A diagram showing the six steps of research, arranged in a cycle: Ask, Design, Collect, Classify and summarise, Analyse, Report. Step 2 (Design) is highlighted, captioned “Types of study designs”.
Chapter 2 introduced four types of research questions: descriptive, relational, repeated-measures and correlational. This chapter discusses the types of research studies needed to answer these RQs, while Chapter 5 to Chapter 9 discuss the details of designing these studies and collecting the data.
[59] studied the percentage of people in Hong Kong wearing face masks in various situations. A descriptive RQ is being asked: the population is “residents of Hong Kong”, and the outcome is (for example) “the percentage who wear face masks when taking care of family members with fever”. Answering this RQ requires a descriptive study.
Observational studies are used for RQs with no intervention. They are commonly-used, and sometimes are the only type of research design possible. Observational studies do not have an intervention, and hence have conditions (Definition 2.42) rather than treatments.
Among Australian teenagers with a common cold, is the average duration of cold symptoms shorter for teens taking a daily dose of echinacea compared to teens taking no medication?
This RQ has a between-individuals comparison, so is a relational RQ. If the researchers do not impose the taking of echinacea (that is, the individuals make this decision themselves), the study is observational. The two conditions are “taking echinacea”, and “not taking echinacea” (Figure 4.5).
A diagram of an observational study with a relational RQ showing two conditions: echinacea and no echinacea, with dashed lines indicating steps not under researcher control.
[60] recorded the weights of university students at the beginning of university, and then after \(12\) weeks from the same students. The comparison is within individuals; this is a repeated-measures (paired) RQ. Since the researchers do not impose anything on the students, there is no intervention (Figure 4.7).
The outcome is the average weight. The response variable is the weight of individuals. The within-individuals comparison is the week of the university semester (\(1\) and \(12\)).
A diagram of an observational study with a repeated-measures RQ, showing weight measured at weeks 1 and 12 with dashed lines indicating steps not under researcher control.
[70] explored the relationship between time since death, and the concentration of sodium in synovial (knee) fluid. This is a correlational RQ as groups are not being compared. The time since death is the explanatory variable, and the concentration of sodium in synovial fluid is the response variable. The researchers do not impose the time since death, so there is no intervention (Figure 4.9).
A diagram of an observational study with a correlational RQ, showing time since death as the explanatory variable and synovial sodium as the response variable, with dashed lines indicating steps not under researcher control.
Experimental studies, or experiments, are used for RQs with an intervention, and are commonly-used. Well-designed experimental studies can establish a cause-and-effect relationship between the response and explanatory variables. However, using experimental studies is not always possible. In general, well-designed experimental studies are more likely to be internally valid than observational studies. Experimental studies have an intervention, and hence treatments (Definition 2.41).
In an experimental study, the unit of analysis (Definition 2.53) is the smallest collection of units of observations that can be randomly allocated to separate treatments.
This RQ uses a within-individuals comparison (before and after walking \(400\ms\)) so is a repeated-measures (and paired) RQ. The study has an intervention if researchers impose the \(400\ms\) walk on the subjects (Figure 4.13). The outcome is the average heart rate. The response variable is the heart rate for each individual man.
[82] studied leaf-drip irrigation, exploring the relationship between the water pressure and flow rate. This is a correlational RQ, where the hydraulic pressure time is the explanatory variable, and the flow rate is the response variable. The researchers imposed nine different values for water pressure, so there is an intervention (Figure 4.15).
True experiments are commonly used to answer relational RQs. An example of a true experiment is a randomised controlled trial, often used in drug trials.
The echinacea study (Section 2.11) could be designed as a true experiment. The researchers would allocate individuals to one of two groups, and then decide which group took echinacea and which group did not (Figure 4.19).
These steps may happen implicitly: researchers may allocate each person at random to one of the two groups (echinacea; no echinacea). This is still a true experiment, since the researchers could decide to switch which group receives echinacea; ultimately, the decision is still made by the researchers.
A diagram of a true experimental study showing researchers allocating individuals to two groups (echinacea and no echinacea) and treatments to those groups.
Quasi-experiments are similar to true experiments (i.e., answer relational RQs) but treatments are allocated to groups that already exist (e.g., may be naturally occurring).
The echinacea study (Section 2.11) could be designed as a quasi-experiment. The researchers could find two existing groups of people (say, from Suburbs A and B), then decide to allocate people in Suburb A to take echinacea, and people in Suburb B to not take echinacea (Figure 4.22).
A diagram of a quasi-experimental study showing researchers allocating treatments to existing groups (echinacea and no echinacea), with dashed lines indicating steps not under researcher control.
Figure4.22.Quasi-experimental studies: researchers do not allocate individuals to groups, but do allocate treatments to groups. The dashed lines indicate steps not under the control of the researchers.
A researcher wants to examine the effect of an alcohol awareness program (based on [61]) on the average amount of alcohol consumed per student in a university Orientation Week. She runs the program at University A only, then compares the average amount of alcohol consumed per person at two universities (A and B).
This study is a quasi-experiment since the researcher did not (and can not) determine the groups: the students (not the researcher) would have chosen University A or University B for many reasons. However, the researcher did decide whether to allocate the program to University A or University B.
In experimental studies, researchers create differences in the values of the explanatory variable through allocation, and then note the effect this has on the values of the response variable. In observational studies, researchers observe differences in the values of the explanatory variable, and observe the values of the response variable.
Importantly, only well-designed true experiments can show cause-and-effect. Nonetheless, well-designed observational and quasi-experimental studies can provide evidence to support cause-and-effect conclusions, especially when supported by other evidence. Although only true experimental studies can show cause-and-effect, true experimental studies are often not possible for ethical, financial, practical and/or logistical reasons.
The advantages and disadvantages of each study type are discussed later (Section 8.2), after these study types are discussed in greater detail in the following chapters.
Many studies report that the bacteria in the gut of people on the autism spectrum is different from the bacteria in the gut of people not on the autism spectrum [55][50], and suggest the bacteria may contribute whether a person is autistic. These studies were observational, so the suggestion of a cause-and-effect relationship may be inaccurate.
Other studies [83] suggest that people on the autism spectrum are more likely to be “picky eaters”, which contributes to the differences in gut bacteria.
Forward direction (Subsection 4.6.1): the values of the explanatory variable are obtained, and the study determines what values of the response variable occur in the future. All experimental studies have a forward direction.
Backward direction (Subsection 4.6.2): the values of the response variable are obtained, then the study determines what values of the explanatory variable occurred in the past.
Directionality is important for understanding cause-and-effect relationships. If the explanatory variable occurs before the outcome is observed, a cause-and-effect relationship may be possible. That is, studies with a forward direction are more likely to provide evidence of causality.
[35] compared \(25\) people with legionella infections with \(75\) similar people without the infection, and found that recent (past) use of potting mix was associated with an increase in the risk of contracting illness.
Research studies are sometimes described as “prospective” or “retrospective”, but these terms can be misleading [72] and their use not recommended [7].
Observational studies with a forward direction are often called cohort studies. Both experimental studies and cohort studies can be expensive and tricky: tracking individuals (a cohort) into the future is not always easy, and the ability to track some individuals into the future may be lost (drop-outs): plants or animals may die, people may move or decide to no longer participate, etc. Forward-directional observational studies:
may add support to cause-and-effect conclusions, since the comparison occurs before the outcome (only well-designed experimental studies can establish cause-and-effect).
[11] studied dogs and cats who had been recommended to receive intermittent nasogastric tube (NGT) aspiration for up to \(36\hs\text{.}\) Some pet owners did not give permission for NGT, while some did; thus, whether the animal received NGT was not determined by the researchers (the study is observational). The researchers then observed whether the animals developed hypochloremic metabolic alkalosis (HCMA) in the next \(36\hs\text{.}\)
Since the explanatory variable (whether NGT was used) was recorded at the start of the study, and the response variable (whether HCMA was observed) was determined within the following \(36\hs\text{,}\) this study has a forward direction.
Observational studies with a backward direction are often called case-control studies. The “cases” are often individuals with a certain disease, and then the controls are those without the disease (see Example 4.27). Researchers find individuals with specific values of the response variable (cases and controls), and determine values of the explanatory variable from the past. Case-control studies:
The response variable (whether the respondent had smnd) is assessed when the study begins, and whether subjects had exposure to metals (explanatory variable) is determined from the past. This observational study has a backward direction.
[25] asked older Australians their opinions of their own food security, and recorded their living arrangements. Individuals’ responses to both the response variable and explanatory variable were gathered at the same time. This observational study is non-directional.
Choosing the type of study is only one part of research design; many other decisions must be made also. The purpose of these decisions is to ensure researchers can confidently study the relationship between the response and explanatory variables (internal validity) in the population of interest (external validity) from studying one of the many possible samples. This is related to the idea of bias.
Various types of bias are possible, some of which are studied later. Maximising internal and external validity reduces bias. Bias may occur during research design, sample selection (Section 6.7), data collection (Section 6.7), analysis, or interpretation of results (Chapter 8). This book only discusses some possible biases.
Three types of research studies are: descriptive studies (for studying descriptive RQs), observational studies (for studying relationships without an intervention), and experimental (for studying relationships with an intervention).
Observational studies can be classified as having a forward direction (cohort studies), backward direction (case-control studies), or no direction (cross-sectional studies). Experimental studies always have a forward direction. Relational RQs with an intervention can be classified as true experiments or quasi-experiments. Cause-and-effect conclusions can only be made from well-designed true experiments.
Ideally studies should be designed to be internally and externally valid. In general, experimental studies have better internal validity than observational studies.
In a true experiment, the researchers apply treatments to groups that they have determined; in a quasi-experiment, the researchers apply treatments to groups that they have not determined.
[20] studied the time needed for organic waste to turn into compost. For some batches of compost, earthworms were added. In other batches, earthworms were not added to the waste. One RQ asked whether the composting times for waste with and without earthworms was the same or not.
[46] studied recycled concrete beams. Beams were divided into three groups, different loads were then applied to each group, then the shear strength needed to fracture the beams was measured. Is this a quasi-experiment or a true experiment? Explain.
A research study compared the use of two different education programs to reduce the percentage of patients experiencing ventilator-associated pneumonia (VAP). Paramedics from two cities were chosen to participate. Paramedics in City A were allocated to receive Program 1, and paramedics in the other city to receive Program 2.
[21] compared “the effectiveness of alternating pressure air mattresses vs. overlays, to prevent pressure ulcers” (p. \(2\,099\)). Patients were provided with alternating pressure air overlays (in 2001) or alternating pressure air mattresses (in 2006). The number of pressure ulcers were recorded.
[36] compared four weight-loss diets, using \(811\) overweight adults each randomly assigned to one diet. The diets used comparable foods. The authors state (p. 859):
The primary outcome was the change in body weight after \(2\) years in […] comparisons of low fat versus high fat and average protein versus high protein and in the comparison of highest and lowest carbohydrate content.
