Quasi-Experimental Research Research Methods in Psychology

Table Of Content

• Natural Experiments:
• Differences Between Quasi-Experiments And True Experiments
• Examples of quasi-experimental designs
• What are the different quasi-experimental study designs?
• Quasi-experimental Designs That Use a Control Group but No Pretest
• Quasi-Experimental Design: Methods & Applications
• Workplace financial education and change in financial knowledge: A quasi-experimental approach (Horwitz et al.,

To get the true effect of the intervention of interest, we need to control for the confounding variable. Consider a study evaluating the effectiveness of teaching modern leadership techniques in start-up businesses. Instead of artificially assigning businesses to different groups, researchers can observe those that naturally adopt modern leadership techniques and compare their outcomes to those of businesses that have not implemented such practices. Consequently, it is easier to estimate the true effect of the variable of interest on the outcome of interest. Quasi-experimental research compares groups with different circumstances or treatments to find cause-and-effect links.

Natural Experiments:

Regression to the mean all but guarantees that their scores will be higher even if the training program has no effect. A closely related concept—and an extremely important one in psychological research—is spontaneous remission. This is the tendency for many medical and psychological problems to improve over time without any form of treatment. If one were to measure symptom severity in 100 common cold sufferers today, give them a bowl of chicken soup every day, and then measure their symptom severity again in a week, they would probably be much improved. This does not mean that the chicken soup was responsible for the improvement, however, because they would have been much improved without any treatment at all.

Differences Between Quasi-Experiments And True Experiments

The study by Grant et al et al uses a variant of the SWD for which individuals within a setting are enumerated and then randomized to get the intervention. Individuals contributed follow-up time to the “pre-clinic” phase from the baseline date established for the cohort until the actual date of their first clinic visit, and also to the “post- clinic” phase thereafter. Studies using a wait-list partial randomization design are also included in Table 2 (24, 27, 42). These types of studies are well-suited to settings where there is routine enumeration of a cohort based on a specific eligibility criteria, such as enrolment in a health plan or employment group, or from a disease-based registry, such as for diabetes (27, 42). It has also been reported that this design can increase efficiency and statistical power in contrast to cluster-based trials, a crucial consideration when the number of participating individuals or groups is small (22). The true experimental design may be impossible to accomplish or just too expensive, especially for researchers with few resources.

Examples of quasi-experimental designs

A group of severely depressed people today is likely to be less depressed on average in 6 months. In reviewing the results of several studies of treatments for depression, researchers Michael Posternak and Ivan Miller found that participants in waitlist control conditions improved an average of 10 to 15% before they received any treatment at all (Posternak & Miller, 2001)[2]. Thus one must generally be very cautious about inferring causality from pretest-posttest designs. Another alternative explanation for a change in the dependent variable in a pretest-posttest design is regression to the mean.

However, a number of important confounding variables, such as severity of illness and knowledge of software users, might affect both outcome measures. Thus, if the average length of stay did not change following the intervention but pharmacy costs did, then the data are more convincing than if just pharmacy costs were measured. Quasi-experiments are studies that aim to evaluate interventions but that do not use randomization. Similar to randomized trials, quasi-experiments aim to demonstrate causality between an intervention and an outcome.

Quasi-experimental Designs That Use a Control Group but No Pretest

Environmental assessment of interventions to restrain the impact of industrial pollution using a quasi-experimental ... - BMC Public Health

Environmental assessment of interventions to restrain the impact of industrial pollution using a quasi-experimental ....

Posted: Thu, 14 Oct 2021 07:00:00 GMT [source]

The authors administered a 50-question survey to participants and non-participants within 72 hours of the program start (pre-survey) and end (post-survey). The survey questions assessed financial knowledge, financial satisfaction, money beliefs, and money behaviors. The authors conducted statistical analyses to compare differences in outcomes between program participants and non-participants. Question 1 is new and addresses the issue of clustering, either by design or through the organizational structure responsible for delivering the intervention (Box 3).

This question avoids the need for separate checklists for designs based on assigning individual and clusters. A “yes” response can be given to more than one response item; the different types clustering may both occur in a single study and implicit clustering can occur an individually allocated nonrandomized study. Researchers interested in studying the impact of a public health campaign aimed at reducing smoking rates may take advantage of a natural experiment. By comparing smoking rates in a region that has implemented the campaign to a similar region that has not, researchers can examine the effectiveness of the intervention. Imagine a study aiming to determine the effectiveness of math apps in supplementing traditional math classes in a school. Randomly assigning students to different groups might be impractical or disrupt the existing classroom structure.

Understanding these differences is crucial for researchers when selecting the most appropriate research method for their study. As well, when a complex intervention is related to a policy or guideline shift and implementation requires logistical adjustments (such as phased roll-outs to embed the intervention or to train staff), QEDs more truly mimic real world constraints. As a result, capturing processes of implementation are critical as they can describe important variation in uptake, informing interpretation of the findings for external validity. However, QEDs are often conducted by teams with strong interests in adapting the intervention or ‘learning by doing’, which can limit interpretation of findings if not planned into the design.

The researchers might also manipulate the value of the child care subsidies in order to determine if higher subsidy values might result in different levels of maternal employment. You can use two-group tests, time-series analysis, and regression analysis to analyze data in a quasi-experiment design. In a true experiment, some participants would eat junk foods, while the rest would be in the control group, adhering to a regular diet. Recruiting participants and properly designing a data-collection attribute to make the research a true experiment requires a lot of time and effort, and can be expensive if you don’t have a large funding stream. This design involves measuring the dependent variable(s) before and after an intervention or event, but without a control group.

The comparison group consisted of non-participants identified by employers and received a financial education booklet in lieu of classes. The comparison group consisted of 56 employees, 7 from Company A and 49 from Company B. The study sample was predominantly female, and a plurality of employees had incomes between $38,521 and $101,582. However, the comparison group members had a lower percentage of bachelor’s degree recipients than the program group. There are certain situations where the use of a quasi-experimental design is more suited to the study. This is especially true for studies where it would be unethical to withhold treatment from a subject on a random basis. In such situations, researchers can utilize quasi-experimental design to circumvent any ethical issues.

These quasi-experimental designs offer researchers flexible alternatives to traditional experiments, allowing exploration of cause-and-effect relationships in various contexts. This design utilizes a cutoff point or threshold to determine which participants receive the treatment or intervention. It assumes that participants on either side of the cutoff are similar in all other aspects, except for their exposure to the independent variable. At this design stage, the first step at improving internal validity would be focused on selection of a non-equivalent control group(s) for which some balance in the distribution of known risk factors is established. This can be challenging as there may not be adequate information available to determine how ‘equivalent’ the comparison group is regarding relevant covariates. This means that each person has an equivalent chance of being assigned to the experimental group or the control group, depending on whether they are manipulated or not.

For example, a cohort study can study intervention and comparator groups concurrently, with information about the intervention and comparator collected prospectively (PCS) or retrospectively (RCS), or study one group retrospectively and the other group prospectively (HCS). These different kinds of cohort study are conventionally distinguished according to the time when intervention and comparator groups are formed, in relation to the conception of the study. Some studies are sometimes incorrectly termed PCS, in our view, when data are collected prospectively, for example, for a clinical database, but when definitions of intervention and comparator required for the evaluation are applied retrospectively; in our view, this should be an RCS.

In order to assure subject safety, all researchers should have their project reviewed by the Institutional Review Boards (IRBS). True experiments, in which all the important factors that might affect the phenomena of interest are completely controlled, are the preferred design. Often, however, it is not possible or practical to control all the key factors, so it becomes necessary to implement a quasi-experimental research design. In a classic 1952 article, researcher Hans Eysenck pointed out the shortcomings of the simple pretest-posttest design for evaluating the effectiveness of psychotherapy.

Researchers typically employ it when evaluating policy or educational interventions, or in medical or therapy scenarios. This method involves using statistical tests to determine whether the results of a study are statistically significant. Inferential statistics can help researchers make generalizations about a population based on the sample data collected during the study.