The child welfare field requires more and better evidence about what works to support children and families. Conducting rigorous impact evaluations is one way to generate that evidence. As an alternative to randomized controlled trials (RCTs) (also known as experimental designs), quasi-experimental designs (QEDs) can also yield reliable evidence about program effectiveness and may be more appropriate in certain contexts.

Key Findings and Highlights

This report defines QEDs, summarizes their benefits and challenges, and provides an overview of four common QEDs (matched-group, interrupted time series, difference-in-difference, and regression discontinuity). It uses child welfare examples to show where and how you might employ these evaluation designs. In this report, we highlight a few main points:

• Sometimes, RCTs are impossible, impractical, or not appropriate. This may occur because there isn’t excess demand for services or the type of program isn’t a good fit for an RCT. In those cases, QEDs can offer an alternative evaluation method.
• QEDs do not randomly assign people to treatment and comparison groups. Instead, QEDs use other methods to identify a comparison group.
• QEDs try to make the comparison group as similar as possible to the treatment group based on pre-intervention (i.e., baseline) characteristics such as age, gender, race and ethnicity, or other characteristics relevant to the context and the intervention.

Recommendations

Establishing evidence of which programs work in child welfare, and which do not, is important. This information helps us provide effective services so that children can grow up in safe and nurturing environments. We recommend you use impact evaluations to test the effectiveness of well-defined interventions. Impact evaluations can take the form of an RCT or a QED. When an RCT is not appropriate, we suggest you explore the possibility of using a QED to test your program’s effectiveness.

This resource is part of the Roadmaps to Building Evidence in Child Welfare series—a collection of instructional resources about conducting child welfare evaluations. You can find more practical guidance on data, evaluation, and evidence in the child welfare field from CWEST in materials from the Child Welfare Evidence Building Academy, a program of trainings for child welfare agency staff, practitioners, and evaluators.