The child welfare field needs to know more about what works to support children and families. Conducting an impact evaluation is one way to learn what works. Randomized controlled trials are the most reliable impact evaluations. This report is intended to help child welfare administrators and staff understand more about RCTs. You will learn about what RCTs are, the value of conducting RCTs, the ethics of RCTs, what it takes to implement an RCT, and ways to customize an RCT to meet your needs.

Key Findings and Highlights

An RCT evaluation uses a random process to decide who is offered a program (the treatment group) and who is not offered the program (the comparison group). A random process means the decision of whether they are in the treatment or comparison group does not depend on who they are or how they were referred.

To implement an RCT, you should have some things in place:

1. a well-developed program model that answers the who, what, why, and how of your program
2. strong collaboration and communication among the agency, service providers, program developers, evaluators, funders, and caseworkers
3. a well-managed eligibility and referral process for identifying and referring individuals eligible for the program
4. data to measure baseline characteristics, program participation, and outcomes

Once the program is ready for the RCT, there are many different types of RCTs that one could choose, including individual, cluster, randomized waitlist, or step wedge cluster RCT. There are also many ways to customize an RCT, including choosing the randomization ratio, blocks, flow of randomization, and stratification. 

Finally, once the RCT has begun it is important to monitor randomization. Monitoring an RCT is an important step to ensuring a good quality study. You should use program and administrative data to make sure people randomized to the treatment group get offered the program and those in the control group do not get offered the program.

Recommendations

Establishing evidence of which programs work in child welfare, and which do not, is important. You want to produce evidence that what you are doing works. The best way to do this is to conduct an RCT if possible. RCTs are basically lotteries that you can use to tell if your program works. The information in this report can help you get started as you work with an evaluator to evaluate your program.

This resource is part of the Roadmaps to Building Evidence in Child Welfare series—a collection of instructional resources about conducting child welfare evaluations. You can find more practical guidance on data, evaluation, and evidence in the child welfare field from CWEST in materials from the Child Welfare Evidence Building Academy, a program of trainings for child welfare agency staff, practitioners, and evaluators.