Strong, well-executed evaluations are essential to a child welfare system that truly improves outcomes for children, youth, and families. However, limited evidence and too few rigorous studies prevent the field from identifying what works. Persistent barriers continue to limit the high-quality, systematic evaluations needed to understand program implementation, impact, and cost.

In this series of three briefs, we describe the need for rigorous evaluation of policies, programs, and practices affecting the experiences and outcomes of those served by the child welfare system in the United States, which includes child protective services, family preservation and reunification services, out-of-home care, and adoption and guardianship services supported by Title IV-E and Title IV-B of the Social Security Act.

In the Context and Landscape brief, we identify the current evidence on the effectiveness of child welfare programs and services; in the Hurdle for Agencies brief, we discuss the challenges child welfare agencies face limiting the development of systematic evidence; and in the Design Challenges brief, we discuss challenges to using various evaluation designs. Through these briefs, we hope to begin a conversation in the field about how to overcome the challenges to building knowledge of the effectiveness of child welfare services. Our aim is to equip families, child welfare agency staff, program developers, legal representatives, and advocates with insights that can guide systemwide improvements to these services.

Why This Matters

The child welfare services system needs to understand what works for whom and under what conditions to improve the lives of the children, youth, and families they serve. But limited evidence limits the field in achieving positive outcomes. Well-designed and well-executed evaluations can help improve outcomes for those served and provide guidance to practitioners on how to deliver services, but the child welfare field has suffered from too few evaluations of this caliber. Furthermore, many challenges and barriers hinder high quality, systematic evaluation needed to answer critical questions about program implementation, outcomes, and cost. And ultimately, the purpose of generating evidence is to be used and applied, which sits outside the purview of the research field.

Key Takeaways

Challenges to conducting rigorous evaluations of child welfare programs and services arise because of the following:

• Many programs are not ready for evaluation. The target population is not well-defined or identified and data are not available to estimate its size and flow.
• Impact evaluations of small programs may not be feasible, especially in rural states and for relatively small populations with unique needs.
• Resistance to randomized controlled trials (RCTs) is common but lack of outcome data eliminates many quasi-experimental designs.
• Agencies need to build evaluation planning into every stage of program development and implementation.
• Federal, state, and county agencies should invest in evaluation capacity, especially given the evaluation requirements of the Family First Preservation Services Act. In particular, Congress should increase federal funding for evaluation, creating an evaluation pool from Title IV-E funds. How We Did It

The findings in these briefs are based on years of discussions with agency administrators, program directors, and frontline workers on whether and how to evaluate their programs. Many of these discussions did not lead to conducting evaluations due to the challenges identified in these briefs.