Economic trends have made postsecondary education increasingly important to self-sufficiency, but research suggests that far too many young people in foster care will not have the educational credentials needed to succeed in this economy without additional supports. Specifically, young people in foster care enroll in college at lower rates than their peers and are less likely to persist through the end of their first year when they do enroll.
Policymakers’ efforts to expand access to postsecondary education among young people in foster care and to increase their postsecondary educational attainment have led to the creation of both state and federal programs that address financial barriers to pursuing a college degree. These efforts include the federal Chafee Education and Training Voucher (ETV) program, state tuition waiver programs, and targeted college scholarships.
Additionally, a growing number of college success programs are providing students who have experienced foster care with wraparound services and supports so they can succeed in school and graduate. However, little progress has been made in building the evidence base for these programs.
This brief presents the results of a formative evaluation of one college success program—the Seita Scholars program at Western Michigan University (WMU). It describes the program and the students it serves. It also explains how we tested the program’s logic model and the implications of our test results for rigorous evaluation of the program.
Primary Research Questions
- Does the Seita Scholars program have a coherent logic model?
- Is the program being implemented with fidelity to its logic model?
- Does the program have data to measure the services it provides and their intended outcomes?
- Is it likely that an evaluation would be able to detect impacts under present conditions?
The purpose of this report is to describe what we learned about the Seita Scholars program from our formative evaluation activities and share our assessment of whether this program and others like it could be rigorously evaluated.
Key Findings and Highlights
The Seita Scholars program has been providing Western Michigan University (WMU) students who are or were in foster care with comprehensive supports since the 2008–09 academic year. Students must apply to the program upon admission to WMU. Seita Scholars receive a $5,000 scholarship each semester to cover their cost of attendance and receive one-on-one coaching provided by campus coaches.
From 2008 to 2017, 454 students participated in the Seita Scholars program. The Seita Scholars are more racially diverse than WMU’s student population. On average, Seita Scholars appear less prepared for college-level work than the average WMU student as measured by high school GPAs and ACT scores. Campus coaches are an important resource for Seita Scholars, and the support they provide goes far beyond help with academics or other education-related concerns. Both the retention rate (i.e., the percentage of students who completed the spring semester of their first year and returned for the fall semester of their second year) and the six-year graduation rate are lower for Seita Scholars than for other WMU students. However, Seita Scholars are more likely than other WMU students to transfer to other schools. Their average GPA is also well above the 2.0 they are required to maintain.
Although the program’s implementation is largely consistent with its logic model and the program could reasonably be expected to achieve its intended outcomes, it would be difficult to identify a sufficient number of students who are eligible for but not participating in the Seita Scholars program and not receiving similar services from another program. Consequently, we have concluded that it would not be feasible to rigorously evaluate the Seita Scholars program at Western Michigan University, although replicating and evaluating the program at other postsecondary institutions might be possible.
Researchers from the Urban Institute and Chapin Hall at the University of Chicago engaged in two sets of activities as part of the formative evaluation. First, we conducted a pair of site visits. The purpose of these site visits was to learn about (1) the program’s logic model; (2) the way program leaders, frontline staff, program partners, and other stakeholders understand the program; (3) program participants’ experiences with the program; and (4) program or administrative data we could use. During the site visits we interviewed program leaders, frontline staff, university partners, and other stakeholders and conducted focus groups with student participants. Second, we analyzed program and administrative data on the characteristics of program participants, the services they receive, and their academic outcomes.
Developers of college success programs should consider incorporating coaching in college success programs using a model that focuses not only on education, but also on a range of life domains and should think about offering a scholarship as part of their college success program to attract potential students who might not otherwise apply to their school. Program developers should also cultivate institutional support and seek input from campus community members who can help guide decisionmaking about program development. Finally, program developers should take steps to ensure that participating in the program does not have unintended negative consequences, such as stigmatizing participants or precluding them from making choices available to other students.
Evaluators of college success programs will need to consider designs other than the traditional individual-level randomized controlled trial given the difficulty of identifying a comparison group of students who are eligible for but not participating in the program nor receiving other similar services. They should also begin working with program developers from the beginning rather than after the program has been designed and implementation has begun.