The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
The text below is an excerpt from the complete document. Read the full report in PDF format.
Abstract
In previous work (Chaplin and Capizzano 2006), we evaluated a summer learning intervention that receives both federal and private funding, the Building Educated Leaders for Life (BELL) Accelerated Learning Summer Program. That study found the program to be effective using the gold standard of research methods, random assignment. This observation and interview-based process study details BELL's activities during the summers of 2004 and 2005. It examines whether program components were implemented with fidelity (i.e., as they said they would be in program documents) and describes implementation issues that may affect whether the BELL program can be replicated in other sites.
Overview
In previous work (Chaplin and Capizzano 2006), we conducted an evaluation of a summer learning intervention and found it to be effective using the gold standard of research methods, random assignment. Given those positive results, this new paper describes the specific elements of the successful program so it can be replicated, and investigates potential barriers to implementation and replication. As with most evaluation studies our study only estimated impacts of the program overall. This means that we cannot distinguish which elements caused the positive impacts. However, we can say that the elements described here were present in this successful program and thus of potential importance.
This observation and interview-based process study describes activities that occurred during the summers of 2004 and 2005 in an academically-oriented summer program that receives both federal and private funding: the Building Educated Leaders for Life (BELL) Accelerated Learning Summer Program. The BELL program is an innovative, academic summer program with a strong, asset-based youth development approach. It employs well-developed curricula in both reading and math and contains features of positive developmental settings outlined by the National Research Council and Institute of Medicine in the report, Community Programs to Promote Youth Development (2002). Most importantly, as noted above, the program has demonstrated effectiveness in reducing summer learning loss among low-income children based on both a rigorous random assignment external evaluation (Chaplin and Capizzano 2006) and two internal evaluations (T. Cooper 2002, 2003).
This report serves two purposes. First, it allows researchers to examine whether BELL's program components are implemented with fidelity (i.e., as they said they would be in program documents). This examination allows the research team to better understand the nature of the BELL intervention in practice and assist with the interpretation of outcome study results. Second, this report describes implementation issues that may affect whether the BELL program can be replicated in other sites. Issues that affect program replication often include difficulty in obtaining sustainable funding, high-quality program staff, and physical space, among others. In addition, it is also important to understand if there are any unique characteristics associated with the existing BELL sites that may not be easily reproduced. These characteristics may include an abundance of partners in the community from which to draw resources, a lack of competing programs, or other environmental characteristics that help to support the program.
This process report was developed as part of a larger study that included a random assignment impact evaluation of this summer program. Students were randomly assigned to treatment and control groups. Only the treatment group was given access to the BELL summer program. Impacts were estimated by comparing outcomes for the treatment and control groups.
Random assignment was done in both the summers of 2004 and 2005. However, student survey response rates were very low in 2004 so the impact report cited above is based on the 2005 data. This background is important for two reasons. First, random assignment in 2004 had some impacts on program operations, as noted below. Second, the impact results reported above are based on the 2005 program, which differed somewhat from the 2004 program. This report describes both the 2004 and 2005 BELL summer programs and how they differed.
End of excerpt. The entire report is available in PDF format.
Related Research
Impacts of a Summer Learning Program: A Random Assignment Study of Building Educated Leaders for Life (BELL)
The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Usage, posting and reprint of materials on the UI web site:
Most publications may be downloaded free of charge from the web site in PDF format. This information may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required.
Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact paffairs@urban.org.
If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.
Latest Research
Browse by Topic:
PDF
Printer-friendly summary
