urban institute press

Examining Comprehensive School Reform / Chapter One

School Reform cover imageAn Introduction to Comprehensive School Reform
Daniel K. Aladjem and Kathryn M. Borman

Urban school reformers for decades have focused on ways to improve educational outcomes for underserved and disadvantaged students. School reform has involved multiple actors at multiple levels. Local school- and district-based reformers have been assisted by a strikingly large number of federal and state policies that have evolved over the past 40 years. These policies have shifted from targeting individual students for additional assistance through Title I and other similar programs to developing and institutionalizing universal high standards governing teaching and learning for all students, emphasizing schools as the most effective sites for change. Many now assert that systemic reform strategies involving national, state, and local policies are required for long-term change (Borman and Associates 2005).

Over the past 15 years, comprehensive school reform (CSR) has emerged as a key instantiation of the "whole-school" approach to reform (Borman et al. 2004). CSR generally involves school-level adoption and implementation of externally developed, research-based school reform models and approaches. While no exact count exists, data suggest that CSR has been tried in thousands of schools nationwide.

While CSR implementation strategies, funding streams, and other structures and processes may have changed since the first models were designed in the late 1980s,1 three characteristics distinguish CSR from others.

First, model developers provide initial training or orientation to help school-level educators understand the model's organizing framework and goals.

Second, each model has a blueprint or set of specifications that vary from being more or less "prescriptive." For example, Success for All is a highly scripted model endorsed by many educators working with the least prepared students (Borman et al. 2003). In contrast, the Modern Red SchoolHouse instructional modules, at least in their earliest manifestations, were developed as site-specific curricula by teachers at participating schools.

Third, each model developer (and his or her organization) is highly sensitive to market forces, modifying, enlarging, and expanding its target audiences-state departments of education, districts, or schools—as market forces change.

The goals of the federal CSR program are to support the implementation of comprehensive school reform, especially in high-poverty Title I schools, and to improve efforts that enable all children, particularly low-achieving students in low-performing schools, to meet challenging academic standards. Schools that receive federal CSR funds must adopt approaches that address components outlined in the No Child Left Behind Act (NCLB). For all intents and purposes, this set of components can serve as a de facto definition of CSR.

  • CSR is based on scientific research and has been replicated in diverse schools;
  • CSR integrates instruction, assessment, classroom management, and professional development into a schoolwide reform plan;
  • CSR prioritizes professional development for teachers and staff;
  • CSR sets measurable goals for student performance;
  • CSR is supported by school faculty, administrators, and other staff;
  • CSR in turn, supports school faculty, administrators, and other staff;
  • CSR involves parents and the community;
  • CSR is assisted by an established comprehensive school reform entity;
  • CSR is regularly evaluated on implementation, as well as on students' outcomes; and
  • CSR coordinates federal, state, local, and private resources to help support and sustain reform.

In chapter 4, "The Influence of States and Districts on Comprehensive School Reform," Naida Tushnet and Donna M. Harris address these components in some detail, applying them to their research. Their analysis makes clear that CSR must accommodate NCLB to remain a viable alternative in reforming school practices.

Challenges and Opportunities for Comprehensive School Reform

A number of challenges confront schools and school districts using CSR models in whole-school reform, such as finding financial support for implementation. However, No Child Left Behind and, most recently, other federal policies aimed at high school reform demand increasingly higher levels of student performance as well as increasingly challenging curricula. As a result, avenues of opportunity remain open for those undertaking CSR implementation, since CSR is holistic and in line with rigorous standards. In this section, we attempt to look ahead in our analysis of the possibilities and constraints now affecting CSR adoption.

First, it has become increasingly clear to those undertaking school reform that contextual factors in the state, district, school, and classroom matter greatly (Rowan, Barnes, and Camburn 2004). In addition to the characteristics of the CSR model itself (e.g., more or less prescriptive; more or less reliant upon teachers' and principals' leadership, etc.), context arguably most affects how reform will be implemented and sustained over time. In fact, Borman and her colleagues with the National Longitudinal Evaluation of Comprehensive School Reform argue that aligning national, state, district, and school policies constitutes one of the "necessary conditions" in determining CSR outcomes (Borman et al. 2004).

Second, at this writing, how states will assist so-called low-performing schools under NCLB remains unclear. In a recent report, Laguarda (2003) argues that under NCLB's accountability provisions, states are increasingly under pressure to provide "meaningful assistance" to struggling schools. In the nine states included in Laguarda's intensive study of state-sponsored assistance to low-performing schools,2 virtually all relied upon Title I funds to support assistance efforts. Following the discontinuation of support through CSR Demonstration legislation and the Obey-Porter bill, CSR implementation has largely relied on funds from Title I sources. Whether a systemic approach to assisting low-performing schools will emerge remains unclear. Currently, most states providing assistance rely upon teams of local state or district staff, as well as field consultants charged with addressing failing schools' needs. Rather than using specific CSR packages or models, these reformers are likely to use sets of "best practices" that address the state's approach to reading and language arts, mathematics, and most recently, science. Taken together, the effects of NCLB and the end of explicit CSR funding suggest that CSR faces an uncertain future in the early years of the 21st century.

Studies Of Comprehensive School Reform

Most of the chapters in this volume draw upon research on the impact and outcomes of CSR models. The one exception is chapter 2, "The Development of Comprehensive School Reform Models," by Sally Kilgore, who draws upon her experience managing and guiding the CSR model Modern Red SchoolHouse. Kilgore explores CSR from the outset of its development and notes that CSR has been characterized by ambitious goals and by contrasting and conflicting views of how to realize them. Chapter 2 also examines the history and development of the CSR movement, tracing it from its inception in the early 1990s to the present.

Four of the chapters (3, "Comprehensive School Reform and State Accountability Systems," 6, "Mandates and Support of Comprehensive School Reform" 7, "Comprehensive School Reform vs. No Child Left Behind," and 12, "Comprehensive School Reform Models vs. Instructional Practices") are based on research conducted under the auspices of the National Longitudinal Evaluation of Comprehensive School Reform (NLECSR). The NLECSR is a quantitative and qualitative study of behavior, decisions, and processes, as well as outcomes, employing a quasi-experimental design with matched treatment and comparison schools. Having begun in the fall of 2000, NLECSR concluded in the spring of 2006.

A major purpose of the NLECSR was to determine the effects of CSR models on student achievement in more than 650 elementary and middle schools (grades 3 through 8), identifying the most effective components of CSR models as well as describing the situations and populations for which specific CSR models are most effective. We also are noting the contextual supports that contribute to a CSR model's effectiveness. The 650 participating schools are located in 21 districts, primarily urban, across 16 states. NLECSR involves longitudinal surveys of district administrators (64), principals (650), and teachers (about 5,000) over the course of three years, as well as the collection of student record data (achievement and enrollment) for those districts, schools, and classes.

To complement national survey data, NLCSR researchers also conducted qualitative research in 34 "high-performing" and "high-potential" CSR schools, located in five districts. These qualitative case studies are contributing to an understanding of both the implementation and the effects of the CSR models' key components and overall effectiveness. NLECSR researchers observed classes to evaluate instruction; interviewed teachers and administrators about instruction, implementation, and their experiences with the CSR effort; and collected extant documents about reform, student achievement, and school demographics from each school.

Three research questions, focusing on outcomes and implementation, drive the NLECSR.

  • How effective are specific externally developed, research-based CSR models in improving the achievement of all students?
  • How are model characteristics related to the success of model implementation and to improvement in teaching and learning, in what settings, and with what students?
  • What supporting conditions and strategies are necessary to effectively implement and sustain CSR models in schools and school districts?

To enhance a thorough understanding of CSR effects and to do so in the context of rigorous research, sponsors, including the U.S. Department of Education and the Bill & Melinda Gates Foundation, funded multiple studies beginning in 2000. In September 2000, the Department of Education awarded grants to six groups of researchers. Among those, in addition to American Institutes for Research and the University of South Florida, were the Success for All Foundation (SFAF), and Policy Studies Associates (PSA). The following year (2001), the Department of Education contracted with WestED to conduct an evaluation of CSR schools that received federal funding. SFAF is conducting a mixed-methods randomized experiment of Success for All under Geoffrey Borman's leadership; in chapter 9, "School-Level Factors in Comprehensive School Reform," he and his coauthors draw upon results from year two of this ongoing study. PSA's research, also quasi-experimental and mixed methods, looks more directly at the district's role in supporting CSR. Brenda J. Turnbull shows in chapter 5, "Comprehensive School Reform as a District Strategy," how district support for CSR diminished over observed years of implementation. As accountability concerns increased, CSR schools were pressured to abandon their models or combine them with other district-sanctioned approaches to curriculum and instruction. Many CSR schools also changed leadership, often due to district policies. Nonetheless, despite limited and dwindling district support, many schools continued to use CSR principles or practices as a strategy for school improvement.

Bringing an educational reform effort to scale and sustaining it past the period of funding remain challenging issues. Chapter 10, "Partnerships in Middle Grades Comprehensive School Reform," emphasizes that sharing resources, expertise, and responsibility adds value to the work that individual schools, districts, and educational organizations could accomplish on their own. Drawing upon data from three different studies, including a qualitative study funded by the W. K. Kellogg Foundation, H. Dickson Corbett, Cheri Fancsali, Pritha Gopalan, Alexandra Weinbaum, and Bruce L. Wilson suggest that educational partnerships can positively influence scale-up and sustainability as initiatives mature.

Roughly contemporaneous with the Department of Education's efforts, the Bill & Melinda Gates Foundation launched its own school reform initiative. Though not explicitly defined as a CSR effort, many of the programs funded by the Gates Foundation are CSR models. In early 2001, the foundation contracted with the American Institutes for Research and SRI International to conduct a large-scale, quasi-experimental, mixed-methods study of its efforts to reform high schools. In chapter 11, "Comprehensive School Reform in High Schools," Don Dailey, Becky Smerdon, and Barbara Means demonstrate the foundation's effectiveness in creating innovative schools by remaining open and flexible in their approach to supporting high school reforms.

Our Approach

The studies reported in this volume take a number of different methodological approaches to studying CSR. Both the researchers (who are in most cases the authors of the chapters) and the editors approached our work informed by a range of social science disciplines including sociology, political science, and policy analysis. However, underneath it all, the work represented here is fundamentally policy analytic research and evaluation.

Cuban (2001) has described the primary targets of CSR as the roughly one-third of schools whose students historically have performed poorly. These schools face particularly intractable problems of high staff turnover, inexperienced principals and teachers, limited resources, and racial strife, as well as fragmented curricula and low-quality instruction. We can identify three problems as being of central interest and importance to the analyses in this volume. The myriad other problems can all be defined as subsets of these three.

First, low-performing schools in urban settings often suffer from a lack of coherence among the instructional and related activities under-taken by school personnel. This lack of coherence includes, for example, fragmentation of the curriculum and the school day, poorly related or incompatible instructional strategies, inconsistent behavioral expectations, and a lack of shared values. These problems at the school and classroom levels collectively result in low expectations for students and generally low academic performance and achievement.

The second problem facing these schools is the lack of information, knowledge, and skills needed for effective reform. Principals, teachers, and school staff have tremendous information, knowledge, and skills but lack the time and resources to exploit them, and especially to gain new information, knowledge, and skills to undertake the radical transformations required by CSR. In the best-case scenario, most school professionals will have a year of graduate education plus a few additional courses. This formal education will likely have focused on daily instructional tasks. It is highly unlikely that school professionals will have much training in organizational theory, management, or strategic change.

Last, there are strong disincentives to enter and remain in professional careers in schools, and for those who do remain, there are precious few incentives to develop advanced pedagogical skills, let alone the skill to undertake a complex organizational transformation.

The U.S. Department of Education, New American Schools, CSR model developers, and others posit that CSR offers a solution to this problem (although they might define it somewhat differently). The overriding question for the studies presented in this volume and for others committed to CSR is whether this confidence in CSR is well founded. On its face, CSR seems promising, and research on some selected models is promising (Borman et al. 2003; Herman et al. 1999). CSR explicitly addresses the first two prongs of the problem by offering ways to foster program coherence in schools and by providing external technical assistance that should solve the information problems facing school professionals. CSR should address the third prong as well, but changing the incentives accorded principals and teachers is beyond the control of any single school. The extent to which CSR model staff work with state, district, and school staff will determine, in part, the extent to which CSR can successfully change the incentive system.3

The range of activity related to comprehensive school reform is impressive. Thousands of schools have implemented CSR programs over the past decade, using either self-developed models or externally developed models. This breadth of activity has spawned an almost equally wide variety of research into CSR. Despite the popularity of many CSR schools and models, and despite the research base on which these models rest, Herman and her colleagues' (1999) literature review demonstrated that the research on different CSR models' effects on student outcomes is relatively weak. Indeed, of the 24 models reviewed, only 3 had strong evidence of positive effects on student achievement. Similarly, Borman and his colleagues' recent review (2003) of studies on 29 widely implemented CSR models found that research evidence on CSR's effectiveness is somewhat inconsistent, with only 3 models showing strong evidence of improving student performance (a different 3, however, than those identified by Herman).

Our Purpose

While policymakers, practitioners, and researchers have paid much attention to CSR for over a decade, the debate on CSR has only recently begun to accelerate. A number of long-term studies are concluding as this volume is being prepared. Most will produce a number of journal publications in the next one to two years, drawing more attention to the debate and providing a clearer picture of CSR's implementation and impact. Journal articles, however, cannot pull together varied points of view and multiple sources of data to address the full system of influences around a reform like CSR. As schools and districts face increasing pressure to improve student achievement under NCLB, several important issues for CSR arise:

  • What support is necessary from external model providers?
  • What are the responsibilities of external model providers?
  • How does CSR intersect with state accountability systems?
  • How can states build coherent systems, or must CSR necessarily conflict with state accountability regimes?
  • How might the imperatives of accountability divert stakeholders' attention?
  • What support is necessary from local school districts?
  • What conditions are necessary to implement and sustain reform?
  • What factors produce positive student outcomes?

These issues are addressed in the chapters that follow.

The policy question that drives CSR and other school reform research is how best to improve America's schools, especially for those that schools have not traditionally served well—impoverished urban children. Our intent is not only to build knowledge for knowledge's sake, but to engineer a better society, characterized by schools that nurture the talents of all children.

Notes

1. See chapter 2 in this volume.

2. The states included in Laguarda's study are Arkansas, California, Florida, Indiana, Nevada, North Carolina, Vermont, West Virginia, and Wyoming.

3. One might argue that CSR need not address each of these aspects of the problem. If CSR is to be truly comprehensive, however, it seems not unreason-able to hold CSR models to a standard that requires addressing the full problem, not just two-thirds of it.

References

Borman, G. D., G. M. Hewes, L. T. Overman, and S. Brown. 2003. "Comprehensive School Reform and Achievement: A Meta-Analysis." Review of Educational Research 73(2): 125-230.

Borman, K. M., and Associates. 2005. Meaningful Urban Education Reform: Confronting the Learning Crisis in Mathematics and Science. Albany: SUNY Press.

Borman, K. M., K. Carter, D. K. Aladjem, and K. C. Le Floch. 2004. "Challenges for the Future of Comprehensive School Reform." In Putting the Pieces Together: Lessons from Comprehensive School Reform Research, edited by C. T. Cross (109-50). Washington, DC: George Washington University Press.

Cuban, L. 2001. "Improving Urban Schools in the 21st Century: Dos and Don'ts, or Advice to True Believers and Skeptics of Whole School Reform." Paper presented at the OERI Symposium on Comprehensive School Reform Research and Evaluation, Denver, July 18-20.

Herman, R., D. Aladjem, P. McMahon, A. O'Malley, S. Quinones, and D. Woodruff. 1999. An Educator's Guide to Schoolwide Reform. Washington, DC: American Institutes for Research.

Laguarda, K. 2003. "State-Sponsored Technical Assistance to Low-Performing Schools: Strategies from Nine States." Paper presented at the annual meeting of the American Educational Research Association, Chicago, April 23.

Rowan, B., C. Barnes, and E. Camburn. 2004. "Benefiting from Comprehensive School Reform: A Review of Research on CSR Implementation." In Putting the Pieces Together: Lessons from Comprehensive School Reform Research, edited by C. T. Cross (1-52). Washington, DC: George Washington University Press.


Examining Comprehensive School Reform, edited by Daniel K. Aladjem and Kathryn M. Borman, is available from the Urban Institute Press (paper, 6" x 9", 368 pages, ISBN 0-87766-733-0, $29.50).

 

2011 Urban Institute | Contact Us | Privacy Policy