Counting High School Graduates when Graduates Count

This paper is a revised version of work presented at the annual meetings of the National Economic Association on January 3, 2003, in Washington DC. Please direct correspondence to Christopher B. Swanson, The Urban Institute, Education Policy Center, 2100 M Street NW, Washington DC 20037, cswanson@ui.urban.org.

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

TABLE OF CONTENTS

Abstract

1. Introduction

2. No Child Left Behind, Accountability, and High School Graduation

3. Measurement Considerations

4. Alternative Graduation Indicators
4.1. Longitudinal Graduation Rate
4.2. NCES High School Completion Rate
4.3. The Greene Method - Cohort Graduation Rates
4.4. Cumulative Promotion Index (CPI)

5. Data and Methods

6. Results
6.1. National Estimates - Graduation Rates and Indicators of Indicator Quality
6.3. State Estimates - Graduation Rates and Indicators of Indicator Quality
6.4. Graduation Rates and District Characteristics - Minority Enrollment and Size

7. Discussion and Conclusions

8. References

9. Tables and Figures
Figure 1: Frequency Distributions for Alternative High School Graduation Rate Indicators, High School Class of 1999-2000
Table 1: Estimated Graduation Rates and Quality Indicators for Alternative Measurement Approaches, High School Class of 1999-2000
Table 2: Estimated Graduation Rates and Quality Indicators by State - NCES Method
Table 3: Estimated Graduation Rates and Quality Indicators by State - Greene Method
Table 4: Estimated Graduation Rates and Quality Indicators by State - CPI Method
Table 5: Estimated Graduation Rates and Quality Indicators for Alternative Measurement Approaches by District Racial and Ethnic Minority Composition
Table 6: Estimated Graduation Rates and Quality Indicators for Alternative Measurement Approaches by District Enrollment in Grades 9-12
Table 7: Estimated 1999-2000 Graduation Rates for the 100 Largest School Districts, Total PK-12 Enrollment

ABSTRACT

The No Child Left Behind Act of 2001 (NCLB) requires that the nation's public elementary and secondary school systems be held accountable for achieving high levels of educational proficiency for all students. This law represents an unparalleled extension of the federal role into the realm of local educational accountability. Specifically, NCLB mandates that all states establish performance-based accountability systems that include: clear standards and goals for improvement; rigorous methods of measuring progress towards established performance targets; and high-stakes consequences for schools and districts that fail to make sufficient progress in reaching the goal of universal student proficiency. While achievement testing will be the central component of these state accountability systems, high school graduation rates are also a requirement indicator of performance at the secondary level. This report seeks to inform the on-going debate over high school graduation rates with particular attention to the ways in which No Child Left Behind has effectively both redirected attention toward graduation rates and reshaped the contours of that debate. We begin by briefly introducing the provisions of NCLB that pertain to high school graduation and discussing their implications from a measurement perspective. Next we present several distinctive strategies for developing a high school graduation indicator that are broadly consistent with the new federal requirements for accountability. The advantages and disadvantages of each strategy are addressed. In the empirical portion of this study we construct these proposed high school graduation indicators using information from the Common Core of Data, the U.S. Department of Education's census of local educational agencies and schools. These indicators are created at the district level for the high school class of 2000, the most recent year for which complete data are available. We then systematically compare the results generated by the respective measures, with particular attention to: their point estimates and distributions; the quality of the indicators from an informational perspective (e.g. coverage of school districts or student population); and differential performance of the indicators when disaggregated by state and by agency characteristics such as minority enrollment and district size. In each of these areas, analyses reveal substantial differences among the three alternative graduation rate indicators examined. In particular, two troubling findings emerge regarding graduation indicators that rely heavily on information about high school dropout. First, these indicators cannot be calculated for many states and districts due to missing information on the number of students who drop out of high school. Second, in cases where values can be computed, dropout-dependent measures appear to dramatically overestimate graduation rates compared to alternative indicators. We conclude by discussing several lessons this study may offer for future research and the policy implications for measuring graduation rates under conditions of high-stakes accountability.

1. INTRODUCTION

The relationship between levels of educational attainment and individual economic and social outcomes has been well documented. Individuals with higher levels of education (and more advanced credentials) enjoy higher income, more stable employment, and less dependency on public assistance. The more well-educated are also less likely to experience a variety of detrimental social outcomes, including early childbearing, reports of ill health, incarceration, or criminal victimization. Until about the 1970s, a high school diploma was generally viewed as a credential that would ensure a reasonably secure and well-paying job. But with increasing regularity, discussions of the returns to education have focused on attaining a college education as a prerequisite for economic success in today's labor market. This shift of attention is largely justifiable. Rates of college matriculation have increased considerably over the past 20 years as have the economic returns to a bachelor's degree, often termed the "college premium" (Murphy & Welch 1989; Juhn & Murphy 1995). As a larger segment of the population completes college and enters the workforce with a postsecondary degree, it is reasonable to anticipate that the college degree will supplant a high school diploma with regard to hiring decisions and also as a more generalized indicator of educational and social status.

In light of the ascendancy of the college credential, there may be a temptation to view the high school diploma as becoming economically and socially obsolete. On the other hand, however, it is imperative that we not loose sight of the relevance that a high school education continues to hold for young adults, even today. Much like the college premium noted above, high school graduates consistently outperform non-graduates on a variety of social and economic indicators, often by substantial margins (Chaplin & Lerman 1997). Obtaining a high school diploma can also represent a critical first step in gaining access to a college education and, eventually, earning the bachelor's degree that will further enhance an individual's prospects for advancement in an increasingly competitive society.

One of the reasons that the issue of high school completion has garnered relatively minor attention in recent years could be the common perception that high school dropout rates have historically been on the decline and have stabilized at rather low levels during the past decade. In fact, widely-reported estimates based on results from the Current Population Survey (CPS) place the dropout rate among 16-24 year olds at about 11 percent nationwide in 2000, a level that some might view as nearing the limit of the high school dropout rate practically attainable (Kaufman, Alt & Chapman 2001). Despite this appearance of a growing consensus, a dissenting perspective has arisen from a body of research that has challenged the accuracy and reliability of commonly-reported statistics on high school completion and dropout (see Chaplin 2002, Greene 2002a). These studies cite several factors that tend to systematically inflate the high school completion rates commonly reported by the U.S. Department of Education, Census Bureau and other authoritative sources, particularly when these estimates are based on survey data as is the case for the CPS.¹ These potential sources of bias include: the frequent decision to count GED's along with regular diploma recipients as high school graduates, sample undercoverage for certain segments of the population (e.g. those incarcerated), and self-reporting bias regarding levels of educational attainment.²

Just as high school completion arguably remains an important predictor of an individual's future economic success, it also represents a key indicator of performance for educational systems. In addition to producing higher-achieving students, effective schools, districts or states are expected to retain a greater proportion of students until completion of secondary schooling. Consequently, such educational units should strive to achieve higher graduation rates. Public and scholarly attention to high school dropout and completion has been rejuvenated in recent years as handfuls of states and large districts around the country have introduced new educational accountability systems. This interest has been largely motivated by the fear that imposing high stakes testing, exit exams for graduation, and policies ending social promotion would create pressure for low-performing students to exit (or be removed from) the system and produce increased dropout rates (Lillard & DeCicca 2001; Herbert & Hauser 1999; Bonsteel & Rumberger 1999; Haney 2000). In addition, the trend toward more systematic forms of accountability on a national scale further raises the stakes for measuring high school completion accurately and consistently. Specifically, new federal educational legislation authorized in the No Child Left Behind Act requires that all high schools explicitly take completion rates into account (along with achievement test scores) when measuring their current level of performance and the progress they have made towards reaching their long-term performance goals.

This paper seeks to inform the debate over high school graduation rates with particular attention to the ways in which No Child Left Behind has effectively both redirected attention toward graduation rates and reshaped the contours of that debate. Since one of our main interests in this study is considering the impacts of these federal policies, we focus throughout the paper strictly on the public education system. Although subject to performance pressures from market forces, private schools are not directly subject to the performance-based accountability requirements of NCLB or other federal statutes. We begin by briefly introducing the provisions of NCLB that pertain to high school graduation and discuss their implications from a measurement perspective. Next we present several distinctive strategies for developing a high school graduation indicator that are broadly consistent with the new federal requirements for accountability. The advantages and disadvantages of each strategy are addressed. In the empirical portion of this study we construct these proposed high school graduation indicators using information from the Common Core of Data, the U.S. Department of Education's census of local educational agencies and schools. These indicators are created at the district level for the high school class of 2000, the most recent year for which complete data are available. We then systematically compare the results generated by the respective measures, with particular attention to: their point estimates and distributions; the quality of the indicators from an informational perspective (e.g. coverage of school districts or student population); and differential performance of the indicators when disaggregated by state and by agency characteristics such as minority enrollment and district size. We conclude by discussing several lessons this study may offer for future research and the policy implications for measuring graduation rates under conditions of high-stakes accountability.

This report is available in its entirety in the Portable Document Format (PDF).

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Usage, posting and reprint of materials on the UI web site:

Most publications may be downloaded free of charge from the web site in PDF format. This information may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required.

Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact paffairs@urban.org.

If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.