Urban Institute: Education Policy

The Impact of Teacher Experience

paffairs@urban.org ( Jennifer King Rice) — Tue, 21 Sep 2010 00:00:00 EST

Many occupations recognize employees years of experience as a relevant factor in human resource policies. In education, teacher experience is a cornerstone of traditional single-salary schedules; it drives teacher transfer policies that prioritize seniority; and it is commonly considered a major source of inequity across schools and, therefore, a target for redistribution. The underlying assumption is that experience promotes effectiveness. But is this really the case? Do students attain higher levels of achievement when taught by more experienced teachers? Over 40 years of teacher productivity research suggests that the simple assumption that more is better requires greater nuance; experience effects are complex and depend on a number of factors. Recent evidence from CALDER studies provides new insight into the effects of teacher experience.

Scaling the Digital Divide: Home Computer Technology and Student Achievement

paffairs@urban.org ( Jacob Vigdor, Helen Ladd) — Tue, 10 Aug 2010 00:00:00 EST

Does differential access to computer technology at home compound the educational disparities between the rich and the poor? Authors examine the arrival of broadband service in North Carolina between 2000 and 2005, using data on the state's public school students and earlier surveys documenting broad racial and socioeconomic gaps in home computer access and use. The introduction of home computer technology is associated with modest but statistically significant and persistent negative impacts on student math and reading test scores, thus broadening math and reading achievement gaps. They conclude that home computer technology is put to more productive use in households with more effective parental monitoring.

What Makes Special-Education Teachers Special? : Teacher Training and Achievement of Students with Disabilities

paffairs@urban.org ( Li Feng, Tim Sass) — Tue, 10 Aug 2010 00:00:00 EST

This paper examines the impact of pre-service preparation and in-service formal and informal training on the ability of teachers to promote academic achievement among students with disabilities. Using student-level longitudinal data from Florida over a five-year span the authors estimate "value-added" models of student achievement. There is little support for the efficacy of in-service professional development courses focusing on special education. However, teachers with advanced degrees are more effective in boosting the math achievement of students with disabilities than are those with only a baccalaureate degree. Also pre-service preparation in special education has statistically significant and quantitatively substantial effects on the ability of teachers of special education courses to promote gains in achievement for students with disabilities, especially in reading.

Teacher Attitudes about Compensation Reform : Implications for Reform Implementation

paffairs@urban.org ( Dan Goldhaber, Michael DeArmond, Scott DeBurgomaster) — Tue, 10 Aug 2010 00:00:00 EST

Reform advocates and policymakers concerned about the quality and distribution of teachers among schools support proposals of alternative compensation for teachers in hard-to-staff schools and subject-areas. But the successful implementation of such proposals depends on teacher attitudes. Results from a 2006 survey of teachers in Washington State linked to school and district data confirm that teacher opinion about pay reform is not uniform, and illustrate teacher preferences for different pay structures vary substantially by individual and workplace characteristics. Policymakers interested in implementing new pay systems should carefully assess teacher opinion in determining where (and how) they invest in them.

Scrambling the Nest Egg: How Well Do Teachers Understand Their Pensions, and What Do They Think about Alternative Pension Structures?

paffairs@urban.org ( Michael DeArmond, Dan Goldhaber) — Mon, 09 Aug 2010 00:00:00 EST

This paper addresses two questions: How well do teachers understand their current pension plans? And, what do they think about alternative plan structures? The data come from administrative records and a 2006 survey of teachers in Washington State. The results suggest Washington's teachers are fairly knowledgeable about their pensions, though new entrants and mid-career teachers appear to be less knowledgeable than veteran teachers. As for teachers' preferences for plan structure, the survey suggests that when it comes to investing additional retirement savings, a plurality of teachers favor defined contribution plans which offer more portability and choice, but more risk than traditional defined benefit plans. All else equal, teachers newer to the profession are more likely than veteran teachers to favor a defined contribution structure.

Measure for Measure: The Relationship between Measures of Instructional Practice in Middle School English Language Arts and Teachers' Value-Added Scores

Mon, 09 Aug 2010 00:00:00 EST

In this study, the authors ask what classroom practices, if any, differentiate teachers with high impact on student achievement in middle school English Language Arts from those with lower impact. The study further explores the extent to which value-added measures signal differences in instructional quality. Even with the small sample used in the analysis, there is evidence that high value-added teachers have a different profile of instructional practices than do low value-added teachers. Teachers in the top quartile as measured by value-added scores score higher than second-quartile teachers on all 16 elements of instruction that were measured. The differences are statistically significant for a subset of practices including explicit strategy instruction.

School Principals and School Performance

paffairs@urban.org ( Damon Clark, Paco Martorell, Jonah Rockoff) — Mon, 09 Aug 2010 00:00:00 EST

This paper uses data from New York City to estimate how the characteristics of school principals relate to school performance, as measured by students' standardized exam scores and other outcomes. There is little evidence of any relationship between school performance and principal education and pre-principal work experience, but some evidence that experience as an assistant principal at the principal's current school is associated with higher performance among inexperienced principals. There is mixed evidence on the relationship between formal principal training and professional development programs and school performance, with the caveat that the selection and assignment of New York City principals participating in these programs make it hard to isolate their effects. The positive returns to principal experience suggest that policies which cause principals to leave their posts early (e.g., via early retirement or a move into district administration) will be costly, and the tendency for less-advantaged schools to be run by less experienced principals could exacerbate educational inequality.

Effective Schools: Managing the Recruitment, Development, and Retention of High-Quality Teachers

paffairs@urban.org ( Tara Beteille, Demetra Kalogrides, Susanna Loeb) — Mon, 09 Aug 2010 00:00:00 EST

Teachers are systematically sorted across schools. Often, schools serving the lowest-achieving students staffed by the least-skilled teachers. While teachers' school preferences account for some of the sorting, school practices are also likely to be a key factor. The authors examine the relationship between a school's effectiveness during a given principal's tenure and the retention, recruitment and development of its teachers. Three key findings emerge about principal effectiveness. More effective principals: (1) are able to retain higher-quality teachers and remove less-effective teachers; (2) are able to attract and hire higher-quality teachers to fill vacancies; (3) have teachers who improve at a greater pace than those in schools with less effective leadership (there is some evidence for this, albeit weak). These findings drive home the importance of personnel practices for effective school leadership.

Teacher Mobility, School Segregation, and Pay-Based Policies to Level the Playing Field

paffairs@urban.org ( Charles Clotfelter, Helen Ladd, Jacob Vigdor) — Mon, 09 Aug 2010 00:00:00 EST

Research has consistently shown that teacher quality is distributed very unevenly among schools to the clear disadvantage of minority students and those from low-income families. Using information on teaching spells in North Carolina, the authors examine the potential for using salary differentials to overcome this pattern. They conclude that salary differentials are a far less effective tool for retaining teachers with strong pre-service qualifications than for retaining other teachers in schools with high proportions of minority students. Consequently, large salary differences would be needed to level the playing field when schools are segregated.

Before or After the Bell? : School Context and Neighborhood Effects on Student Achievement

paffairs@urban.org ( Paul A. Jargowsky, Mohamed El Komi) — Mon, 09 Aug 2010 00:00:00 EST

This paper explores the relative effects of school and neighborhood characteristics on student achievement in Texas. School variables are more robust and explain a greater degree of the variance in test scores than neighborhood characteristics. Neighborhood level variables, as a group, are statistically significant even in the presence of school variables. The particular pattern of effects varies by the manner in which the school context was controlled, by poverty status, move status, and location in the conditional achievement distribution. But neighborhood always mattered. Even if neighborhood conditions are less robust than school context effects, concern about neighborhood conditions is still justified.

What Makes for a Good Teacher and Who Can Tell?

paffairs@urban.org ( Douglas Harris, Tim Sass) — Mon, 09 Aug 2010 00:00:00 EST

Are there important determinants of teacher productivity that are not captured by teacher credentials but that can be measured by subjective assessments? And would evaluating teachers based on a combination of subjective assessments and student outcomes more accurately gauge teacher performance than student test scores alone? Using data from a midsize Florida school district, this paper explores both questions by calculating teachers' "value added" and comparing those outcomes with subjective ratings of teachers by school principals. Teacher value-added and principals' subjective ratings are positively correlated and principals' evaluations are better predictors of a teacher's value added than traditional approaches to teacher compensation focused on experience and formal education. Also, teachers' subject knowledge, teaching skill, and intelligence are most closely associated with both the overall subjective teacher ratings and the teacher value added.

Teacher Career Paths, Teacher Quality, and Persistence in the Classroom : Are Schools Keeping Their Best?

paffairs@urban.org ( Dan Goldhaber, Betheny Gross, Daniel Player) — Mon, 09 Aug 2010 00:00:00 EST

Most studies that have fueled alarm over the attrition and mobility rates of teachers have relied on proxy indicators of teacher quality, even though these proxies correlate only weakly with student performance. This paper examines the attrition and mobility of early-career teachers of varying quality using value-added measures of teacher performance. Unlike previous studies, this paper focuses on the variation in these effects across the effectiveness distribution. On average, more effective teachers tend to stay in their initial schools and in teaching. But the lowest performing teachers, who are generally the most likely to transfer between schools, appear to "churn" within the system, and teacher mobility appears significantly affected by student demographics and achievement levels.

Teachers' Perceptions of Their Working Conditions: How Predictive of Policy-Relevant Outcomes?

paffairs@urban.org ( Helen Ladd) — Mon, 09 Aug 2010 00:00:00 EST

This study uses data from North Carolina to examine the extent to which survey based perceptions of working conditions are predictive of policy-relevant outcomes, independent of other school characteristics such as the demographic mix of the school's students. Working conditions emerge as highly predictive of teachers' stated intentions to remain in or leave their schools, with leadership emerging as the most salient dimension. Teachers' perceptions of their working conditions are also predictive of one-year actual departure rates and student achievement, but the predictive power is far lower. These weaker findings for actual outcome measures help to highlight both the strengths and weaknesses of using teacher survey data for understanding outcomes of policy interest.

Principal Preferences and the Unequal Distribution of Principals across Schools

paffairs@urban.org ( Eileen Horng, Demetra Kalogrides, Susanna Loeb) — Mon, 09 Aug 2010 00:00:00 EST

This study uses longitudinal data from one large school district - Miami-Dade County Public Schools, to investigate the distribution of principals across schools. Schools serving many low-income, non-white, and low-achieving students have principals with less experience, less education, and who attended less selective colleges. This distribution of principals is partially driven by the initial match of first-time principals to schools at the beginning of their careers and is exacerbated by systematic attrition and transfer away from these schools. Supplementing the data with surveys of principals, the authors find principals' stated preferences for school characteristics mirror observed distribution and transfer patterns. Principals prefer to work in easier to serve schools with favorable working conditions which also tend to be schools with fewer poor, minority and/or low-achieving students.

Triangulating Principal Effectiveness: How Perspectives of Parents, Teachers, and Assistant Principals Identify the Central Importance of Managerial Skills

paffairs@urban.org ( Jason Grissom, Susanna Loeb) — Mon, 09 Aug 2010 00:00:00 EST

While the importance of effective principals is undisputed, few studies have addressed what specific skills principals need to promote school success. This study draws on unique data combining survey responses from principals, assistant principals, teachers and parents with rich administrative data to identify which principal skills matter most for school outcomes. Factor analysis of a 42-item task inventory distinguishes five skill categories, yet only one of them, the principals' organization management skills, consistently predicts student achievement growth and other success measures. Analysis of evaluations of principals by assistant principals confirms this central result. The analysis argues for a broad view of instructional leadership that includes general organizational management skills as a key complement to the work of supporting curriculum and instruction.

Estimating Principal Effectiveness

paffairs@urban.org ( Steven Rivkin, Gregory F. Branch, Eric A. Hanushek) — Sat, 07 Aug 2010 00:00:00 EST

This paper presents preliminary estimates of key elements of the market for school principals, employing rich panel data on principals from Texas State. The consideration of teacher movements across schools suggests that principals follow patterns quite similar to those of teachers - preferring schools that have less demands as indicated by higher income students, higher achieving students, and fewer minority students. Looking at the impact of principals on student achievement, there are some small but significant effects of the tenure of a principal in a school. Moreover, the variation in principal effectiveness tends to be largest in high-poverty schools, consistent with hypothesis that principal ability is most important in schools serving the most disadvantaged students. Principals who stay in a school tend to be more effective than those who move to other schools.

Teacher Layoffs: An Empirical Illustration of Seniority vs. Measures of Effectiveness

Wed, 28 Jul 2010 00:00:00 EST

In the face of unavoidable teacher layoffs, policymakers must juggle a variety of issues in choosing the best criteria for laying off teachers. The standard approach in most school districts relies on measures of seniority. Analyzing data on 4th and 5th grade teachers in New York City public schools, CALDER researchers find substantial differences in which teachers get cut under a seniority-based layoff policy versus a policy based on teacher effectiveness (value-added). The authors model the two layoff scenarios to respond to a (fictional) budget shortfall equivalent. The bottom line is that teacher layoffs based on teacher performance, preferably multiple performance measures, lead to a more effective workforce and improved student performance.

Constrained Job Matching: Does Teacher Job Search Harm Disadvantaged Urban Schools?

paffairs@urban.org ( Eric A. Hanushek, Steven Rivkin) — Wed, 16 Jun 2010 00:00:00 EST

Search theory suggests early career job changes lead to better matches that benefit both workers and firms, but this may not hold true in teacher labor markets characterized by salary rigidities, barriers to entry, and substantial differences in working conditions. Education policy makers are particularly concerned that teacher turnover may have adverse effects on the quality of instruction in schools serving predominantly disadvantaged children. Although these schools experience higher turnover, on average, than other schools, the impact on the quality of instruction depends on whether more productive teachers are more likely to depart. In Texas, the availability of matched panel data of students and teachers enables the isolation of teachers' contributions to achievement. Teachers who remain in their school tend to outperform those who leave, particularly those who exit Texas public schools entirely. This gap is larger for schools serving mainly low income students evidence that high turnover is not nearly as damaging as many suggest.

School Accountability and Teacher Mobility

paffairs@urban.org ( Li Feng, David Figlio, Tim Sass) — Wed, 16 Jun 2010 00:00:00 EST

This study is the first to exploit policy variation within the same state to examine the effects of school accountability on tacher job changes. Using student-level data from Florida State the authors measure the degree to which schools and teachers were "surprised" by the change in the school grading system (in summer of 2002)— what they refer to as an "accountability shock"— by observing the mobility decisions of teachers in the years before and after the school grading change. They find over half of all schools in the state experience an accountability "shock" due to this grading change. Also, teachers are more likely to leave schools facing increased accountability pressure— and even more likely to leave schools shocked downward to a grade of "F". They are less likely to leave schools facing decreased accountability pressure. Moreover, schools facing increased pressure experience an increase in the quality of teachers who leave or stay and schools with no accontability shock experience no significant change to the quality of teachers that leave or stay.

Competitive Effects of Means-Tested School Vouchers

paffairs@urban.org ( David Figlio, Cassandra M.D. Hart) — Wed, 16 Jun 2010 00:00:00 EST

Voucher options like tuition tax credit-funded scholarship programs have become increasingly popular in recent years. This study examines the effects of private school competition on public school students' test scores in the wake of Florida's Corporate Tax Credit Scholarship program which offered scholarships to eligible low-income students to attend private schools. The authors examine whether students in schools exposed to a more competitive private school landscape saw greater improvement in their students' test scores after the introduction of the program, than did students in schools that faced less competition. Greater degrees of competition are associated with greater improvements in students' test scores following the introduction of the program. The findings are not an artifact of pre-policy trends; the degree of competition from nearby private schools matters only after the announcement of the new program, which makes nearby private competitors more affordable for eligible students. Also, schools that we would expect to be most sensitive to competitive pressure see larger improvements in their test scores as a result of increased competition.

New Estimates of Design Parameters for Clustered Randomization Studies : Findings from North Carolina and Florida

paffairs@urban.org ( Zeyu Xu, Austin Nichols) — Wed, 16 Jun 2010 00:00:00 EST

The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to provide empirical information needed to design adequately powered studies that randomize schools using data from Florida and North Carolina. The authors assess how different covariates contribute to improving the statistical power of a randomization design and examine differences between math and reading tests; differences between test types (curriculum-referenced tests versus norm-referenced tests); and differences between elementary school and secondary school, to see if the test subject, test type, or grade level makes a large difference in the crucial design parameters. Finally they assess bias in 2-level models that ignore the clustering of students in classrooms.

Using Value-Added Measures of Teacher Quality

paffairs@urban.org ( Eric A. Hanushek, Steven Rivkin) — Mon, 26 Apr 2010 00:00:00 EST

Can value-added measures provide valuable information to assess the quality of teachers and to create incentives for improvement? CALDER researchers tackle this important and timely question by describing the analytic framework of value-added measures, by identifying methodological concerns about value-added estimation and ways to mitigate them, and by discussing the policy uses of value-added estimates of teacher effectiveness.

Distribution of Benefits in Teacher Retirement Systems and Their Implications for Mobility

paffairs@urban.org ( Robert M. Costrell, Michael Podgursky) — Fri, 23 Apr 2010 00:00:00 EST

Defined benefit pension systems concentrate benefits on career teachers and impose costs on mobile teachers. This study analyzes the magnitude of these effects. Compared to a neutral system, often about half of an entering cohort's net pension wealth is redistributed to teachers who separate in their fifties from those who separate earlier, with some variation across states. This implies large costs for interstate mobility. Teachers who split a thirty-year career between two pension plans often lose over half their net pension wealth compared to teachers who complete a career in a single system. Likely explanations include the relative influence of senior versus junior educators in interest group politics and a coordination problem between states.

How Career Concerns Influence Public Workers' Effort : Evidence from the Teacher Labor Market

paffairs@urban.org ( Michael Hansen) — Fri, 23 Apr 2010 00:00:00 EST

This study presents a generalization to the standard career concerns model and applies it to the public teacher labor market. The model predicts that optimal teacher effort levels decline with both tenure at a school and experience, all things being equal. Using administrative data from North Carolina spanning 14 school years through 2008, the study finds significant changes in teacher sick leave consistent with the generalized career concerns model. There is evidence that observed behaviors cannot be due to the endogeneity of teacher mobility decisions alone as well as evidence suggestive of teacher shirking. In sum, teachers exert considerable discretion over their own effort levels in response to these incentives, with important policy implications.

Assessing the Potential of Using Value-Added Estimates of Teacher Job Performance for Making Tenure Decisions

paffairs@urban.org ( Dan Goldhaber, Michael Hansen) — Fri, 23 Apr 2010 00:00:00 EST

Reforming teacher tenure is an idea that appears to be gaining traction with the underlying assumption that one can infer, to a reasonable degree, how well a teacher will perform over her career based on estimates of her early-career effectiveness. In this paper, the authors explore the potential for using value-added models to estimate performance and inform tenure decisions. There is little evidence that the variation of teacher effects change over teacher careers, but strong evidence that prior year estimates of job performance predict student achievement, even when there is a multi-year lag between the two.

Principal Effectiveness and Leadership in an Era of Accountability: What Research Says

paffairs@urban.org ( Jennifer King Rice) — Fri, 23 Apr 2010 00:00:00 EST

In an era of greater school accountability, leadership matters. For decades, principals have been recognized as vital to the effectiveness of schools, but strong empirical evidence on the extent to which, and the ways in which, school leaders matter has not been available. CALDER researchers have advanced our knowledge in this area by skillfully drawing on rich state longitudinal databases. This brief synthesizes new findings on the effectiveness and distribution of principals, the characteristics of good leadership, and how best to prepare principals for this increasingly demanding job.

Rural Schools Need Realistic Improvement Models

paffairs@urban.org ( Luke C. Miller, Michael Hansen) — Thu, 22 Apr 2010 00:00:00 EST

Race to the Top's prescribed models for turning around the nation's lowest-performing schools are designed for urban areas and leave rural districts out of the high-stakes money game. This omission needs to be fixed.

Reform Ideas in New Book Aim for Teacher Excellence

paffairs@urban.org ( The Urban Institute) — Tue, 01 Dec 2009 00:00:00 EST

An unmistakable sense of urgency runs throughout Creating a New Teaching Profession, with the top scholars and practitioners who coauthor the book underscoring that current systems for training, hiring, retaining, and rewarding teachers not only are imperfect, but are detrimental to building the best teacher workforce possible. Contributors to the book propose such major reforms as remaking longstanding teacher training systems and using private-sector approaches to modernize recruitment and compensation.

Ambitious Reform Efforts Evaluated in New Book on America's High Schools

paffairs@urban.org ( The Urban Institute) — Thu, 19 Nov 2009 00:00:00 EST

Eighteen education policy experts put the past decade's surge in high-school reform efforts to the test in Saving America's High Schools from the Urban Institute Press. Led by coeditors Becky Smerdon and Kathryn Borman, the team of authors size up national reform trends and draw on at least five years of research in Baltimore, New York City, Chicago, Ohio, and North Carolina.

Widening the Net: National Estimates of Gender Disparities in Engineering

paffairs@urban.org ( Clemencia Cosentino de Cohen, Nicole Deterding) — Thu, 22 Oct 2009 00:00:00 EST

This paper explores the causes behind the severe underrepresentation of women in engineering. Based on national data on undergraduate engineering programs, this study presents cross-sectional estimates of male and female student retention. Contrary to widespread beliefs, the study found that overall and in most disciplines there is no differential attrition by gender. Instead, results suggest that gender disparities in engineering are largely driven by inadequate enrollment (not inadequate retention) of women. The paper concludes that outreach within institutions of higher education, across institutions (into two-year colleges, middle and high schools), and into K-12 curricular reformare needed to address what is, at its very core, a recruitment problem.

Retention Is Not the Problem: Women aren't being drawn to engineering in the first place.

paffairs@urban.org ( Clemencia Cosentino de Cohen) — Thu, 22 Oct 2009 00:00:00 EST

"Study Seeks to Improve Retention Among Women Engineering Students," declares a 2008 news release announcing a grant to four universities. Countless other articles cite female retention as a grave problem. This focus on retention drives a host of strategies to increase the number of women engineers. But is low retention behind the problem? Are women underrepresented in engineering because they enroll only to eventually drop out? The answer, as documented in the July 2009 Journal of Engineering Education, is a resounding "No!"

The Influence of School Administrators on Teacher Retention Decisions

Thu, 27 Aug 2009 00:00:00 EST

When given the opportunity, many teachers choose to leave schools serving poor, low-performing, and minority students. While substantial research has documented this phenomenon, far less effort has gone into understanding what features of the working conditions in these schools drive this relatively high turnover rate. This paper explores the relationship between school contextual factors and teacher retention decisions in New York City. The methodological approach separates the effects of teacher characteristics from school characteristics by modeling the relationship between the assessments of school contextual factors by one set of teachers and the turnover decisions by other teachers within the same school. Teachers perceptions of the school administration have by far the greatest influence on teacher-retention decisions. This effect of administration is consistent for first-year teachers and the full sample of teachers and is confirmed by a survey of teachers who have recently left teaching in New York City.

Are Teacher Absences Worth Worrying about in the U.S.?

paffairs@urban.org ( Charles Clotfelter, Helen Ladd, Jacob Vigdor) — Wed, 26 Aug 2009 00:00:00 EST

Using data from North Carolina, this paper examines the frequency, incidence, and consequences of teacher absences in public schools, as well as the impact of an absence disincentive policy. The incidence of teacher absences is regressive: when schools are ranked by the fraction of students receiving free or reduced-price lunch, schools in the poorest quartile averaged almost one extra sick day per teacher than schools in the highest income quartile, and schools with persistently high rates of teacher absence were much more likely to serve low-income than high-income students. The authors find that absences are associated with lower student achievement in elementary grades and that the demand for discretionary absences is price-elastic. Estimates suggest that a policy intervention which simultaneously raised teacher base salaries and broadened financial penalties for absences could both raise teachers' expected income and lower districts' expected costs.

Supplemental Education Services Under No Child Left Behind: Who Signs Up, and What Do They Gain?

paffairs@urban.org ( Carolyn J. Heinrich, Robert H. Meyer, Gregory W. Whitten) — Wed, 26 Aug 2009 00:00:00 EST

Schools that have not made adequate yearly progress in increasing student academic achievement are required, under No Child Left Behind (NCLB), to offer children in low-income families the opportunity to receive supplemental educational services (SES). In research conducted in Milwaukee Public Schools, the authors explore whether parents and students are aware of their eligibility and options for extra tutoring under NCLB, and who among eligible students registers for SES. Using the best information available to school districts, the authors estimate the effects of SES in increasing students reading and math achievement. They find no average impacts of SES attendance on student achievement gains and use qualitative research to explore possible explanations for the lack of observed effects.

Leaving No Child Behind: Two Paths to School Accountability

paffairs@urban.org ( Analia Schlosser, David Figlio, Cecilia Elena Rouse) — Tue, 25 Aug 2009 00:00:00 EST

The relatively poor academic achievement of black and Hispanic students has been a national concern since the passage of the Elementary Secondary and Education Act (ESEA) in 1963. Frustrated with relatively slow progress in closing these educational gaps, the most recent reauthorization of the ESEA, the No Children Left Behind Act of 2001 (NCLB) attempts to employ rigorous accountability standards to speed progress. At about the same time, Florida implemented a change in its A+ Plan for Education that focused on the educational gains of low-performing students. This paper examines whether either of these accountability systems improved the academic outcomes of black, Hispanic and economically disadvantaged students in Florida. Schools labeled as failing or near-failing in Floridas system tend to boost performance of students in these subgroups, while schools presented with incentives under NCLB to improve subgroup performance appear to be much less likely to do so. However, Hispanics appear to benefit from the NCLB sub-grouping requirements if they attend schools with low accountability pressure under Floridas grading system.

The Achievement Consequences of the No Child Left Behind Act

paffairs@urban.org ( Thomas S. Dee, Brian A. Jacob) — Tue, 25 Aug 2009 00:00:00 EST

The No Child Left Behind (NCLB) Act has compelled states to design school accountability systems based on annual student assessments. The effect of this Federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). This study identifies the impact of NCLB by relying on comparisons of the test-score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Results indicate that NCLB generated statistically significant increases in the average math performance of 4th graders (effect size = 0.22 by 2007) as well as improvements at the lower and top percentiles. However, the authors do not find consistent evidence that NCLB generated similarly broad improvements in reading achievement or achievement among 8th graders.

Supplemental Educational Services and Student Test Score Gains: Evidence from a Large, Urban School District

Tue, 25 Aug 2009 00:00:00 EST

This study examines the effect of SES on student test score gains and whether subgroups of students benefit more from NCLB tutoring services, using information about students enrolled in 3rd through 8th grades in 121 elementary and middle schools from 2003-04 to 2007-08. A total of 17 elementary and middle schools were required to offer SES at some point during the period under study, and 9,861 student-year pairings in the sample were eligible to receive SES. The authors find consistently significant and positive average effects of SES on test score gains in mathematics. Results in reading tend to be insignificant. SES tutoring does not appear to disproportionately benefit a particular racial/ethnic group or ability level. Female students and students with disabilities appear to benefit more from participating in SES. SES has a significant, cumulative effect on students in both mathematics and reading. They also demonstrate that not accounting for content area of tutoring can cause downward bias in estimates of the SES treatment effect. These findings are qualified on a couple of dimensions.

Which Students are Left Behind? The Racial Impacts of the No Child Left Behind Act

paffairs@urban.org ( John M. Krieg) — Mon, 24 Aug 2009 00:00:00 EST

The No Child Left Behind Act imposes sanctions on schools if the fraction of each of five racial group of students demonstrating proficiency on a high stakes exam falls below a statewide pass rate. This system places pressure on school administrators to redirect educational resources from groups of students most likely to demonstrate proficiency towards those who are marginally below proficient. Using statewide observations of 3rd and 4th grade math tests, this paper demonstrates that students of successful racial groups at schools likely to be sanctioned gain less academically over their subsequent test year than comparable peers at passing schools. This effect is stronger at schools more likely to suffer from NCLB sanctions and is robust to non-random sample selection.

"Going Down With the Ship?" The Effect of School Accountability on the Distribution of Teacher Experience in California

paffairs@urban.org ( David P. Sims) — Mon, 24 Aug 2009 00:00:00 EST

Many school accountability programs, including the No Child Left Behind (NCLB) Act are built on the premise that the threat of sanctions attached to failure will produce higher student achievement. However, the stigma associated with failing schools and the expected costs of possible future sanctions may lead experienced teachers to leave these schools for other opportunities. This may undermine the programs improvement efforts. Particularly it may lead failing schools to rely on a higher proportion of novice teachers. This study looks at elementary and secondary schools in California from 2002-2006 to determine the effect of failing to meet academic performance thresholds on teacher experience under the NCLB accountability system. Because failing schools differ in important ways from schools that meet performance targets, the author takes advantage of the racial subgroup rules to compare groups of schools that may have different failure probabilities despite similar profiles. The author finds that failure to meet AYP is associated with decreases in aggregate teacher experience and increases in the proportion of novice teachers.

Status versus Growth: The Distributional Effects of School Accountability Policies

paffairs@urban.org ( Helen Ladd, Douglas Lauen) — Mon, 24 Aug 2009 00:00:00 EST

Although the Federal No Child Left Behind program judges the effectiveness of schools based on their students achievement status, many policy analysts argue that schools should be measured, instead, by their students achievement growth. Using a ten-year student-level panel data set from North Carolina, the authors examine how school-specific pressure associated with the two approaches to school accountability affects student achievement at different points in the prior-year achievement distribution. Achievement gains for students below the proficiency cut point emerge in response to both types of accountability systems, but more clearly in math than in reading. In contrast to prior research highlighting the possibility of educational triage, the authors find little or no evidence that schools in North Carolina ignore the students far below proficiency under either approach. They find that the status, but not the growth, approach reduces the reading achievement of higher performing students. Results suggest that the distributional effects of accountability pressure depend not only on the type of pressure for which schools are held accountable (status or growth), but also the tested subject.

Left Behind By Design: Proficiency Counts and Test-Based Accountability

paffairs@urban.org ( Derek Neal, Diane Whitmore-Schanzenbach) — Mon, 24 Aug 2009 00:00:00 EST

Many test-based accountability systems, including the No Child Left Behind Act (NCLB), place great weight on the numbers of students who score at or above specified proficiency levels in various subjects and often provide incentives for teachers and principals to target children near current proficiency levels for extra attention, but the same systems provide weak incentives to devote extra attention to students who are proficient already or who have little chance of becoming proficient in the near term. Using fifth grade test scores from the Chicago Public Schools, this paper shows both the introduction of NCLB in 2002 and similar district level reforms in 1996 generated noteworthy increases in reading and math scores among students in the middle of the achievement distribution. However, the least academically advantaged students did not score higher in math or reading following the introduction of accountability. There is mixed evidence of score gains among the most advantaged students. Also, results suggest that the choice of the proficiency standard in such accountability systems determines the amount of time that teachers devote to students of different ability levels.

The Qualifications and Classroom Performance of Teachers Moving to Charter Schools

paffairs@urban.org ( Celeste Carruthers) — Fri, 26 Jun 2009 00:00:00 EST

Do charter schools draw good teachers from traditional, mainstream public schools? Using a panel dataset of all North Carolina public school teachers from 1997-2007, this research paper finds nuanced patterns of teacher quality flowing into charter schools. High rates of inexperienced and uncertified teachers moved to charter schools, but among certified teachers changing schools, the on-paper qualifications of charter movers were better or no different than the qualifications of teachers moving to comparable mainstream schools. Estimated measures of classroom performance for a subset of grade 3 - 5 teachers show that charter movers were more effective in math and reading instruction, relative to other mobile teachers. Charter movers compared less favorably, however, to non-mobile teachers and colleagues within their sending schools. The distribution of classroom performance among future charter teachers, adjusted for sampling error, was significantly lower than the distribution for exclusively mainstream teachers.

Assessing the Potential of Using Value Added-Estimates of Teacher Job Performance for Making Tenure Decisions

paffairs@urban.org ( Dan Goldhaber, Michael Hansen) — Wed, 15 Apr 2009 00:00:00 EST

Using individual teacher and student-level longitudinal data from North Carolina, this research brief presents selected findings from work examining the stability of value-added model estimates of teacher effectiveness, focusing on their implication for teacher tenure policies and making high stakes personnel decisions. Findings show year-to-year correlations in teacher effects are modest, but pre-tenure estimates of teacher job performance do predict estimated post-tenure performance in both math and reading, and would therefore seem to be a reasonable metric to use as a factor in making substantive teacher selection decisions.

The Stability of Value-Added Measures of Teacher Quality and Implications for Teacher Compensation Policy

paffairs@urban.org ( Tim Sass) — Wed, 15 Apr 2009 00:00:00 EST

There is little doubt that teacher quality is a key determinant of student achievement, but finding ways to identify and reward the best teachers has proven illusive. This research brief considers the stability of value-added measures of teacher effectiveness over time and the resulting implications for the design and implementation of performance-based teacher compensation schemes.

The Texas FERPA Story

paffairs@urban.org ( Dan O'Brien) — Wed, 15 Apr 2009 00:00:00 EST

This research brief describes the legal and operational structure of the Texas longitudinal data system related to recent changes in the Family Educational Rights and Privacy Act of 1974 (FERPA)which establishes the rights of parents to access their children's educational records and protects the confidentiality of student informationthat more closely align law and practice. The U.S. Department of Education's FERPA Final Regulations Amendments took effect January 8, 2009.

The Narrowing Gap in New York City Teacher Qualifications and Implications for Student Achievement in High-Poverty Schools

Wed, 15 Apr 2009 00:00:00 EST

This important research explores the effects of district policy interventions on the distribution of teacher qualifications and student achievement. Authors use a 5-year span of individual teacher- and student-level longitudinal data from New York City (NYC) from 2000 through 2005 to estimate the differences in the effectiveness of teachers entering NYC schools through different pathways to teaching. The study finds that the gap between the qualifications of NYC teachers in high-poverty and low-poverty NYC schools has narrowed substantially since 2000, mostly ensuing from the city's concentrated effort to match exceptionally capable teachers with very needy students and the virtual substitution of newly hired uncertified teachers in high-poverty schools with new hires from alternative certification routes: NYC Teaching Fellows and Teach for America.

Do Disadvantaged Urban Schools Lose Their Best Teachers?

paffairs@urban.org ( Eric A. Hanushek, Steven Rivkin) — Tue, 14 Apr 2009 00:00:00 EST

This research brief examines differences in teacher effectiveness by school transition status and school characteristics in a large urban school district in Texas, using estimates of teacher effectiveness based on teacher contributions to student learning outcomes across classrooms. This research finds little or no evidence to support the view that more effective teachers have higher exit probabilities. In fact, the study finds that teachers who exit are significantly less effective, on average, than those who stay.

Overview of Measuring Effect Sizes: The Effect of Measurement Error

Tue, 14 Apr 2009 00:00:00 EST

This research brief estimates the overall extent of test measurement error and how this varies across students using New York City student- level longitudinal data across grades 3-8 from 1999- 2007. Results reinforce the importance of accounting for measurement error, as it meaningfully increases effect size estimates associated with teacher attributes. There are important differences in teacher effectiveness that are systematically related to observed teacher attributes. Such effects are important in the formulation and implementation of personnel policies. Also, effect sizes as traditionally measured have led analysts to understate the magnitudes of effects because the standard deviation of observed scores overstates the dispersion of true achievement in the student population.

DCPS Human Capital Initiatives : Before the District of Columbia City Council

paffairs@urban.org ( Jane Hannaway) — Fri, 06 Feb 2009 00:00:00 EST

Testimony of CALDER Director Jane Hannaway before the D.C. City Council on the human capital initiatives of the District of Columbia's Public Schools, given January 16, 2009. Hannaway describes CALDER's work on teacher quality addressing three main findings: (1) Teachers are the most important school factor that affects student learning, and the variation in effectiveness across teachers is large; (2) The variation in teacher effectiveness is greater within schools than the variation between schools; and (3) The variation in teacher effectiveness, both within and between schools, is a management problem that begs for attention. Hannaway argues at least some of this variation is a civil rights problem that demands policy attention and urges DCPS to continue to pursue new human capital management strategies to ensure teacher quality for all students.

Accountability Policies : Implications for School and Classroom Practices

paffairs@urban.org ( Jane Hannaway, Laura Hamilton) — Tue, 21 Oct 2008 00:00:00 EST

This paper reviews the research literature associated with the implications of performance-based accountability policies for school and teacher behaviors. It examines what is known about both possibly productive responses, such as focused effort on valued subjects, and non-productive responses, such as teaching to the test, induced by performance-based accountability systems.