Value-Added Analysis and Education Policy

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

No. 1 in the series from CALDER. The text below is an excerpt from the complete document. Read the full paper in PDF format.

Abstract

This brief describes estimation and measurement issues relevant to estimating the quality of instruction in the context of a cumulative model of learning. It also discusses implications for the use of value-added estimates in personnel and compensation matters. The discussion highlights the importance of accounting for student differences and the advantages of focusing on student achievement gains as opposed to differences in test scores. Despite potential shortcomings, value-added analysis can provide valuable information for use in evaluating and compensating teachers. The key is not to be cavalier about the information contained in value-added estimates but to understand the pieces that go into producing estimates of teacher quality.

Introduction

Education researchers have recently recognized that easily quantifiable characteristics—including postgraduate education, experience, college quality, certification, and even scores on standardized tests—do not capture much of the variation in the quality of instruction as measured by the contribution to learning. This finding provides empirical support for outcome-based approaches that measure and, in some cases, reward teacher quality based on student outcomes. Such approaches eliminate the need to identify the precise relationship between teacher characteristics and the type of training undertaken by focusing on classroom performance. The No Child Left Behind (NCLB) mandate that students must be tested annually provides the necessary achievement data, and increasing numbers of districts and states are adopting pay-for-performance plans and accountability systems in which student test outcomes occupy a central role.

The success of outcome-based policies hinges on several factors, but the validity of the teacher quality measures is perhaps the most important. As the research literature discusses in great detail, the determinants of both student and teacher choices and the allocation of students among classrooms complicate efforts to estimate the contributions of teachers to learning. The imprecision of tests as measures of achievement, failure of some examinations to measure differences throughout the skill distribution, and limited focus of the tests on a small number of subjects further complicate efforts to rank teachers and schools based on the quality of instruction.

Yet despite these potential drawbacks, value-added analysis may still provide valuable information to use in personnel decisions and teacher compensation structures. The fact that value-added estimates will never measure precisely the quality of instruction in a classroom does not imply that they have no productive uses. Rather, recognizing the methodological issues can facilitate more informed uses of standardized test results and the development of stronger assessments.

This brief describes estimation and measurement issues relevant to estimating the quality of instruction in the context of a cumulative model of learning. It also discusses implications for the use of value-added estimates in personnel and compensation matters. The discussion highlights the importance of accounting for student differences and the advantages of focusing on student achievement gains as opposed to differences in test scores. It also recognizes, however, that the value-added framework does not address all potential impediments to consistently estimating the quality of instruction. Specific methods mitigate some problems and not others; none may resolve all potential problems. Acquiring a clearer understanding of these issues can improve the methods used to estimate added teacher value and how these estimates are used.

(End of excerpt. The entire paper is available in PDF format.)

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Usage, posting and reprint of materials on the UI web site:

Most publications may be downloaded free of charge from the web site in PDF format. This information may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required.

Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact paffairs@urban.org.

If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.