Many test-based accountability systems, including the No Child Left Behind Act (NCLB), place great weight on the numbers of students who score at or above specified proficiency levels in various subjects and often provide incentives for teachers and principals to target children near current proficiency levels for extra attention, but the same systems provide weak incentives to devote extra attention to students who are proficient already or who have little chance of becoming proficient in the near term. Using fifth grade test scores from the Chicago Public Schools, this paper shows both the introduction of NCLB in 2002 and similar district level reforms in 1996 generated noteworthy increases in reading and math scores among students in the middle of the achievement distribution. However, the least academically advantaged students did not score higher in math or reading following the introduction of accountability. There is mixed evidence of score gains among the most advantaged students. Also, results suggest that the choice of the proficiency standard in such accountability systems determines the amount of time that teachers devote to students of different ability levels.