Nonprofits face growing demands to demonstrate their impact. Their ability to report on program performance is essential to organizational legitimacy and financial survival. This report chronicles the evaluation experiences of four youth-serving nonprofits that participated in the East of the River Initiative, a multi-year effort to increase the capacity of agencies to assess their performance. We detail key successes and challenges with the goal of sparking a dialogue between nonprofits, funders, and technical assistance providers about the proper value of evaluation in the sector.