The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
A Northern Virginia Urban League program that provided employment assistance to pregnant and parenting teens routinely collected information on whether the teens had found jobs, a major goal for the program. When the program broke out the employment success data by race and ethnicity, it found that a high proportion of Hispanic clients were not obtaining employment. This led the program to review its procedures and make a number of revisions, including hiring a young Hispanic woman as a caseworker. The employment outcome for Hispanics subsequently rose from about 50 percent to about 60 percent.
The Montgomery County (Maryland) Jewish Social Service Agency's home health care program regularly tracked the number of problem incidents, such as the number of client accidents, falls, and suicide attempts. It found that many senior citizens had falls at home, often resulting in hospital stays. Staff subsequently studied the patterns of falls, learning they occurred because of obstacles cluttering the home. The agency modified its procedures for home visits to have the visiting nurse review safety issues with the client, including identifying and correcting potential danger spots. This modification resulted in substantially fewer client falls.
New York City's Department of Sanitation has measured the cleanliness of city streets regularly for many years. It uses that information to help allocate cleanup crews. The share of streets rated as clean has increased remarkably over the years. The outcome information is also reported publicly in the Mayor's annual management report, which in the past has displayed a map of the city showing both clean and dirty areas.
The United States Postal Service has tracked the delivery times for various origins to various destinations over several years. Keeping these times as low as feasible is a major Postal Service objective. The Service used the tracking information to identify those origins and destinations with unexpectedly high delivery times. This focus helped the Postal Service drive down delivery times considerably.
The United States Internal Revenue Service recently had a second employee listen in on sampled taxpayer telephone calls to its toll-free lines, to monitor the accuracy of customer service representatives' responses to taxpayers questions. The tabulations for one year found that one particular error type occurred in all of its 15 field offices and twice as often as any other type. The IRS developed guidance material and training for its customer service representatives on these issues, reducing this type of error in later years.
The common ingredient in all these examples is that each program had installed a systematic process to regularly monitor the quality and results of its services to the publicregardless of whether the program was a government or private nonprofit agency. The scale of the monitoring effort varied greatly, from that of small nonprofit agencies to very large federal agencies. Nevertheless, each agency used the outcome information to help improve services to its customers.
Unfortunately, throughout the United States, outcome measurement has been used primarily to assess agency accountabilityindeed, a legitimate and important use of outcome information. However, to make measurement efforts really worthwhile, public and private agencies also need to analyze and use the information to help them improve services. This transforms "outcome measurement" into "outcome management."
The first step, of course, is to introduce and implement outcome measurement. At the federal level, the Government Performance and Results Act of 1993 requires outcome measurement for all major programs in all federal agencies. Full implementation began in 1999, but the effort still has not fulfilled its goal to improve services. The U.S. Office of Management and Budget's new Performance Assessment Rating Tool (PART) process attempts to link the measurement effort to program management, but outcome management is still widely perceived as primarily an accountability and budgeting tool.
A substantial number of states and local governments have also introduced some form of outcome measurement. Local governments in particular have effectively used outcome information to improve services. (A classic example is the New York City's Department of Sanitation street cleanliness example given above.)
Nonprofit service organizationsencouraged by United Way of America, national associations such as the American Red Cross and Boys and Girls Clubs of America, and private charitable foundations that provide funds to the nonprofitshave also begun to introduce some form of outcome measurement.
To switch the focus from the critical eye of accountability to the more positive and beneficial focus on service improvement, funders and the organizations themselves need to provide both incentives and support for such an effort. Small steps, such as those outlined in the six guides described in this newsletter (especially the reports on analysis and use of outcome information), can help.
Developing and implementing outcome measurement takes hard work. Only if the products of that hard work are used by officials and their staffs to improve services will the real potential of the outcome measurement movement be realized.
The Outcome Management for Nonprofit Organizations guidebooks are available online at http://www.urban.org/Series/NonprofitManagement