Harry Hatry is a distinguished fellow at the Urban Institute in Washington, D.C., where he is the is director of the Institute's Public Management Program. Since the early 1970s, he has been a leader in developing procedures that allow nonprofit organizations and federal, state, and local government agencies to track how well they are performing their services—including social, education, mental health, economic development, sanitation, parks and recreation, transportation, police, fire, corrections, and environmental protection services. In recent years, he has also worked to improve performance measurement and management in other countries, including Thailand and Hungary.
He has authored or co-authored numerous books, reports, and articles describing performance measurement procedures, including publications for the U.S. Departments of Education and Justice, the International City/County Management Association, and United Way of America. Key works in which he has been an author include How Effective Are Your Community Services? Procedures for Measuring Their Quality; The Guide to Program Outcome Monitoring for the U.S. Department of Education; Measuring Program Outcomes: A Practical Approach (for United Way of America); Service Efforts and Accomplishment Reporting: Its Time Has Come—An Overview (for the Governmental Accounting Standards Board); Practical Program Evaluation for State and Local Governments; and Program Analysis for State and Local Governments.
Hatry has received awards for his work in performance measurement and evaluation from a number of organizations, including the American Society for Public Administration and the National Academy of Public Administration.
He spoke with Richard Deutsch about how performance measurement can improve service delivery:
Hatry:
It's the attempt in a systematic way to regularly provide data to agency officials about the outcomes and the efficiency of their programs. That information should provide guidance for managers to improve, to find out where problems exist, to see where they are doing well and where they are not doing well. They can then take action and use later performance measurement reports to see if the desired outcomes occurred.
Deutsch:
In your book you stress the importance in performance measurement of differentiating between program outputs and outcomes. Why?
Hatry:
One of the chronic problems in performance management is for officials and managers to distinguish the physical outputs the agency itself produces-the number of reports prepared, number of miles of roads maintained, number of clients served-to distinguish that from the results of the agency's efforts, the actual outcomes-the change in behavior, attitudes, or condition of the agency's customers. Both types of measurement are important for managers, but in past decades the focus has been on the outputs. The new, commonsense approach is concerned primarily with provision of benefits to the public.
Deutsch:
If this is a commonsense approach, why wasn't it prevalent in the past?
Hatry:
Managers basically felt that they knew whether their programs were doing well or not. They felt their judgments were sufficient. A number of things have happened in the last few decades: there's a recognition now that you need more than that, that the manager's viewpoint isn't sufficient for tracking progress; money for social programs is tighter; social problems have increased: population, poverty, racial discrimination, and other problems that are difficult to track were not perceived in the old days as being as much of a problem as they are today. There's been increasing pressure in recent years-from the public, business leaders, politicians-demanding managers show us what we're getting for our money. And computer technology has advanced to the point where important performance data can quite feasibly be collected and processed.
Deutsch: So we're at a point where the intellectual basis for this movement has been evolving over a couple of decades and now, finally, federal legislation is implementing this?
Hatry:
Yes, but we shouldn't play down the role of the local and state governments. Local governments were probably into performance measurement first. Local governments in places such as Charlotte (North Carolina), Dayton (Ohio), Savannah (Georgia), and Sunnyvale (California) were doing this in the 1970s. They were all doing major pieces of outcome measurement back then. Even then, they were thinking about customer satisfaction with government services. State governments, particularly Oregon and Texas, were beginning efforts in the early 1990s, if not before. A number of private, nonprofit service agencies have also begun making considerable efforts to assess the outcome of their services.
Deutsch:
How does your latest book advance discussion of the issue?
Hatry:
The book provides an extensive series of recommendations on instituting not only performance measurement but also performance management. It is written for any agency that delivers public services, including the private, nonprofit sector. The book addresses basic technical issues, major issues of the performance management process, and identifies obstacles and limitations of performance measurement.
Deutsch:
You say in the book that performance measurement can tell a manager about a program's outcome, but it doesn't tell you why that outcome was achieved.
Hatry:
Yes, but we make the point that a well-conceived performance measurement system can provide a lot of clues. If you are doing surveys of customers, for example, we urge that surveys also ask respondents for suggestions for improvement-and ask why customers give bad marks to certain program characteristics. Disaggregating by organizational units, and by various demographic characteristics, can help identify where things are going well and where not. However, this does not replace the need for in-depth evaluations that attempt to identify causes. But those are difficult to do and expensive.
Deutsch:
You also make the point that performance measurement is not simply a tool for government but is also important for private nonprofits.
Hatry:
Yes. United Ways have been leaders in using performance measurement, and this is a good example of a place where funders have been asking what communities are getting for their money. Local United Ways are now beginning to encourage the measurement of outcomes to provide some of the answers. A number of other organizations, such as in the health field, are seeking to encourage or even require outcome measurement.
Deutsch:
You discuss the relationship between performance measurement and results-based budgeting, the idea of trying to show exactly what the public gets-or expects to get-for its money.
Hatry:
We really don't know now and very possibly never will know the exact relationships between resources budgeted and outcomes. We don't know how much money is required to increase customer satisfaction by X percentage points or to reduce crime by Y percent. But at least we should be able to record what we've done, where we are, and see whether we've made any difference afterwards.
