Harry Hatry is director of the Urban Institute's Public Management Program. Since the early 1970s, he has been a leader in developing procedures for evaluation of nonprofit and government services. In recent years, he has also worked to improve performance measurement and management in other countries, including Thailand and Hungary.
He has authored or co-authored numerous books, reports, and articles describing performance measurement procedures, including publications for the U.S. Departments of Education and Justice, the International City Management Association, and United Way of America. Hatry has received awards for his work in performance measurement and evaluation from a number of organizations, including the American Society for Public Administration and the National Academy of Public Administration.
He spoke with Richard Deutsch about how performance measurement can improve service delivery:
Hatry:
It's the attempt in a systematic way to regularly provide data to agency officials about the outcomes and the efficiency of their programs. That information should provide guidance for managers to improve, to find out where problems exist, to see where they are doing well and where they are not doing well. They can then take action and use later performance measurement reports to see if the desired outcomes occurred.
Deutsch:
In your book you stress the importance in performance measurement of differentiating between program outputs and outcomes. Why?
Hatry:
One of the chronic problems in performance management is for officials and managers to distinguish the physical outputs the agency itself produces-the number of reports prepared, number of miles of roads maintained, number of clients served-to distinguish that from the results of the agency's efforts, the actual outcomes-the change in behavior, attitudes, or condition of the agency's customers. Both types of measurement are important for managers, but in past decades the focus has been on the outputs. The new, commonsense approach is concerned primarily with provision of benefits to the public.
Deutsch:
If this is a commonsense approach, why wasn't it prevalent in the past?
Hatry:
Managers basically felt that they knew whether their programs were doing well or not. They felt their judgments were sufficient. A number of things have happened in the last few decades: there's a recognition now that you need more than that, that the manager's viewpoint isn't sufficient for tracking progress; money for social programs is tighter; social problems have increased: population, poverty, racial discrimination, and other problems that are difficult to track were not perceived in the old days as being as much of a problem as they are today. There's been increasing pressure in recent years-from the public, business leaders, politicians-demanding managers show us what we're getting for our money. And computer technology has advanced to the point where important performance data can quite feasibly be collected and processed.
Deutsch:
Congress passed the Government Performance and Results Act in 1993. It stressed outcome measurement and public management accountability for the federal government. What's been accomplished?
Hatry:
It's amazing what has been done. Since 1997, the action has really picked up. With considerable continuing interest in Congress and oversight by the Office of Management and Budget, federal agencies are taking the requirements seriously. That said, it's still in the very early stages. As of today, no annual program performance report has yet been produced. The first ones are due in March 2000, the first reports to Congress and the President on actual achievement on annual performance plans and the extent to which agencies have been able to meet their outcome targets. We're still in the very early stages of a very complex piece of business.
Deutsch:
So we're at a point where the intellectual basis for this movement has been evolving over a couple of decades and now, finally, federal legislation is implementing this?
Hatry:
Yes, but we shouldn't play down the role of the local and state governments. Local governments were probably into performance measurement first. Local governments in places such as Charlotte (North Carolina), Dayton (Ohio), Savannah (Georgia), and Sunnyvale (California) were doing this in the 1970s. They were all doing major pieces of outcome measurement back then. Even then, they were thinking about customer satisfaction with government services. State governments, particularly Oregon and Texas, were beginning efforts in the early 1990s, if not before. A number of private, nonprofit service agencies have also begun making considerable efforts to assess the outcome of their services.
Deutsch:
How does your latest book advance discussion of the issue?
Hatry:
The book provides an extensive series of recommendations on instituting not only performance measurement but also performance management. It is written for any agency that delivers public services, including the private, nonprofit sector. The book addresses basic technical issues, major issues of the performance management process, and identifies obstacles and limitations of performance measurement.
Deutsch:
You say in the book that performance measurement can tell a manager about a program's outcome, but it doesn't tell you why that outcome was achieved.
Hatry:
Yes, but we make the point that a well-conceived performance measurement system can provide a lot of clues. If you are doing surveys of customers, for example, we urge that surveys also ask respondents for suggestions for improvement-and ask why customers give bad marks to certain program characteristics. Disaggregating by organizational units, and by various demographic characteristics, can help identify where things are going well and where not. However, this does not replace the need for in-depth evaluations that attempt to identify causes. But those are difficult to do and expensive.
Deutsch:
You also make the point that performance measurement is not simply a tool for government but is also important for private nonprofits.
Hatry:
Yes. United Ways have been leaders in using performance measurement, and this is a good example of a place where funders have been asking what communities are getting for their money. Local United Ways are now beginning to encourage the measurement of outcomes to provide some of the answers. A number of other organizations, such as in the health field, are seeking to encourage or even require outcome measurement.
Deutsch:
You discuss the relationship between performance measurement and results-based budgeting, the idea of trying to show exactly what the public gets-or expects to get-for its money.
Hatry:
We really don't know now and very possibly never will know the exact relationships between resources budgeted and outcomes. We don't know how much money is required to increase customer satisfaction by X percentage points or to reduce crime by Y percent. But at least we should be able to record what we've done, where we are, and see whether we've made any difference afterwards.
