What Is CPM, and Why Is It Important?
By its focus on results or outcomes, CPM can help communicate to the public an agency's accomplishments and performance in comparison with similar agencies.
An explosion of effort in performance measurement by all levels of government in the United States, and internationally, has occurred in recent years. In addition to the public sector's increased interest, the private nonprofit sector has also demonstrated growing interest in performance measurement. As agency staff collect data (and, frequently, report it on the Internet), they—and citizens they serve—have begun asking how their agency is doing as compared with other similar agencies. This has led to what will become a major aspect of the information about agency performance—how agencies "stack up" against other similar agencies.
Comparative performance measurement (CPM) can be used to improve the management and operations of a particular agency or function, to improve policy and resource allocation decisions, and to communicate to the public what is being accomplished and what community needs should be addressed. In particular, CPM involves
- measuring the performance of an agency, jurisdiction, or area of concern,
- obtaining performance information from similar agencies or jurisdictions,
- comparing the performance of the agencies, operations, or jurisdictions,
- identifying differences in performance and the reasons for these differences,
- learning from these differences how performance can be improved, and
- applying what is learned.
As a rule, CPM should follow internal performance measurement and comparison of performance among different geographic or functional divisions of the agency. This is largely because a clear understanding of performance within an agency is prerequisite to comparing its performance with other agencies. The remainder of this chapter discusses why CPM is important, describing how it builds on other public management initiatives and identifying potential pitfalls.
How Can CPM Be Used?
Public sector agencies face several inherent challenges as they seek to improve outcomes and provide more effective and efficient services. First among these barriers is the lack of any real competitive pressure. Second, it is often difficult to measure and communicate what is being accomplished by public sector agencies and to create a sense of public accountability. Third, the public sector has limited substantive research available as to what works in achieving meaningful results. The role CPM can play in addressing these is discussed in the following sections.
CPM as a Motivator for Improved Performance
One of the most substantial barriers to effective management and service delivery in the public sector is the lack of competition. In the private sector (and many other endeavors of life, such as athletics, the arts, and even academic achievement), competition drives innovation and product or service improvement. Private companies compete both in terms of quality and price. Consumers determine which products provide the best value—the best quality at the best price. Those products determined by consumers to be the best value achieve comparative advantage and succeed, while products of poor value quickly fade away.
It is difficult to motivate real improvement in public sector services without these competitive or market forces. Public sector agencies are typically free from the pressures of achieving and maintaining comparative advantage. Increasingly, however, some public agencies face the possibility that substantial portions of their work will be contracted out, or privatized. CPM sends the same kind of signals the marketplace provides for private sector goods and services. By comparing the performance of similar agencies, CPM creates similar competitive pressures.
CPM can also serve to motivate improved individual performance. Public sector employees, like most people, can be motivated by competition. However, it is often difficult to communicate to employees their agency's purpose, accomplishments, and performance. CPM can help demonstrate how well employees are performing compared with others. In particular, an agency's performance compared with others can serve as a basis for recognizing the employees of that agency. It can also spark the competitive spirit of agency employees as they strive to out-perform other agencies. The performance levels of other better-performing agencies can be used to establish targets for improvement. CPM can also be effective in identifying issues or barriers to improved performance, in identifying ideas and solutions to those issues or barriers, and in communicating to employees that service is an important element of what an agency does.
CPM as an Accountability Tool
Another substantial barrier to effective public sector management and service delivery is the difficulty in clearly measuring and communicating what is being accomplished, and how well. This task is important in creating accountability to elected officials and the taxpayers who support government services, as well as to the customers who are direct recipients of those services.
Public sector accountability is at best indirect. Consumers can indicate their degree of satisfaction or dissatisfaction with public sector services through voting or choosing whether or not to live or operate their businesses in a certain city, county, or state. However, these decisions are infrequent, difficult to tie directly to individual agency performance, and may often send mixed messages about conflicting consumer objectives.
The advent of performance measurement, especially when outcome information is reported regularly in an understandable way, has been a major step forward in demonstrating accountability for results. CPM can build on performance measurement and add another dimension to accountability—comparisons among similar agencies.
By its focus on results or outcomes, CPM can help communicate to the public an agency's accomplishments and performance in comparison with similar agencies. Furthermore, through development of common customer survey questions and instruments, CPM can be used to compare the satisfaction level of citizens or customers with the services provided by different agencies.
CPM as a Tool for Determining Best Practices
Much less recognized is the fact that CPM is a powerful tool for assessing what does and does not work. Industries that have achieved remarkable improvement have done so by identifying and implementing "best" practices. The medical field, as an example, has focused on comparing the performance of different approaches and treatments, and applying what is learned to improve service quality. The automotive industry has emphasized comparing the safety performance of various vehicles and safety devices (using both test and actual experience data). The resulting information has then been used to identify best practices and substantially improve the quality of today's cars.
Government and nonprofit agencies can follow this model. CPM can provide the base of information for analysis, determining what factors lead to better performance. While substantial differences usually exist among agencies delivering similar services, typically there are more similarities than differences. For example, local police departments throughout the United States operate in basically the same way. Each deploys field officers to patrol and respond to citizen calls and detectives to investigate crimes. While these police agencies may operate in different environments (such as different demographic settings), considerable similarity exists as to their missions, organizational structures, and activities. By using CPM to identify the most successful agencies, the best or most effective practices can be identified and shared.
What Does CPM Add to Public Management Improvement Efforts?
CPM builds on other public management improvement efforts, including performance measurement, privatization, managed competition, and Total Quality Management (TQM). Each of these is discussed in the following sections.
Performance Measurement
Performance measurement, although in use for several decades, has only recently become a common tool by which the public sector can better understand and communicate what it is accomplishing. Historically, the public sector has focused on inputs and activities, paying less attention to the outcomes or results achieved by those inputs, activities, programs, and strategies. Today, the focus is shifting much more to what is actually being accomplished.
As public sector organizations and agencies implement performance measurement, they compare their current and past performances. Some are also setting targets against which future performance will be compared. These comparisons demonstrate real progress in public sector management. However, such comparisons provide little information on achievable results or new approaches to or methods of improving performance.
By comparing its performance with others, an agency can determine how much improvement can be made and gain new ideas. CPM provides this outside comparison. For example, an agency may be celebrating a reduction in the amount of time required to process client applications—from an average of 10 days to 8 days. This may be an achievement well worth celebrating, but until the organization compares its new performance level with similar agencies, it has only a partial picture of what can be accomplished. Similar agencies may be processing client applications in just 2 days. However, comparisons of such indicators should be made with caution, since the quality as well as the speed of service may differ.
Privatization and Managed Competition
Privatization of public sector services attempts to subject public sector services to the pressures of private competition and take advantage of the benefits that such competition brings—improved quality and efficiency. However, many of the services provided by the public sector are unique, and it is often difficult to adapt them to a competitive marketplace.
Privatizing such services may mean the provider is a private sector company, but may not create competitive pressures and the associated benefits. Additionally, services delivered by the private sector, many of which are provided for the "public good," can potentially be compromised or lost. Fortunately, CPM has the potential to provide the motivational benefits of privatization. Comparing performance among similar agencies creates competitive pressure for increased quality and efficiency.
Managed competition has been an effective way of bringing private sector competitive pressure to bear on those public sector services that are the same as or similar to services provided in the private sector. In its most common form, managed competition pits a public sector organization and its employees against private sector contract providers. For example, a public sector organization and private contractors may each bid on collecting trash from a certain area of a city, providing janitorial and building maintenance services for a public building, or providing landscaping maintenance for a public park. The work will be well defined in a set of bid specifications and awarded to the lowest bidder, whether public or private sector.
While managed competition offers substantial benefits, its application is very limited. Many of the services provided by the public sector are unique to the public sector. Managed competition can be applied only to those services the private sector currently provides or can be enticed to deliver.
CPM does not have these limitations and can be applied to most public agencies and services. While CPM may not create the same competitive intensity, it has the potential to generate substantial competitive pressure. Additionally, where services are provided by a private contractor, CPM can be used to determine if the contractor is doing a good job.
Total Quality Management (TQM)
TQM focuses on determining the cause of poor product quality within an organization and working to substantially reduce or eliminate those causes. TQM relies heavily on a detailed understanding of the processes involved in providing a service or accomplishing a result. This effort uses such analytic tools as statistical process control, work flow analysis, cycle time reduction, system redesign, quality function deployment, and benchmarking.
CPM can be used to identify areas in need of improvement—where the tools of TQM can most effectively be applied.
What Are the Potential Limitations of CPM?
As is true of any tool, comparative performance measurement has its limitations. Perhaps the most significant of these is the fact that no two jurisdictions or organizations are completely comparable. Each has unique characteristics and situations that make it different from all others. As a result, it is impossible to find organizations that are exactly comparable. Therefore, effort should be put into identifying those that are "roughly" comparable—those that are most similar to each other. Unique features then become factors potentially explaining differences in performance, approach, or method.
Another key limitation is that CPM, like any tool, can be misused. In many cases, comparisons have been incomplete, used poor quality data, or were biased to support a particular point of view. To be useful, CPM must be conducted as thoroughly and carefully as possible. Valid comparisons, even if rough, require substantial effort and commitment.
A third limitation is that the process of determining performance indicators for comparison, identifying comparable agencies, collecting and checking performance information, and analyzing and reporting the comparative information requires a substantial amount of time, effort, and money. These costs should be well understood prior to undertaking a CPM effort.
Fourth, CPM data, like performance measurement data, do not in themselves explain why performance is different—why the results are high or low, good or bad—relative to other organizations. Before assumptions can be made about performance, the reasons for high and low performance levels must be examined.
These reasons include:
- Factors not accounted for in the CPM effort that might have caused a high or low level of performance on an indicator. Examples are unusually good or bad weather, new state or federal requirements, major changes in resources, regional or demographic differences, and changes in clients being served.
- The data for a particular agency may not conform exactly to the data definition used by other agencies or the CPM data collection effort. Even with the most rigorous effort to collect consistent data, such variations will inevitably occur.
The final limitation is the concern that an agency will be identified and reported as performing less well than other similar agencies. This assessment could have a negative impact on the agency and could make the agency, or others, less willing to measure and report performance.
The Focus of This Book
This book focuses on public sector, interjurisdictional CPM efforts. It is primarily intended for practitioners—public managers and employees at all levels of government—looking for ways to improve the services provided by their organization or agency. Managers and employees of nonprofit, service-providing organizations, which often function in ways similar to government agencies, are considered part of this audience. Nonprofit service organizations, encouraged by such national associations as the United Way of America and by their funders, have begin using outcome measurement. National associations of nonprofits and their members are beginning to develop common data elements for CPM purposes in order to encourage improvements and identify successful practices. This is likely to become an area of considerable growth for CPM.
Chapter 2 discusses the different types of and approaches to comparative performance measurement, with numerous examples. The remaining chapters explore establishing the scope of a CPM effort in terms of what to compare (chapter 3), preparing for data collection (chapter 4), collecting performance information and making it comparable (chapter 5), analyzing the results of the effort (chapter 6), reporting the results (chapter 7), and applying the information (chapter 8).
Comparative Performance Measurement, by Elaine Morley, Scott P. Bryant and Harry P. Hatry, is available from the Urban Institute Press (paper, ISBN 0-87766-700-4, $28.00). To obtain a copy call (202) 261-5687 or 1-877-UIPRESS.
