urban institute nonprofit social and economic policy research

Surveying Clients about Outcomes

Read complete document: PDF


PrintPrint this page
Share:
Share on Facebook Share on Twitter Share on LinkedIn Share on Digg Share on Reddit
| Email this pageE-mail
Document date: August 31, 2003
Released online: August 31, 2003

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Note: This report is available in its entirety in the Portable Document Format (PDF).


Contents

Preface

Acknowledgments

Introduction
About This Guidebook

Section I: Survey Basics
What Are Client Surveys?
Limitations
Challenges

Section II: Initial Decisions
Step 1: Identify What Information Is Needed
Step 2: Determine Which Clients to Survey, and How Many
Step 3: Decide Who Should Design, Conduct, and Analyze the Survey

Section III: Developing the Questionnaire
Step 4: Prepare the Questionnaire
Step 5: Conduct a Pretest
Step 6: Decide When and How Often to Administer the Survey

Section IV: Administering the Survey
Step 7: Choose How to Administer the Survey
Step 8: Take Actions to Achieve a High Response Rate
Step 9: Ensure That Surveys Do Not Harm Clients
Step 10: Administer the Survey

Section V: Collecting and Using the Data
Step 11: Enter and Tabulate the Data
Step 12: Analyze the Results
Step 13: Prepare and Disseminate Reports
Step 14: Encourage Use of the Survey Information

Section VI: Cost and Time Requirements
Initial Decisions
Developing the Questionnaire
Administering the Survey
Collecting and Using the Data

References

Appendices

Exhibits


Introduction

Executives, program managers, and those who fund nonprofit organizations have an inherent interest in learning about service outcomes—how their services affect client condition, situation, or behavior. Collecting and using such results are essential to the practice of outcome management. Good outcome management, of course, helps to improve service delivery, make services more effective, and modify or eliminate ineffective activities. It also helps to assess organizational achievement and value.

The first guide in this series, Key Steps in Outcome Management, identifies various methods for obtaining outcome information, such as reviewing agency records, using trained observers (such as caseworkers), and administering client surveys. Each has its benefits and costs. Systematic client surveys, for example, can be more expensive than other methods, but are likely to yield some of the most important kinds of outcome information for a nonprofit organization. Although few nonprofit organizations regularly survey their clients, feedback from clients is likely to be a major, if not the major, source of information on service outcomes.

This guide is intended to encourage the use of client surveys to measure service outcomes on a routine basis.1 Extensive and complicated surveys are not necessary. At the same time, some aspects of the process can be technically challenging. That is why this guide contains more detail on procedures than others in the series.2 Nonprofits do not have to become survey experts, as technical expertise and support are available from a variety of sources. But nonprofits must understand what surveys involve, recognize good survey practice, and make decisions about the role that staff members—as opposed to technical specialists—ought to play. This knowledge helps ensure that useful, high-quality information is collected and provides reasonable confidence that the information collected will stand up to external scrutiny.

About This Guidebook

This guidebook includes the following sections:

Section I: Survey Basics outlines some features, limitations, and challenges of the survey process for nonprofits.

Section II: Initial Decisions includes steps 1 through 3 of the client survey process, involving actions taken before survey development begins.

Section III: Developing the Questionnaire includes steps 4 through 6, covering drafting the survey instrument and pretesting.

Section IV: Administering the Survey includes steps 7 through 10, covering distributing and administering the survey, and handling response rates and client confidentiality.

Section V: Collecting and Using the Data includes steps 11 through 14, involving entering, tabulating, and analyzing the data.

Section VI: Cost and Time Requirements discusses the efforts required for implementing and maintaining a client survey process.

Exhibit 1 summarizes the key steps to conducting client surveys.


EXHIBIT 1
Key Steps to Conducting
Effective Client Surveys

Section II: Initial Decisions

Step 1. Identify what information is needed

Step 2. Determine which clients to survey, and how many

Step 3. Decide who should design, conduct, and analyze the survey

Section III: Developing the Questionnaire

Step 4. Prepare the questionnaire

Step 5. Conduct a pretest

Step 6. Decide when and how often to administer the survey

Section IV: Administering the Survey

Step 7. Choose how to administer the survey

Step 8. Take actions to achieve a high response rate

Step 9. Ensure that surveys do not harm clients

Step 10. Administer the survey

Section V: Collecting and Using the Data

Step 11. Enter and tabulate the data

Step 12. Analyze the results

Step 13. Prepare and disseminate reports

Step 14. Encourage use of the survey information


Note: This report is available in its entirety in the Portable Document Format (PDF).


1 The guide draws from several sources on survey design and practice, including Don A. Dillman, Mail and Internet Surveys: The Tailored Design Method; Harry P. Hatry et al., Customer Surveys for Agency Managers; the American Statistical Association's "What Is a Survey?" series of brochures; Kathryn E. Newcomer and Timothy Triplett, "Planning and Using Surveys to Collect Useful Program Information"; Boys & Girls Clubs of America, Youth Development Outcome Measurement Tool Kit, 1998; and the web site of the American Association for Public Opinion Research, http://www.aapor.org.

2 Another guide in this series, Finding Out What Happens to Former Clients, also discusses client surveys as a means of gathering information on program outcomes.


Topics/Tags: | Health/Healthcare | Nonprofits


Usage and reprints: Most publications may be downloaded free of charge from the web site and may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact publicaffairs@urban.org.

If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.

Disclaimer: The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute.

Email this Page