The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Note: This report is available in its entirety in the Portable Document Format (PDF).
Why Follow Up with Clients?
About This Guidebook
SECTION I: Before Starting Follow-Up
Step 1: Decide What Information Is Needed and from Whom
Step 2: Determine Timing of Follow-Up
Step 3: Choose How to Administer the Survey
SECTION II: While Clients Are in Service
Step 4: Assess Condition and Behavior at Entry
Step 5: Inform Clients about the Need for Follow-Up
Step 6: Obtain Current Contact Information
Step 7: Establish Good Relations with Clients
Step 8: Obtain Client Consent for Follow-Up
Step 9: Discuss Follow-up Procedures with Clients at Exit
SECTION III: While Conducting Client Follow-Ups
Step 10: Maintain Contact with Former Clients
Step 11: Offer Clients Incentives for Participating
Step 12: Administer the Follow-up Questionnaire
SECTION IV: After Outcome Information Becomes Available
Step 13: Analyze and Report Follow-up Outcomes
Step 14: Take Action
SECTION V: Other Key Issues
Maintaining Client Confidentiality
Reducing Costs of Follow-Up
Selecting a Client Sample for Follow-Up
1. Steps for Following Up with Former Clients
2. Sample Outcome Indicators Based on Follow-up Information
3. Sample Client Follow-up Tracking Log
4. Sample Client Contact and Update Form
5. Sample Consent Form for Post-Service Follow-Up
6. Follow-up Procedures to Discuss with Clients at Exit
7. Tips for Locating Former Clients
8. Tips for Obtaining Follow-up Information from Former Clients
9. Questions That Follow-up Data May Answer
10. Guidelines for Protecting Client Confidentiality during Telephone Follow-Up
11. Sample Assignment of Follow-up Responsibilities
12. Tips for Reducing Follow-up Costs
Outcome measurement offers an invaluable tool for monitoring the impact of our services and for tracking our clients' progress toward desired program goalswhether to provide safe and affordable housing to the disabled and disadvantaged, or to rehabilitate those with addictions or past criminal involvement. Volunteers of America, one of the largest nonprofit faith-based social services organizations in the country, delivers a broad range of services nationally to more than 1.6 million individuals annually. Touching so many lives, we feel a strong commitment to ensuring the highest possible quality and effectiveness of the services we deliver.
As Volunteers of America begins implementing outcome measures in our approximately 100 different types of programs, one of our most difficult challenges is determining the long-term impact of our services. Do the newly employed retain their jobs and self-sufficiency? Do the parents and children who have been reunited remain together and continue healthy functioning? To measure these lasting effects, clients must be tracked after they leave our programs. Yet collection of follow-up data often strains the resources of community-based nonprofits, such as our local offices. Gathering these data may require staff efforts to be diverted from ongoing service delivery, and former clients may be difficult to locate or reluctant to provide follow-up information.
This report tackles these major obstacles to obtaining data on client outcomes. Drawing from the lessons learned by a number of other community based nonprofits, including Volunteers of America local offices, it offers practical advice on methods to secure this data cost-effectively. The information provided on using follow-up data to identify best practices and areas where program improvement efforts are needed is particularly valuable.
With technical assistance resources, such as this report, and commitment from organizational leadership, nonprofits can successfully implement outcome evaluation systems that will help ensure the long-term effectiveness of their services to improve the lives they touch.
Charles W. Gould
Volunteers of America
What constitutes program success? If a client stops abusing substances while participating in a treatment program, but resumes a few months after completing treatment, did the program succeed? If a teenager receives pre- and post-natal support and guidance after her first pregnancy, but reports a second pregnancy six months after giving birth to her first child, was the support effective? If a foster parent reports a missing child three months after completing a youth development program, did the program succeed? These examples illustrate the importance of
following up with clients after they complete or leave services.
The outcome of a program designed to improve clients' conditions or behaviors and sustain this improvement beyond the period of service provision cannot be adequately assessed at the time the client leaves service. Information on the client's status at some point in time after the client has left servicethree, six, nine, or 12 months, for exampleis a considerably more valid basis for assessing program results.
Since many nonprofit programs seek to help clients enjoy long-term success, it makes senseand is organizationally strategicto obtain regular feedback from clients. Such information provides not only an assessment of program effectiveness, but also a solid basis for identifying needed improvements to services. While many nonprofits do not follow up with clients post-service, some others, such as vocational rehabilitation and job training programs, have followed up and reported on the number and earnings of clients who retained their job for specified periods after placement.
After-service follow-ups can be done efficiently, successfully, and at reasonable cost. This guide to tracking clients offers step-by-step procedures, model materials (including planning tools and feedback forms), and suggestions for keeping costs low. The manual is primarily geared to nonprofit managers and other professional social service staff who are most likely to design and implement a process for following up with clientsand who ultimately will apply this information to their organization's programs and practices.
This manual is not intended for programs that (1) provide services that are not expected to lead to longer-term effects, such as homeless shelters or food kitchens that offer mostly "one-day" help; or (2) serve long-term clients who receive regular and ongoing treatment, such as the institutionalized.
The grantmaking community, which can provide the needed moral and resource support for outcome measurement by nonprofits, is another major audience for this guide. Funders report that they want meaningful and measurable results of their grants, not only to highlight the accomplishments of grantees, but also to illustrate their own accountability in disbursing funds.
Why Follow Up with Clients?
Following up with former clients helps nonprofits determine whether improvements
in clients' behavior or condition have been sustained.1 In fact, the process of assessing program outcomes has many other uses for an organization. These include the following:
- Assessing whether clients have maintained program results after leaving service
- Discovering what contributes to long-term success
- Motivating staff by letting them know the extent to which their former clients have achieved satisfactory results
- Identifying problems that may lead clients to leave the program before completion
- Identifying common post-service problems that may indicate a need for program modifications
- Helping former clients stay connected in case additional services or referrals are needed
- Meeting funder requirements to assess the impact of grants
- Marketing the organization's services
|In the FieldUsing Follow-up Information
The Family Leadership Program of Crossway Communities originally taught a single curriculum
to its participating adults, all of whom had diverse educational backgrounds. Using information from its annual alumni survey, the Leadership Program subsequently designed two separate curricula and divided its class into individuals with some college education or a bachelor's degree, who need additional training to get them back on their feet; and individuals who need more education and life skills training to succeed. The organization reports increased enthusiasm for the leadership program since the change.
|In the FieldUsing Follow-up Information
Before implementing a client follow-up process, the Alexandria Resource Mothers (ARMS) program of the Northern Virginia Urban League set a goal of graduation or completion of vocational training by 80 percent of its participating teen mothers. The actual number achieving this goal was 60 percent. An analysis of feedback obtained through follow-up telephone surveys with clients revealed that young Latina mothers were more reluctant than teen mothers of other ethnic backgrounds to stay in school. As a result, ARMS hired a recent graduate of the programa young Latin-American womanto serve as a resource for participating Latina mothers. In addition, all resource mothers were trained to take a more culturally sensitive approach to discussing continuing education with Latinas. The ARMS program coordinator feels that collecting and using client follow-up information has not only improved this program for parenting teens, but also serves to motivate staff by letting them know how their efforts help clients.
Nonprofits are concerned that follow-up procedures will require additional staff, increase the burden on current staff, or will require additional training or use of consultants. They sometimes feel that locating former clients will be too difficult or that they will be unwilling to participate, thus shifting resources from the primary mission to help clients in service. The follow-up procedures recommended in this guide attempt to reduce these problems.
The proposed procedures differ in many ways from full "program evaluation" studies by professional organizations or universities, which can be very expensive. Follow-ups can be done inexpensively, especially if the organization maintains a good relationship with its clients while they are in service.
About This Guidebook
This guide presents 14 key stepsfor use by all types of nonprofitsto conducting
effective follow-up with clients (see exhibit 1).2 The steps are grouped into four sections, with a fifth section on overarching issues.
SECTION I. Before Starting Follow-Up describes what needs to be done before the follow-up process even begins.
SECTION II. While Clients Are in Service covers what preliminary steps should be taken while clients are in programs and receiving services.
SECTION III. While Conducting Client Follow-Ups identifies key steps after clients have left services to increase the likelihood that follow-up will be successful.
SECTION IV. After Outcome Information Becomes Available provides guidance on how to analyze and report data obtained from the clients, as well as how to use the information to
SECTION V. Other Key Issues discusses in more detail such important issues as maintaining client confidentiality, training program staff, and reducing the costs of following up with former clients.
1 "Nonprofit" refers to community-based organizations that directly serve clients.
2 Limited literature exists on procedures for following up with former clients on an ongoing basis. See for example David P. Desmond, James F. Maddux, Thomas H. Johnson, and Beth A. Confer, "Obtaining follow-up interviews for treatment evaluation," Journal of Substance Abuse Treatment
12 (1995): 95-102; and Elizabeth A. Hall, "Homeless Populations: Strategies for Maximizing Follow-up Rates" (paper presented at Center for Substance Abuse Treatment Addictive Treatment for Homeless Technical Assistance Workshop, November 29-30, 2001, Bethesda, Maryland). Detailed manuals on following up with clients have also been produced by federal agencies, including the U.S. Department of Health and Human Services' Center for Substance Abuse Treatment, and the U.S. Department of Education's Division of Adult Education and Literacy (as part of the National Reporting System for Adult Education).
This guide was written by Ritu Nayyar-Stone and Harry P. Hatry.
The guide benefited greatly from the assistance, comments, and suggestions of Michael Hendricks, Evaluation Consultant, Volunteers of America; Laura F. Skaff, Director of Research and Evaluations, Volunteers of America; and Linda Lampkin, the Urban Institute.
The authors also thank the following individuals who provided highly useful information on their follow-up procedures. Their input has made this report more comprehensive and richer for our audience: Francine Feinberg, Director, Meta House; Elizabeth A. Hall, UCLA, Integrated Substance Abuse Programs; John Korsmo, University of Wisconsin, Milwaukee; Doreen Mulz, Volunteers of America, Southwest California; Florene Price, Alexandria Resource Mothers Project; Greg Zinser, President and CEO, Vista Hill; and Larry Hitchison, Outcome Director, Vista Hill.
The editors of the series are Harry P. Hatry and Linda Lampkin. We are grateful to the David and Lucile Packard Foundation for their support.