The Benefits and Challenges of Designing a Cost Study with Community Partners
“How much does it cost?” is one of the most common questions policymakers, funders, researchers, and program administrators ask when considering a new program or policy. A cost study (PDF) can help answer this question, but cost estimates are only as good as the information collected. Without input and guidance from staff members and administrators directly involved in the programming, the answer may lack critical insight.
Traditionally, researchers conduct cost studies using rigid protocols to account for all anticipated elements of a program’s operations and staff time. But this design may ignore community expertise and therefore miss important elements, be burdensome for service providers who may not see any direct benefit, and perpetuate a hierarchical approach to research.
We did something different for the Family-Centered Community Change (FCCC) evaluation: we took a community-engaged approach and invited program partners to co-design a cost study.
Considerations in a community-engaged cost study
FCCC is a two-generation effort that integrated existing supports for adults and children and developed new family-focused services in Buffalo, New York; Columbus, Ohio; and San Antonio, Texas.
Throughout the evaluation, we collaborated with community-based staff from each local partnership to shape the scope of the study, design the methodology and data collection tools, and interpret our results. This approach was more likely to generate information that local service partners could use for future program planning and fundraising. It also provided better context to interpret the results.
Our community-engaged cost study included the following steps:
- establishing a study scope and expectations around data collection that were realistic for each community, accomplished through several meetings and iterations with local staff
- convening focus groups with staff at various roles to ask for input on the methods and data collection instruments, helping us ensure our instruments would capture the activities relevant to each partner’s efforts and be easily understood by staff members regardless of their organization, service sector, or FCCC community
- soliciting feedback from staff to review and contextualize the research findings to help us understand why certain costs might be higher than others before reporting publicly
But these benefits came with trade-offs. The community-engaged cost study presented several challenges:
- For community-based partners, collaborating with the research team required more effort up-front and in the follow-up period.
- Data collection and analysis took longer than typical to accommodate meetings and focus groups with community-based partners.
- Results were less precise and comparable. To accommodate partners’ preferences and schedules, we collected information over different data collection periods and with different tracking frequencies. We were only able to consistently capture the cost of staff time, instead of a more comprehensive range of costs. We also only estimated costs at the program level, rather than understanding costs incurred for each family served (which would have been a more precise approach).
Takeaways for future community-engaged research
What can other researchers learn from our work? We recommend the following strategies:
- Incorporate community partners at all phases of the study. In a community-engaged cost study, researchers can engage program partners regularly. Collaboration can be structured around important research milestones, including study design, data collection, and interpretation of results. To facilitate this process, researchers should send regular updates and documents for communities to review.
- Be transparent about what can and cannot change. Certain structures or expectations may need to be in place to create a methodologically strong cost study. The research team should be transparent about what cannot change and explain why. Community partners and researchers should agree on how feedback will be processed and how community partners’ recommendations can be incorporated into the study.
- Acknowledge that community-based partners may have limited time. Community-based partners are often juggling multiple obligations. Researchers should acknowledge their partners’ time commitments and constraints and be flexible, which may mean anticipating a longer study timeline.
- Consider creating alternative products for community use. Cost study reports are technical and may provide more detail than community members need or want. Consider working with the community to develop additional products that present the data in ways that work for them—fact sheets, presentations, and podcasts are some creative options.
The Urban Institute has the evidence to show what it will take to create a society where everyone has a fair shot at achieving their vision of success.
(10'000 Hours / Getty Images)