Since March 2020, COVID-19 has forced many child care programs across the country to close, and dozens of surveys of child care providers and families have been conducted. States and local leaders are trying to understand how programs and their staff are faring and what it will take to reopen and recover from the pandemic.
But with the urgency for answers, are surveys being developed and tested the way they should be? Writing good survey questions can take time and resources. Developing surveys quickly is challenging, especially in a time of such change and unfamiliarity. Keeping in mind best practices for survey writing may help. Our new brief highlights five tips for national, state, and local agencies seeking answers:
- Clearly identify who you are targeting. Child care providers vary in many ways. Who do you want to respond? Both center-based and home-based providers? Are you including Head Start and public prekindergarten programs? What about before- and aftercare for school-age kids? Do you want to survey child care directors or the larger workforce, including all teaching staff? The questions you ask largely depend on the people responding and their positions and roles. Know there may be limitations to your findings if you cannot reach all groups; for example, if the survey only includes accredited providers or those that accept subsidies.
- Keep your survey short and the focus narrow. Although it’s hard to cut questions you think are important, long surveys are challenging to complete and can turn people away. What are your study goals? What do you ultimately want to know? People may have 10 to 15 minutes to fill out a web survey, but not 30. It’s better to prioritize and use a shorter survey than to ask extra questions that would be nice to have but aren’t essential for the study. Chances are some respondents won’t complete the whole survey if it’s too long, and you’ll be left with lots of missing data.
- Use specific and appropriate time frames. Will your survey only focus on experiences during the pandemic? If you want to collect information about a time period before COVID-19, it’s best to provide a specific reference point. Even then, some people will have a difficult time recalling events and information. Current crises can make it harder for respondents to focus on past experiences. In general, memories for details are often short-lived, so time frames should be reasonable, such as in the past week or month depending on the topic.
- Be careful with word choice. Survey questions and response options should have the same meaning for all respondents and be clear in intent.Surveys nowadays commonly ask about employment, such as asking a child care worker if they lost their job because of COVID-19. How do we define “losing a job”? There are several possible scenarios that could misconstrue findings: the worker could be laid off, furloughed (technically still employed but not working or earning pay), on paid administrative leave (temporarily not working but earning a paycheck), or performing some remote work for pay but not performing regular duties. More specific questions about changes in pay, work location (on site or remote), total hours worked, and time spent working directly with children could be helpful to explain how COVID-19 has affected their employment.
- Test your survey! Before launching a survey, test it at different stages. Early on, ask several experts to review draft questions and provide feedback on the wording of questions and priorities. Perform cognitive testing with several child care providers or people similar to your target population. They will point to confusing or sensitive questions and response options and suggest better ways to phase them. Once your survey protocols are complete, pretest them as you would administer them in the field. If it’s a web survey, send test links to several people outside your research team to make sure the display is clear and skip patterns work. Fix any issues you observe.
Although you may feel the need to rush to get a survey in the field, take the necessary time to write clear and concise questions, solicit expert feedback, and test your survey. Otherwise, you risk missing or inaccurate data and an incomplete picture of the situation.
Surveys are a crucial way state and local leaders can understand child care providers’ needs during the pandemic. Asking the right questions is key.