by Timothy Triplett

What is it used to measure?

Sample design allows researchers to get estimates from a population so large that every member cannot be interviewed.

How does it work?

When designing a sample, researchers should first ask if they must interview every member of a population. If the population of interest is quite large, then a sample almost certainly needs to be selected. However, even if the entire population can be surveyed, selecting a smaller number of subjects may provide reasonably precise estimates of the entire population at a much-reduced cost. Many studies spend too much money trying to interview an entire population, while better estimates would have resulted from getting a high response rate from a sample. In general, sample sizes of 1,000, 500, and even 200 or fewer can provide sufficient precision as long as the sample has been selected at random from the overall population.

To do separate analysis on a subgroup of the population, researchers either choose a larger overall random sample or select at random additional respondents that meet the definition of the subgroup. How many interviews to complete depends on a number of things (e.g., types of analysis planned, variability of key variables, total population), most of all on how much precision the study needs. While researchers always try to achieve the highest quality analysis possible, they recognize that most surveys do not need the same level of precision. Many successful surveys have published results based on samples of 500 or less.

All good sample designs have one common component: they are randomized, selected so that probability of selection can be determined. This is true for a simple random sample, a stratified random sample with unequal probability of selection for some subgroups, or a more complex dual-frame design with two different samples. Without randomization a sample is usually described as a "convenience sample", composed of population members that are easily located and willing to participate. Convenience samples are not recommended for evaluations that involve making an inference about the population as whole, but they can be justified when trying to interview hard-to-find populations or when the evaluation objective is not to produce estimates but to learn more about some key issues.

Research examples

"2002 NSAF Sample Design"

Customer Surveys for Agency Managers: What Managers Need to Know

"2002 NSAF Collection of Papers"