2002 NSAF Variance Estimation

Report No. 4 in the 2002 NSAF Methodology Series

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Note: This report is available in its entirety in the Portable Document Format (PDF).

1. INTRODUCTION

This report describes the methods and estimated sampling errors for statistics from the 2002 National Survey of America’s Families (NSAF). The first chapter gives an overview of the sample design and summarizes the precision of the survey estimates for both children and adults. This chapter also discusses computing sampling error estimates for differences estimated from NSAF data collected in 2002 (Round 3) and the earlier two rounds (Round 1 in 1997 and Round 2 in 1999).

The rest of the report describes the methodology used to create these types of estimates of sampling variability. Chapter 2 reviews the two main methods of computing sampling errors or variances of estimates from surveys with complex survey designs like the NSAF. Chapter 3 discusses the replication method of variance estimation for the NSAF in more detail. Chapter 4 discusses software available for computing sampling errors.

1.1 Overview of the Survey

The NSAF collected information on the economic, health, and social dimensions of the wellbeing of children, adults under the age of 65, and their families in 13 states and the balance of the nation. The Urban Institute selected these study areas (see figure 1-1) before the first survey in 1996 because they represent a broad range of fiscal capacity, child well-being, and approaches to government programs. Data were also collected in the balance of the nation to permit estimates for the United States as a whole. The sample design is briefly outlined here. The complete details are provided in the 2002 NSAF Sample Design, Report No. 2.

The primary goal of the survey in all three rounds was to obtain social and economic information about children in low-income families (with incomes below 200 percent of the poverty threshold) since the impact of new federalism on these children was likely to be greatest. Secondary goals included obtaining similar data on children in higher income families, plus adults under age 65 (with and without children).

One major change in the design for the 2002 survey is that Milwaukee County in Wisconsin, which had been a separate study area with its own sample in 1997 and 1999, was no longer a separate study area. The Milwaukee County study area was included with the rest of Wisconsin as a single study area.

Another major change in the sample design was to eliminate the nontelephone sample in the study areas representing specific states. The NSAF uses two frames—a random digit dialing (RDD) sample of households with telephones and an area sample conducted in person for those households without telephones. In 1997 and 1999, this dual-frame approach (see Waksberg et al. 1997) was used for each study area and for the nation. In 2002, only a national area sample was selected, and the same dual-frame method was used to produce national estimates. No separate sample of households without telephones was selected in the study areas, so estimates for the study areas are based only on the RDD sample selected. The weights of the RDD sample in the study areas were adjusted to reduce the potential bias associated with not sampling households without telephones in the study areas.

The national area sample consisted of a subsample of primary sampling units (PSUs) used in the 1999 survey. The 1999 PSUs were subsampled in the study areas, while all the PSUs from the balance of the United States were retained. Since block groups with very high telephone coverage rates as of the 1990 Census were excluded in the Round 1 and Round 2 sampling, this same restriction was carried forward to the 2002 survey of nontelephone households.

Another new feature introduced in the 2002 survey was the subsampling of refusal cases. A random sample of telephone numbers was selected, and only those identified in the subsample were followed to completion if they refused the screening interview. The numbers that were not subsampled were classified separately to make it easier to compute response rates.

In the RDD sample, a screener-based subsampling of households was used to sample low-income households at a higher rate than other households. A very short income question was asked during the RDD screening interview, and those that reported an absence of children or incomes above 200 percent of the poverty threshold were subsampled.

Within both the RDD and area samples, household members were subsampled to reduce the respondent burden. If there were multiple children under age 6, one was randomly selected. The same was done for children 6 to 17 years old. Data were collected from the most knowledgeable adult (MKA) in the household for the sampled child. During the MKA interview, data were also collected about the MKA and about his/her spouse or partner. Most questions asked about the MKA were repeated in reference to the spouse or partner; however, some questions on health insurance and health care usage were asked in reference to only one of the two. The target of these questions was randomly assigned to either the MKA or his/her spouse or partner. Questions asked only about the MKA were those related to feelings, religious activities, and opinions.

Other adults in households with children were subsampled, as were adults in adult-only households. Adults were eligible only if they would not have been the MKA for other children in the household if those children had been selected. Self-response was required for sampled adults—with proxy data collected about his/her spouse or partner (if living in the same household). Data were not collected directly from the spouse of a sampled adult. As in the MKA interview, there were also some questions related to feelings, religious activities, and opinions that were asked only about the sampled adult.

Note: This report is available in its entirety in the Portable Document Format (PDF).

The nonpartisan Urban Institute publishes studies, reports, and books on timely topics worthy of public consideration. The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.

Usage, posting and reprint of materials on the UI web site:

Most publications may be downloaded free of charge from the web site in PDF format. This information may be used and copies made for research, academic, policy or other non-commercial purposes. Proper attribution is required.

Copyright of the written materials contained within the Urban Institute website is owned or controlled by the Urban Institute. Posting UI research papers on other websites is permitted subject to prior approval from the Urban Institute—contact paffairs@urban.org.

If you are unable to access or print the PDF document please contact us or call the Publications Office at (202) 261-5687.