Introduction
The 1999 National Survey of America's Families (NSAF), like its 1997 counterpart, is a survey of the economic, health, and social characteristics of children, adults under the age of 65, and their families. NSAF data collection was conducted for the Urban Institute and Child Trends by Westat, a nationally renowned survey research firm.
Interviews in 1999 were obtained from over 42,000 households, yielding information on more than 109,000 persons under age 65. The scope and design of the 1997 survey was similar, with over 44,000 interviewed households and again about 109,000 nonelderly persons.
These large representative samples have been taken for both 1997 and 1999 in each of 13 targeted states1 and in the balance of the nation. The 13 targeted states are Alabama, California, Colorado, Florida, Massachusetts, Michigan, Minnesota, Mississippi, New Jersey, New York, Texas, Washington, and Wisconsin (see figure 1). Collectively these 13 states account for over half of the U.S. population and represent a broad array of government programs, fiscal capacity, demographic characteristics, and child well-being.
Figure 1
Targeted NSAF States
The NSAF sample is representative of the civilian, noninstitutionalized population under age 65. Data for the 1999 survey were obtained from February to October 1999. As with virtually all household surveys, some important segments of the population (e.g., the homeless) could not be sampled because of their living arrangements and therefore are not included in the survey results. A small fraction of the sample consisted of what may be called "linguistically-isolated" households, where neither English nor Spanish was spoken by any person within the household. Individuals in these living arrangements could not be interviewed either.2
Survey Process
The NSAF sample had two parts: The main sample consisted of a random-digit dial (RDD) survey of households with telephones. This was supplemented with a second (area probability) sample of households without telephones. In both the RDD and area samples, interviewing was conducted in two stages: first, a short, five-minute screening interview was conducted in order to determine household eligibility. Following the screener interview, if the household was eligible and retained as a sample household, a more detailed, 27- to 45-minute extended interview was conducted to ask about the main survey items of interest.
Sample Design. The NSAF sample was drawn separately for each of the 13-state study areas and for the balance of the nation. Telephone households were subsampled, with the subsampling rates depending on the presence of children in the household and their response to a single household income-screening question. All households screened as having children and low incomes were given a full interview, while higher-income households with children and all households without children (but with someone under 65) were subsampled before in-depth questioning. Households with only adults age 65 and over were screened out of the survey. In all, 147,623 telephone households were screened for the 1999 survey. After screening, extended interviews were conducted in 40,873 of these telephone households.
In the area sample, households within sampled blocks were screened and all nontelephone households with someone under 65 were interviewed. Because only a small fraction of households do not have a telephone, block groups from the 1990 census that had a very high percentage of telephone households were eliminated from the area sampling frame. A special coverage adjustment was made during the weighting process to account for excluding persons in nontelephone households in these block groups. For this portion of the sample, screening interviews were conducted with 39,817 households in 1999. Because only persons without telephones were eligible, extended interviews were conducted after screening in just the 1,487 nontelephone households identified, making 40,873 telephone + 1,487 nontelephone = 42,360 interviewed households altogether. In 1997, the proportion of telephone to nontelephone household interviews was very similar, with 42,973 telephone + 1,488 nontelephone = 44,461 households in that year.
For the 1999 NSAF, both the telephone and area samples contained partial overlaps of sampling units used in the 1997 NSAF. For the telephone sample, this meant that about 60 percent of the telephone numbers used in 1999 were drawn from the 1997 sample of telephone numbers. Numbers from the 1997 round were sampled at different rates, depending upon the 1997 screener result code. In general, those telephone numbers that had completed screeners from 1997 were retained at higher rates than telephone numbers that resulted in noncontact, refusal, or nonworking screener result codes. Of the 1,384 area segments interviewed in 1999, 1,265 had come from the 1997 area sample.
Data Collection. Interviewing by telephone interviewers was carried out from centralized facilities using computer-assisted telephone interviewing (CATI) technology. In-person interviewers provided cellular telephones to connect respondents in nontelephone households to the interviewing centers for the CATI interview; hence, the interviews were conducted in essentially the same way in both telephone and nontelephone households.
Adults with Children. For households with children under the age of 18, up to two children could be sampled for in-depth study: one under the age of 6 and another between the ages of 6 and 17. Interviews were conducted with the most knowledgeable adult (MKA)—that is, the adult in the household who was most knowledgeable about the health care, education, and well-being of the sampled child. It was possible to have different MKAs for each child, although it was more common to have a single respondent (usually the mother) for both children. Overall, for the 1999 survey, 29,917 extended interviews were conducted with the primary caregivers of children, netting information on children. In the 1997 NSAF there were 28,331 such extended interviews and information obtained on children.
Adults without Children. In households with children, in addition to the MKA, one or two additional adults under 65 (those who did not have any of their own children under the age of 18 living with them) were sampled and interviewed. Similarly, in households without children, depending on the size of the household one or two adults (between the ages of 18 and 64) were randomly selected for interview. For the 1999 survey, 16,788 extended interviews were conducted with childless adults. In the 1997 NSAF there were 20,168 such extended interviews. Altogether, combining interviews with MKAs and with adults without children, information was obtained directly or by proxy on adults in the 1999 NSAF - about the same as the adults obtained in the 1997 survey.
As with the 1997 NSAF, the 1999 survey features an oversample of families with incomes below 200 percent of the federal poverty level. Forthcoming reports in the 1999 NSAF Methodology Series provide details on the considerable success achieved here. See especially 1999 NSAF methodology reports Nos. 2 and 3.
The interviews of most knowledgeable adults contained questions on the sampled children, the MKA and his or her spouse/partner, and their families. The childless adult interviews, for the most part, covered the same questions as the MKA interviews; except that detailed questions on children were not asked. As was true in the 1997 survey, questions were asked on the topics listed below.
Economic Security
Child Support
Employment and Earnings
Family Income
Food Security
Housing and Economic Hardship
Welfare Program Participation
Health and Health Care
Health Care Coverage
Health Care Use and Access
Health Status/Limitations
Child Well Being
Child Behaviour Problems
Child Care Use
Child Education and Cognitive Development
Child Social and Positive Development
Family Environment
Family Stress
Family Structure
Parent/Adult Psychological Well-Being
Other Areas
Attitudes on Welfare, Work, Raising Children
Demographics
Household Composition
Social Services Issues
Some questions covered a family's circumstances at the time of the survey; others were about the 12 or more months prior to the interview or about calendar year 1998. Some new questions were asked in the 1999 survey, as detailed in report no. 1 in the 1999 NSAF Methodology Series. These were needed to cover changes in public programs between 1997 and 1999, such as the introduction of the State Children's Health Insurance Program (SCHIP). Improvements were made in some 1997 questions when it was realized that they were not operating as expected. A significant improvement was made for 1999, for instance, in the way the survey asked about nativity, leading to better estimates of immigration status.
Estimation Methods
Responses to NSAF items were weighted to provide approximately unbiased aggregate estimates for each study area and for the country as a whole. The weights were applied to all survey items in an effort to:
- Compensate for differential probabilities of selection for households and persons;
- Reduce biases occurring where nonrespondents have different characteristics than respondents;
- Adjust, to the extent possible, for undercoverage in the sampling frames and in the conduct of the survey; and
- Reduce the variance of the estimates by using auxiliary information.
The weighting can be described as involving three stages for both the random-digit dialing and in-person components of NSAF to produce person and family weights:
- The first stage was the computation of the base weight. The base weight is the inverse of the probability of selection, which accounts for the unequal screening rates. This weight also includes an adjustment for the planned exclusion of nontelephone households from the area sampling frame and for the subsampling of persons in selected households.
- The second stage was an adjustment for unit nonresponse (entire households and persons who did not respond to the survey). This was done by adjusting the weights of respondents in particular groups to account for the nonrespondents in those groups.
- In the third stage, the nonresponse adjusted weights were post-stratified so that the NSAF sample estimates agreed with independent population totals3 derived from U.S. Census Bureau sources on the number of persons by age, education, ethnicity, gender, race, and housing tenure. This was done for each study area and the nation as a whole.
These three stages incorporate screener data to create household weights and extended interview data to create the person and family weights. The weights account for the unequal probability of sampling (at both the household and person levels) and include adjustments for nonresponse and undercoverage. The final result is a series of estimates consistent with Census Bureau population totals that reduce biases due to undercoverage and nonresponse. In some cases, the adjustment to Census Bureau population controls may also reduce the sampling error of the estimates.
Another estimation issue is how to handle item nonresponse—that is, where an interview was obtained but some specific questionnaire items were missing. For most questions the item nonresponse rates were very low, often less than 1 percent. As is the case with any household survey containing questions about sensitive information (such as income and mortgage amounts), the NSAF occasionally encountered significant levels of item nonresponse. In particular, nonresponse rates for items related to income were in neighborhood of 20 percent—consistent with what is found elsewhere (e.g., the March Current Population Survey).
For estimates presented in Snapshots II, nearly all questions on employment, earnings, and family income were imputed when missing, as were selected items from the sections on health care coverage and health care use and access. The imputation of missing responses was intended to meet two goals. First, it makes the data easier to use; for example, imputing missing income responses permits the calculation of family income and poverty measures for all sample families. Second, the imputation partially adjusts for bias, since the characteristics of nonrespondents may differ from those of respondents.
Missing responses have been imputed at the person level (except for the economic hardship and housing items, which were imputed at the household level). The method used to make the imputations for missing responses in the NSAF was a standard "hot deck."
In a hot-deck imputation, the value reported by a respondent for a particular question is given to a similar person who failed to respond to that question. The hot-deck approach to imputing missing values is the most common method used to assign values for missing responses in large-scale household surveys. For example, it is the method used for the March Current Population Survey (CPS), the source of the official annual estimates of the poverty population.
Sampling Errors and Precision
As in all surveys, the estimates from the NSAF are subject to both sampling and measurement errors. Sampling errors can be directly quantified and are discussed here. Nonsampling errors are assessed separately.
The sample of households and persons selected for the NSAF is just one of many possible samples that could have been selected. The standard error of an estimate is a measure of the uncertainty of that estimate due to the fact that the sample selected is just one of the many possible samples that could have been chosen (thus, it is also often called sampling error).4
This uncertainty in survey estimates due to sampling can also be expressed as a margin of error. For estimates based on large sample sizes, the chance that an estimate would differ from a complete census count (which has no sampling error) by more than the margin of error being employed for the Snapshots is less than 10 percent. The margin of error being used thus corresponds to a 90 percent confidence interval. That is, we can be 90 percent confident that the NSAF estimate, plus or minus the margin of error, will cover the complete census count.
Despite the fact that specific details are given elsewhere on each estimate's margin of error, it still may be worth showing how reliable the NSAF is overall. This is done in table 1, where upper bound margins of error are given for both 1997 and 1999 NSAF proportions. Upper-bound margins of error are also provided for differences between proportions from one round of the survey to another.
The margins of error reported in the table are averages that vary both across states and across statistics. Still, they can be considered rough approximations of the actual sampling margin of error. They are, however, conservative in the sense that they assume that our estimate is for a 50 percent attribute. Margins of error get smaller as we move away from a 50 percent attribute. In calculating the margin of error of a difference, the conservative practice of assuming independence has been employed between the two surveys. In later work with the NSAF, better estimates of sampling error will be made and the Web pages cited above reloaded.
Nationally, the 1999 NSAF margins of error were designed to be somewhat lower5 than those that were achieved for 1997 and this is borne out in table 3. By design, NSAF achieved quite small margins of error for the 13 targeted states. In fact, the margins of error were about the same in both rounds. The one instance in which this was not true was for data on low-income children. In part because of the decline in overall poverty rates, the state samples of low-income children shrank, leading to an increase in the maximum margin of error from 3.4 percent to 3.9 percent. Even so, the small margins of error for state-level estimates for low-income children and adults remain one of the main achievements of the NSAF design in both 1997 and 1999.
Table 1. Maximum Margins of Error, 1997 and 1999 NSAF
(90 percent confidence interval upper bounds for proportions and differences of proportions, 1999 versus 1997.) |
| Item |
1997 |
1999 |
Difference |
| Children |
|
National |
|
All |
1.0 |
0.9 |
1.3 |
|
Low-income |
1.4 |
1.4 |
2.0 |
|
State |
|
All |
2.2 |
2.3 |
3.2 |
|
Low-income |
3.4 |
3.9 |
5.2 |
|
Adults |
|
National |
|
All |
0.8 |
0.7 |
1.1 |
| Low-income |
1.2 |
1.1 |
1.6 |
|
State |
|
All |
2.0 |
1.8 |
2.7 |
|
Low-income |
3.0 |
3.2 |
4.4 |
| Note: Margin of error of difference calculated conservatively, assuming independence across survey waves. |
The margins of error computed for each estimate can be used as rough approximations for testing statistical hypotheses as to whether an individual state, for example, was different in some way from the nation as a whole or whether a particular state had changed significantly from 1997 to 1999. In the report itself, results are identified as statistically significant in various ways. All of these statements are being made at the 10 percent level and imply that a 90 percent confidence interval constructed around the difference does not include zero—the point of no difference. Since multiple comparisons are being made, a Bonferroni adjustment was considered, but has not been done since the comparisons were all specified ahead of time. Still, 1 out of every 10 statistically significant comparisons for differences (on the average) will not represent a real change.
Nonsampling Errors and Biases
While sampling error is usually associated with the precision of estimates from a survey, nonsampling errors are usually (but not exclusively) associated with bias in survey estimates. Bias is simply the degree to which an estimated survey statistic (such as a proportion, total, or mean) differs from the true population value. Biases in survey estimates can arise for many reasons. Below, we provide some examples of NSAF methodological work focusing on three potential sources of bias: undercoverage, nonresponse, and problems relating to measurement.
Undercoverage. Survey estimates may be biased if certain elements of the population are not given a chance to be sampled for the survey. For example, a survey that relied exclusively on telephones to conduct interviews would exclude households without telephones. This would not be a problem if the NSAF were only to be used to produce estimates for persons living in telephone households. But since the NSAF has a primary focus on low-income families and families without telephones make up a disproportionate percentage of the low-income population, the NSAF sample design included households without telephones.
The NSAF, like all household surveys, suffers from a net shortfall or undercoverage relative to independent Census Bureau figures. However, these effects appear to be consistent with those achieved in other well-conducted surveys. To assess coverage, we compare survey estimates of different population groups in the NSAF to what the Census Bureau believes to be the right total (after adjusting for the 1990 Census undercount). Ideally, such coverage ratios should be close to 100 percent. For the 1999 NSAF, before adjustment, the coverage ratio of children was 90 percent—somewhat lower than the 94 percent ratio from the 1997 survey. For adults, the coverage ratio was lower, but still good at about 87 percent (up from 86 percent in 1997).
As mentioned earlier, the use of independent population controls in the estimation mitigated the potential undercoverage bias resulting from this source of error. For more information on undercoverage concerns in the NSAF, see report nos. 3 and 14 in the 1997 NSAF Methodology Series and report no. 3 in the 1999 NSAF Methodology Series (forthcoming).
Nonresponse. Unit nonresponse occurs when sampled units (such as households and families in the NSAF) do not respond to the survey. As a practical matter, unit nonresponse increases the cost of getting the same number of completed interviews. From a statistical standpoint, unit nonresponse raises concerns about possible biasing of estimates, since respondents and nonrespondents may be systematically different from one another for key survey measures.
It is important to note that a low response rate is not itself a direct indicator of the actual magnitude of nonresponse bias. Estimates will not suffer from nonresponse bias when there are no differences between respondents and nonrespondents. Typically, survey methodologists focus on response rates as proxy indicators of the presence of nonresponse bias because there is usually little information available on nonrespondents that can be used to judge the magnitude of differences between respondents and nonrespondents.
The household response rates achieved in the NSAF were quite respectable. The survey used several measures of nonresponse. The one most comparable to surveys like the CPS has the 1997 NSAF response rate at just under 70 percent and that for the 1999 NSAF at about 64 percent. Part of the reason that the 1999 survey rates were lower than for 1997 was the overlap we built in to the interviewing between rounds. RDD response rates have been declining across the board in recent years for many surveys and NSAF appears to be no exception.
Arguably, both of these rates are quite good, especially given the consensus that survey response rates have been falling in recent years. Nonetheless, concerns about potential nonresponse bias led us to conduct several special studies to understand what impact this level of response might have on statistics of interest. We only cite three of the studies below:
- For the 1997 survey, initially interviewed NSAF households were compared with the characteristics of nonresponding households that were later reached with supplemental efforts. These analyses showed no evidence of large or systematic nonresponse errors in the 1997 NSAF statistics examined (see report no. 7 in the 1997 NSAF Methodology Series).
- Another effort to assess nonresponse bias was possible because the overlap between rounds gave us situations where we obtained an interview in one round, but not the other. The results so far indicate that for the items examined, NSAF nonresponse was largely ignorable - i.e., leading to virtually no bias after adjustment (Black and Safir 2000).
- In a third study, a modeling approach was taken in estimating the potential nonresponse bias. Based on the model, we were able to conclude that for nearly two-thirds of the households that refuse at screening, the nonresponse is completely ignorable after adjustment and does not lead to any bias. Based on other evidence, the remaining refusal cases probably contribute no or minimal bias as well (Scheuren 2000).
Because of nonresponse bias concerns in the NSAF and the inadequacy of looking exclusively at response rates as an indicator of bias due to nonresponse, further analyses are underway. These show similar results and will be reported on at a later time in the 1999 NSAF Methodology Series.
Measurement Error. Measurement error arises due to the imperfect nature of the data collection process in surveys. The interviewer, the respondent, the questionnaire, and the mode of data collection are all potential sources of measurement error.
Interviewer Errors. Interviewers can introduce measurement error if, for example, they vary in the way they deliver questions to respondents and in the way they record the answers obtained. For both rounds of the NSAF, heavy reliance was placed on extensive interviewer training and monitoring procedures. For example, about 10 percent of each interviewer's work was silently monitored for quality control purposes.
Respondent Errors. Measurement error may also arise if respondents differ in their abilities and motivation to answer specific survey questions. For example, due to the use of a partial overlap of units sampled in the 1997 NSAF, some of the 1999 NSAF respondents participated in both rounds of the survey. Their previous participation could have caused them to either change the behavior being measured or to provide responses that are in some way different from what they would have been had the respondents not participated previously; this is sometimes called a panel effect.
To test for panel effects (Wang, Cantor and Safir 2000), we compared 1999 survey estimates of those respondents who were interviewed in both rounds of the survey with survey estimates of respondents from the fresh cross-section (who were not interviewed in the 1997 NSAF). We controlled for differences in basic demographic and household characteristics. In general, we found little evidence of panel conditioning effects. Even in those cases where we did find statistically significant differences, the magnitudes of the differences were small enough that estimates for the overall sample for 1999 are largely unaffected.
Questionnaire Errors. The NSAF questionnaire relied heavily on earlier survey instruments as a source of the questions used. We believe that this approach has resulted, for the most part, in reliable results comparable to other national surveys on similar topics. Every effort was made to keep question wording unchanged from round to round to improve estimates of change. As noted earlier, however, improvements had to be made in some questions between the 1997 and 1999 NSAF when we realized that they were not operating as expected. A significant improvement was made, for example, in the way the survey asked about where persons were born, leading to improved information on immigrant status in the 1999 survey. For more on these issues, see report no. 10 in the 1997 NSAF Methodology Series and report no. 1 in the 1999 NSAF Methodology Series.
Mode Errors. The mode of data collection can lead to measurement error. For example, respondents may be less likely to provide truthful responses to sensitive questions in telephone- than in self-administered surveys (see de Leeuw and van der Zouwen 1988, for example). For the NSAF, we believe the use of cellular telephones to conduct interviews with nontelephone households reduced the potential for differential measurement error due to mode differences between the two parts of the survey.
External Validation
Despite all the efforts made in the NSAF (only a few of which have been described here), nonsampling errors could not be eliminated. The ability to quantify the effect of these errors within the surveys themselves is very limited, unlike the case for sampling errors. To deal with this, an extensive effort was made to compare the NSAF with other surveys whenever possible. Overall, nonsampling errors in the NSAF were reasonably well controlled and do not appear to have led to more than minor inconsistencies between the NSAF and other surveys that set out to obtain similar information.
To illustrate this, below we make some comparisons with corresponding CPS estimates for both the 1997 and 1999 NSAF.6 Two fairly typical examples are given in tables 2 and 3, where the survey used for comparison is the CPS:
- Table 2 compares the distribution of employment earnings for nonelderly adults (18 to 64 years of age) as estimated in the 1997 and 1999 NSAF and the CPS (for the previous year; that is, 1996 and 1998). A remarkable degree of closeness exists, given that there is sampling and nonsampling error in both surveys.
- Table 3 displays the household size distribution estimated from the 1997 and 1999 NSAF and CPS samples. Again, we see considerable closeness and no differences of any substantive significance, although the smaller percentage of one-person households found by the NSAF is worth noting.
In summary, for these and many other comparisons not shown (family composition, work experience, income, and poverty by key demographic characteristics), the NSAF estimates are very close to the CPS estimates—for the most part, well within normal sampling variation.
| Table 2. Earnings from Employment: Distributions for Adults 18 to 64 Years Old, 1997 and 1999 NSAF and CPS Compared (in percent) |
|
Earnings |
1997 (1996 Earnings) |
1999 (1998 Earnings) |
| |
NSAF |
CPS |
NSAF |
CPS |
|
Under $10,000 |
35.4 |
36.1 |
33.5 |
34.7 |
|
$10,000-14,999 |
9.9 |
9.7 |
8.6 |
8.8 |
|
$15,000-24,999 |
17.5 |
17.7 |
17.3 |
16.6 |
|
$25,000-34,999 |
13.9 |
13.3 |
14.1 |
14.0 |
|
$35,000-49,999 |
12.6 |
11.4 |
13.4 |
12.6 |
|
$50,000-74,999 |
7.5 |
7.0 |
8.8 |
8.5 |
|
$75,000+ |
3.3 |
3.8 |
4.4 |
4.9 |
|
Total |
100.0 |
100.0 |
100.0 |
100.0 |
| Sources: Current Population Survey (CPS), and National Survey of America's Families (NSAF) information from Urban Institute tabulations. Data may not add to totals due to rounding. |
| Table 3. Household Size: Distributions for 1997 and 1999, NSAF and CPS Compared (in percent) |
|
Household Size |
1997 |
1999 |
| |
NSAF |
CPS |
NSAF |
CPS |
|
1 Adult |
13.2 |
13.5 |
13.0 |
14.1 |
|
2 Adults |
57.4 |
56.8 |
57.3 |
57.1 |
|
3 Adults |
18.8 |
19.1 |
19.4 |
18.7 |
|
4+ Adults |
10.5 |
10.7 |
10.3 |
10.2 |
|
Total |
100.0 |
100.0 |
.0 |
100.0 |
| Sources: Current Population Survey (CPS), and National Survey of America's Families (NSAF) information from Urban Institute tabulations. Data may not add to totals due to rounding. |
Concluding Comments
A brief overview of the 1999 and 1997 data collection and estimation procedures for the NSAF has been shown here. The methods used to minimize errors and compensate for those that are unavoidable in data collection have been described. We have also discussed the survey's resulting reliability—both in terms of sampling and nonsampling errors. We were particularly encouraged at how comparable CPS and NSAF estimates appear to be.
For the most part, the calculated sampling margins of error—in the Snapshots II elsewhere on this Web page—offer a solid guide to how reliable NSAF information is and how it might be used (either on its own or in combination with other measures). More complete information on the 1997 NSAF's design, data collection procedures, and measures of quality has already appeared in an extensive series of methodological reports, available online. The 1999 NSAF series of reports is now being issued. To check on their availability, consult the NSAF Web page.
Over about the last year and a half, virtually the entire 1997 NSAF dataset has been released in an anonymous form for public use. Beginning this November with a data file focusing on children, researchers will be able to obtain access to the 1999 NSAF, which will, again, be anonymous. Initially, this access will mainly be provided through an on-line web tabulator program, entitled CrossTabMaker.
Notes and Acknowledgments
This summary of NSAF survey methods and data reliability parallels that done earlier by Genevieve Kenney, Fritz Scheuren, and Kevin Wang to accompany the 1997 Snapshots report. Mike Brick of Westat contributed importantly to both.
As noted above, in keeping with the conventions used for the 1999 Snapshots, the margin of error values shown here are for 90 percent confidence intervals. For the 1997 Snapshots, the margins of error were set at 95 percent and hence are somewhat wider.
Endnotes
1. Milwaukee was also designated as a study area in its own right, so Wisconsin can be viewed as consisting of two study areas, Milwaukee and the balance of the state.
2. However, NSAF field procedures did allow for the use of proxy respondents or "facilitators" to interview sampled respondents who could not be interviewed in English or Spanish. We still had a loss of between 1 percent and 2 percent of the sample due to this barrier. From the interviewer notes, these were often Chinese-, Korean-, or Russian-speaking households.
3. The totals used were based on the 1990 decennial census counts, carried forward by the Census Bureau to 1997 using birth and death records, plus information on net migration. As is done in the Current Population Survey (CPS), the control totals were adjusted to account for the 1990 decennial net Census undercount. Other Census Bureau population estimates controlled were the percentage of persons by home ownership (used in the derivation of the weights for children and adults) and education level (used in the derivation of the weights for adults). Occasionally, the variables needed in this weighting were missing. When this occurred, the missing responses had to be imputed, although this was rarely necessary. Race for persons of Hispanic origin is an exception here and had to be imputed quite frequently, since many Hispanic respondents answered the race and ethnicity questions with the designation "Hispanic." For this reason, while we are comfortable with the race data overall and with the designation of Hispanic origin, we do not recommend that the race data for Hispanics be used separately.
4. The sampling error introduced because of the imputation of missing responses to specific questions is not estimated by the jackknife method used elsewhere in the survey. See report no. 4 in the 1997 NSAF Methodology Series for more details on the jackknife. For the most part, though, this understatement should be very small, as is discussed in report no. 10 in the 1997 NSAF Methodology Series.
5. By design, the overall national estimates were improved between the two rounds. This was done chiefly by substantially increasing the sample taken in the balance of the United States. Thus, nationally, the margin of error for low-income children dropped slightly, even though a drop was not achieved in most of the target states.
6. For details on the extensive external validation efforts undertaken, see reports no. 6 and 15 in the 1997 NSAF Methodology Series and report no. 6 in the 1999 Methodology Series (forthcoming).
References
1997 and 1999 National Survey of America's Families Basic Snapshot II Tables, The Urban Institute Web site
1997 and 1999 National Survey of America's Families Methodology Reports, The Urban Institute Web site.
Wang, K, Cantor, D., and Safir, A. (2000), "Panel Conditioning in an Random Digit Dial Survey," Proceedings of the Survey Research Methods of the American Statistical Association, Alexandria, VA.
de Leeuw, E. D., and van der Zouwen, J. (1988), "Data Quality in Telephone and Face-to-Face Surveys: A Comparative Meta-Analysis," Telephone Survey Methodology, John Wiley and Sons.
Black, T., and Safir, A. (2000), "Non-Response Bias in the NSAF," Proceedings of the Survey Research Methods of the American Statistical Association, Alexandria, VA.
Scheuren, F. (2000), "Quality Assessment of Quality Assessment," Proceedings of the Survey Research Methods of the American Statistical Association, Alexandria, VA.
Latest Research
Browse by Topic:
Printer-friendly summary
