Extending the Utility of Federal Data Bases. Response Rate and Other Quality Issues


The surveys listed in the Task 2 report, and which will be analyzed further in the balance of this report, are conducted by federal agencies, or carried our under contract for the agencies. Knowledgeable statistical staffs monitor the survey operations and the quality of the work is generally quite high. There is particular stress on attaining high response rates, and we believe that all of the surveys do about as well as can be expected, given that, with the exception of the vital statistics system, the ACS, and the U.S. Census, the survey responses are voluntary rather than mandatory.

To say that the response rates are acceptable does not mean there are no potential nonresponse problems. All of the major surveys use poststratification in the final stage of weighting to reduce sampling errors, and to compensate as much as possible for nonresponse and undercoverage. There are almost always separate poststratification cells for blacks, Hispanics, and all other race/ethnic groups and NHANES has such cells for Mexican-Americans. The minority subgroups are almost always combined into categories like "total Hispanics" or "total other races" (which includes American Indians and Alaska Natives). Subdomains such as Puerto-Ricans, Cuban-Americans, Central-Americans, etc., are thus combined into a single class, with identical weights. Similarly, all Asians and Pacific Islanders get identical weights. If, in fact, some of these subgroups have lower response rates than the overall rate for the race/ethnic class, and are not separately adjusted, they will be underrepresented in the statistics. A similar situation exists with undercoverage. For example, if illegal aliens tend to avoid reporting (as seems likely) and if a higher proportion of Mexican-Americans are here illegally than in other Hispanic subpopulations (as is also likely), then the uniform weighting will slightly understate Mexican-Americans and overstate other Hispanic subgroups.

Unfortunately, there is not much that can be done to adjust for such occurrences. Agencies already make strenuous efforts to attain high response rates and it is unlikely that further exhortation to improve will be effective. Users of the data, however, should be aware of such limitations in drawing conclusions from the statistics.