Extending the Utility of Federal Data Bases. Definition of Adequate Precision

05/01/2000

The level of precision that is considered adequate for a survey should reflect the kinds of analyses to be made and the effect that errors in the statistics are likely to have on practical uses of the information. When differences of 10 or 20 percent will have trivial effects on policy decisions, fairly large sampling errors are tolerable. In other instances, even small errors could have an important adverse effect on uses of the data. The situation is further complicated by the fact that although precision may be satisfactory for most statistics relating to the total population of the subgroup, it may be inadequate for subdomains such as age-sex subgroups, low-income persons, persons in each region of the U.S., etc. It is frequently found that no matter how large the sample for a particular survey, there will be some desirable analyses for which the sample is insufficient. Some examples are separate studies of babies, teenagers, or the elderly; examination of data for persons with income below the poverty level; the rural population; etc.

Consequently, there is no simple or single standard of reliability that is applicable to all studies that may be carried out. Although most large surveys have multiple objectives, in each case, the principal uses of the data should be considered along with the consequences of errors in the data. An important part of the consideration should be whether there is a need for special treatment of certain subdomains. In Section 2.2 we give examples of how some of the major U.S. surveys approach the problem. The budget that is likely to be available should, of course, also be taken into consideration. Another factor that may play a role is the existence of significant nonsampling errors in the data collection system, since there is no reason to incur the cost of a very large sample size if the main quality problem is poor reporting by the respondents rather than sampling error. There is no point in establishing unrealistic standards that cannot be achieved. Section 2.2 contains a few examples of how these and other considerations have been instrumental in establishing standards for some of the major U.S. multipurpose sample surveys, which in turn have determined the sample sizes.