Deriving State-Level Estimates from Three National Surveys: A Statistical Assessment and State Tabulations. B. Precision of Estimates

It is impossible to define a single level of precision that is necessary for all estimates. The level of precision that is necessary depends on the use of the estimates. Different Federal agencies have different standards for their data. Some have standards that only determine the level of precision for estimates to be used in analyses, while others have standards for precision for publication. For example, the National Center for Health Statistics has a requirement that coefficients of variation (the standard error of an estimate divided by the mean) not exceed 30 percent. The Center reports and interprets the estimates that have at least this level of precision. Less precise estimates may be reported but are not interpreted.

The precision of a direct estimator is a function of two parameters, the standard deviation of the population distribution and the effective sample size. The precision of an estimate for a characteristic that is highly variable in the population will be less than that for a characteristic that is fairly consistent across the population. The variability of the characteristic is measured by the standard deviation. Similarly, a larger effective sample size will provide more accurate estimates than a smaller effective sample size.

When estimating percentages (as for all four variables examined in Section III of this report), the characteristic is dichotomous, a binomial variable (e.g., in poverty, not in poverty). In this case the standard deviation is a simple function of the percentage with the characteristic. The standard deviation is,

where P is the percentage with the characteristic in the population. The closer the true percentage (e.g., percent in poverty) is to 50 percent, the larger the standard deviation. The closer the percentage is to either 0 or 100 percent, the smaller the standard deviation. For example, the standard deviation when P = 50 percent is 0.50, while the standard deviation when P = 1 percent is 0.10.

The effective sample size is the actual sample size divided by the design effect. The design effect is a factor that reflects the effect on the precision of a survey estimate due to the difference between the sample design actually used to collect the data and a simple random sample of respondents. National in-person household surveys, such as the three considered here, are conducted as stratified, multi-stage, clustered, area-probability surveys. By clustering the sampled households in a limited number of geographic areas, the cost of data collection is significantly reduced. However, respondents in the same cluster are likely to be somewhat similar to one another. As a result, a clustered sample will generally not reflect the entire population as "effectively." Before selecting the sample of clusters, the country is stratified based on characteristics believed to be correlated with the survey variables of greatest interest. This stratification produces more precise survey estimates for targeted domains than an unstratified design. The design effect reflects all aspects of the complex sample design. While the design effect is different for each variable, experience with these surveys indicates that the variables under study will have reasonably similar design effects.

View full report

"deriving.pdf" (pdf, 2.83Mb)