Deriving State-Level Estimates from Three National Surveys: A Statistical Assessment and State Tabulations. V. IMPLICATIONS OF THE FINDINGS


Given the relatively low precision requirements used in the previous section, it is possible to estimate the proportion of the total population in a state with a characteristic for almost all states from either survey. For the CPS, this is also true for children and, except for Alaska, the elderly. The CPS is only able to support estimates for blacks for about half of the states, and for Hispanics in 30 to 40 percent of the states. Given the smaller sample size of the SIPP, its ability to support such estimates for subpopulations is more limited than the CPS. For children and the elderly, the SIPP can support estimates for the majority of states. It can produce estimates for blacks that meet the specified levels of precision for about 20 states and for Hispanics in less than 10 states.

The 1996 CPS permits analyses of all of the selected characteristics for the subgroups examined at the specified levels of precision for eight states -- California, Florida, Illinois, Massachusetts, New Jersey, New York, Pennsylvania, and Texas. The SIPP permits analyses for six states -- California, Florida, Illinois, New Jersey, New York, and Texas. The binding constraint for the data for a number of states is the sample size for Hispanics. If the selected characteristics for Hispanics are not included in assessing which states meet all of the criteria, 16 states are added for the CPS and three states are added for the SIPP. For the SIPP, work disability among those aged 65 to 69 also caused several states to fail to meet all of the criteria.

It is important to repeat that these precision requirements, used in this document are quite arbitrary. If more precise estimates are desired, the number of states meeting the cut-off will obviously be reduced.

We also examined a variety of approaches that could be used to improve state-level estimates. These include supplementing the state samples for states with insufficient samples; combining data from multiple rounds of the same survey; combining data from the three surveys; and using indirect model-dependent estimators.

For several reasons, it may be misleading, or even counterproductive, to require an estimate to meet a standard level of precision to be considered useful. First, using a standard may create the illusion that estimates just meeting the standard are error free, and those that fall just below the standard are entirely uninformative. Second, decision makers often have little choice but to use the best information available, even if it is poor, and an estimate that has "substandard" precision may be the best available. Third, estimates that have low precision can sometimes be usefully combined with other imprecise information to obtain more useful results. The most obvious way to combine imprecise estimates is to combine two separate estimates of the same statistic from different surveys or different rounds of the same survey, as we discussed in Section IV.B and Section IV.C. An alternative method is to use econometric modeling to understand the variable's determinants rather than measuring the variable itself.4

To illustrate this last point, consider an analysis of state poverty rates for children using survey-based state time series on the estimated child poverty rate. For smaller states, much of the variation in the estimates over time will be due to measurement imprecision, and the individual estimates for these states would be of little interest in themselves. Nonetheless, the data series for all states can provide information to the modeling effort, which would focus on understanding how various state-level factors (demographic, economic, and program) affect child poverty rates. This effort would improve our understanding of how specific program changes affect child poverty even if we cannot precisely determine how a specific change in a specific state affected that state's child poverty rate.

In sum, the use of the statistic must be considered in combination with the level of precision to determine the validity of an estimate. This observation lends itself to "rules of thumb" for different types of analyses, but precludes ironclad standards.

View full report


"deriving.pdf" (pdf, 2.83Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®