Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys. STANDARDIZED EMPIRICAL COMPARISONS


In this chapter we present findings based primarily on the extensive standardized empirical comparisons that were described in the preceding chapter. These findings, which cover all eight surveys, include our principal comparative estimates of income. For these estimates the study uses income data for 2002 (HRS and MCBS income for 2003 were deflated with the CPI-U) that covers a calendar year, except for the rolling reference period in ACS. We compare the survey estimates of income along several dimensions, as no single measure captures the full breadth of what good income data should provide. We look in turn at aggregate income and its distribution by quintile, the location of quintile boundaries, per capita income by quintile, estimates of the poor and near poor, employment and earned income, unearned income, and program participation. These comparisons focus on the five general population surveys that are conducted by the federal government and designed to provide representative estimates of the full civilian noninstitutional population: the CPS, ACS, SIPP, MEPS, and NHIS. More limited comparisons that include the PSID are interspersed among the findings on the five surveys. Comparative estimates of the population without health insurance coverage—the uninsured—and its distribution by income are presented for the CPS, SIPP, MEPS, NHIS, and PSID. Separate analyses of the two surveys of restricted populations—that is, the HRS and MCBS—are included near the end of the chapter and followed by an analysis of internal inconsistencies relevant to the income data in the NHIS, MEPS, and SIPP.

The purpose of the comparisons presented in this chapter is not to establish statistically significant differences or demonstrate that alternative estimates are statistically the same.  Our efforts to adjust for differences in universe, income concept, and family definition, and our earlier findings on living arrangements underscore the importance of nonsampling error in comparative estimates across surveys.  Furthermore, the surveys included in the study have large samples, for the most part, which means that small differences may be statistically significant yet unimportant from a policy perspective. Given these considerations, we felt that our fixed resources were better spent in furthering our understanding of the surveys, the differences we were observing, and the impact of various design features than in calculating statistics that would provide only marginal value-added at best.

View full report


"report.pdf" (pdf, 4.33Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®