Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys. Method of allocation

12/23/2008

As explained earlier, we use the term allocation to encompass all methods of filling in missing values besides editing (rarely useful for dollar amounts). Most of the surveys utilize “hot deck” imputation methods to allocate missing values. Hot deck methods involve matching the records with missing values to “donor” records on the basis of a typically large number of characteristics. The missing values are assigned from the donor records. Using other respondents as donors helps to ensure that the allocated values are plausible and have an appropriate distribution. Often, multiple variables may be assigned from the same donor to ensure that there is some internal consistency among the allocated values.

TABLE VI.7

DISTRIBUTION OF TOTAL INCOME AND ALLOCATED INCOME BY SOURCE: FIVE SURVEYS
Source of Income CPS ACS SIPP MEPS NHIS
Percentage Distribution of Total Income
Total 100.00 100.00 100.00 100.00 NA
Wages and Salaries 77.71 75.91 71.84 82.64 NA
Self-employment 5.07 6.16 10.71 NA NA
Asset Income 4.10 5.01 2.41 3.47 NA
Social Security or Railroad Ret. 6.02 5.62 6.44 5.70 NA
SSI 0.40 0.43 0.59 0.63 NA
Welfare 0.10 0.13 0.16 0.08 NA
Pensions 3.85 4.82 6.05 3.94 NA
Percentage Distribution of Allocated Income
Total 100.00 100.00 100.00 100.00 NA
Wages and Salaries 72.69 74.16 64.03 83.85 NA
Self-employment 6.63 8.08 13.08 NA NA
Asset Income 7.50 5.53 4.52 5.04 NA
Social Security or Railroad Ret. 6.26 5.89 6.27 5.43 NA
SSI 0.33 0.41 0.50 0.12 NA
Welfare 0.08 0.13 0.18 0.03 NA
Pensions 3.98 4.42 9.46 3.77 NA

Source: Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the 2003 NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

A potentially important distinction among allocation methods is whether they make use of “partial” information on the missing amounts. Partial information could include prior wave values (SIPP), detailed income brackets reported in lieu of a dollar amount (MEPS and NHIS), or wage rates and hours worked (used for annual wages in MEPS). Arguably, allocations that make use of partial information are qualitatively different from allocations that rely solely on covariates of the missing items.

The CPS relies heavily on hot deck imputation methods to allocate missing values due to item non-response, and the allocation flags for the income items do not indicate the use of partial information in any form of allocation. The allocation flags in the ACS public use file do not differentiate among types of allocation, so we assume that no ACS allocations use partial information.

In the SIPP, income from earnings may be allocated using a set of procedures specific to labor force items that makes use of prior wave data. Such imputations are designated with a single code (EPPFLAG = 1), so situations in which the respondent provided the equivalent item in a prior wave could not be differentiated from situations where the respondent provided only related items.62 In examining the frequency with which the survey allocations of income utilize partial information, we will treat these labor force allocations as using partial information. We will do the same with allocations identified as logical edits, allocations from prior wave data, and the ratio adjustments that we made in order to fill in missing months for sample members who had not yet joined the sample or who were not interviewed in those waves. Logical edits are identified by codes of 3 on the allocation flags, and allocations using prior wave data are identified by codes of 4. The ratio adjustments, as we explained in Chapter III, use data from other waves to allocate amounts to the missing waves. Finally, allocations also include the procedures used to impute what the Census Bureau calls “Type Z non-respondents.” These are non-responding persons in responding households. All data for Type Z non-respondents are imputed using hot deck methods; we treat these as allocations without partial information.

For MEPS, we consider all income allocations from reported brackets as making use of partial information, and we do the same with allocations of wage and salary income from hourly wage rates and hours worked.63 Allocations from reported brackets are coded 2 on the allocation flags while allocations of wage and salary income from hourly wage rates and hours worked are identified by a code of 4 on the wage allocation flag. Hot deck imputations that do not use partial information are identified by codes of 5 or 6.

For the NHIS, allocations that were based on detailed bracketed values provided by respondents instead of actual dollar amounts are coded 3 on the allocation flag. We count these allocations as based on partial information.

When we divide allocated dollars of total income into allocations performed with or without partial information, we find that allocations with partial information dominate the allocations for SIPP and MEPS (Table VI.8). In each of these surveys, allocations without partial information account for about 7 percent of total income while allocations with partial information account for 25 percent of total income in SIPP and 36 percent in MEPS. Allocations with partial information represent only 2 percent of total income in the NHIS while allocations without partial information account for 30 percent of total income.64 In both the CPS and ACS, we classified no allocations as using partial information, leaving 34 percent of total income in the CPS and 18 percent in the ACS as allocated without partial information.

TABLE VI.8

ALLOCATION OF TOTAL INCOME BY USE OF PARTIAL INFORMATION: FIVE SURVEYS
Estimate CPS ACS SIPP MEPS NHIS
Amount of Total Income ($billions) 6,468.4 6,346.3 5,766.2 6,257.7 6,115.2
Percent of Dollars Allocated: 34.2 17.6 32.4 42.7 32.4
  With Partial Information 0.0 0.0 25.4 35.6 2.2
  Without Partial Information 34.2 17.6 6.9 7.1 30.2

Source:  Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the 2003 NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

Given that we have no direct measure of the quality of the allocated data obtained with the use of partial information, we hesitate to assert that non-response to the income questions is as much less of a problem in the SIPP and MEPS as these results might be read to suggest.

The proportion of total income allocated using partial information does not vary by income quintile in the SIPP whereas it does in MEPS, rising from 25 percent in the lowest quintile to 38 percent in the highest quintile (Table VI.9). In the ACS, as we have noted, there is a modest reduction in the percentage of dollars allocated without partial information as the income quintile increases, and SIPP appears to show the same, but with much lower allocation rates. The lowest quintile in MEPS appears to have a relatively high rate of allocation without partial information to complement its comparatively low rate of allocation with partial information. The NHIS does not show a clear relationship between income quintile and allocation with or without partial information.

Differences in the percentage of income allocated by source increase dramatically for SIPP and MEPS when we divide allocations into those generated with or without partial information (Table VI.10). Because of the labor force allocation procedures used for wages and salaries and self-employment income in the SIPP, these two sources emerge with high rates of allocation with partial information and very low rates of allocation without partial information. The same is true of Social Security income in SIPP, although the reasons are not as obvious, and we do not see the same phenomenon in MEPS. For wages and salaries in MEPS, the combination of bracketed amounts and predictions based on wage rates and hours worked account for a very high rate of allocation with partial information and very low rate of allocation without partial information.  Asset, pension and welfare income have high rates of allocation without partial information in both SIPP and MEPS.

TABLE VI.9

PERCENT OF TOTAL INCOME ALLOCATED WITH OR WITHOUT PARTIAL INFORMATION BY QUINTILE:
FIVE SURVEYS
Family Income Quintile CPS ACS SIPP MEPS NHIS
Total Income in Billions of Dollars
(Quintile)Lowest 370.5 368.7 391.4 360.0 356.0
(Quintile)Second 774.1 778.4 750.8 808.4 687.1
(Quintile)Third 1,090.2 1,087.4 1,008.8 1,144.7 1,020.3
(Quintile)Fourth 1,446.8 1,415.8 1,307.2 1,461.8 1,479.1
(Quintile)Highest 2,786.7 2,696.0 2,308.0 2,483.0 2,572.7
Percent of Dollars Allocated with Partial Information
Lowest 0.0 0.0 24.8 24.6 2.3
Second 0.0 0.0 25.5 31.7 2.4
Third 0.0 0.0 25.0 34.6 1.1
Fourth 0.0 0.0 25.3 37.3 1.8
Highest 0.0 0.0 25.8 37.9 2.8
Percent of Dollars Allocated without Partial Information
(Quintile)Lowest 35.1 21.8 8.6 11.5 31.7
(Quintile)Second 33.6 20.1 7.5 7.1 32.2
(Quintile)Third 32.9 18.7 7.2 6.5 28.2
(Quintile)Fourth 32.5 17.2 6.7 6.2 27.1
(Quintile)Highest 35.6 16.1 6.5 7.3 32.0

Source:   Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the 2003 NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS.

TABLE VI.10

PERCENT OF INCOME ALLOCATED WITH AND WITHOUT PARTIAL INFORMATION:
FIVE SURVEYS
Source of Income CPS ACS SIPP MEPS NHIS
Percent of Dollars Allocated with Partial Information
Total Income (NHIS family income) 0.0 0.0 25.4 35.6 2.2
Wages and Salaries (NHIS earnings) 0.0 0.0 24.6 38.9 0.0
Self-employment 0.0 0.0 34.8 NA NA
Asset Income 0.0 0.0 38.6 35.5 NA
Social Security or Railroad Ret. 0.0 0.0 24.8 13.8 NA
SSI 0.0 0.0 15.3 0.0 NA
Welfare 0.0 0.0 12.4 0.0 NA
Pensions 0.0 0.0 18.7 18.9 NA
Percent of Dollars Allocated without Partial Information
Total Income (NHIS family income) 34.2 17.6 6.9 7.1 30.2
Wages and Salaries (NHIS earnings) 32.0 17.2 4.3 4.4 31.8
Self-employment 44.7 23.1 4.7 NA NA
Asset Income 62.6 19.4 22.2 26.5 NA
Social Security or Railroad Ret. 35.5 18.5 6.7 26.9 NA
SSI 28.0 16.7 12.2 7.9 NA
Welfare 29.2 17.9 23.5 13.5 NA
Pensions 35.4 16.2 31.9 22.0 NA

Source:   Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the 2003 NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS. Note: Estimates in the top and bottom panels sum to the estimates reported in Table VI.5.

View full report

Preview
Download

"report.pdf" (pdf, 4.33Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®