Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys. Approximation and rounding

12/23/2008

One reason to examine rounding in the context of policy analytic use of income data is that the heaping of incomes at well-spaced values can distort the results of policy simulations involving the use of income thresholds to establish program eligibility. An eligibility threshold that lies near an income amount with excessive heaping will produce dramatically different results depending on whether the threshold falls just below or just above that amount. If the former, a simulation will mildly understate the impact of a small change in policy; if the latter, a policy simulation will grossly overstate the impact of the policy change.

TABLE VI.13

PERCENT OF INCOME ALLOCATED BY CALENDAR MONTH AND FAMILY INCOME QUINTILE: 2003 ACS
Family Income Quintile
Month Lowest Second Third Fourth Highest Total
Jan 22.3 22.2 19.7 17.5 17.1 18.5
Feb 23.7 20.9 19.2 18.3 18.0 19.0
Mar 25.6* 24.6* 22.9* 21.7* 22.3* 22.8*
Apr 27.1* 26.2* 24.7* 24.1* 24.0* 24.6*
May 26.2* 25.3* 24.7* 23.6* 23.8* 24.2*
Jun 25.5* 24.0* 22.2* 22.3* 20.4* 21.9*
Jul 24.4* 21.8 20.5 17.9 18.9 19.6*
Aug 23.9* 22.2 21.5* 19.3* 19.8* 20.5*
Sep 23.2 21.7 20.7 18.2 18.4 19.4
Oct 24.1* 22.1 20.0 18.3 18.7 19.6*
Nov 23.8 22.0 20.1 19.3* 18.6 19.7*
Dec 23.7 21.9 20.8 19.4* 18.7 19.8*

Source: U.S. Census Bureau, from the 2003 ACS. Note: Three highest allocation rates for each source are indicated in bold.

*Statistically significant from January at the .05 level or greater.

Another reason that rounding is a concern when assessing income data for policy work is that a high level of rounding suggests inaccuracy or a lack of precision in the data. This may reduce the analyst’s confidence in the data source or the results that it produces.

We examined the extent of rounding in reported incomes below $52,500 for earnings, wages and salaries, Social Security benefits, other retirement income, total personal income, and total family income in the five general population surveys, which allow reported income to be separated from allocated income.65 Earnings and total family income are the only income amounts collected in the NHIS and, therefore, the only amounts on which all five surveys could be compared. Except in the SIPP, where all annual amounts are built up from monthly values, Social Security benefits reported at the person level will have been collected as a single value. Most respondents reporting wages and salaries, retirement income, and even earnings are likely to have supplied a single value in response to one question even though multiple questions were asked.

The results show the differential impact of few versus many income questions. The NHIS relies on a single, person-level question to collect earnings and a single, family-level question to collect total family income. In this survey, 40 percent of the reported earnings and 36 percent of the reported family incomes below $52,500 are multiples of $5,000, and 23 percent of the earnings and 21 percent of the total family incomes are multiples of $10,000 (Table IV.14). At the opposite extreme, the SIPP, with numerous monthly questions, shows very little rounding when amounts are aggregated to the annual level. Except for earnings, fewer than 1 percent of the reported amounts are divisible by $5,000.

TABLE VI.14

REPORTING OF ROUNDED VALUES BY SOURCE OF INCOME BY SURVEY AMONG POSITIVE DOLLAR AMOUNTS BELOW $52,500
Income Source and Level of Rounding CPS ACS SIPP MEPS NHIS
Earnings
Percent divisible by $5,000 27.8 29.6 1.3 18.6 39.8
Percent divisible by $10,000 15.8 17.4 0.8 9.7 22.9
Percent of income in range 82.1 82.4 88.8 81.8 80.9
Wages and Salaries
Percent divisible by $5,000 27.2 29.7 0.9 NA NA
Percent divisible by $10,000 15.4 17.4 0.6 NA NA
Percent of income in range 82.2 82.7 89.4 NA NA
Social Security
Percent divisible by $5,000 0.6 4.3 0.3 6.9 NA
Percent divisible by $10,000 0.4 1.9 0.1 3.6 NA
Percent of income in range 100.0 100.0 100.0 100.0 NA
Retirement Income
Percent divisible by $5,000 4.5 8.0 1.0 7.4 NA
Percent divisible by $10,000 2.7 4.3 0.5 3.7 NA
Percent of income in range 95.6 95.4 99.0 100.0 NA
Total Personal Income
Percent divisible by $5,000 13.7 19.7 0.6 9.7 NA
Percent divisible by $10,000 7.8 11.5 0.4 5.1 NA
Percent of income in range 84.6 84.2 90.8 85.5 NA
Total Family Income
Percent divisible by $5,000 11.0 16.2 0.6 11.1 35.6
Percent divisible by $10,000 6.2 9.5 0.4 6.1 20.9
Percent of income in range 66.9 66.0 77.7 72.0 60.3

Source:   Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 CPS ASEC supplement, the 2001 SIPP panel, the 2002 Full-year Consolidated MEPS-HC, and the 2003 NHIS, and prior 12 months income, inflation-adjusted to calendar year 2002, from the 2002 ACS. Note:  Allocated amounts are excluded from each source.  Family income for the NHIS is based on the NHIS family, which is the level at which family income was reported.

Among the remaining three surveys, the ACS shows the most rounding, with 30 percent of both earnings and wage and salary income, 20 percent of total personal income, and 16 percent of total family income being divisible by $5,000. The fractions are much lower for Social Security and other retirement income (4 and 8 percent, respectively).

On wages and salaries as well as earnings, the CPS is only marginally better than the ACS, with 27 to 28 percent of the amounts being divisible by $50,000 and 15 to 16 percent being divisible by $10,000. For Social Security benefits, however, the CPS approaches the SIPP, with only 0.6 percent of the reported amounts being multiples of $5,000. On total family income the CPS compares to MEPS, both of which have 11 percent of responses divisible by $5,000. MEPS shows less rounding than the CPS on earnings and total personal income but resembles the ACS on Social Security and retirement income, where 7 percent of the reported responses are divisible by $5,000.

We also examined rounding in the PSID, but because the PSID income variables do not include all of the items reported in Table VI.14 or, for some items, apply to a narrower universe, we present the PSID results separately. Most of the PSID variables in Table VI.15 were constructed (by PSID staff) from the responses to multiple questions, and, as we have seen, this tends to reduce the observed level of rounding.

The most disaggregated variable, the family head’s wages and salaries, combines responses to a single wage and salary question over potentially multiple jobs. Since most workers have only one job, however, most of the values of this variable reflect a single response. Not surprisingly, this variable exhibits the highest degree of rounding, with 25 percent of the responses divisible by $5,000 and 14 percent divisible by $10,000. These values compare closely to what we found for CPS wages and salaries (27 percent and 15 percent), although the CPS variable includes all workers. It is possible that we would see less rounding in the CPS wages and salaries if we limited them to family reference persons, defined as is done in the PSID, which would be the male in a husband-wife family. We suggest this because it is plausible that there is more rounding in the reporting of wages and salaries for non-principal earners than for the principal earner (who is more likely to be the CPS respondent or spouse of the respondent).

TABLE VI.15

REPORTING OF ROUNDED VALUES BY SOURCE OF INCOME AMONG POSITIVE DOLLAR AMOUNTS BELOW $52,500: PSID
Income Source and Level of Rounding PSID
Combined Earnings of Head and Wife
Percent divisible by $5,000 18.6
Percent divisible by $10,000 10.7
Percent of income in range 59.4
Family Head's Wages and Salaries
Percent divisible by $5,000 24.9
Percent divisible by $10,000 13.9
Percent of income in range 73.8
Social Security Income of Family
Percent divisible by $5,000 2.4
Percent divisible by $10,000 1.0
Percent of income in range 100.0
Combined Transfer Income of Head and Wife
Percent divisible by $5,000 4.5
Percent divisible by $10,000 2.2
Percent of income in range 97.9
Labor Income of Heads and Wives
Percent divisible by $5,000 22.8
Percent divisible by $10,000 12.5
Percent of income in range 78.0
Total Family Income
Percent divisible by $5,000 5.5
Percent divisible by $10,000 3.1
Percent of income in range 58.9

Source:   Mathematica Policy Research, from tabulations of calendar year 2002 income from the 2003 PSID. Note: Allocated amounts are excluded from each source.  Family income is based on the PSID family.  Wives include unmarried partners.

The labor income of heads and wives incorporates additional components of wage and salary income that the PSID collects in separate fields, including overtime pay, bonuses, and tips. Collecting wage and salary income in this way ought to reduce the amount of rounding, even though most families may report only one amount for the head and one for the wife or partner, if present. This wage and salary income is counted separately for family heads and wives/partners. We do see less rounding, but only by a modest amount:  23 percent is divisible by $5,000 and 12.5 percent is divisible by $10,000. Adding income from an unincorporated business and pooling the incomes of heads and wives/partners to create a combined earnings amount for each family reduces the rounding further, down to 19 percent divisible by $5,000 and 11 percent divisible by $10,000. None of the other four surveys shows a marked reduction in rounding between wages and salaries and earnings, although it is possible that it is the pooling of heads’ and wives’ earnings rather than the addition of self-employment income that accounts for most of the reduction in the PSID.

With 2.4 percent divisible by $5,000 and 1.0 percent divisible by $10,000, family Social Security income in the PSID shows about half as much rounding as personal Social Security income in the ACS but more than the CPS or SIPP. The combined transfer income of the head and wife/partner shows the same level of rounding as retirement income in the CPS, with 4.5 percent divisible by $5,000 and 2.2 percent divisible by $10,000. Transfer income would include retirement income (other than Social Security) as well as a number of other sources. Lastly, total family income in the PSID exhibits only half the level of rounding as total family income in the CPS and MEPS and one-third as much as total family income in the ACS, with 5.5 percent divisible by $5,000 and 3.1 percent divisible by $10,000.

View full report

Preview
Download

"report.pdf" (pdf, 4.33Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®