Measuring Income and Poverty in Four Surveys: An Overview
This report is available on the Internet at:http://aspe.hhs.gov/sp/reports/2008/mipfs/report.html
- Introduction and Summary
- CY2002 Person-Level Income by Source
- CY2002 Family Definitions and Poverty Status
- CY2002 Person-Level Income by Source for Poor and Elderly Populations
- Appendix A
- Appendix B
- Table 1
- Table 2
- Table 3
- Table 4
- Table 5
- Table 6
- Table 7
- Table 8
- Table 9
- Table 10
- Table 11
- Table 12
- Table 13
- Table 14
- Table 15
- Table 16
Policy makers use national surveys to paint a picture of the U.S. population along a variety of dimensions, such as demographic characteristics, poverty status, receipt of benefits from public programs, and health insurance. Inferences are drawn about need and eligibility for a range of Federal benefit and health programs based on sources and amounts of income, and the incidence of poverty, as estimated in National surveys.
If major surveys are equally successful in capturing income, then for the same time period, populations and income types, consistently defined income estimates and poverty rates across surveys will be highly similar – varying somewhat due to sampling error. If consistently defined income estimates differ significantly among surveys, then policy makers’ conclusions can depend on which survey is used. It is important to understand whether variations in the results produced by different surveys are significant enough to imply different policy alternatives.
This paper constructs comparable measures of income and poverty and examines whether the same picture of the U.S. population is presented by four major Federal surveys: the Annual Social and Economic Supplement to the Current Population Survey (CPS), the Survey of Income and Program Participation (SIPP), the Medical Expenditure Panel Survey Household Component (MEPS), and the National Health Interview Survey (NHIS). Standard errors were not calculated, since when presenting the findings of policy analysis, point estimates are given without standard errors.
Each survey covers the civilian non-institutional population, although the surveys differ significantly in design, periodicity, income question detail and income data processing and post-processing. However, despite differences, estimates of total income can be constructed for the same time period and unit of analysis. This paper focuses on empirical estimates, that is, how quantitative results vary depending on the survey used. Analysis of the impact that specific survey differences have on empirical results is beyond the scope of this paper, but important differences are noted, and an Appendix provides some detail on survey design, methodology, and income measurement. Another study, funded by the Office of the Assistant Secretary for Planning and Evaluation, is systematically examining the quality of income data collected on eight surveys, including the four in this paper4.
The number of persons with income, the dollar amounts and the amounts per person – person-level data – are compared among surveys for calendar year 2002. Comparisons are made for all persons, the poor and the elderly. Every attempt is made to construct comparable estimates, and unavoidable differences are noted on tables or in the text. However, it is not possible to adjust directly for one major difference, NHIS treatment of unmarried partners who pool resources as a family. Official poverty estimates do not treat unmarried partners as a family. Because NHIS asks only a “family income” amount, there is no person-level income data with which to construct family income for the family definition used in official poverty estimates, which come from CPS.
This paper has five sections, starting with this introduction. The second section describes income amounts and recipiency for each income source and compares estimates of earners among surveys. The third section compares CPS and NHIS family definitions across the total population and for the poor. The fourth section describes income amounts and recipiency for each income source for the poor and for the elderly. The last section summarizes findings and suggests some possible next steps.
Three of the surveys, CPS, SIPP and MEPS, collect information on dollar amounts of income by source for each person in the survey. The fourth survey -- NHIS -- collects information on receipt of income from various sources for each person, but not the dollar amounts. As noted above, measures of income by source and income recipiency by source were constructed for calendar year 2002 as comparably as possible5. 2.1. All Income For All Ages. Table 1 shows aggregate income by source and its’ percent distribution by source for the three surveys collecting income amounts by source (and by person). There are large differences among the surveys in overall total income and total income by source for the same time period. SIPP has over seven hundred billion dollars less total income than CPS. This difference is more than accounted for by almost nine hundred billion dollars less wages and salaries in SIPP compared to CPS. SIPP has higher amounts of self-employment income, pensions, SSI and public assistance than CPS, and about half the property income. MEPS has somewhat lower aggregate income than CPS, but higher wages and salaries and SSI, and much lower self-employment income.
While aggregate income amounts vary significantly, the distribution by source is remarkably consistent. Earned income (wages and salaries and self-employment) dominates the picture, accounting for 83 percent of total income in each survey. Earned income’s actual share varies less than one percentage point among the surveys, from 82.5 to 83.1 percent of total income. The surveys differ in the proportion of earnings attributable to self-employment as compared to wages and salaries -- SIPP shows the most self-employment earnings, at 10.6 percent of aggregate income, and MEPS the least, at 1.8 percent of aggregate income. The remaining 17 percent of income in each survey is accounted for by Social Security (5.6 to 6.4 percent), other pensions (3.9 to 5.6 percent), property income6 (2.4 to 4.2 percent), and means-tested SSI and public assistance (0.5 to 0.8 percent), as well as a range of other income sources (including Veterans benefits, Unemployment Insurance, Workers Compensation and child support) that together account for the balance (1.8 to 2.5 percent). SIPP has the highest shares for Social Security, other pensions, and public assistance, and the lowest shares for property income.
|Billions of Dollars||Percent Distribution by Source|
|Wages and Salaries||5,073.0||4,181.8||5,179.9||77.7%||71.9%||81.4%|
|Interest and Dividends||204.0||102.7||178.0||3.1%||1.8%||2.8%|
|Rents, Royalties and Estates||70.3||34.2||52.1||1.1%||0.6%||0.8%|
Note: Total and detail in all tables exclude small negative amounts of self-employment and rental income; Social Security lines in all tables include Railroad Retirement.
Tables 2 and 3 show unduplicated recipients, the recipiency rate – percent of the population with that income source – and the recipiency rate as a percent of the CPS recipiency rate by income source. CPS is used as a standard of comparison since it has the most widely-used income data and is the source of official poverty statistics. Table 3 also shows the average amount per recipient for each income source. In all tables the recipient numbers for any category count each person only once -- someone receiving income from more than one source in the category is counted as one person, although persons may be in multiple categories. Total persons with income and with earnings are also unduplicated.
There is greater disparity among the surveys in the number of recipients for each income source than in the percent distribution of income by source. Generally, the highest recipiency is found in SIPP, which collects the most detailed income data. The lowest recipiency is found in NHIS, which has the fewest questions about sources and types of income, and gets amounts only for earnings and total family income. The NHIS public use file has no imputed values for item non-response on recipiency questions, but item non-response rates are low and do not account for the difference.8
The main exception is wages and salaries, which MEPS finds received by 160.0 million persons compared to 141.3 and 143.0 million in SIPP and CPS, respectively. The other exception is IRA withdrawals, which MEPS estimates are received by more than twice as many persons as SIPP. The overall recipiency rates suggest SIPP is finding more sources of income per person than the other surveys.
The lower dollar aggregates in SIPP, combined with its higher recipiency rates, result in SIPP having the lowest amounts of income per recipient for all income, and for almost all sources of income. Average amounts per recipient are highest in CPS and lowest in SIPP for total income, wages and salaries and Social Security. Average amounts per recipient are highest in MEPS and lowest in SIPP for property income, SSI and public assistance. SIPP has the highest self-employment income per recipient, and CPS has the highest pension income per recipient. Both CPS and MEPS find about twice as much other income per recipient as SIPP, and close to three times as much property income as SIPP.
2.2. Income for Persons 18 or Older.
The surveys differ in their treatment of the income of persons under 18 (see Appendix B for details). To determine the impact of these differences, the tables above were repeated for the adult population, excluding some 73 million children. Virtually no change occurred in aggregate income amounts, percent distribution of income by source, or relative recipiency rates for any of the surveys; and total income, earnings, and wages and salaries per recipient increased only 1.5 to 2.6 percent. However, recipients and recipiency rates changed more sharply.
Table 4 shows the result of restricting the universe to adults. In CPS, SIPP and MEPS, persons with any income decreased 4.3 to 5.8 million, compared to 9.2 million in NHIS. In CPS, SIPP and MEPS, persons with wages and salaries and the total with earnings decreased 2.7 to 4.0 million, in contrast to no change in the NHIS, which asks earnings only for persons 18 or over. In CPS, SIPP and MEPS, recipients decreased less than one million for all other income sources except interest and dividends. In NHIS, recipients decreased over a million for Social Security and public assistance, and four million for “all other” income.9
When the universe is restricted to adults, overall income recipiency rates increase 19.2 to 22.5 percentage points, to 87.9 percent in NHIS, and 92.4 to 96.0 percent in CPS, MEPS and SIPP. The largest increases in recipiency are in wages and salaries (15.3 to 18.0 percentage points), any earnings (16.6 to 18.0 percentage points), and interest and dividends (6.7 to 14.7 percentage points). Recipiency of other income sources increased by lesser amounts in all surveys, with recipiency of public assistance decreasing slightly in NHIS.
The picture of income recipiency that emerges for adults thus differs substantially from that for persons of all ages. Nonetheless, the highest recipiency is still found in SIPP, and the lowest in NHIS, except for wages and salaries, earnings, and IRA withdrawals. MEPS has 157.8 million adults with earnings, compared to 151.2 and 147.8 million in SIPP and CPS, and 144.6 million in NHIS. MEPS also estimates more than twice as many adults as SIPP had income from IRA withdrawals. Overall recipiency rates still suggest SIPP is finding more sources of income per person than the other surveys.
2.3 Composition of Earnings. All four surveys find earned income to be the dominant component – 83 percent – of aggregate income in the United States, and wages and salaries to be the single most important income source for adults. As shown above in Table 4, three surveys -- CPS, SIPP and NHIS – find relatively similar numbers of adult wage earners in 2002, from 134.7 to 139.7 million. However, MEPS has 157.2 million adults reporting wage and salary income, 17.5 million more than CPS, which is the official source of national employment and labor force data. There is a mirror image difference in adults with self-employment income. Three surveys -- CPS, SIPP and NHIS – find fairly significant numbers of self-employed adults in 2002, ranging from 12.0 to 20.0 million, while MEPS has only 4.2 million. Adults whose self-employment is the sole source of earnings show a similar pattern -- CPS, SIPP and NHIS have 7.9 to 13.8 million, compared to 620 thousand in MEPS. MEPS appears to be an outlier both in the total number of employed adults and in the composition – wages and salaries versus self-employment – of their earnings.
When these findings were shared with MEPS staff, they advised that the wage and salary income variable on the public use file (despite its name) reflected both wage and salary and self-employment earnings. They suggested reclassifying earned income based on data in the JOBS file, which is a separate research file with a record for each job mentioned in each interview and/or on employment status on the public use file. However, since the employment status variables on the public use file classify only the current main job and do not allow for multiple jobs or a combination of wages and salaries and self-employment, this approach proved less than useful.
An identifier match was performed to merge data from the public use and JOBS files in order to determine whether an improved estimate of wage and salary recipiency could be constructed. However, the suggested reclassification of wages and salaries based on JOBS data was not in fact possible because persons reporting earned income on the public use file were sometimes not the same individuals as those with JOBS file records of employment, and because large numbers both worked for others and were self-employed. There is no way to assign income amounts to persons with employment in JOBS but no reported earnings. There also is no way to split wage income into wages and salaries and self-employment for persons who both worked for others and were self-employed in JOBS, but reported only wages. Additional uncertainty about use of JOBS data results from the fact that the identifier match found over 6 million people reporting earned income, mostly wages, on the public use file who had no employment records in the JOBS file. The differences in covered population would preclude valid comparisons of income by source.
Table 5 shows persons in MEPS classified by whether they reported earnings and/or had JOBS employment records, and age. Some 6.6 million persons had reported earnings on the public use file (MEPS income) and no JOBS employment records, and another 2.6 million had employment records but no earnings. This is over 9 million persons in one but not the other file -- either with earned income but no employment records, or with employment records but no earned income – in addition to 154 million persons in both files. Taken together, MEPS finds over 163 million unduplicated persons all ages reporting earnings and/or a job, compared to 151 million in CPS, or 155 million in SIPP. Thus after taking JOBS file data into account, MEPS provides a picture of the employed population even more at variance from CPS, SIPP and NHIS than the picture based on income data in the public use file alone.
Restricting the comparisons to adults has little impact on the findings. There are two million adults who report no earned income but have JOBS employment records and over six million adults with average annual earnings of $16,277 who have no JOBS employment records10. This gives a total of 160.1 million adults who reported earnings and/or were identified as employed in MEPS, compared to 144.6 to 151.2 million in CPS, SIPP and NHIS.
Table 6 shows adults classified by type of employment -- whether only wages and salaries, only self-employment, or both -- and whether earnings were reported. No attempt has been made to reclassify reported earned income or impute earnings based on JOBS data. The employment status variable included at staff recommendation is on the public use file and is based on JOBS data. It shows substantially the same pattern as JOBS data, but only describes the current main job and does not allow for multiple jobs or a combination of wages and salaries and self-employment. Apparently it has not been used to edit the income data, with which it has significant inconsistencies.
Note: MEPS employment status variables classify the current main job only; coding does not allow for multiple jobs or both wages and salaries and self-employment.
In CPS, SIPP and NHIS, screeners and edits make it virtually impossible for a person to have employment but no earnings (with the minor exceptions of unpaid family workers and self-employed with losses) or to have earnings without employment. However, this is not the case in MEPS.
Poverty status is determined by comparing family income to a set of poverty thresholds that vary with family size, age of the householder (for one and two-person families), and number of children. Differences in how well income is measured affect poverty status directly, by changing the income amounts that are summed to family income for comparison to the poverty thresholds. Differences in family definitions affect poverty status in two ways. Changing who is considered part of a family can change family size. More importantly, if the persons being included or excluded from the family have income, changing who is in the family can change which persons’ income is included in family income and whether family members are counted as poor.
Two of the surveys -- CPS and SIPP -- use the CPS family definition employed in official poverty counts. Under this definition, a family consists of persons related by blood or marriage. The third survey -- NHIS -- uses a broader definition that treats unmarried couples (of the same or opposite sex) who pool resources as a family, and includes foster children. MEPS, the fourth survey, uses the NHIS definition but also constructs CPS families to use in calculating person and family weights.
Additional differences must also be taken into account in order to present findings that are analytically comparable across surveys. First, the MEPS public use file includes 6.3 million persons in 3.5 million families of “undefined size” that lack data for one or more family members11. Tables 7 through 10 exclude these MEPS families of “undefined size”. Later, in Table 11, these families of “undefined size” are reintroduced to permit appropriate comparisons of the demographic characteristics of the poor.
Second, family weights for families of size one (single individuals) in MEPS produce a significantly different population count12 than the person weights for these same cases. Differences in MEPS family counts under person and family weights result from post-stratification of families (see Appendix B) as well as from the difference between person and family universes in MEPS.13 For accurate comparisons, all persons and families by family size in Tables 7 through 10 are calculated using person weights for each survey.
3.1. Differences in Overall Family Counts. Table 7 shows persons and families, by family size, for both CPS and NHIS family definitions in the four surveys. Under the broader NHIS definition, there are fewer families of larger average size. The greatest differences are for one and two-person families, which account for about 40 percent of all persons (38.6 percent in NHIS to 42.7 percent in MEPS). Single persons decrease 7.1 million between the two definitions in MEPS, 9.3 million if NHIS is compared to SIPP, and 11.2 million when compared to CPS. Persons in families of two increase 2.7 million between the two definitions in MEPS, 7.4 million when NHIS is compared to SIPP, and 5.1 million from NHIS to CPS. Detailed tabulations (not shown) found the NHIS definition added 5.5 million unmarried adults either to other single persons to create families, or to existing related families, in both MEPS and in NHIS.
Regardless of the family definition used in comparing surveys, the largest differences remain the number of single individuals. Comparing counts under the CPS family definition, MEPS has more single persons than SIPP or CPS, despite the exclusion of 2.0 million singles in families of “undefined size”. Comparing counts under the NHIS family definition, MEPS has more singles than NHIS, (again despite the excluded 2.0 million), even though the NHIS treats as singles close to one million students living in college dormitories who are included in the parental family in CPS, SIPP and MEPS.
Notes: 1. Family sizes calculated from persons, using person weights, by family size. 2. Excludes families of “undefined size” in MEPS.
Table 8 shows percent distributions of the counts of persons and families in Table 7, to provide clearer comparisons given that total persons range from 278 to 286 million. Findings are unchanged – the NHIS family definition reduces singles, and increases couples and larger families.
Notes: 1. Family sizes calculated from persons, using person weights, by family size. 2. Excludes families of “undefined size” in MEPS.
The impact of alternative family definitions on the number, size and composition of families in the overall population leads to a further question, What is their impact on the number, size and composition of families below the poverty threshold? Table 9 shows poor persons14 and families, by family size, for both CPS and NHIS family definitions in the four surveys. These comparisons are complicated by the fact that the accuracy of income measurement and the amount of income reported varies among the surveys, with the result that surveys using the same family definition nonetheless find different numbers of persons and families below the poverty threshold. Nonetheless, compared to the CPS family definition, the NHIS definition produces fewer poor families of larger average size. Specifically, the NHIS has one million fewer poor families than the CPS, although it has four million more poor persons.
Holding income measurement constant and comparing the two alternative family definitions in MEPS, single poor persons decrease 1.5 million, poor persons in families of two to four decrease 1.3 million and poor persons in larger families increase by 65 thousand.15 In NHIS, despite the much higher total count of poor, there are 1.8 million fewer single poor persons than in SIPP, and 2.6 million fewer than in CPS, with correspondingly higher counts of poor persons in larger families. Detailed tabulations (not shown) found that over 80 percent of unmarried partners included in families by the NHIS definition have earnings. In MEPS, they increased family income as well as size, resulting in fewer poor. This suggests that the high count of poor in NHIS does not result from the broader family definition being used, and in fact would be still higher, if the CPS family definition were used when family income is ascertained.
Notes: 1. Family sizes calculated from persons, using person weights, by family size 2. Excludes families of “undefined size” in MEPS
Notes: 1. Family sizes calculated from persons, using person weights, by family size 2. Excludes families of “undefined size” in MEPS
3.2. Differences in the Poverty Population. Table 11 shows poor persons16 and poverty rates, by demographic sub-groups, for both CPS and NHIS family definitions in the four surveys. As was the case with Table 9, these comparisons are complicated by the differences in income reporting that result in surveys with the same family definition finding different numbers of poor persons. As noted above, MEPS data in Table 11 (and subsequent tables) contain all persons on the MEPS public use file. These include 2.3 million poor persons in families of “undefined size”, 36 percent of all persons in families of “undefined size” on the public use file. Their inclusion is required for appropriate comparisons of demographic characteristics, since they are included when MEPS demographics and poverty rates are post-stratified to match the CPS.
Table 11 shows that the decline in numbers of poor persons and poverty rates resulting from use of the broader NHIS family definition is not spread evenly across demographic groups. Comparing the definitions in the same survey, that is, using the same income data, gives the clearest illustration. In MEPS, the NHIS family definition decreases the number of poor persons by 2.8 million, of which 1.1 million are males and 1.7 million are females. Children in poverty decrease by 1.3 million and non-aged adults by 1.5 million, while elderly poor change by less than ten thousand.17 Compared to CPS, NHIS finds an additional 4.0 million persons in poverty, and 5.4 million more than SIPP. The additional poor in NHIS counts are more likely to be male, Hispanic or non-aged adults, as compared to female, white or Black, or children. For one group, the elderly, there is virtually no difference between counts of poor in CPS, NHIS or under either family definition in MEPS, although SIPP finds fewer poor elderly than other surveys.
The patterns for poverty rates reflect those for numbers of poor. In MEPS, use of the broader NHIS definition lowers the overall poverty rate by almost a percentage point, and results in lower poverty rates for all groups except the elderly. The poverty rates decrease most for females, Hispanics and Blacks, and children, decreases less for males, whites and non-aged adults and do not change for the elderly. NHIS poverty rates are higher than in any other survey overall and for each demographic group (except for Blacks in SIPP), but not uniformly so. NHIS poverty rates compared to other surveys increase less for elderly, females, children and Blacks, and more for males, Hispanics, whites and non-aged adults.
Looking at counts of poor and poverty rates across surveys, SIPP has the smallest count of poor and the lowest overall poverty rate using the CPS family definition. Poverty rates in SIPP are relatively higher for women, children and Blacks, and relatively lower for males, Hispanics and the elderly. There are 1.4 million fewer poor in SIPP than in the CPS, 1.1 million fewer males and 1.4 million fewer non-aged adults, but more children and Hispanics. The counts and poverty rates in MEPS are virtually the same as those in CPS, a result MEPS guarantees by re-weighting the public use file to match CPS poverty rates for CPS-defined families by age, sex, and race/ethnicity.18 The NHIS finds 4.0 million more poor than the CPS, almost evenly split between male and female. Almost half of the additional poor in the NHIS compared to CPS are Hispanic, and almost all (80 percent) are non-aged adults. Less than 100 thousand are elderly.
Notes: 1. The three race/ethnicity categories are mutually exclusive but non-exhaustive, that is, lines for White and Black exclude Hispanics and the three categories do not include other minority groups 2. Includes families of “undefined size” in MEPS
The poor and the elderly both are demographic groups of interest for policy research. The two populations are of roughly equal size (with a small overlap of 3.1 to 3.7 million) but account for quite different shares of aggregate income. The poor, approximately 12 percent of total population, have less than two percent of aggregate income. In contrast, the elderly account for 11 to 12 percent of aggregate income. This section employs the CPS family definition to measure poverty in CPS, SIPP and MEPS, and as noted above, includes all persons on the MEPS public use file.
4.1. Income of the Poor. Table 12 shows aggregate income by source and the percent distribution by source for persons in poverty in the three surveys collecting income amounts by source. The surveys report very similar amounts of aggregate income for the poor – $112 to $120 billion – despite somewhat different counts of poor. Average amounts of income for poor persons with income are also similar, ranging from $6,376 per recipient in SIPP to $6,722 per recipient in MEPS. Amounts and distribution by source, however, vary substantially between surveys as well as differing from the distribution by source for the total population. There is much more variation among surveys in the relative importance of different income sources for the poor than for the population as a whole. Earned income (wages and salaries and self-employment) ranges from 50.5 percent in CPS to 59.4 percent in MEPS, in contrast to the uniform 83 percent of aggregate income for the total and the adult population found in all surveys. The share of wages and salaries has an even greater range, 46.5 percent in CPS to 59.3 percent in MEPS, compared to 72 to 81 percent for the total population. In all surveys the second largest component of income of the poor is Social Security, accounting for 16.6 percent of income in MEPS and 23.8 percent in CPS, followed by means-tested SSI, ranging from 8.8 percent in CPS to 12.3 percent in MEPS.
These three sources (earnings, Social Security and SSI) account for 83.1 percent of income of the poor in CPS, 83.7 percent in SIPP, and 88.3 percent in MEPS. Means-tested public assistance and a range of income sources including Veterans benefits, child support, Unemployment, and Workers Compensation account for most of the balance, with very little contributed by other pensions or property income.
Table 13 shows unduplicated poor recipients, and recipiency rates (percent with that income source) by income source. There is as much disparity among the surveys in the number of recipients for each income source as in the percent distribution of income by source. Generally, NHIS has the highest number of recipients as well as the highest recipiency rates, including earnings and wages and salaries, Social Security and pensions, but not property income or SSI. NHIS classifies many more persons as poor than the other three surveys, but with somewhat different demographic characteristics. Additional poor in NHIS include many persons 18 to 64 (3.2 to 4.6 million compared to the other surveys) and less than a million more children (200 to 800 thousand), thus shifting the demographic mix towards persons working or otherwise receiving income.
The demographic mix of the poverty population in SIPP compared to that in the CPS has a somewhat higher share of children, and lower share of non-aged adults and elderly, which might lead to lower recipiency rates. Nonetheless, the overall recipiency rate for income among the poor in SIPP is higher than CPS, particularly for self-employment, pensions, SSI and public assistance. In contrast, MEPS poverty rates by demographic group are designed and in fact are close or identical to those in CPS. It would be reasonable for the pattern of receipt of income by source for the poor in MEPS to also be highly similar to that in CPS. However, recipiency by income source for the poor in MEPS is not close to CPS. Recipiency is higher for wages and salaries, earnings, and SSI, and lower for self-employment, property income, and “all other”.
Another way of comparing the picture of the poverty population provided by different surveys is to look at the share of income and of income recipients accounted for by the poor. These relative shares standardize for the different levels of aggregate income by source in the different surveys. Table 14 shows the relative shares of total income and income recipients by source represented by the poor in the three surveys collecting income amounts by source. The poor account for the majority of public assistance income and recipients in the three surveys. However, the relative share of public assistance income ranges from 49.4 percent in SIPP to 72.5 percent in MEPS, and the proportion of recipients ranges from56 to 66 percent. The income source with the second largest share accounted for by the poor is SSI. The proportion of total SSI estimated to be received by the poverty population is virtually identical among surveys – 37.8 or 37.9 percent -- although the dollar aggregates differ substantially, and there is some range – 39.5 percent in SIPP to 44.4 percent in CPS – in the percent of SSI recipients accounted for by the poor. The third income source estimated to have significant relative shares accounted for by poor is Social Security – 5.6 to 6.8 percent of dollars and 9.2 to 10.9 percent of recipients. The poor account for negligible shares of other individual sources of income (detail not shown).
Notes: 1. Property income includes interest and dividends, IRAs, and rents, royalties, and estates. 2. Unduplicated recipients of property income not calculated.
4.2 Income of the Elderly. Table 15 shows aggregate income by source and its’ percent distribution by source for persons age 65 and over in the three surveys collecting income amounts by source. The survey differences in overall amounts of income for the elderly are not nearly as large as for the total population. This is due in part to the fact that the largest differences among surveys are in earned income, which is much less important as a source of income for the elderly than for the population as a whole. However, income by source for the elderly varies much more among surveys than for the population as a whole. While CPS has the highest and SIPP the lowest amounts of aggregate income for the total population, MEPS has the highest level of aggregate income for the elderly, and SIPP still has the lowest.
The most important source of income for the elderly is Social Security, which is highest in CPS at $307.0 billion and lowest in MEPS at $286.3 billion. The share of income accounted for by Social Security ranges from 36.8 percent in MEPS to 42.6 percent in SIPP. The second most important source varies among surveys – in SIPP it is other pensions at 26.3 percent, in MEPS it is wages and salaries at 25.3 percent, and in CPS, other pensions at 20.0 percent. When Social Security, other pensions and IRAs are combined to constitute retirement income, total amounts range from $453.0 billion in CPS to $499.7 billion in SIPP, and the share of total income provided by retirement income varies from 60.3 percent in MEPS to 74.7 percent in SIPP.
Total earned income (wages and salaries and self-employment) of the elderly has greater range than retirement income for dollar amounts but less variation in the share of income. Earnings of the elderly are lowest in SIPP– $117.6 billion and 17.0 percent of income – and highest in MEPS – $206.8 billion and 26.6 percent – with CPS at $156.7 billion and 21.4 percent. The relative importance of wages and salaries and self-employment also varies sharply among surveys. SIPP, which has substantially lower amounts and shares of earnings for the elderly than CPS or MEPS, has much higher amounts and shares of self-employment income. Property income (interest, dividends, rents, royalties etc.) also differs substantially among surveys, ranging from $55.6 billion and 8.0 percent in SIPP to $102.6 billion and 14.0 percent in CPS. SSI and all other sources together account for approximately two percent of the income of the elderly. Very little of the SSI income reported in the three surveys is received by elderly persons; the majority (76 to 81 percent) of SSI income is reported by persons under 65.
Notes:1. Retirement income includes Social Security (including Railroad Retirement), other pensions, and IRA withdrawals. 2. Small amount (under 0.3) of Public Assistance included in All Other category.
Table 16 shows unduplicated recipients, and recipiency rates (percent with that income source) by income source for the population aged 65 and over. There is somewhat less disparity among the surveys in number of recipients for each income source than in the percent distribution of income by source. Nonetheless, there are sizeable differences among surveys. MEPS has three million fewer elderly receiving Social Security than SIPP, and CPS and NHIS have two million fewer Social Security recipients than SIPP. MEPS also has 3.5 million more elderly with earned income from a job or self-employment than CPS, and over three million more than SIPP or NHIS. SIPP finds other pension received by over half the elderly, compared to 40 percent in MEPS and 34 percent in CPS. As with the income amounts, the picture of the sources and composition of support for the elderly differ significantly from survey to survey.
Overall, the highest recipiency is found in SIPP. SIPP also has a higher recipiency rate than CPS, MEPS or NHIS for each individual income source except wages and salaries and IRA withdrawals. For both of these sources MEPS has much higher numbers (and rates) of recipiency. As with the overall and adult populations, recipiency rates suggest SIPP is finding more sources of income per person than the other surveys.
In evaluating policy options, analysts must be aware of the strengths and weaknesses of potential or alternative sources of estimates. Indeed, this study finds that measures of income and income recipiency vary substantially among the surveys even when comparable estimates of family income in 2002 are constructed for each survey. Policy analysts also have a responsibility to inform policy officials of the impact that alternative sources of estimates are likely to have on findings and consequent policy options due to differences in income measures and other survey characteristics.
At the same time, policy analysts may not be able to use the surveys with the best income data because other essential data are not also collected by those surveys. If detailed health data is needed, the CPS – the official source of income and poverty statistics -- cannot be used. Similarly, policy analysis for many Federal programs frequently requires information on each person in a family in order to calculate eligibility of units that are smaller than the family. Without this information, “what-if” scenarios looking at persons who are eligible but not participating, or who would become eligible if the program is changed, cannot be constructed. The NHIS cannot be used for such analysis since it lacks person-level income data; has family income measured only as a range; and uses a definition of “marriage” not currently in use for any Federally-funded transfer programs.
The different survey definitions of a family have significant impacts on estimates of family size, family income and poverty rates, all of which are important for policy analysis. The broader NHIS family definition used in MEPS and NHIS greatly reduces estimates of single persons, increases family size and income, and appears to reduce the number of persons in poverty at a given level of income reporting. Policy analysis based on either of those surveys must be sensitive to the choice of family definition. Also, as discussed in the paper, additional issues arise in developing comparable poverty estimates for MEPS that need to be resolved.
While there are large differences among the surveys in estimates of total income and income by source for the U.S. population for this time period, the distribution by source is remarkably consistent. At the same time, the surveys provide sharply different counts of adults with earnings – wages and salaries and self employment -- the most important source of income for adults in the United States. There are even greater differences among the survey estimates of the number with wages and salaries. Examination of populations important for policy purposes also finds very different pictures of target populations. The size and demographic composition of the poverty population as well as its sources of income varies considerably among surveys.
The elderly, for whom survey estimates of aggregate income and poverty are closer than for adults of all ages, have much more variation in income by source than the population as a whole. The relative importance of retirement and earned income varies sharply among surveys. For the elderly, the income source and impact on retirement decisions and well-being after retirement are critical policy concerns.
Earnings account for 83 percent of all income, and atypical patterns naturally raise questions. MEPS income and JOBS data taken together find over 163 million unduplicated persons all ages (160 million adults) reporting earnings and/or a job, compared to 151 million all ages (148 million adults) in CPS, or 155 million all ages (151 million adults) in SIPP. These large differences in employment require an explanation, especially given that the principal focus of the CPS is National employment and labor force data, for which it is the official source.
CPS, SIPP and NHIS have screeners and edits that make it impossible (with minor exceptions) for a person to have employment but no earnings or to have earnings without employment. This is not the case in MEPS, which has 6.6 million persons with reported earned income but no employment record and 2.6 million persons with employment records but no reported earnings (and no imputed earnings). For another 25 million persons, the source of earnings – wages and salaries or self-employment – differs between the public use file income data and their type of employment in JOBS file records. Clearly increased attention to editing of MEPS employment and earned income data for consistency would enhance data quality and utility.
Different estimates of the poverty rates of important demographic groups suggest that design, estimated costs and estimated impacts of policy options for assisting low income populations would vary depending on the survey used. For example, program interventions involving a primarily Hispanic population must deal with cultural and language issues that would not apply to other populations. SIPP tends to have the lowest poverty rates, suggesting that it is more successful than the other surveys in capturing income at the lower end of the income distribution. This presumably reflects the greater detail of its’ income questions, which were designed by policy analysts to improve income information about low income persons. SIPP is therefore likely to yield the lowest cost estimates, whereas NHIS has significantly higher poverty rates, except for children and Blacks, and is likely to yield quite different results.
The authors greatly appreciate the assistance provided by MEPS staff.
|File||2003 Annual Social and Economic Supplement||2001 Panel||2002 Full Year File||2003 NHIS|
|Total Income (CPS Definition)||PTOTVAL> 0||TPTOTINC >0 and/or EEDFUND = 1||TTLP02X||Any item|
|Wages and Salaries||WSAL-YN; OI-YN (15,16)||TPMSUM1> 0 and/or ER15 > 0; (for $, also TPMSUM2;TMLMSUM)||WAGEP02X||PSAL|
|Self-employment||SEMP-YN; FRSE-YN; OI-YN (17, 18)||TBMSUM1 > 0 (for $, also TBMSUM2)||BUSNP02X||PSEINC|
|Interest and Dividends||INT-YN; DIV-YN; OI-YN (5,6) (See Note 1)||TINTINC > 0 and/or TDIVINC > 0 and/or TRNDUP2 > 0||INTRP02X; DIVDP02X||PINTRSTR and/or PDIVD|
|Rents, Royalties, Estates and Trusts||RNT-YN; SUR-SC1(8); SUR-SC2(8); OI-YN (7,8)||ER37; ER53; TJACLR > 0 and/or TJACLR2 > 0 and/or TOACLR > 0; TRNDUP1 > 0||TRSTP02X|
|IRA Income||RET-SC1(7); RET-SC2(7)||ER42||IRASP02X|
|Pensions||SUR-SC1(except 5,8); SUR-SC2(except 5,8); DIS-SC1(except 1,6,8); DIS-SC2(except 1,6,8); RET-SC1(except 5,7); RET-SC2(except 5,7); OI-YN (2,10,13)||ER30; ER31; ER32; ER34; ER35; ER36; ER38; ER13||PENSP02X||PPENS; POPENS|
|Social Security and Railroad Retirement||SS-YN; SUR-SC1(5); SUR-SC2(5); DIS-SC1(6); DIS-SC2(6); RET-SC1(5); RET-SC2(5); OI-YN (1)||ER01A; ER01K; ER02||SSECP0X||PSSRR|
|SSI||SSI-YN||ER03A; ERO3K; ER04||SSIP02X||PSSI|
|Public Assistance (cash only)||PAW-YN; OI-YN (3,4)||ER20; ER21;ER23; ER24||PUBP02X||PTANF|
|Unemployment Compensation||UC-YN; OI-YN (11,12)||ER05; ER07||UNEMP02X||(see Note 5)|
|Workers Compensation and Veterans Benefits||WC-YN if WC-TYPE = 1,2; DIS-SC1(1,8); DIS-SC2(1,8); OI-YN (9); VET-YN||ER10; ER12; ER14; ER08||WCMP02X; VETSP02X||(see Note 5)|
|Child Support and Alimony||CSP-YN; ALM-YN||ER28; ER29; ER26||CHILDP02X; ALIMP02X||PCHLDSP (see Note 5)|
|Financial Assistance, Educational Aid (non-VA) and Other Cash Income||FIN-YN; ED-YN; OI-YN (19), WC-YN if WC_TYPE=3,4||ER50; ER51; ER55; ER56; ER75; EEDFUND = 1||CASHP02X; OTHRP02X||PINCOT (see Note 5)|
Notes:1. CPS recipiency of dividends restricted to non-zero dollar amounts -- positive value for DIV-YN is based on asset ownership and includes over two million persons with zero dividend income 2. Pension category includes disability and survivors benefits, and own disability insurance 3. Workers Compensation and Veterans Benefits category includes Black Lung and excludes own insurance, and survivors benefits from Workers Compensation and Black Lung 4. Four items – Unemployment Compensation, Workers Compensation and Veterans Benefits, Child Support and Alimony, and Financial Assistance, Educational Aid (non-VA) and Other Cash Income – are aggregated in the analysis in the category “All Other”.5. In NHIS, alimony, Veterans Benefits, Workers Compensation, Unemployment Compensation and contributions from family/others are included in PINCOT, which is shown here as the category financial assistance/other cash income.
|Current Population Survey||Survey of Income and Program Participation|
|Design Summary||Annual cross-section household survey at fixed point in time collecting detailed prior calendar year income and current labor force and demographic data||Longitudinal panel survey collecting detailed monthly income and a broad range of other data through interviews at 4 month intervals, with most recent panels started in 2001 and 2004|
|Universe||Resident civilian noninstitutionalized population of the US plus military living with civilian family members on or off base||Resident civilian noninstitutionalized population of the US plus military living with civilian family members on or off base|
|Exclusions||Unrelated children under 15 in group quarters e.g. shelters||Unrelated children under 15 in group quarters e.g. shelters|
|Sample Size||78,300 interviewed households in 2003 (2002 income)||28,000 interviewed households in 2001 panel (post-reduction)|
|Sample Design||Multi-stage sampling design at the State level with sub-State samples in NY and CA||Multi-stage sampling design at the National level with sample drawn at beginning of each new panel|
|Oversamples||Hispanics, non-Whites, and Whites with children under 19||Low income areas|
|Data Collection||Data collected once from each household through personal and phone interviews February through April, with bulk of data collection in March (2002 forward)One person per household responds for household||Data collected from each household through 9 personal and phone interviews at 4 month intervals (2001 panel) -- rotation groups provide continuous fieldworkEach person 15 or over responds for self|
|Longitudinal||Movers followed to new locations and move-ins interviewed while living with sample personsInstitutionalized not interviewed and data for decedents is last interview outside an institution before death|
|Response Rates and Attrition||2003 Supplement month response to CPS-1 (underlying monthly survey): 92.2% – supplement data is imputed for CPS-1 responders who refuse supplement||2001 panel initial response: 87.7%Average 6.5% attrition wave to wave (4 months) after wave 2|
|Medical Expenditure Panel Survey||National Health Interview Survey|
|Design Summary||Longitudinal panel survey collecting monthly event-level health data and prior calendar year income data through interviews at 6 month intervals, with panels starting annually||Annual cross-section household survey collecting prior month and year health data and prior calendar year income data continuously during the year (rolling sample)|
|Universe||Resident civilian noninstitutionalized population of the US (military living with civilian family members excluded but their income included in the income of their civilian family)||Resident civilian noninstitutionalized population of the US (military living with civilian family members excluded but their income included in the income of their civilian family)|
|Exclusions||Unrelated minors (usually under age 18) in households or group quarters if not foster children||Unrelated minors (usually under age 18) in households or group quarters if not foster children|
|Sample Size||14,700 interviewed families in 2002 (2002 income)||Over 36,000 interviewed families in 2003 (2002 income)|
|Sample Design||Sub-sample of persons interviewed by NHIS during previous year||Multi-stage sampling design at the National level with 1/4 of annual sample drawn quarterly|
|Oversamples||Blacks, Hispanics, Asians, and families predicted to have income less than 200% of poverty||Blacks and Hispanics|
|Data Collection||Data collected from each household through 5 personal and phone interviews at approximately 6 month intervals in fairly continuous fieldwork-One person per family responds for family||Data collected once from each household through personal interviews in continuous fieldwork – 1/52 of sample per week-One person knowledgeable about family health responds for family but any adult present may respond for self|
|Longitudinal||Movers followed to new locations and move-ins interviewed while living with sample persons-Institutionalized not interviewed but data for decedents collected until death|
|Response Rates and Attrition||Initial response (including frame): 70%Average 2% attrition wave to wave (6 months) after wave 1||2003 response: 89.2%|
|Current Population Survey||Survey of Income and Program Participation|
|Definitions Differ From CPS||Self-employment is monthly draw plus net profit (cash basis) and never negative-Includes lump-sum or non-periodic payments such as IRA withdrawals-Excludes educational benefits|
|Amount Detail For Persons||Over 50 sources and up to 24 amounts of income||Up to 60 sources and amounts of income|
|Person and Family Totals||Sum of detail for persons and family||Sum of detail for persons and family|
|Persons Covered||All persons for Social Security, SSI and TANF-Persons 15 or over for all other income sources-For persons under 15, Social Security, SSI and TANF assigned to a representative payee or guardian||All persons for Social Security, SSI and TANFPersons 15 or over for all other income sources-For persons under 15, Social Security, SSI and TANF assigned to a representative payee or guardian|
|Reference Period||Prior calendar year||Monthly and 4-month periods in prior 4 months|
|Recall Length||Average 14 ½ months maximum 15 ½ months||Average 3 months maximum 5 months|
|Medical Expenditure Panel Survey||National Health Interview Survey|
|Definitions Differ From CPS||Internal Revenue Service definitions used for tax filers Note 1 -Wages omit “above the line” item such as 401(k) contributions -Self-employment earnings other than sole proprietorships and farm reported with rents, royalties, estates and trusts- Includes taxable lump-sum or non-periodic payments- Excludes tax exempt interest for tax filers||Earnings amount includes net income from rental property and unemployment or worker’s compensation- Recipiency data groupings conform to CPS definitions – rental income is grouped with dividends, estates and trusts|
|Amount Detail For Persons||Up to 16 sources and amounts of income-Taxable income sources not person-level for joint return filers -- primary filer allocates amounts between self and spouse-Additional data on employment in separate JOBS file||One bracket for total earnings from all sources-Recipiency but no amounts for up to 10 other sources – public use file has no imputations of recipiency-Brackets on public use file are $5,000 wide below $25,000 and $10,000 wide from $25,000 to $75,000|
|Person and Family Totals||Sum of detail for persons and family||One bracket for total income of NHIS-type familyBrackets on public use file are $5,000 wide below $25,000 and $10,000 wide from $25,000 to $75,000-NCHS creates 5 files of imputations and recommends that analyses be performed 5 times and results averaged|
|Persons Covered||Persons 16 or over, and persons under 16 who report filing a tax return, for taxable income source-All persons for all other income sources||Persons 18 or over for earnings|
|Reference Period||Prior calendar year||Prior calendar year|
|Recall Length||Average 15 months maximum 18 months||Average 18 months maximum 23 months|
|Current Population Survey Annual Supplement||Survey of Income and Program Participation|
|Earnings Top-Codes||Wages and salary from longest job top-coded at $200,000-Other wage and salary, self-employment and farm earnings top-coded at $35,000, $50,000 and $25,000 respectively-Value when top-coded is average across top-coded records in each of 12 demographic cells for each earnings source (some cells empty or collapsed)||Each monthly earnings amount (main job, other wages and salary, and self-employment including farm) top-coded at $12,500 (equivalent to $150,000 annually) if 4-month sum from that source exceeds $50,000-Value when top-coded is average across top-coded records in each of 12 demographic cells for each earnings source (some cells empty or collapsed)|
|Other Income Top-Codes||Larger of 97th percentile of dollar values or 99.5th percentile of persons 15 or over (whether or not have income)-Value when top-coded is average across top-coded recordsNo top-codes for Social Security, SSI, TANF, unemployment benefits, Workers Compensation and Veterans payments||Larger of 97th percentile of dollar values or 99.5th percentile of persons 15 or over (whether or not have income)-Value when top-coded is top-codeNo top-codes for Social Security, SSI, TANF, unemployment benefits, Workers Compensation and Veterans payments|
|Poverty Status Family and Date||Persons related by blood, marriage or adoption (CPS family)-Interview month (usually March) an average of 3 months after income reference year||Persons related by blood, marriage or adoption (CPS family)-Tabulations use December for construction of family units and consequent family composition and income amounts|
|Poverty Status in Paper||2002 poverty status of CPS family as of the following March (usually) based on CPS income for year||December, 2002 based on pre-tax money income for year|
|Weights and Date||Person, family and household weights for subsequent March for all data collection months||Person, family and household weights for each month, year, and full panel|
|Universes||Anyone with person weight has family and household weights – 3 universes the same||Anyone with person weight has family and household weights – 3 universes the same|
|Person Control Totals||Age, sex, race/ethnicity, and State of residence||Age, sex, race/ethnicity, and marital and family status of householder|
|Family Control Totals||None||None|
|Income Control Totals||None||None|
|Medical Expenditure Panel||National Health Interview Survey|
|Earnings Top-Codes||Wages and salary and self-employment each top-coded at 99th percentile – amount not documented-Top-coded amounts replaced with a “smeared” value||Total earnings top-coded at $75,000+ bracketValue if top-coded is $75,000+ bracket|
|Other Income Top-Codes||Other income sources top-coded at the 99th percentile-Person totals separately top-coded at 99th percentile-Top-coded amounts replaced with a “smeared” value||Family income top-coded at $75,000+ bracket-Value when top-coded is $75,000+ bracket, which includes 28% of persons on 2003 (2002 income) file|
|Poverty Status Family and Date||Persons related by blood, marriage or adoption (CPS family)-Alternative NHIS family also calculated December 31 of income reference year||Persons related by blood, marriage or adoption, including foster relationships and unmarried (opposite or same sex) partners (NHIS family)1 to 12 months after income reference year (rolling sample)|
|Poverty Status in Paper||Calendar year poverty status of CPS-type family as of December 31 based on pre-tax money income for 2002-Calendar year poverty status of NHIS-type family as of December 31 based on pre-tax money income for 2002||Bracket for 2002 poverty status of NHIS-type family as of interview month based on pre-tax money income for year Brackets on public use file are 25% wide below 200% of poverty and 50% wide from 200 to 500% of poverty|
|Weights||Person, family and household weights for December of calendar year and full panel||Person, family and household weights for calendar quarter of interview -- average across quarters is effectively mid-June|
|Universes||Person universe restricted to original NHIS sample personsCPS-type family universe excludes part of person universe but includes move-ins related by blood or marriage (meeting CPS family definition)-Broader NHIS-type family universe adds unmarried partner move-ins and others (meeting NHIS family definition) to CPS-type family universe||Anyone with person weight has family and household weights – 3 universes the same Some families with household weights are refusals but family weights not adjusted to compensate|
|Person Control Totals||Age, sex, race/ethnicity, Census Region, and MSA/non-MSA (and income)||Age, sex, and race/ethnicity-Weights prior to post-stratification also on file|
|Family Control Totals||Family type (spouse present or not), family size, age, sex, and race/ethnicity of reference person, MSA/non-MSA, and region for CPS-type familiesCPS has no family control totals so family counts depend on CPS method of calculating family weights note 2||None|
|Income Control Totals||CPS poverty rates for persons in CPS-type families as of December 31 – crossed with demographic control totals when person weights calculated||None|
2. The CPS family weight is the person weight of the reference person after a family equalization process that ensures husbands and wives have the same weights while overall age, sex, and race/ethnicity control totals are maintained. The method of family equalization depends on household composition and the sex of the reference person: either the householder weight is used for the spouse or partner, or the weights of the householder and spouse or partner are averaged, or a separate ratio adjustment is calculated.
5. Appendix A gives the variables used for recipiency and amounts for each income source for each survey. Exhibits 1 to 3 in Appendix B summarize survey differences in periodicity, question detail, recall interval, control totals and weighting, as well as definitional and other differences affecting income data.
7.CPS money income excludes lump sums, including irregular withdrawals from IRAs, that are included in SIPP and MEPS. A small amount ($3.3 billion) of IRA distributions were reported in the CPS but are excluded in this paper. Actual IRA distributions net of rollovers in CY2002 were $123.3 billion. (Bryant and Sailer (2006). “Accumulation and Distribution of Individual Retirement Arrangements, 2001-2002", Statistics of Income Bulletin, Spring, 2006. Internal Revenue Service, Washington, D.C.)
8. Weighted item non-response rates in NHIS for adults range from 2.6 percent for wages and salaries to 4.2 percent for “all other” income sources, except for interest and dividends, which have item non-response rates of about seven percent.
11. There are data for these 6.3 million persons, but not for one or more other family members; in fact, 3 million persons with data are in families with no data for the family reference person. The number and income of the 6.3 million persons can be determined but not the size or income of the complete family. MEPS public use file poverty status for these persons does not and cannot take into account family members with no data, and is based only on the family members present on the file.
13. In MEPS, only original sample persons have person weights, not move-ins joining MEPS families after sample selection. Both have NHIS-type family weights, and most have CPS-type family weights, except move-ins not related by blood or marriage. As measured by family weights, there are 10.4 million in the CPS-type family and 13.0 million in the NHIS-type family universes who are not in the person universe. In addition, the 6.4 million original sample persons in families of “undefined size” have no family weights, so are in the person but not family universe.
18. Poverty status was independently calculated for all persons based on family income and size, and matched poverty codes on the public use files for CPS and SIPP but not MEPS. The MEPS algorithm for poverty status apparently bases family size and income on persons ever in the family, including those deceased, rather than those present December 31. As a result, MEPS and CPS poverty rates do not quite match, although MEPS person weights are adjusted to match CPS poverty rates.