Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys. Income Data in a Policy-Analytic Context

12/23/2008

       The data requirements for policy analysis are not the same as the requirements for more general research. They are both different and more extensive. Whatever the issue being addressed, good income information for policy work is likely to require several additional qualities, outlined below.

In policy analyses, income is often used to determine potential eligibility for a new or existing program. Program benefits or charges to the participant may vary with income. Understanding the impact at different points in the income distribution is important. As a result, policy work requires income expressed in dollar amounts, not fixed income brackets.

The concept of poverty has an official definition and an official source of measurement in the CPS. Poverty status is important in policy evaluation and in public debate and, therefore, must be expressed on the same basis as is done in official statistics. Departures from official concepts may be useful for a variety of purposes, but they need to be tied back to the official statistics.

Policy analysis of health issues usually requires data on health insurance status and utilization of health care. Analyses of means-tested programs frequently require estimates of earnings separately from total income, as earned income is often treated differently than unearned income. Policy analysis on issues affecting the elderly requires data on current retirement contributions and pension coverage.

Policy analysis may deal with individuals or part of a family, and may compare different filing units. To analyze units below the family level requires income data for each person. Health insurance provides a relevant example—the filing unit may be children up to a certain age based on family income, as in the State Children’s Health Insurance Program (SCHIP); or a worker, spouse, and dependent children, under a private health insurance plan; or it may include children outside the household in the case of divorce. Frequently, the construction of filing or eligibility units is one of the most challenging aspects of policy analysis and one that is exacerbated by limitations of the data.

Weaknesses in the data underlying policy proposals and cost estimates bring the validity of an initiative into question, even though such weaknesses may not be directly relevant to the estimates. Significant inconsistencies within a survey provide the basis for challenges to the proposals themselves based on the unreliability of the estimates. Differences with alternative estimates of totals on which surveys should seemingly agree invite challenges as well. For example, while the nature of population estimation provides some leeway (the Census Bureau revises historical population estimates each year), significant differences in population totals can hurt credibility.

Efficient policy design requires detail on benefits or insurance coverage already in place and, along with it, the administrative systems with which persons already interact. Participation in Social Security, Medicare, Supplemental Security Income (SSI), Food Stamps, Medicaid, and welfare is particularly important.

Typically the policy process has tight time frames, particularly when legislation is being written or negotiated or a vote is impending. Unexpected developments growing out of the need to secure votes or satisfy specific constituencies may require new analyses—sometimes with substantial changes—with very quick turnaround times. Immediate access to the data on which the analyses are based is critical, as is the ability to conduct needed analyses without restriction.

With regard to the income data specifically, accuracy at the lower end of the income distribution is more important than accuracy at the upper end. Income measures that are of particular significance include the number and composition of the poor and near-poor and the magnitudes of key income sources, such as earnings and program benefits. In addition to income, employment has consistently been an area of policy concern, both as a source of self-support and the source of most health insurance coverage. Accuracy in the measurement of employment is also critical. Lastly, randomness as measured by standard errors is not nearly as important as possible bias. The findings of policy analysis, and budget estimates, are presented as point estimates without standard errors. Bias, on the other hand, leads to consistent over- or under-estimates.

View full report

Preview
Download

"report.pdf" (pdf, 4.33Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®