Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys. Issues Raised by Rolling Samples


The different routes to income measurement taken by the ACS and NHIS raise questions about policy relevance and data quality. From a policy perspective, it would be desirable for the income reference period to align as closely as possible with the reference period for other policy-relevant variables, such as health insurance coverage, health status, health care utilization, and program participation. In the NHIS, key health policy variables refer to the time of the survey or the past 12 months, for the most part. However, from the perspective of data quality it would be better to ask the annual income question for whatever reference period respondents can more easily address, and for policy uses it is better to have a reference period that is aligned with official poverty estimates. Faced with a difficult task, respondents may give lower quality responses or mentally change the question to something they can more readily answer. It has been suggested, for example, that the poor measurement of health insurance coverage in the CPS arises from the difficulty of the task that respondents are being asked to perform. With respect to income measurement explicitly, the fact that the statements of annual income supplied by financial institutions refer to the previous calendar year suggests that respondents would find it easier to report their incomes for the prior calendar year than the past 12 months. Conventional wisdom has suggested that respondents are most aware of their income for the prior calendar year when they are engaged in pulling together the financial records needed to prepare their tax returns (for those who file). But for how long might the prior calendar year income remain salient? Will respondents be able to recall this income as easily in December of the following year as in the early part of the year?

One way to approach assessing the difficulty that respondents face in dealing with a rolling reference period or a fixed reference period but varying recall interval is to examine patterns of non-response. If respondents find it easier to report their incomes for the previous calendar year than for the past 12 months, then we ought to see a decline in response rates to the income questions as the interview date moves farther from the end of the calendar year. We explored this possibility with ACS data and obtained unexpected findings, which are reported in Chapter VI. Similarly, if respondents are challenged by a growing recall interval, then response rates to the income questions in NHIS ought to decline over the course of the survey year. We explored this question as well but found only a modest decline in response rates. 

View full report


"report.pdf" (pdf, 4.33Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®