Families on TANF in Illinois: Employment Assets and Liabilities. Limitations of the Study

06/10/2003

The findings presented in this report have several potential limitations common to studies based on data from sample surveys and state administrative records. The limitations related to the survey findings include bias resulting from nonresponse by some sample members, recall errors in responses to survey questions, and misreporting in responses to sensitive questions. Despite our strong efforts to minimize the incidence of these problems, 22 percent of the sample members did not complete an interview, either because they were not reached at all or because they ended the interview before completion. In addition, some sample members who did complete an interview undoubtedly provided erroneous answers to certain questions. We adjusted for survey nonresponse by weighting the respondents up to the full study population on the basis of characteristics obtained from state TANF administrative records. However, the weights are based on just three characteristics (county of residence, age, and receipt of a positive TANF grant), and the weighted survey respondents may differ from the full population in other characteristics.

The limitations related to the findings based on data from TANF records, Unemployment Insurance earnings records, and criminal records arise from the absence of data on out-of-state activities and incomplete coverage of in-state activities (e.g., some jobs are not covered by the Unemployment Insurance system, and information on some arrests and convictions is not forwarded to database administrators). Limitations in administrative data may also arise when clients misreport information to authorities (e.g., misreporting of educational attainment or marital status by case heads to TANF caseworkers).

Nevertheless, the incidence and magnitude of these limitations are likely to be no greater than in other similar studies of welfare populations. Furthermore, our survey-based findings are likely to be more reliable than the norm because of a short survey field period (which reduced the risk of recall error), a high response rate, and the use of survey weights to adjust for the nonresponse that did occur.

View full report

Preview
Download

"report.pdf" (pdf, 1.55Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®