Families on TANF in Illinois: Employment Assets and Liabilities. Assessment of the Weighted Survey Respondents

06/10/2003

We assessed the representativeness of the weighted sample of survey respondents by comparing them with the entire 33,495 cases in the sampling frame. The comparisons were based on the same 12 characteristics that we used to test for differences between survey respondents and nonrespondents. Those include demographic characteristics of grantees and cases as of November 2001, the TANF benefit amount in November 2001, and employment and earnings in 2000 and 2001. Characteristics that were expressed as percentages in the comparison of survey respondents and nonrespondents are expressed as case counts in this analysis. Thus, we are comparing the weighted count of survey respondents that possess the selected characteristic with the count of cases in the sampling frame that possess the characteristic. The results of the tests are presented in Table A.5.

Table A.5
Comparison of Weighted Survey Respondents with All Members of the Sampling Frame for MPRS 2001-02 Survey of Single-Parent TANF Cases in Illinois
Measure Weighted Survey Respondents (a) Entire Sampling Frame (b) Difference (c=ab) Error Rate (100Xc/b)
Case Heads
Number female 33,308 32,954 354 1.1%
Number age 28 or younger 17,344 17,499 -155 -0.9%
Number African American, non-Hispanic 27,594 27,305 289 1.1%
Number never married 27,396 28,028 -632 -2.3%
Number w/o high school diploma/GED 16,566 16,517 49 0.3%
Number employed at least 1 qtr., 2000-01 26,218 25,624 594 2.3%
Mean annualized earnings per head, 2000-01 $3,066 $3,015 $51 1.7%
Cases
Number living in Cook County 26,913 27,123 -210 -0.8%
Number with $0 TANF benefit, 11/01 2,859 2,859 0 0.0%
Mean TANF benefit per case, 11/01 $237.23 $240.80 -$3.57 -1.5%
Mean number of children per case 2.34 2.46 -0.13 -5.3%
Mean duration of TANF spell per case (mos) 14.68 15.46 -0.78 -5.0%
Number of TANF Cases 416 33,495    
Source: Administrative data on the TANF caseload maintained by the Illinois Department of Human Services and Unemployment Insurance records maintained by the Illinois Department of Employment Security. Characteristics were measured in November 2001, unless otherwise indicated.
Frame: All TANF cases in Illinois that: (1) were eligible for a cash grant in November 2001, (2) were classified by DHS as single-parent, and (3) included the grantee (i.e., the person to whom the benefit was issued) as a member of the case (thus excluding child-only cases). Some of the cases (9 percent) in the sampling frame did not receive a positive cash grant despite being classified as eligible for a cash grant (see Table C.4 for reasons).
Sample: A stratified random sample was selected from the sampling frame. The sampling strata were Cook County and downstate (all areas outside of Cook County). Cases were selected into the sample at the same rate in the two strata, resulting in a sample of 532 cases, of which 416 (78 percent) responded to the survey. Survey weights were developed to adjust for nonresponse and sampling error. The weighted count of survey respondents is 33,495, which is the number of cases in the sampling frame.

Table A.5 shows a high degree of correspondence between the weighted survey respondents and all of the cases in the sampling frame. For ten of the twelve characteristics considered the weighted count of survey respondents differs from the count of all cases in the sampling frame by 2 percent or less. Most notably, the difference with respect to the grantee characteristic, "age 28 or younger," is just one percent. Recall that this is the characteristic on which survey respondents and nonrespondents differ most sharply. The survey weights virtually eliminate the effect of nonrandomness in survey response along this dimension.

The survey weights mitigate, but fall short of eliminating, the effects of nonrandomness in survey response with respect to two characteristics of TANF cases: the number of children per case and the duration of the current spell on TANF. For both of these characteristics the error rate in the sample of weighted respondents is 5 percent relative to all cases in the sampling frame. These error rates, while not trivial, are less than half of the relative difference in these characteristics between survey respondents and nonrespondents.

To summarize, the weighted survey respondents resemble very closely the cases in the sampling frame. This correspondence is a consequence of a high survey response rate, the general absence of large systematic differences between survey respondents and nonrespondents, and the survey weights. Consequently, we can be confident in drawing inferences about the sampling frame from statistics computed on the basis of the survey data.

Endnotes

1. The routine TANF benefit issuance cycle for November 2001 extended from October 19 to November 8. The sampling frame was identified upon the completion of benefit issuance on November 8. Four factors accounted for 95 percent of the zero benefit cases: recoupment of prior overpayments, failure to cooperate with eligibility determination, participation in Illinois' Work First program, and sanctioning.

2. A case whose November TANF benefit was processed prior to November 8, 2001, may have closed by the time the sampling frame was identified following completion of routine benefit processing on November 8. We excluded cases such as this (recent TANF leavers) from the sampling frame in order to maximize the proportion of survey respondents that were current TANF recipients when interviewed.

3. For coding industry responses we used the 1987 Standard Industrial Classification Manual. For coding occupation responses we used the 2000 Standard Occupational Classification Manual.

4. The characteristics were obtained from TANF administrative records for November 2001 maintained by the Illinois DHS and from Unemployment Insurance earnings records for the years 2000 and 2001 maintained by the Illinois Department of Employment Security.

5. The six cells and their associated survey weights are: 1) zero TANF benefit, Cook County, 95.96; 2) zero TANF benefit, downstate, 93.48; 3) positive TANF benefit, Cook County, less than or equal to age 28, 69.17; 4) positive TANF benefit, Cook County, greater than age 28, 93.60; 5) positive TANF benefit, downstate, less than or equal to age 28, 70.38; 6) positive TANF benefit, downstate greater than age 28, 98.24.

View full report

Preview
Download

"report.pdf" (pdf, 1.55Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®