Families on TANF in Illinois: Employment Assets and Liabilities. Comparison of Survey Respondents and Nonrespondents

06/10/2003

In a survey that achieves less than a 100 percent response rate there is a risk that respondents may be systematically different from nonrespondents. Such differences would imply that the respondents should not be regarded as a random subsample of the full survey sample. If the survey data are not adjusted to mitigate these differences, such as by weighting the survey respondents, it may be inappropriate to draw inferences about the sampling frame from statistics computed on the basis of the survey data.

To assess whether the sample members who responded to the survey are different from those who did not respond, we regressed each of 12 characteristics of the case or case head (i.e., the TANF grantee) on a 0/1 variable that indicates the sample member's response status: 1 for respondents and 0 for nonrespondents.(4) The regression coefficient on the indicator variable is the difference between the two groups in a characteristic. We conducted a t-test to determine whether the estimate of the coefficient is significantly different from zero for each of the 12 characteristics. The results of that analysis are presented in Table A.2.

Table A.2 shows significant differences between survey respondents and nonrespondents in only three of the twelve selected characteristics. Cases with grantees who were 28 years old or less were more likely to participate in the survey than were cases with older grantees. Consistent with that difference, respondent cases tended to have fewer children and to have been on TANF for fewer consecutive months than nonrespondent cases.

To reduce the bias that systematic differences between survey respondents and nonrespondents might introduce to inferences based on the survey data, we developed survey weights that directly correct for the age difference. The development of the weights is described in the next section. We expected the weights to indirectly mitigate differences in age-related characteristics, such as the number of children on the case and the duration of the current TANF spell. Those expectations were realized, as discussed in the final section of this appendix.

Table A.2
Comparison of Respondents and Nonrespondents to MPRS 2001-02 Survey
of Single-Parent TANF Cases in Illinois
Measure Survey Group Difference
Respondents Non-Respondents Amount P Value
Case Heads        
Percentage female 99.5 98.3 1.2 0.171
Percentage age 28 or younger 58.9 44.0 14.9*** 0.004
Percentage African American, non-Hispanic 82.5 82.8 -0.3 0.939
Percentage never married 83.4 80.2 3.2 0.415
Percentage w/o high school dip./GED 50.0 47.4 2.6 0.623
Percentage employed at least 1 qtr., 2000-01 78.3 77.6 0.7 0.867
Mean annualized earnings, 2000-01 $2,955 $3,406 -$451 0.319
Cases        
Percentage living in Cook County 80.5 82.8 -2.2 0.589
Percentage with $0 TANF benefit, 11/01 7.2 6.0 1.2 0.660
Mean TANF benefit, 11/01 $241.34 $258.17 -$16.83 0.222
Mean number of children 2.28 2.55 -0.27* 0.098
Mean duration of TANF spell (mos.) 14.40 17.11 -2.71*** 0.004
Number of TANF Cases 416 116    
Source: Administrative data on the TANF caseload maintained by the Illinois Department of Human Services and Unemployment Insurance records maintained by the Illinois Department of Employment Security. Characteristics were measured in November 2001, unless otherwise indicated.
Sample: Stratified random sample of TANF cases in Illinois that: (1) were eligible for a cash grant in November 2001, (2) were classified by DHS as single-parent, and (3) included the grantee (i.e., the person to whom the benefit was issued) as a member of the case (thus excluding child-only cases). Some of the cases (9 percent) in the study population did not receive a positive cash grant despite being classified as eligible for a cash grant (see Table C.4 for reasons). The sampling strata were Cook County and downstate (all areas outside of Cook County). Cases were selected into the sample at the same rate in the two strata.
*/**/*** Significantly different from zero at the .10/.05/.01 level.

View full report

Preview
Download

"report.pdf" (pdf, 1.55Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®