In a survey that achieves less than a 100 percent response rate there is a risk that respondents may be systematically different from nonrespondents. Such differences would imply that the respondents should not be regarded as a random subsample of the full survey sample. If the survey data are not adjusted to mitigate these differences, such as by weighting the survey respondents, it may be inappropriate to draw inferences about the sampling frame from statistics computed on the basis of the survey data.
To assess whether the sample members who responded to the survey are different from those who did not respond, we regressed each of 12 characteristics of the case or case head (i.e., the TANF grantee) on a 0/1 variable that indicates the sample member's response status: 1 for respondents and 0 for nonrespondents.(4) The regression coefficient on the indicator variable is the difference between the two groups in a characteristic. We conducted a t-test to determine whether the estimate of the coefficient is significantly different from zero for each of the 12 characteristics. The results of that analysis are presented in Table A.2.
Table A.2 shows significant differences between survey respondents and nonrespondents in only three of the twelve selected characteristics. Cases with grantees who were 28 years old or less were more likely to participate in the survey than were cases with older grantees. Consistent with that difference, respondent cases tended to have fewer children and to have been on TANF for fewer consecutive months than nonrespondent cases.
To reduce the bias that systematic differences between survey respondents and nonrespondents might introduce to inferences based on the survey data, we developed survey weights that directly correct for the age difference. The development of the weights is described in the next section. We expected the weights to indirectly mitigate differences in age-related characteristics, such as the number of children on the case and the duration of the current TANF spell. Those expectations were realized, as discussed in the final section of this appendix.
|Percentage age 28 or younger||58.9||44.0||14.9***||0.004|
|Percentage African American, non-Hispanic||82.5||82.8||-0.3||0.939|
|Percentage never married||83.4||80.2||3.2||0.415|
|Percentage w/o high school dip./GED||50.0||47.4||2.6||0.623|
|Percentage employed at least 1 qtr., 2000-01||78.3||77.6||0.7||0.867|
|Mean annualized earnings, 2000-01||$2,955||$3,406||-$451||0.319|
|Percentage living in Cook County||80.5||82.8||-2.2||0.589|
|Percentage with $0 TANF benefit, 11/01||7.2||6.0||1.2||0.660|
|Mean TANF benefit, 11/01||$241.34||$258.17||-$16.83||0.222|
|Mean number of children||2.28||2.55||-0.27*||0.098|
|Mean duration of TANF spell (mos.)||14.40||17.11||-2.71***||0.004|
|Number of TANF Cases||416||116|
|Source: Administrative data on the TANF caseload maintained by the Illinois Department of Human Services and Unemployment Insurance records maintained by the Illinois Department of Employment Security. Characteristics were measured in November 2001, unless otherwise indicated.
Sample: Stratified random sample of TANF cases in Illinois that: (1) were eligible for a cash grant in November 2001, (2) were classified by DHS as single-parent, and (3) included the grantee (i.e., the person to whom the benefit was issued) as a member of the case (thus excluding child-only cases). Some of the cases (9 percent) in the study population did not receive a positive cash grant despite being classified as eligible for a cash grant (see Table C.4 for reasons). The sampling strata were Cook County and downstate (all areas outside of Cook County). Cases were selected into the sample at the same rate in the two strata.
*/**/*** Significantly different from zero at the .10/.05/.01 level.