We assessed the representativeness of the weighted sample of survey respondents by comparing them with the entire 33,495 cases in the sampling frame. The comparisons were based on the same 12 characteristics that we used to test for differences between survey respondents and nonrespondents. Those include demographic characteristics of grantees and cases as of November 2001, the TANF benefit amount in November 2001, and employment and earnings in 2000 and 2001. Characteristics that were expressed as percentages in the comparison of survey respondents and nonrespondents are expressed as case counts in this analysis. Thus, we are comparing the weighted count of survey respondents that possess the selected characteristic with the count of cases in the sampling frame that possess the characteristic. The results of the tests are presented in Table A.5.
|Measure||Weighted Survey Respondents (a)||Entire Sampling Frame (b)||Difference (c=ab)||Error Rate (100Xc/b)|
|Number age 28 or younger||17,344||17,499||-155||-0.9%|
|Number African American, non-Hispanic||27,594||27,305||289||1.1%|
|Number never married||27,396||28,028||-632||-2.3%|
|Number w/o high school diploma/GED||16,566||16,517||49||0.3%|
|Number employed at least 1 qtr., 2000-01||26,218||25,624||594||2.3%|
|Mean annualized earnings per head, 2000-01||$3,066||$3,015||$51||1.7%|
|Number living in Cook County||26,913||27,123||-210||-0.8%|
|Number with $0 TANF benefit, 11/01||2,859||2,859||0||0.0%|
|Mean TANF benefit per case, 11/01||$237.23||$240.80||-$3.57||-1.5%|
|Mean number of children per case||2.34||2.46||-0.13||-5.3%|
|Mean duration of TANF spell per case (mos)||14.68||15.46||-0.78||-5.0%|
|Number of TANF Cases||416||33,495|
|Source: Administrative data on the TANF caseload maintained by the Illinois Department of Human Services and Unemployment Insurance records maintained by the Illinois Department of Employment Security. Characteristics were measured in November 2001, unless otherwise indicated.
Frame: All TANF cases in Illinois that: (1) were eligible for a cash grant in November 2001, (2) were classified by DHS as single-parent, and (3) included the grantee (i.e., the person to whom the benefit was issued) as a member of the case (thus excluding child-only cases). Some of the cases (9 percent) in the sampling frame did not receive a positive cash grant despite being classified as eligible for a cash grant (see Table C.4 for reasons).
Sample: A stratified random sample was selected from the sampling frame. The sampling strata were Cook County and downstate (all areas outside of Cook County). Cases were selected into the sample at the same rate in the two strata, resulting in a sample of 532 cases, of which 416 (78 percent) responded to the survey. Survey weights were developed to adjust for nonresponse and sampling error. The weighted count of survey respondents is 33,495, which is the number of cases in the sampling frame.
Table A.5 shows a high degree of correspondence between the weighted survey respondents and all of the cases in the sampling frame. For ten of the twelve characteristics considered the weighted count of survey respondents differs from the count of all cases in the sampling frame by 2 percent or less. Most notably, the difference with respect to the grantee characteristic, "age 28 or younger," is just one percent. Recall that this is the characteristic on which survey respondents and nonrespondents differ most sharply. The survey weights virtually eliminate the effect of nonrandomness in survey response along this dimension.
The survey weights mitigate, but fall short of eliminating, the effects of nonrandomness in survey response with respect to two characteristics of TANF cases: the number of children per case and the duration of the current spell on TANF. For both of these characteristics the error rate in the sample of weighted respondents is 5 percent relative to all cases in the sampling frame. These error rates, while not trivial, are less than half of the relative difference in these characteristics between survey respondents and nonrespondents.
To summarize, the weighted survey respondents resemble very closely the cases in the sampling frame. This correspondence is a consequence of a high survey response rate, the general absence of large systematic differences between survey respondents and nonrespondents, and the survey weights. Consequently, we can be confident in drawing inferences about the sampling frame from statistics computed on the basis of the survey data.
1. The routine TANF benefit issuance cycle for November 2001 extended from October 19 to November 8. The sampling frame was identified upon the completion of benefit issuance on November 8. Four factors accounted for 95 percent of the zero benefit cases: recoupment of prior overpayments, failure to cooperate with eligibility determination, participation in Illinois' Work First program, and sanctioning.
2. A case whose November TANF benefit was processed prior to November 8, 2001, may have closed by the time the sampling frame was identified following completion of routine benefit processing on November 8. We excluded cases such as this (recent TANF leavers) from the sampling frame in order to maximize the proportion of survey respondents that were current TANF recipients when interviewed.
3. For coding industry responses we used the 1987 Standard Industrial Classification Manual. For coding occupation responses we used the 2000 Standard Occupational Classification Manual.
4. The characteristics were obtained from TANF administrative records for November 2001 maintained by the Illinois DHS and from Unemployment Insurance earnings records for the years 2000 and 2001 maintained by the Illinois Department of Employment Security.
5. The six cells and their associated survey weights are: 1) zero TANF benefit, Cook County, 95.96; 2) zero TANF benefit, downstate, 93.48; 3) positive TANF benefit, Cook County, less than or equal to age 28, 69.17; 4) positive TANF benefit, Cook County, greater than age 28, 93.60; 5) positive TANF benefit, downstate, less than or equal to age 28, 70.38; 6) positive TANF benefit, downstate greater than age 28, 98.24.