Studies of Welfare Populations: Data Collection and Research Issues. Data Used in Leaver Studies

06/01/2002

Studies of welfare leavers rely heavily on two types of data: state administrative records and direct surveys of welfare leavers.(3) Each source can provide valuable but limited information about some aspects of the well-being of welfare leavers.

Administrative Data

Twenty-one of the 49 leaver studies we review use administrative data as shown in Table 12-4. States have data systems used in administering programs, such as TANF, and these databases can be used in conducting leaver studies. Typically state welfare program data can provide information on the timing of receipt of welfare benefits, the value of the grant, the number of people (adults and children) in the case, as well as some demographic characteristics of recipients, usually race, age, number and ages of children, and whether a case is single parent or two parent. Of course, availability of TANF data is critical to conducting a leaver study because the data allow one to define who is a leaver. In addition, this information can be used to determine who among a group of leavers returns to welfare and to develop some basic characteristics for conducting subgroup analysis. One also can examine records on participation prior to the month of exit to assemble a history of receipt.(4) This information also can be used to analyze subgroups based on being a long-term or short-term recipient, although none of the studies we review have carried out such a subgroup analysis.

 

TABLE 12-4
Studies Using Administrative Data
State Exit Cohort Period of Follow up After Exit Programs Covered(a)
Arizona-1 1Q98 1 year Employment, Temporary Assistance for Needy Families (TANF), FoodStamps, childcare subsidy, childsupport, child welfare
Arizona-2 4Q96 1 year Employment, TANF, Food Stamps, Medicaid
California-Los Angeles Co. 3Q96 1 year Employment
California-San Mateo Co. 1997 1 year Employment, TANF, Food Stamps, Medicaid
District ofColumbia 4Q98 18 months TANF, Food Stamps, Medicaid
Florida c 3 years Employment, TANF, Food Stamps
Georgia-1 1997 1 year Employment, TANF
Georgia-2 IQ97 1 year Employment, TANF
Illinois-2 3Q97-4Q98 1 year Employment, TANF, Food Stamps, Medicaid, the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), childcare subsidy, family case management services, drug and alcohol treatment services, child support, child welfare(b)
Maryland-1 October 1996-September 1997 1 year Employment, TANF
Maryland-2 October 1996-September 1997 2 years Employment, TANF
Maryland-3 October 1996-March 1998 18 months Employment, TANF
Mississippi IQ98 6 months Employment, TANF
Missouri-1 4Q96 2 years Employment, TANF, Food Stamps, Medicaid
New York-2 1Q97 1 year Employment, TANF, Food Stamps, Medicaid
North Carolina-1 September 1996(d) 30 months Employment, TANF, Food Stamps
Ohio-Cuyahoga Co. I 1996 1 year TANF
Ohio-Cuyahoga Co. 2 3Q96 1 year Employment, TANF, Food Stamps, Medicaid
Washington-4 4Q97 2 years Employment, TANF, Food Stamps, Medicaid, childcare subsidy, child support programs, child welfare(b)
Wisconsin-1 July 1995-1996 15 months Employment, TANF, Food Stamps, Medicaid
Wisconsin-2 n.a. n.a. Employment, TANF, Food Stamps, Medicaid
a TANF refers to cash assistance. For studies that predate the implementation of TANF, the use of the term TANF in the table indicates Aid to Families with Dependent Children (AFDC) cash assistance.
b Child abuse and neglect referrals and out-of-home placements. Women, Infants, and Children (WIC).
c The AFDC Component exited in February 1995.
d Florida uses a TANF Cohort instead of an exit cohort. The study chose a random sample of people who began receiving TANF benefits with the implementation of TANF. The study tracks their employment, TANF, and Food Stamp history over three years.

State program data also can include information on participation in the Food Stamp and Medicaid programs linked to TANF program data. Table 12-4 shows that the majority of study areas (9 out of 15) have a study that includes both of these sources of data. Other types of program data also may be available to be linked to TANF data. Only three of the studies listed here have made use of additional program data. Examples of the types of data they examine include childcare subsidies, receipt of child support payments, and involvement in the child welfare system. Information from such programs provides a richer description of the well-being of leavers.

By their nature, program data do not contain information on families who no longer receive program benefits. Consequently, there is no way to determine if leavers who do not return to the caseload and are not participating in other programs from which data are available are finding jobs. To address this problem, many leaver studies use additional administrative data, linking their welfare program records to data from state unemployment insurance (UI) systems. If a leaver is working for an employer that reports wages to the state UI system, then these linked records can reveal whether a leaver is working in a given quarter and how much that leaver earned. Because the employment and earnings of welfare leavers are a key outcome for policy makers and researchers, linking administrative data from the welfare system with data from the state UI system is vital. Nineteen of the 21 studies link their program data with state UI data.

Note that using administrative data to assess the status of welfare leavers often requires researchers to link information across various data systems. In general, researchers use Social Security numbers to link information on welfare leavers with information from other sources such as UI earnings records. If there is a discrepancy in an individual's Social Security number across data systems, then no match can be made. Goerge and Lee (this volume: Chapter 7) provide a detailed discussion of techniques that can be used to improve the quality of matched data between administrative data systems.

Overall, the greatest strength of administrative data is that they provide accurate information on program participation for all leavers who continue to reside in the state. Information on employment and earnings from UI records also is reliable; however, leavers who work outside the state or in jobs that do not generate UI wage reports(5) will not be picked up in a state's UI system.(6) Thus, administrative data on employment probably understate employment among leavers. The greatest weakness of administrative data is their failure to provide information on many aspects of well-being and changes in family structure. Thus, they provide a limited picture of the status of TANF leavers.

Survey Data

Surveys of welfare leavers are particularly good at obtaining information that is beyond the scope of administrative data systems. For example, in addition to employment and wage information, a survey can obtain data on job characteristics--nonwage benefits, training, and work-related expenses. Surveys also can elicit information on changes in a leaver's personal characteristics and household composition as well as what sort of hardships the leavers have faced. Furthermore, leavers can be surveyed even if they have moved across state lines. Thirty-two of the 49 leaver studies we review use data collected from surveys of welfare leavers. Features of the 32 studies are listed in Table 12-5.

 

TABLE 12-5
Leaver Studies Using Surveys
State/Study Exit Cohort Timing of Survey Postexit Sample Size Response Rate (%) Type of Survey Respondents Paid
Leaver Studies:
Arizona 1Q98 12-18 months 821 72 Phone/in person Yes
District of Columbia 4Q98 1 year 277 61 Phone/in person Yes
Idaho-1 3rd and 4th Q97 6 months 477 17 Mail No
Idaho-2 3rd and 4th Q97 10 months 53 47 Mail No
Illinois-1 December 1997 or June 1998 4-11 months 427 31 Phone Yes
Illinois-2 December 1998 6-8 months 514 51 Phone/in person Yes
Kentucky January-November 1997 1-11 months 560 17 Phone No
Massachusetts 1st and 2nd Q97 3 months(a) 341 53 In person Yes
Michigan July 1998 12 months 126 85 In person No
Mississippi IQ98 6 months 405 87 Phone/mail/inperson No
Missouri-2 4Q98 30 months 878 75 Phone/in person Yes
Montana March 1996- September 1997 1-18 months 208 (c) Phone No
North Carolina-2 July 1998 5 months 315 77 Phone Yes
New Jersey February-October 1998 n.a. 453 45 In person No
New Mexico July 1996-June 1997 n.a. 88 12 Mail No
New York-1 November 1997 6 months 126 22 Phone No
Oklahoma October 1996-November 1997 2-20 months 292 53 Phone Yes
Pennsylvania March 1997-January 1998 1-11 months 169 47 Phone No
South Carolina-1 n.a. n.a. 2,002(b) 77 Phone/in person No
South Carolina-2 2Q97 1 year 391 76 Phone/in person No
South Carolina-3 3Q97 I year 403 76 Phone/in person No
Tennessee n.a. n.a. 2,500 51 Phone No
Texas November 1997 6 months 1,396 42 Phone/mail No
Virginia n.a. n.a. 171 46 Phone No
Washington-1   n.a. 65 C In person No
Washington-2 December 1997-March 1998 12-18 months 560 31 Phone No
Washington-5 October 1998 6 months 987 72 Phone/in person Yes
Wisconsin-3 1Q98 6-9 months 375 69 (d) No
Wyoming n.a. n.a. 200 32 Phone No
Caseload Studies:(e)
Indiana n.a. n.a. 847 71 Phone/in person No
Iowa n.a. n.a. 162 85 In person No
Washington-3 n.a. n.a. 592 52 Phone No
a This study surveyed respondents every 3 months for a year. The study includes the results of the interviews at months 3 and 12.
b This study is an analysis of five surveys performed in South Carolina. These five surveys have a total sample of 2002 cases. The overall response rate of these five surveys was 77 percent.
c (Response rate not reported.
d Survey mode not described.
e These studies took a random sample of people who began receiving benefits when Temporary Assistance for Needy Families (TANF) was implemented in the state. At the time of the survey, these recipients may or may not have been receiving TANF benefits.

Surveys of welfare leavers generally collect information on a sample of families who left TANF during a specific timeframe by interviewing them months after their exit. The choice of how long after exit to interview respondents has advantages and disadvantages. The sooner the time period is to the exit from welfare, the more able a recipient is to recall information on the circumstances around leaving, such as reason for leaving and specifics of his or her first job. The later the interview takes place from the exit, the more information about a family's transition can be gathered. The actual range of time of the interview after exiting in these studies varies from 3 months (Massachusetts) to 30 months (Missouri).

Most studies gather survey information using telephone interviews, but many also conduct some in-person interviews. This combination method ensures that leavers without telephones are included in the study. Three studies (two from Idaho and one from New Mexico) used mail surveys; this method is not recommended because the common problems with all surveys (described as follows) are magnified in mail surveys.

Overall, the strength of survey data is the breadth of information they contain. However, survey data have their own shortcomings. First, surveys rely on respondents to answer questions accurately and truthfully.(7) Second, survey data are collected for only a sample of welfare leavers; therefore, any assessment of the well-being of leavers based on surveys is subject to sampling error. Finally, and potentially most seriously, even if the sample of leavers accurately reflects all leavers, not all sampled families will respond to the survey. That is, a researcher only will be able to contact and interview a subset of the original sample. If the leavers who respond to the survey are very different from the nonrespondents, then the survey data will suffer from nonresponse bias and not accurately represent the status of leavers. The best way to reduce nonresponse bias is to have a high response rate. A large literature is available on increasing response rates (see Cantor and Cunningham, this volume: Chapter 2; Singer and Kulka, this volume: Chapter 4, and Weiss and Bailar, this volume: Chapter 3). (See Table 12-5 for response rates in the leaver studies examined here.)

View full report

Preview
Download

"01.pdf" (pdf, 472.92Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"02.pdf" (pdf, 395.41Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"03.pdf" (pdf, 379.04Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"04.pdf" (pdf, 381.73Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"05.pdf" (pdf, 393.7Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"06.pdf" (pdf, 415.3Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"07.pdf" (pdf, 375.49Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"08.pdf" (pdf, 475.21Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"09.pdf" (pdf, 425.17Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"10.pdf" (pdf, 424.33Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"11.pdf" (pdf, 392.39Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"12.pdf" (pdf, 386.39Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"13.pdf" (pdf, 449.86Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"14.pdf" (pdf, 396.87Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®