Evaluating Two Welfare-to-Work Program Approaches: Two-Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites. Sample Sizes and Data Sources

12/01/1997

The findings in this report on participation in employment-related activities, program costs, and employment, earnings, and welfare impacts for single-parent AFDC recipients cover a two-year follow-up period. At this writing, two years of follow-up data are available only for those individuals randomly assigned to a research group through December 1992, while random assignment continued for an additional 6 to 13 months in the three sites examined in the report. The site samples thus represent 50 to 63 percent (depending on the site) of the eventual single-parent AFDC recipient samples that will be analyzed as part of the evaluation.(12)

The following paragraphs describe the data sources and the sizes of the samples examined for each type of analysis in the report. Appendix Table A.1 presents a complete breakdown of the sample sizes, by data source, site, and research group.

AFDC and Unemployment Insurance Administrative Records Data

Employment, earnings, and welfare impacts were computed using automated county and state AFDC administrative records and state unemployment insurance (UI) records data. AFDC and UI records were available for all 11,977 sample members for whom two years of follow-up were available.(13) The administrative records sample is depicted on Figure 2.4 by the largest circle and includes all sample members in this report.

Two-Year Client Surveys

Some client opinions and participation rates examined throughout the report are based on results compiled from a survey administered to a sample of individuals in all three research groups approximately two years after random assignment.(14) In Figure 2.4, the client survey sample is represented by the circle with horizontal lines. The survey sample was randomly selected from the larger report sample, but it intentionally oversampled certain subgroups to produce a large enough sample for special analyses to appear in later reports. The survey sample was thus a stratified, random sample. For this report, the survey sample was weighted to replicate the demographic characteristics of the entire report sample.

 

Figure 2.4:
Three-Site Schematic Depiction
of Quantitative Data Sources in the Evaluation.

Figure 2.4: Three-Site Schematic Depiction of Quantitative Data Sources in the Evaluation.

SOURCE: See Appendix Table A.1.

Survey respondents were asked about issues such as their participation in training and education activities, if they had received a GED or high school diploma in the past two years, their perceptions of the JOBS program, and their expectations for the future. Interviews included in this report were conducted with individuals randomly assigned between March 1992 and December 1992 in Atlanta and Grand Rapids and between September 1991 and December 1992 in Riverside. The responses of 1,389 sample members in Atlanta, 832 sample members in Grand Rapids, and 1,586 sample members in Riverside are included in this report.(15) Ninety-one percent of fielded surveys in Atlanta, 90 percent in Grand Rapids, and 75 percent in Riverside were completed.

JOBS and Income Maintenance Case File Data

Findings on the LFA and HCD patterns of participation in program activities presented in Chapters 5 and 6 are based on material collected from the review of the JOBS and income maintenance case files of 1,093 single-parent AFDC recipients randomly assigned to the two program groups in the three sites. Case file data were collected for a stratified, random subsample that was demographically representative of the entire report sample. As displayed in Figure 2.4, the case file sample (the circle with vertical lines) is, by and large, a subsample of the above Two-Year Client Survey sample. This overlapping group is represented in Figure 2.4 by the area with both vertical and horizontal lines.

In reviews of case files, MDRC staff recorded sample members' enrollment in activities, length of stay in JOBS, changes in JOBS-mandatory status, sanctions, and deferrals over a 24-month period(16) using standard coding procedures, so that welfare recipients' actions and statuses could be compared across the sites and research groups. Case file documents consulted included standard program forms, case notes, and correspondence between the individuals, their caseworkers, and JOBS activity providers. Note that because individuals in the control group were not eligible for services through JOBS, no case file reviews were conducted for control group members.(17)

Cost Data Sources

The cost analysis used data drawn from state, county, and local fiscal records, program participation records, supportive service payment records, administrative records, Two-Year Client Survey responses, and case file participation records. Sample sizes varied by data source and included individuals assigned to the LFA, HCD, and control groups.

Field Research

MDRC staff observed the JOBS programs and interviewed enrollees, case managers, service providers, and program administrators in each of the three sites. Information was collected about a range of issues, such as management philosophies and structure, the degree to which a participation mandate was enforced, the nature of interactions between caseworkers and program participants, the extent to which the program was able to work with all JOBS-mandatory individuals in the site, the availability of services, and the relationships JOBS staff had established with outside service providers and the sites' IM staff. Materials gathered in these visits are used throughout the report, but particularly in Chapters 3 and 4.

JOBS and Income Maintenance Staff Surveys

JOBS case managers and income maintenance (IM) workers and their immediate supervisors were surveyed about their opinions of JOBS, experiences administering the program, and attitudes toward their clients. These surveys were administered in November 1993 in Atlanta and covered all of the 27 JOBS workers employed at the time and 113 IM workers and supervisors selected at random. In Riverside, surveys were administered in October 1993 and covered all of the 71 JOBS workers and 105 IM workers and supervisors selected at random. Survey administration in Grand Rapids occurred in September 1993 and covered all of the 23 JOBS and 120 IM staff members and supervisors. Completion rates ranged from 90 to 100 percent for JOBS staff and from 94 to 100 percent for IM staff.

Adult Basic Education Teacher Surveys and Administrator Interviews

Basic education teachers were surveyed in the three JOBS sites discussed in this report during the fall-winter of 1993. MDRC targeted programs that offered basic education instruction and had enrolled a large number of JOBS participants in the site. All of the full-time teachers in those programs were asked for a description of their program and about issues such as linkages with JOBS, instructional styles, measures of student progress, and class size. The responses of 24 teachers in Atlanta, 79 teachers in Grand Rapids, and 45 teachers in Riverside are included in this report. In addition, while visiting each of the adult basic education institutions included in the teacher survey, an in-person interview was conducted with the program's administrator.

JOBS Enrollees' Characteristics, Attitudes, and Opinions as of Random Assignment

Standard client characteristic data, such as educational background and AFDC histories, were collected by welfare staff during routine interviews with individuals at JOBS orientation, and are available for all individuals in the report sample. Reading and math achievement test scores are also available for 9,060 individuals, representing about 76 percent of the report sample randomly assigned during the time period when the tests were administered.(18) Data on attitudes and opinions about welfare-to-work programs and employment prospects were collected through a brief, client-completed Private Opinion Survey (POS) administered at JOBS orientation, and are available for 6,953 individuals in the three sites, representing a response rate of 91 percent during the period when this instrument was used.