Evaluating Two Welfare-to-Work Program Approaches: Two-Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites. Endnotes


1.  In two of the three sites -- Grand Rapids and Riverside -- random assignment for research purposes also occurred at a point earlier than JOBS enrollment: when individuals were identified as JOBS-mandatory by income maintenance staff. In these two sites, the research groups analyzed in this report, that is, the ones generated at JOBS enrollment-orientation, are 'nested' within one of the previously created research groups. Analyses using the samples randomly created at an earlier point in the path toward JOBS will measure JOBS's deterrence effects prior to orientation (impacts in addition to the ones discussed in this report) and will examine such issues as clients' reasons for not attending JOBS orientations. Those results will be reported in a separate, forthcoming publication.

2.  Family Support Act of 1988.

3.  This exemption reason was eliminated in December 1992.

4.  In two of the National Evaluation of Welfare-to-Work Strategies sites not included in this report, Oklahoma City and Columbus, some sample members did not have IM workers. Instead, an integrated case manager handled all case management functions, including financial and JOBS monitoring functions. The effectiveness of this approach compared with the traditional approach of separate case workers for different functions will be examined in a forthcoming document.

5.  A future National Evaluation of Welfare-to-Work Strategies publication will more closely examine the process by which AFDC recipients came to attend JOBS orientations and, as noted earlier, will estimate the impacts of being referred to JOBS and obligated to enroll in the program by attending a JOBS orientation.

6.  While a number of earlier studies, such the San Diego Job Search and Work Experience Demonstration and the Virginia Employment Services Program Demonstration, utilized side-by-side tests of two different program strategies (see, for example, Goldman, Friedlander, and Long, 1986; and Riccio et al., 1986), no prior evaluation has conducted a side-by-side test of comprehensive program models. These earlier studies restricted access to some program services, such as work experience or basic education activities, to one program group, while permitting the other program group to access these services. Thus, these earlier evaluations were tests of individual service components. In contrast, the National Evaluation of Welfare-to-Work Strategies is a study of two pervasive program philosophies in three sites. In these three sites, sample members in the two program groups received very different messages about the goals of the program and were offered a range of services compatible with that message. Chapter 3 discusses the implementation differences between these two program models.

7.  The GAIN Appraisal test, an instrument developed by the Comprehensive Adult Student Assessment System (CASAS) specifically for use by the California GAIN program, was given to sample members at orientation. According to the designers of the test, individuals who score below 215 have difficulty completing tasks that require more than minimal literacy or computation skills.

8.  Restricting the Riverside HCD sample to those who did not have a high school diploma or GED, regardless of how they performed on the GAIN Appraisal test, would have further complicated Riverside within-site comparisons as well as full-sample cross-site comparisons. In addition, this would have created a group with no operational policy relevance to California or Riverside welfare administrators.

9.  Case heads receiving AFDC for unemployed parents (AFDC-UP) were also randomly assigned in Riverside as part of this evaluation. These individuals, who were primary wage earners (typically male) in two-parent households, were also required to participated in JOBS. AFDC-UP sample members are not included in this report; the effects of the LFA and HCD approaches on Riverside AFDC-UPs will be analyzed in a future publication.

10.  The Riverside GAIN program generally did not mandate participation for women with children aged 2 or under, but it did require participation of two groups of single parents regardless of the age of their youngest child: teen parents, on their own or their parents' AFDC case, who did not have a high school diploma or GED; and individuals who worked more than 15 hours per week while receiving AFDC. The first group, teens, was randomly assigned but is not included in the sample for this report. The second group was included in the random assignment process and in the sample for this report.

11.  In order to facilitate comparisons between the reading achievement test scores of research sample members in Riverside and the other sites, the National Evaluation of Welfare-to-Work Strategies commissioned a team led by Walter Haney, Senior Research Associate at the Center for the Study of Testing, Evaluation, and Education Policy at Boston College, to conduct a calibration study of research sample members' scores on the GAIN Appraisal reading test Form 2 and the TALS document literacy tests. The findings of this study, which are discussed in detail in Haney et al., 1996, were used to estimate the TALS document literacy test score that best corresponds to the GAIN Appraisal score received by each research sample member in Riverside.

12.  Note that some individuals were randomly assigned to a research group as part of the evaluation during this time period, but are not analyzed in this report: JOBS-mandatory individuals randomly assigned prior to JOBS enrollment-orientation in Grand Rapids and Riverside as part of a special study of possible deterrence effects of JOBS; AFDC-UPs and teens randomly assigned at JOBS orientation in Riverside; and individuals in Riverside who were randomly assigned at JOBS orientation and were also part of a six-county random assignment evaluation of GAIN services in California, conducted in the late 1980s. (Individuals who were randomly assigned to the GAIN study program group less than three years prior to attending a JOBS orientation were randomly assigned to an evaluation program group, either the HCD or LFA group, but were not eligible to be assigned to the control group.)

13.  In Atlanta and Grand Rapids, AFDC records were not available for sample members who moved out of state during the follow-up period; in Riverside, AFDC records were not available for sample members who moved out of Riverside County during the follow-up period. UI records were not available for sample members who moved out of state during the follow-up period. In addition, UI records often underrepresented certain types of employment, such as domestic service, which may have been 'off the books.' Finally, while Georgia and California employers were required to provide wage information, employers in Michigan were requested to provide this information.

14.  There were no large differences in response rates across research groups. The presence of large differences would have been a potential source of bias in research group comparisons.

15.  These survey sample sizes reflect regression-adjusted measures, including all impact measuring and some participation and cost measures. For a few surveyed individuals, missing data prevented their inclusion in the regression model. The responses of these individuals were included in measures that were not regression-adjusted, that is, some participation and cost measures. For measures that were not regression-adjusted, sample sizes are 1,391 sample members in Atlanta, 836 sample members in Grand Rapids, and 1,588 sample members in Riverside.

16.  The length of follow-up in the case file reviews varied by individual, ranging from a 24-month period to a 37-month period. For this analysis, activities that occurred more than 24 months after random assignment have been disregarded.

17.  Periodically, MDRC staff reviewed the case file records of control group members to confirm that these individuals were not receiving JOBS services. These reviews found that no members of the control groups included in this report received JOBS services while residing in their county of random assignment.

18.  Among those who did not take the tests, about one-third did not speak English; others were unable to remain for the testing, spoke English but were unable to read or write it, or had other reasons for not taking the tests.