How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Random Assignment Designs

12/01/2001

Four of the sites implemented a three-way random assignment research design in order to test the effectiveness of two different program approaches. In the three-way design, each person was assigned, by chance, to one of two program groups or to a control group. This design is especially powerful because members of both research groups have the same background characteristics and face the same labor market conditions and other environmental factors that can affect a program's success in helping people find jobs and advance toward economic security. In the three-way sites, the relative effectiveness of the two programs can be assessed by comparing outcomes for the program groups with one another directly, that is, without taking the control group into account. To assess the programs' absolute effectiveness, however, it is necessary to compare outcomes for each program with the control group's outcomes.(2)

Three of these four sites (Atlanta, Grand Rapids, and Riverside) operated two programs that were designed  for purposes of the evaluation  to magnify the differences between the employment- and education-focused approaches described in Chapter 1: Labor Force Attachment (LFA) programs, which emphasize rapid job placement as the best way for welfare recipients to develop their work habits and skills, even if the job pays low wages; and Human Capital Development (HCD) programs, which emphasize that welfare recipients have to develop their "human capital," that is, their knowledge and basic skills, through education and training in order to have a better chance of finding and keeping jobs and advancing toward well-paid and secure employment. In each site, the two program models were implemented to maximize the contrast between them, thus making the differences between their effects easier to detect. Figure 2.1 illustrates the process by which welfare recipients and applicants in Atlanta and Grand Rapids were randomly assigned to the research groups.

Figure 2.1
Steps Leading to Random Assignment in Atlanta and Grand Rapids

Figure 2.1 Steps Leading to Random Assignment in Atlanta and Grand Rapids

The random assignment process differed in Riverside because California's welfare rules mandated that only people "in need of basic education"  that is, people who lacked a high school diploma or GED, scored low on a welfare department math or reading literacy test, or required instruction in English as a Second Language (ESL)  could be assigned to the HCD group. This constraint meant that whereas the HCD group included only people determined to need basic education, the LFA group included both such people and people determined not to need basic education. To facilitate direct comparisons between the Riverside LFA and HCD groups in this report, the results for the subgroup of LFA group members determined to need basic education are provided in addition to the results for the full LFA group.(3) A second conse-quence of this constraint in Riverside is that the only way to make direct comparisons between the results of the Riverside HCD program and those of the programs in other sites in the evaluation is to focus on the subgroups of people in those other programs who lacked a high school diploma or GED.(4) Figure 2.2 illustrates the process by which welfare recipients and applicants in Riverside were randomly assigned to the research groups.

Figure 2.2
Steps Leading to Random Assignment in Riverside

Figure 2.2 Steps Leading to Random Assignment in Riverside

Unlike the goal in the other sites that used a three-way design, the goal in Columbus was to test and compare the effectiveness of two different case management models. In the Traditional model, one worker handled the welfare department's employment and training function and another worker handled welfare eligibility and payment issues  often called "income maintenance." Both workers maintained relatively large caseloads. In the Integrated model, a single worker handled both the employment and training and income maintenance functions. In the Integrated model, the worker maintained a smaller caseload than either of the workers in the Traditional model since, on a per client basis, the worker was handling jobs "traditionally" done by two workers.

The remaining three sites in the evaluation (Oklahoma City, Detroit, and Portland) used a two-way random assignment design to test the effectiveness of program models already established in those sites. In other words, instead of implementing a program designed expressly for research purposes, as in the three-way sites, program administrators in each of the two-way sites determined their welfare-to-work program goals and practices and randomly assigned people to a group that entered the program or to a control group.(5) A summary of the research designs in all seven sites is presented in Table 2.1.

 

Table 2.1
Research Designs for the Seven Evaluation Sites
Characteristic Atlanta Grand Rapids Riverside Columbus Detroit Oklahoma City Portland
Type of random assignment Three-way

(2 program groups, 1 control group)

Three-way

(2 program groups, 1 control group)

Three-way

(2 program groups, 1 control group)

Three-way

(2 program groups, 1 control group)

Two-way

(1 program group, 1 control group)

Two-way

(1 program group, 1 control group)

Two-way

(1 program group, 1 control group)

Type of study Differential impacts of HCD and LFA approaches Differential impacts of HCD and LFA approaches Differential impacts of HCD and LFA approaches Differential impacts of Integrated and Traditional case management strategies Net impacts of established program Net impacts of established program Net impacts of established program
Sample composition AFDC applicants and recipients AFDC applicants and recipients; teen parents (ages 18 and 19) AFDC applicants and recipients AFDC applicants and recipients AFDC applicants and recipients; teen parents (ages 18 and 19) AFDC applicants; teen parents (ages 16 to 19) AFDC applicants and recipients
Age of youngest child 3 1 3 3 1 1 1
Point of random assignment Program orientation Program orientation Program orientation Income maintenance office: application or redetermination Program orientation Income maintenance office: application only Program orientation