Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program. Research Design


The study in Columbus uses an unusually strong research design, a random assignment experiment, to estimate program effects. In this design, welfare applicants and recipients were assigned, at random, to one of three groups:

  • the integrated group, whose members were required to participate in the integrated program or face a reduction in their cash grant (a financial sanction);
  • the traditional group, whose members were required to participate in the traditional program or face a financial sanction; or
  • the control group, whose members were neither required nor eligible to participate in any special welfare-to-work program. (Control group members could seek out employment-related services available in the community, and if they did, they could receive child care assistance from the welfare department.)

Because people were assigned to the three groups through a random process, any differences that emerge over time between the groups' outcomes, such as average earnings or welfare payments, can reliably be attributed to the programs.

The three-way design allows researchers to make two types of rigorous comparisons. First, estimates of the net effects of each program can be made by comparing outcomes of the integrated group with outcomes of the control group and by comparing outcomes of the traditional group with outcomes of the control group. (The integrated and traditional groups are also referred to as program groups in this report.) Second, estimates of the differential effects of the programs can be made by comparing outcomes of the integrated group with outcomes of the traditional group. All of these differences in outcomes are referred to as impacts.

In Columbus, 7,242 single-parent welfare applicants and recipients, who were determined to be JOBS-mandatory, were randomly assigned for the evaluation. During the period studied, Columbus mandated participation in the JOBS program from all recipients whose youngest child was at least 3 years old and who did not meet federal exemption criteria. These included working 30 hours or more per week, being ill or incapacitated or caring for an ill or incapacitated household member, being of advanced age, being in at least the second trimester of pregnancy, or living in a remote area that made program activities inaccessible.

The steps leading to random assignment are depicted in Figure 1.1. Between September 1992 and July 1994, IM workers identified new AFDC applicants and ongoing AFDC recipients (who were single parents aged 21 or over) who were JOBS-mandatory. Once an individual was approved to receive welfare, she was randomly assigned to either the integrated, traditional, or control group.(17) People in the integrated group were assigned to an integrated case manager; people in the traditional group were assigned to an IM worker and a traditional JOBS case manager; and people in the control group were assigned to an IM worker (but not to a JOBS case manager). Then, the integrated and traditional JOBS case managers were responsible for sending a letter to each person scheduling her for an orientation session.(18) (See Chapter 3 for more information on the programs' orientation process.)

The fact that random assignment occurred at the welfare office when individuals were referred to JOBS affects how the results in this report should be interpreted. In this design, sample members who did not show up for a JOBS orientation are included in the research sample, and their outcomes, such as earnings and AFDC payments, are averaged together with those of orientation attenders. Since orientation is the gateway to program services, people who did not attend an orientation session could not receive any services. In most of the other sites in the NEWWS Evaluation, random assignment occurred during a JOBS orientation, and thus orientation nonattenders were excluded from the sample. Whereas the evaluations in these other sites test only the effects of the program services and mandates, the evaluation in Columbus captures these program effects plus the effects of a referral to a mandatory welfare-to-work program and any follow-up relating to this referral, such as sanctioning for failing to attend an orientation session.(19)

Steps Leading From Income Maintenance to Random Assignment

Steps leading from income maintenance to random assignment