Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program. Future Research


The final report in the NEWWS Evaluation will track outcomes for sample members in Columbus for up to five years following random assignment. This longer follow-up period is important when evaluating programs that engage many people in education because it can take some time for sample members to put their newly acquired skills to work in the job market. As noted in Chapter 1, however, in October 1997 control group members began to receive program services, and all sample members  from the control group, the integrated group, and the traditional group  began receiving integrated case management. These two changes, which occurred during the fourth or fifth year of follow-up for most sample members (random assignment occurred from 1992 to 1994), may have diminished the differences between the research groups' outcomes.


1.  As noted in Chapter 1, this report refers to cash assistance as AFDC; although the AFDC program has been converted into a block grant to states, AFDC existed throughout the study period for the report.

2.  UI earnings data are collected by calendar quarter (January through March, April through June, and so on). For the research, the quarter during which a sample member was randomly assigned was designated quarter 1. The first follow-up year (called year 1), covers quarters 2 through 5, the second year (year 2) covers quarters 6 through 9, and so on. Monthly AFDC and Food Stamp payments were grouped into quarters and years covering the same periods as earnings quarters and years. See Freedman et al., 2000, for more detail on the methods used in this analysis and on the impacts of the Columbus programs (estimated using two years of follow-up data), and for a more comprehensive comparison of the effects of the Columbus programs with the effects of the other NEWWS Evaluation programs.

3.  Differences in outcomes are considered statistically significant if there is less than a 10 percent probability that they occurred by chance.

4.  In two sites in the NEWWS Evaluation  Grand Rapids and Riverside  random assignment occurred at two separate points: at the point of referral to the welfare-to-work program and at the point of entry into the welfare-to-work program (program orientation). This design allows researchers to calculate separately the impacts of the referral itself and the effects of the program services and mandates. See Knab et al., 2001, for a presentation of findings from this special study.

5.  Three-year earnings impacts for the other education-focused programs were: Atlanta HCD, $1,003, or 11 percent; Grand Rapids HCD, $892, or 10 percent; Riverside HCD, $740, or 14 percent; and Detroit, $848, or 11 percent. Oklahoma City's impact of $12 is not statistically significant. Impacts for the employment-focused programs in the evaluation ranged from $1,292 to $3,152. (These impact findings are from an unpublished MDRC analysis of NEWWS Evaluation data.)

6.  In both programs, longer duration of employment and higher earnings on the job represent about two-thirds of the earnings gain, and an increase in the number of jobs found represents one-third. This decomposition is not exact. It is based on the approximate mathematical equivalence of the "percentage difference" in average total earnings to the sum of the percentage differences in "total quarters employed if employed," "average earnings per quarter employed," and "ever employed." The contribution of each effect is obtained by dividing its percentage difference by the percentage difference in average total earnings. The sum of all three contributions does not equal 100 percent because a small portion of the earnings impact is attributable to interactions among the components. (The integrated program increased "total quarters employed if employed" by 0.13 of a quarter, or 1.9 percent, and increased "average earnings per quarter employed" by $95, or 4.3 percent. Corresponding gains in the traditional program were 0.09 of a quarter, or 1.3 percent, and $88, or 4.0 percent.)

7.  Hamilton and Scrivener, 1999.

8.  Impacts of the other programs ranged from .58 to 1.94 months (from an unpublished MDRC analysis of NEWWS Evaluation data).

9.  Decreases in average three-year AFDC payments for the other education-focused programs were: Atlanta HCD, 6 percent; Grand Rapids, 13 percent; Riverside HCD, 12 percent; Detroit, 3 percent; and Oklahoma City, 4 percent. Decreases for the employment-focused programs in the evaluation range from 8 to 21 percent. (These findings are from an unpublished MDRC analysis of NEWWS Evaluation data.)

10.  The average monthly payment amount for control group members ($333) multiplied by the reduction in number of months of AFDC receipt indicates what the AFDC savings would have been if average monthly payment amounts were the same for program and control group members who remained on welfare. In the integrated program, for example, this calculation ($333 times 2.61 months) yields $869, which represents 81 percent of the $1,079 three-year AFDC savings. The calculation for the traditional program ($333 times 1.71 months) yields $569, which is 70 percent of the $816 three-year AFDC payment impact. The remainder of the impact on three-year AFDC payments may have come from reductions in grant amounts resulting from sanctions or from increased earnings while still on welfare. Alternatively, the overall reduction in months of receipt may have fallen primarily on cases with above-average monthly grant amounts. This decomposition is not exact, since it ignores interactions between grant level and case closure.

11.  The decomposition of the cash assistance payment impact discussed in footnote 10 indicates that $869 of the integrated program's impact on payments was generated because integrated group members spent less time on welfare than their control group counterparts; this figure for the traditional program was $569, $300 less than the figure for the integrated program. The $300 difference exceeds the $264 difference between the two programs' impacts on payments.

12.  Freedman et al., 2000.

13.  Over three years, the three research groups received Food Stamp benefits valued at the following amounts: control group, $6,312; integrated group, $5,616; traditional group, $5,830. Both programs' impact is statistically significant at the .01 level.