Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program. Impacts on Employment and Earnings

06/01/2001

Table 5.1 shows the two programs' impacts on employment and earnings. The first set of columns shows the impacts of the integrated program (integrated-control comparison), and the second set shows the impacts of the traditional program (traditional-control comparison). The last column shows the difference between outcomes of the integrated and traditional programs (integrated-traditional difference).

Table 5.1
Program Impacts on Employment and Earnings
  Integrated-Control Comparison Traditional-Control Comparison  
Outcome Integrated Group Control Group Difference (Impact) Percentage Change Traditional Group Control Group Difference (Impact) Percentage Change Integrated-
Traditional Difference (Impact)
Ever employed, years 1-3 (%) 81.1 78.5 2.6** 3.3 80.7 78.5 2.2** 2.8 0.4
Year1 60.0 60.1 -0.1 -0.2 59.9 60.1 -0.1 -0.2 0.0
Year2 65.2 62.9 2.3* 3.7 64.5 62.9 1.6 2.6 0.7
Year3 68.9 65.3 3.6*** 5.5 67.9 65.3 2.6** 3.9 1.0
Quarters employed, years 1-3 5.75 5.46 0.29*** 5.3 5.69 5.46 0.23** 4.1 0.06
Year1 1.64 1.62 0.02 1.0 1.66 1.62 0.04 2.7 -0.03
Year2 1.97 1.82 0.15*** 8.5 1.94 1.82 0.13*** 7.0 0.03
Year3 2.14 2.02 0.12** 5.8 2.08 2.02 0.06 2.8 0.06
Earnings, years 1-3 ($) 13,208 12,027 1,181*** 9.8 13,027 12,027 1,000** 8.3 181
Year1 2,994 2,914 80 2.8 3,099 2,914 185 6.4 -105
Year2 4,578 3,982 596*** 15.0 4,472 3,982 490*** 12.3 106
Year3 5,635 5,131 505*** 9.8 5,456 5,131 325* 6.3 180
Sample size (total = 7,242) 2,513 2,159     2,570 2,159      
Sources: MDRC calculations from Ohio unemployment insurance (UI) earnings records.
Notes: Estimates were regression-ajusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. "Percentage change" equals 100 times "difference"divided by "control group". Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between outcomes for the program and control groups and to differences between outcomes for the integrated and traditional program groups. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. Year 1 refers to quarters 2 to 5; year 2 refers to quarters 6 to 9; year 3 refers 10 to 13. Because quarter 1, the quarter of random assignment, may contain some earnings and AFDC payments from the period prior to random assignment, it is excluded from follow-up measures.

In the context of Columbus's strong labor market, employment rates were high even without the programs' intervention: As the table shows, 78.5 percent of control group members were employed at some point during the three years after random assignment. They were employed for an average of 5.46 quarters (just over 16 months) and earned an average of $12,027 over the three-year period (this average includes zeros for people with no earnings).

Both programs produced small increases in employment rates and the length of time employed. Over three years, 81.1 percent of the integrated group worked for pay, a 2.6 percentage-point increase, and 80.7 percent of the traditional group worked for pay, a 2.2 percentage-point increase. Integrated group members worked an average of 5.75 quarters, an increase of 0.29 of a quarter (almost a month), and traditional group members worked an average of 5.69 quarters, an increase of 0.23 of a quarter (about two-thirds of a month).

Integrated group members earned on average $13,208 over the three-year period, a $1,181, or 10 percent, increase above the control group. Traditional group members earned on average $13,027, a $1,000, or 8 percent, increase above the control mean. (The $181 difference between the program groups' average earnings is not statistically significant.) These gains are similar to the earnings impacts of the other education-focused programs studied as part of the NEWWS Evaluation.(5) The earnings gains in the Columbus programs were primarily the result of longer duration of employment and higher earnings on the job.(6) In other words, the programs raised total earnings by enabling integrated and traditional group members who would have been employed anyway to obtain better jobs.

As is often found for programs that emphasize building skills prior to finding a job, neither program increased employment levels or earnings during the first year of follow-up. (This indicates that the referral to a mandatory welfare-to-work program did not, on average, spur people to quickly begin a job to avoid the program.) Employment and earnings gains began in the second year of follow-up. By the end of the third year of follow-up, the integrated program's impacts had decreased but remained statistically significant. The traditional program's impacts, in contrast, were less consistent during the third year. (See Appendix Table C.1 for the programs' impacts displayed for each quarter of the follow-up period.) These patterns suggest that the integrated program will likely continue to increase employment and earnings during the fourth year of follow-up, but the traditional program may not.

Contrary to researchers' expectations, more personalized attention, closer monitoring, and the higher rate of participation in program activities in the integrated program did not translate into larger employment and earnings impacts (although quarterly patterns suggest that the integrated program may have more positive results than the traditional program during the fourth year of follow-up). A recently published MDRC analysis of participation in welfare-to-work programs found that although a minimum level of participation is necessary to produce employment impacts, above that threshold there is no linear relationship between participation levels and impacts.(7) In light of this new information, one should not expect that higher participation rates would necessarily yield larger employment and earnings impacts.