How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Sensitivity of the Results


As is common in cost-benefit analyses, assumptions were made in producing the estimates in this analysis. Owing to data limitations, several major assumptions had to be made about the levels of participation in follow-up years 3 to 5, as well as the proportion of participation that was considered in-program or out-of-program. The Five-Year Client Survey asked about participation at any point during the follow-up period, as well as specifically about participation in the last year of the follow-up period. This information was put together with data from the Two-Year Client Survey, which provided more detailed information about participation during the two-year period following program entry, to classify the likelihood of having participated during follow-up years 3 and 4. For the cost-benefit analysis, sample members were considered to have participated during years 3 and 4 if there was a good chance that they could have participated during this period.

In addition, the available data could not provide details on whether participation took place as part of the program or as a self-initiated activity. Therefore, in the cost-benefit analysis, it was assumed that the proportion of in-program participation increased with the number of months of welfare receipt during the period in question. This was the best approximation of the participation patterns seen in the detailed two-year data used in earlier reports.

To measure the effect of these two assumptions, a sensitivity test was conducted (see the following table). In this analysis, the alternative assumptions were that only those with the highest likelihood of having participated in years 3 and 4 did so, and all participation was evenly divided between "in-program" and "out-of-program" for those who received welfare for a portion of the period.(25) The resulting gross and net costs, as well as the net present value from the perspective of government budgets, are presented below. (The welfare perspective is unaffected by program costs.)

Site and Program

Gross Cost Net Cost ($) From the Government Budget Perspective
Program Group ($) Control Group ($) Net Gain or Loss (NPV) ($) Return to Budget per Net Dollar Invested in Program and Nonprogram Activities and Services ($)
Atlanta LFA 6,432 3,006 3,426 -222 0.94
Atlanta HCD 7,843 3,006 4,837 -2,616 0.46
Grand Rapids LFA 8,311 6,513 1,798 3,097 2.72
Grand Rapids HCD 9,993 6,513 3,480 -15 1.00
Riverside LFA 5,280 2,538 2,742 2,123 1.77
Columbus Integrated 6,158 2,433 3,724 703 1.19
Columbus Traditional 5,782 2,433 3,348 -222 0.93
Detroit 5,478 3,751 1,727 -3 1.00
Portland 8,456 5,884 2,572 5,528 3.15

Using these alternative assumptions, gross costs were lower by 5 percent (Portland) to 13 percent (Riverside LFA) for program group members and by 3 percent (Atlanta LFA and HCD and Portland) to 7 percent (Riverside LFA) for control group members. The alternative assumptions resulted in reductions in net costs ranging from 8 percent (Grand Rapids HCD) to 17 percent (Riverside LFA).

As these results show, the point estimates of gross and net costs are indeed sensitive to the assumptions made. However, using these lower cost estimates, the same conclusions were drawn with regard to the ranking of the various sites overall and between program approaches in the NEWWS Evaluation. In addition, the comparisons with the GAIN programs also remain largely unaffected. In the benefit-cost analysis, the lower cost estimates result in very few changes to the bottom line for government budgets: Although there were changes in the magnitude of gains and losses, none of the estimates changed direction. In the sites where there were small losses to government budgets using the original assumptions, the losses are smaller yet, and these programs could be considered to have reached the "break-even" point.