Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. b. Timing of Random Assignment for Applicants

10/01/1996

We recommend that, if welfare reform substantially changes the rules by which eligibility is determined, states perform dual eligibility calculations before random assignment of applicants. They should include in the sample of applicants all cases eligible for welfare under either experimental or control group rules but exclude cases eligible for welfare under neither set of rules. Dual eligibility calculations are particularly justified if twice-ineligible cases represent a large proportion of all applicants. Including twice-ineligible cases in the sample of applicants increases the number of observations for which the likely impact of welfare reform is zero and may make it more difficult to identify statistically significant impacts for eligible applicants. When eligibility rules are the same for the experimental and control groups, only a single eligibility calculation is necessary. All ineligible cases are excluded from the sample of applicants.

None of the random-assignment evaluations we reviewed included dual eligibility calculations before random assignment. Only one of these evaluations was of a waiver package in which the experimental and control groups faced identical rules for welfare eligibility, although in two other instances eligibility rules were only slightly different for the experimental and control groups.

We recognize that, for ease of administration, states may be prepared to perform only a single eligibility calculation, either before or after random assignment. When welfare reform does not change welfare eligibility rules, we recommend that random assignment of applicants occur after eligibility has been determined. When welfare reform changes eligibility rules, we recommend that random assignment of applicants occur before the determination of welfare eligibility and benefits. If a state follows this pattern, there will be no discrepancy between the initial determination of a case's welfare eligibility and benefits and its assigned experimental or control status. Because random assignment before eligibility determination is likely to introduce twice-ineligible cases into the sample of applicants, we recommend that evaluators track all applicant cases (approved and denied) and report cumulative denial rates for applicants in the experimental and control groups. For the group subject to the broader eligibility rules, the denial rate may provide an approximate measure of the share of applicant cases for which the likely impact of welfare reform is zero.