Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. b. Assessing the Performance of a Random-Assignment Method


As required by the terms and conditions of the Section 1115 welfare waivers, all of the evaluators assessed the method of random assignment both through interviews with program staff and through statistical comparisons of the baseline characteristics of experimental and control cases. Neither Colorado's nor Minnesota's evaluations reported any concerns about the implementation of random assignment. California's evaluation experienced some problems in sampling applicants; originally, individuals (rather than cases) were sampled, and some previous recipients were sampled as "new" applicants. In Michigan's evaluation, while some statistically significant differences were detected between the baseline characteristics of the experimental group and the baseline characteristics of the control group, none of these differences exceeded two percentage points. Possible explanations for these differences include mere chance, the random assignment of recipients in groups rather than as individual cases, and the exclusion of denied applicants from the analysis sample.