Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. 3. Analysis and Recommendations


From the perspective of the impact analysis, there is less interest in pooling the applicant and recipient samples than in analyzing each (and particularly applicants) separately. A pooled sample can be selected to give unbiased estimates of impacts on the caseload over the demonstration period. Such impacts, however, are made up of effects on those already on AFDC before welfare reform and of effects on those entering the system only after welfare reform; in general, it is the latter effect (that on applicants) that is of long-term interest. Therefore, we recommend that sample sizes be sufficient to analyze recipients and applicants separately; this typically implies sampling applicants at a higher rate than recipients.

Because an extended sampling period brings large risks along with large administrative costs, we recommend designing the applicant sampling process to reach the target sample over a two- year period, if possible. A shorter sampling period implies a longer follow-up period, less likelihood of major program changes, and some flexibility to extend sampling if goals are not being met.

Finally, applicant sampling rates need to be set carefully to take into account the exclusion of transfers and those who have been through random assignment before. States should use any available historical data on accessions to predict these rates. This is just one example of the usefulness of longitudinal data on accessions and terminations.