Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. 2. State Approaches


Staff members at DHHS typically have specified sample sizes for welfare reform evaluations in the waiver terms and conditions after detailed discussions with the state. These sample size requirements vary according to the state's evaluation objectives and the size of the population being studied. The usual minimum requirements, however, are for the control group to include 2,000 recipient cases and 2,000 approved applicant cases and for the experimental group to be one to two times as large. States may exceed these minimum requirements to improve the precision of their estimates. Usually, sample size requirements do not include specific sample size goals for subgroups. States are required to sample all applicants (not just approved applicants) if the intervention affects eligibility for AFDC, but the sample size requirement is still generally phrased in terms of approved applicants. Thus, the federal requirements generally imply larger sample size requirements when the intervention affects eligibility.(4) Despite this federal guidance, the five evaluations reviewed for this study varied greatly in their planned sample sizes (overall and for key subgroups), as well as in the goals, assumptions, and precision standards used to justify these sample sizes.