Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. a. Avoiding the Spillover of Welfare Reform Policies


In situations in which spillover occurs, control group members receive services they would have not received in the absence of welfare reform. A prime example of spillover is the presence of community effects, in which welfare reform affects members of the control group through changes in community institutions or norms. For example, if welfare reform is accompanied by strong community expectations that welfare recipients work, then control group members may be subject to additional social pressure to obtain employment, even if the formal policies to which they are subject have not changed.

The preceding example suggests that a certain degree of spillover is likely, since a welfare reform program usually will be accompanied by changing attitudes in the community. Indeed, these changes in attitudes and expectations are often both the cause and the intended consequence of a major welfare reform initiative. The resulting contamination of control group policies can be reduced by reminding members of the control group of the policies that apply to them, as well as the policies that do not apply to them (even if neighbors, friends, or relatives are subject to welfare reform provisions). Cases can be given these reminders through mailings, group orientation sessions, or regular meetings with the caseworker. The most effective approach probably is meetings with the caseworker, since they occur on a regular basis and provide information in person.

Spillover also can occur as the consequence of administrative error or manipulation, in which program staff administer welfare reform policies to individuals who are still officially in the control group. For example, a welfare reform-related instructional video could be shown to all research cases, because program staff fail to distinguish the experimental and control groups. This type of spillover can be reduced in several ways. To the extent possible, separate staff can administer welfare reform policies and control group policies. In addition, files for experimental and control cases can be clearly

distinguished through the use of different colored folders or other measures, so welfare reform policies are never accidentally applied to control group members.