Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. C. ENSURING THAT WELFARE REFORM DOES NOT CHANGE CONTROL POLICIES

10/01/1996

Another important issue in implementing a welfare reform evaluation is ensuring that the members of the control group are subject to the welfare policies that would have been in place in the absence of the welfare reform program. Even if random assignment proceeds without error, the implementation of welfare reform may alter control group policies:

  • Services provided to members of the experimental group and other cases subject to welfare reform policies may spill over to cases in the control group.
  • Services provided to members of the experimental group and other cases subject to welfare reform policies may displace services that, in the absence of welfare reform, would have been provided to cases in the control group.

When control group policies are changed, the resulting measures of the impacts of the welfare reform program will be biased, because the control group is subject to policies and situations qualitatively different from those that would have existed in the absence of welfare reform. In spillover, impact estimates may be too small, since control group members receive some welfare reform policies. In displacement, impact estimates may be too large, since control group members fail to receive services they would have received in the absence of the welfare reform.