Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. 1. Advantages of an Experimental Design


The principal advantage of a well-planned and well-executed experimental design is that it ensures that, in other respects than receipt of the treatment, experimental and control cases are alike. The difference in average outcomes between the experimental and control groups is thus an unbiased estimate of the average impact of the program; this is known as internal validity. This eliminates the need to rely on a multivariate statistical model to control for case characteristics.(3) Consequently, the estimation of impacts in an experimental design is straightforward. The central feature of an experimental design, random assignment, does what a multivariate model attempts to do in a nonexperimental design: it controls for differences in characteristics between cases that receive the reform program and those that do not. Random assignment imposes this control more effectively, however, essentially eliminating any possibility of bias from imperfectly controlling for background characteristics.

Another important advantage of an experimental design is that all cases--those receiving the reform program and those not receiving it--coexist in the same site or sites during the same time period. They are therefore exposed to the same economic and other factors that may influence outcome measures independently of the program reforms. This strategy avoids a principal limitation of a quasi-experimental design, which is that cases in one group may be exposed to plant closings, migration, floods, and other economic, social, and natural phenomena that cases in the other group are not exposed to.