Overview of the Final Report of the Seattle-Denver Income Maintenance Experiment. Size of Sample and Strength of Treatment


Other things equal, the larger the sample size, the greater the ability of an experiment to detect treatment effects and the greater the statistical confidence that can be placed in the findings.  Having a larger sample size than all the other income maintenance experiments combined, SIME/DIME produced the results to which one could attach the greatest degree of confidence.

The "strength" of the cash transfer treatment was greater in SIME/DIME than in the other experiments, both in terms of the generosity of the benefit levels and in terms of the length of the treatment period.  The more generous the benefit levels, the greater the expected effects on behavior.  Given this expectation, the more generous the benefit level, the smaller is the treatment group size required to detect the effect of the treatment at any given level of statistical confidence.  In addition, to the extent that it takes time for families to learn what the treatment means and to make adjustments in behavior, longer treatment periods will also increase the likelihood of detectable response.

With respect to treatment generosity, however, a caveat is in order.  Although a stronger treatment is more likely to advance scientific understanding of labor supply behavior, it will not yield better estimates of the probable labor supply responses to a national cash transfer program if the benefits proposed in that program differ substantially from those in the experiment.  Care must be exercised, therefore, in drawing policy implications from the experimental results.