Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. b. Estimating Impacts for Subgroups Defined by Decisions Since Random Assignment

10/01/1996

Even if administrative records and survey data are complete, certain outcomes will only be available for a subset of the research sample that is defined by behavior or events since random assignment. For example, when estimating welfare recidivism rates, the sample must be limited to cases that had left welfare within the follow-up period. This sample is only a part of the entire sample and, if experimental policies induce a different pattern of exits from welfare, the baseline characteristics of experimental and control cases in the subsample will differ. Another example of an analysis using a subsample defined by behavior after random assignment would be an analysis of JOBS participation rates among current welfare recipients.

A random-assignment design may be helpful in dealing with certain problems related to the use of such samples. In particular, experimental status variables may be useful in correcting for sample selection because of decisions since random assignment. For example, if a study was seeking to estimate labor market outcomes for JOBS participants, experimental status could be used in a sample selection procedure as a predictor of whether particular individuals would participate in JOBS.(5)