Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. a. Selecting a Method of Random Assignment

10/01/1996

Random assignment should generate no systematic differences between the baseline characteristics of the experimental group and the baseline characteristics of the control group. The simplest way to accomplish this is to assign cases to the experimental or control group with probabilities equal to the proportion of cases in each group. For example, if the experimental group is expected to equal 35 percent of the cases passing through random assignment, each case passing through random assignment should have a 35 percent chance of selection into the experimental group. Neither the cases themselves nor the program staff administering the program should have any say over who is selected for the experimental group or who is selected for the control group. Otherwise, certain cases may be favored to receive particular policies, leading to systematic baseline differences between experimental cases and control cases.

Random assignment could be accomplished in a manner consistent with these principles in several ways. One approach would be to have a computer generate, for each case, a random number from a uniform distribution between 0 and 1. Suppose, for example, that the state wished to assign 35 percent of cases to the experimental group, 35 percent to the control group, and 30 percent to a nonresearch sample. The computer would compare the random number with the case's probability of selection into the experimental group (0.35 when the probability of selection equals 35 percent). If the number were less than or equal to the probability of selection, the case would be assigned to the experimental group. If the number were greater than the probability of selection, but less than or equal to the probability of being selected into either the experimental group or the control group (0.70 when the probability of selection equals 35 percent for each group), then the case would be assigned to the control group. Otherwise, the case would be assigned to the nonresearch sample.

Another method of random assignment would be to assign cases by the social security number (SSN) of the case head. Since SSNs are not entirely random, only digits at or near the end of the number should be used for random assignment. For example, when the probabilities of selection to the experimental group and the control group are each 35 percent, random assignment could be on the basis of the last two digits of the case head's SSN (00 to 34 resulting in assignment to the experimental group, 35 to 69 resulting in assignment to the control group, and 70 to 99 resulting in assignment to the nonresearch sample). This approach carries a slight risk that, if potential applicants are informed of an SSN-based selection rule in advance, their decision about whether to apply for welfare or whom to identify as the case head might be affected, thereby corrupting the process of random assignment.

A third way to accomplish random assignment is to assign cases on the basis of some other number used for administrative purposes, such as a case number. It would be important to determine the manner in which this number is being generated, and in particular whether program officials have any control over its value. Assuming that particular digits were not subject to the control of program administrators, but were indeed generated randomly, the process of random assignment could proceed in a manner similar to random assignment using the case head's SSN. It would be important to ensure that recipient cases not become aware of the selection rule in advance; otherwise, certain cases might decide to leave welfare prior to random assignment, thereby corrupting the selection process.