Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. a. Types of Data


The three major types of baseline data are (1) data on background characteristics, (2) identifying information on each individual, and (3) contact information.

Background Characteristics. In an experimental evaluation, detail on background characteristics of sample members is less critical than in a nonexperimental evaluation. However, two kinds of data (which may overlap) generally are very useful: (1) characteristics that define subgroups of interest in the analysis (usually including basic demographic and socioeconomic characteristics), and (2) past histories of the outcomes of interest. In a random-assignment evaluation, multivariate regression models of the outcomes are used largely to reduce the variance of the impact estimates. Variables measuring past histories of the outcomes are generally the most important control variables in such multivariate models because they lead to the largest reductions in the variance of the impact estimate. Data on both case characteristics and past history of the outcome (ideally, over several years) are valuable in assessing the importance of threats to the experimental design, such as crossovers, since they permit assessment of whether the cases that are contaminated or lost from the sample are different from those that remain. Such data may provide enough information to adjust for any experimental-control differences. Other types of background information that are of less central importance include variables that can be used to identify statistical models predicting the effects of program components (see Chapter VI for further discussion). Examples could include data on attitudes toward and knowledge of the welfare system or access to services.

Identifying Information. It is critical to collect enough identifying information at the time of random assignment to ensure that each individual in a case can be tracked in the full range of data systems to be used in the evaluation. In particular, each person's social security number must be carefully entered and verified. This is especially challenging for individuals who are part of applicant cases that are denied benefits or decide to withdraw their application, since eligibility workers have less incentive to obtain full and accurate information on such individuals.

Contact Information. If a follow-up survey is planned, information should be collected at the time of random assignment that will make it easier to contact the case head (the primary adult in the case) for an interview at a later date. Such information includes phone numbers and mailing addresses, as well as names, addresses, and phone numbers for several friends or family members who typically know where the sample member can be reached.