Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. c. Initial Contact Information


Lack of contact information from the time of random assignment has been a major problem for the California and Colorado surveys. In California, the initial baseline interviewing did not begin until a year after the demonstration began, in part because of delays in obtaining sample and in part because the development of the survey instrument was delayed, as many stakeholders requested additions and revisions to the survey.(5) The sampling delays occurred because the sample is selected about two months after intake from the state Medicaid data system, and it then takes about another month before the counties forward initial data on sampled cases to the UC-Berkeley Survey Research Center. These data must be processed and samples selected before interviewing can begin. By the time attempts were made to contact sample members, the contact information from the county case files was about a year old. Despite the use of various tracking methods (discussed more later), the Survey Research Center was at a considerable disadvantage because of the delay before the initial contact was made and the lack of information on friends and relatives (since there was no distinct research sample intake at which such information could be collected).

The situation in Colorado was similar. The evaluator sent out letters introducing the survey and contact information forms to be mailed in as soon as possible. However, because the ongoing case sample was selected before the evaluation contractor was chosen, and because it took time to transfer sample information to the contractor, about four to six months had elapsed between the selection of the ongoing case sample and the mailing of the letters. Only 18 percent of the contact forms were completed and returned (but low response rates are not unusual in mail surveys).

In the Wisconsin WNW evaluation, the evaluator planned to obtain contact information in the survey for the process analysis, which was scheduled to occur about four months after sample intake. However, that survey was delayed because the state data system was going through a major revision, and sample information on recipient cases was not made available to MAXIMUS until nine months after the intervention began. The process analysis survey thus occurred 10 to 12 months after enrollment.

The 12-month survey for the Minnesota MFIP evaluation, in contrast, has had the advantage of drawing contact information from a form filled out at the time of random assignment, and has achieved high response rates, as discussed further later.