Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. f. Response Rate Goals and Actual Experience

10/01/1996

In the surveys in California and Colorado, response rates have been low enough to put the usefulness of these surveys for the impact analysis in considerable doubt. For the first Colorado follow-up survey, after nearly four months of survey operations, the response rate for the ongoing sample was 41 percent, much lower than the 60 percent goal; the rate was the same for experimentals and controls. Locating respondents was the key problem. In California, the Wave I English/Spanish survey began a year after random assignment and took 10 months to complete for ongoing cases; the response rate was just under 60 percent, but oversampling allowed UC-Berkeley to reach the desired number of completes.(6) Locating respondents again was the major problem. The response rate for Wave II, which began 18 months after Wave I, has been over 80 percent of those reached in Wave I, or about 50 percent of the original sample. No attempts were made to contact, for Wave II, cases who were not interviewed as part of Wave I.

In the Michigan demonstration, the target response rate for the survey is 80 percent. We have no information on actual experience, but the long follow-up period of four years makes this seem an ambitious goal. In Minnesota, the target for the 12-month followup was 85 percent; for the 36-month followup the target is 80 percent. The first follow-up survey in Minnesota achieved a response rate of 84 percent, very close to the goal; MDRC staff members report that they and RTI decided to end the survey slightly below the target to preserve resources for the second follow-up survey. No target response rate was set for the Wisconsin WNW demonstration. The follow-up survey in Wisconsin has not yet occurred.