Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. b. Survey Methods that Promote High Response Rates

10/01/1996

In the evaluation community, a number of factors are known to be important in obtaining high survey response rates in follow-up surveys of low-income populations:

  • Initial Contact Information. Obtaining contact information from all sample members at the time of demonstration intake, before any of them can "fall through the cracks," is critical in locating respondents later.
  • Updating Contact Information. It is important to update the initial contact information at least every 18 months. If the time between the initial intake and the first followup is longer, it is good practice to contact the respondents just to update contact information.
  • Sophisticated Tracking Methods. Survey research organizations experienced in surveys of low-income populations typically use computerized tracking systems that can check for addresses and phone numbers in multiple databases (such as credit bureau files and motor vehicle registration records). In an evaluation with several rounds of follow-up data collection, good tracking databases can make it worthwhile to attempt to recontact individuals who were nonrespondents in earlier waves, particularly if the response rate on the first wave was less than ideal.(3)
  • Mixed-Mode Interviewing. Mixed-mode surveys, which initially contact respondents by telephone and then in person if telephone attempts are unsuccessful, achieve higher response rates than telephone surveys alone. Field interviewers are more able to contact individuals without phones and also may be able to locate individuals on the basis of information from neighbors. In-person interviews are generally more expensive than phone interviews, however.
  • Respondent Payments. Paying respondents for their time in responding to the survey, even if the payment is small, generally fosters higher response rates and greater cooperation in obtaining quality data. One problem in paying respondents on AFDC, however, has been that such payments generally were counted as income in computing benefits unless a specific federal waiver was obtained.

The lack of any of these factors does not necessarily imply in itself that a survey will have poor response rates, but it may be seen as a risk factor that requires careful monitoring. An additional risk factor exists if an organization that lacks a track record in interviewing low-income populations conducts the survey. Experienced survey organizations have staff from the level of survey director to interviewer who are adept in the techniques of reaching low-income respondents, as well as resources to plan and organize survey operations to achieve high response rates within the time frames needed for timely followup.