Design and Operation of the 2010 National Survey of Residential Care Facilities. Pilot Test


In spring 2008, a small pilot test was conducted in which interviews were completed with five facilities in the greater District of Columbia metropolitan area. The specified goals of the pilot test were:

  • To assess the usability of the CAPI instruments in real-life situations.
  • To assess the feasibility of the questionnaire items across all facility sizes.

Testing of recruiting procedures and obtaining a high response rate were not goals of the pilot test.

Twenty-four facilities were selected from the frame and included a mix among four size strata, urban and nonurban areas, and chain-and nonchain-affiliated facilities. Three experienced interviewers were trained over 2 days from a scripted training guide, interviewer’s manual, and other prepared materials. Each interviewer completed one interview within a few days of training and, shortly thereafter, two interviewers completed one additional interview.

Recruitment began with an initial mailing to the facility that contained a personalized letter signed by the NCHS director, an NCHS confidentiality brochure, and an NSRCF questions-andanswers brochure. Several days later, phone call attempts were made to verify receipt of the package and to gain cooperation. Recruitment proved difficult, with many call attempts required to reach the facility director and set appointments. When contact was made with the director, a screening instrument was used to determine facility eligibility. If the facility was eligible, respondents for the in-person interview were identified and a date and time for the interview was scheduled. An ADCF was then mailed to the facility. A confirmation reminder call also was made to the director 1 to 2 days before the visit. During the in-person interview, the interviewer first completed the facility questionnaire. The interviewer then randomly selected a sample of three to nine residents from the facility’s list of current residents—three residents were sampled in the small and medium facilities, five in large facilities, and nine in very large facilities. Resident questionnaires were then completed with the respondent who was most knowledgeable about the sampled resident.

Of the 24 sampled, a total of 13 facilities completed the entire process. Five of the 24 were ineligible—three because they exclusively served adults with mental disabilities and two because they were no longer in business—for an ineligibility rate of 21%. An additional six facilities firmly refused participation.

Recommendations for changes to the questionnaires were made after the completion of the pilot test. These recommendations addressed redundancies in the resident questionnaire, respondent confusion with terms and definitions, errors in question wording and skip patterns, and concerns about respondent burden. Other improvements included: increasing length of interviewer training; finding better, more efficient approach to recruitment of facilities; reducing individual respondent burden by involving other staff in sampled facility; and streamlining questionnaires by rewording, reordering, and reducing the number of questions to improve efficiency.

View full report


"NSRCFdo.pdf" (pdf, 8.47Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®