Design and Operation of the 2010 National Survey of Residential Care Facilities. National Survey Methods



The national survey followed an approach similar to that for the pretest, in which separate personnel were used in contacting facilities: recruiters to call facilities, determine eligibility, and set appointments; and interviewers to conduct the on-site interviews. Facility recruitment began on March 16, 2010, and continued throughout most of the data collection period; on-site interviewing began on April 26, 2010, and officially ended on November 24, 2010.

Outreach to Provider Organizations

Prior to and during NSRCF data collection, a comprehensive outreach strategy was carried out to effectively spread the word about NSRCF among prospective respondents. A joint letter of support for NSRCF was obtained from these provider organizations that represent the residential care and assisted living industries (Appendix V):

  • LeadingAge, formerly American Association of Homes and Services for the Aging.
  • American Seniors Housing Association (ASHA).
  • Assisted Living Federation of America (ALFA).
  • National Center for Assisted Living (NCAL).
  • Board and Care Quality Forum.

NCHS staff met several times with the Center for Excellence in Assisted Living (CEAL) president and board members representing LeadingAge, ASHA, ALFA, and NCAL (Appendix VI). CEAL is a nonprofit collaborative of 11 national organizations whose aim is to promote high-quality assisted living. The goals of the meetings were:

  • Solicit information on best practices for recruiting facilities to participate in NSRCF.
  • Identify ways to inform their respective provider memberships about the importance of participating in NSRCF.

These board members agreed to raise awareness of NSRCF using selected communication channels with their provider members through their associations’ annual meetings, newsletters, and websites. The Board and Care Quality Forum, which caters largely to small-sized facilities, also agreed to promote the survey and encourage their facilities to participate.

Before and during data collection, short write-ups about NSRCF appeared in the national newsletters of the provider organizations and in the Board and Quality Care Forum newsletter. Survey descriptions and calls for participation were also published in the newsletters of state affiliates of those provider organizations (Appendix VII). NSRCF brochures were distributed at their annual meetings and the survey was mentioned in presentations given by key office holders in these organizations. NCHS created an NSRCF conference exhibit display that was used at AHCA/NCAL’s and AAHSA’s 2009 annual meeting. NCHS staff was also present at these exhibit booths to answer questions about the survey and explain the need for participation. During data collection, follow-up requests were also made to individual state affiliates of the national provider organizations in states with lower-than-expected response rates to encourage their members to participate.

To gauge the success of these outreach activities, respondents at the close of the screener questionnaire were asked: ‘‘Before you received the package about this study, had you heard about this study through newsletters or other information provided by national organizations that support it, such as American Association of Homes and Services for the Aging (AAHSA), American Seniors Housing Association (ASHA), Assisted Living Federation of America (ALFA), National Center for Assisted Living (NCAL), or Board and Care Quality Forum?’’ Overall, 12% of respondents reported hearing about the study. The percentage was highest among the very large facilities (22%) and was lowest among the small facilities (8%). Respondents representing the large and medium-size facilities were somewhat in between, with 22% and 15%, respectively.

Chain Outreach

In February 2010 before data collection began, an NSRCF information package (Appendix VIII) was sent via Federal Express to known chains that had two or more affiliated facilities included in the sample. These packages were mailed to 122 chains with 246 sampled facilities. The purpose of this chain outreach activity was to preempt facility refusals or delayed participation because chain headquarters were unaware of the study. It enabled recruiters to tell respondents that their chain was previously notified about the study whenever these facilities told recruiters that they could not participate without chain approval.

The chain outreach package included the following:

  • Letter signed by NCHS’ Director that provided information about the survey and explained that one or more facilities in their organization would be contacted.
  • NCHS ERB approval letter.
  • Brochure specifically about NSRCF.
  • NCHS’ Confidentiality Brochure— How the National Health Care Survey Keeps Your Information Strictly Confidential.
  • Letter of support from five national residential care provider associations.
The Project Director’s name and contact information was also provided in case chain leadership had additional questions about the study. Only one national chain contacted NCHS and refused participation on behalf of all their facilities.
Recruiter Training
In March 2010, 16 experienced interviewers attended and completed 3½ days of recruiter training that entailed a start-to-finish understanding of the survey and recruitment process. Topics included: an overview of the study; computer applications demonstration; recruitment scenarios; confidentiality procedures; Facesheet and Event Calendar system; conducting calls to facilities; gaining cooperation; administering the screener questionnaire; and appointment setting. In addition, before training began, recruiters received a computer manual, recruiter manual, and a home study exercise that gave them an overview of the national survey and their responsibilities.
Another 2-day recruiter training session was conducted midway through data collection after two recruiters left the project and several others had competing commitments. Six NSRCF interviewers with strong refusal conversion skills, high productivity, and proven success of working well with facilities were retrained as recruiters. One recruiter was also trained to perform interviewer duties due to the limited number of interviewers in one urban area.
Facility Recruitment
Facility recruitment began approximately 5 weeks before interviewer training to allow time for recruiters to set initial appointments. Each recruiter was responsible for contacting a total of about 225 facilities; assigned cases were released in several batches, as recruiter workloads diminished. Figure 1 provides an overview of the recruiter responsibilities.
Figure 1. Recruiter responsibilities
SOURSE: CDC/NCHS, National Survey of Residental Care Facilities
Figure 1. Recruiter responsibilities
First Contact With Sampled Facility
As shown in Figure 1, the first step recruiters performed was a telephone call to the sampled facility. The primary purpose of this call was to confirm the facility director’s contact information (name, title, phone number, and e-mail address) and the facility mailing address in order to mail the advance package. During this call, recruiters did not need to speak to the director because whoever answered the phone could usually provide this information. In about 82% of the sampled facilities, recruiters needed only 1 day to complete this call. In about 6% of the cases, this process took longer than 1 week. The primary reasons cases took longer to complete were:
  • Employee did not answer the call at their main number, requiring multiple callbacks.
  • Recruiter was unsure whether they had reached the sampled facility and needed further confirmation.
  • Telephone number was disconnected; the correct number needed to be identified or determined if the facility was out of business.

Recruiters used the Advance Package Call Script (Appendix IX)to guide them through the process of making these initial calls. This document also contained answers to FAQs and other instructions relating to this call. After confirming or updating the contact information, recruiters entered this information into the Facesheet.

During this recruitment activity, recruiters suspected 186 facilities as out of business, 149 of which were later confirmed out of business.

Advance Package Mail-out

Soon after recruiters completed the initial call to the sampled facility, support staff accessed the Facesheet information and mailed out the Advance Package (Appendix X) via Federal Express (FedEx).

The Advance Package consisted of an NSRCF folder that contained:

  • A letter signed by NCHS’ Director that provided information about the survey, explained its importance, and described how the survey is conducted. The letter contained a list of FAQs and instructions on who to contact for more information.
  • An NCHS ERB approval letter.
  • A brochure specifically about NSRCF.
  • An NCHS Confidentiality Brochure—How the National Health Care Survey Keeps Your Information Strictly Confidential.
  • A letter of support from five national residential care provider associations.

Set Appointment Call

Five business days after the Advance Package was mailed out, recruiters began making callbacks to the sampled facilities. The recruiter’s goals during this call were to:

  • Speak to the director.
  • Confirm that the director received the Advance Package.
  • Answer any questions the director had about the survey.
  • Complete the screener questionnaire to determine facility eligibility.
  • Set the appointment for eligible facilities and provide the director with additional information about the in-person interview.

Before placing these calls, recruiters accessed the FedEx website for the package tracking number, delivery date, and person’s name who signed for it. Then they referred to the Set an Appointment Call Script (Appendix XI) to guide them through these calls. Once the director was reached, recruiters confirmed receipt of the Advance Package and answered any questions about the study referring to the call script and FAQs document (Appendix XII). Next they administered the CAPI screener questionnaire (Appendix XIII) to determine whether the facility met the survey eligibility criteria. This questionnaire took, on average, 10 minutes to complete. The screener questionnaire is also viewable at: NCHS/Dataset_Questionnaires/nsrcf/2010/.

The most challenging part of this activity for recruiters was reaching the director. This challenge was more acute in smaller facilities where directors often provided resident care in addition to their administrative duties, thus ignoring phone calls and voice mails. In other facilities, recruiters frequently reached receptionists who acted as gatekeepers and resisted putting their calls through to the director. E-mail and voicemail messages were also used to reach the director during this process.

Appointment Setting

The CAPI screener instrument automatically determined the facility’s eligibility status and directed recruiters to either end the call if the facility was not eligible or proceed to the last recruiter activity—setting the appointment for the in-person interview. Again, the Set an Appointment Call Script and the online Event Calendar provided recruiters with details for scheduling an exact date and time for the site visit that aligned with a specific interviewer’s availability.

A higher-than-anticipated number of eligible facilities delayed or hesitated in setting their appointment, or refused to do so after completing the screener questionnaire. Of the facilities that agreed to set an appointment during the field period, only 68% were appointed on the same day as they were screened. These delays resulted in recruiters making many callbacks as well as employing other strategies to successfully appoint these cases (for more details, see ‘‘Refusal Aversion and Conversion Strategies’’).

Recruiters also encountered many directors who needed approval from a supervisor or upper-level management within their facility, organization, or chain to participate in the study. The process for obtaining these approvals required some or all of these steps:

  • Send an Advance Package to the person who was qualified to say whether approval was needed.
  • Identify the appropriate person to provide approval.
  • Send the Advance Package to the person authorized to approve participation.
  • Obtain the approval.
  • Convey the approval to the director or key contact at the facility.
These recruitment activities continued throughout data collection. Table A shows the level of effort expended, as measured in elapsed days, from recruiters’ first initiating the Set an Appointment Call to successfully scheduling the initial appointment.
Of the 2,422 initial appointments made during the field period, 401 facilities (17%) rescheduled their appointments. About 5% of appointed cases never completed an in-person facility interview.
Table A. Elapsed time from first appointment call to set appointment
Elapsed time Appointed facilities
Number Percent Cumulative percent
Less than 1 day 432 18 18
1–7 days 450 19 36
8–14 days 252 10 47
15–28 days 333 14 61
29–77days 479 20 80
78 days or more 478 20 100
Total appointed facilities 2,422 100 ...

. . . Category not applicable.

NOTE: Percentages may not add to 100 due to rounding.

Facility Case Transition

For the majority of cases, the shift in responsibility from recruiter to interviewer occurred after recruiters scheduled an appointment for the facility. Some facilities that delayed setting appointments, however, were transferred to the target interviewer, who then assumed responsibility for scheduling the appointment.

Interviewer Training

A 5-day interviewer training session was held in April 2010 with 96 interviewers. Training consisted of plenary sessions for all attendees and smaller concurrent training sessions, each with about 14 interviewers, divided mostly according to assigned region. The major topics covered in training were: overview of the study; computer applications demonstration; confidentiality procedures; Facesheet and Event Calendar system; reminder call; appointment rescheduling; maintaining cooperation; administering the facility, resident selection, resident, and debriefing instruments; procedures for handling ‘‘don’t know’’ responses; and data retrieval. In all, 86 interviewers successfully completed training, nine received a conditional pass, and one failed. Those who received a conditional pass were given specific remedial tasks and retested, and all of these interviewers eventually passed. The one interviewer who failed was dismissed, leaving a total of 95 interviewers to conduct the facility in-person interviews.

After data collection began, some interviewers also were trained via a telephone training session to perform recruiter activities during drive-by visits to hard-to-reach facilities. Drive-bys were used primarily to determine if the facility was still in business, attempt to gain cooperation, screen facilities for eligibility, and schedule an in-person interview, although some also included in-person interviews.

View full report


"NSRCFdo.pdf" (pdf, 8.47Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®