Data quality assurance during the field period was accomplished in a variety of ways. The quality assurance measures included field observations, verification calls, production reports, management meetings, and field memos and calls. These measures were implemented to ensure that data were collected consistently and reliably across the sampled facilities.
Between May 10 and November 3, 2010, NSRCF project staff observed a total of 57 facility interviews conducted by 32 interviewers. These observations served multiple purposes. They provided an opportunity for feedback to interviewers on their performance, determined whether interviewers were administering the questionnaires correctly so that mistakes could be addressed, and identified those who needed further training. In addition, observations provided a mechanism to evaluate the effectiveness of the survey questions, survey protocol, responses to questions, and respondent concerns. When observers detected issues that warranted clarification to field staff, field memos were prepared. These issues were also discussed during weekly team meetings between supervisors and field staff.
Verification calls served to detect interviewer falsification and to receive feedback from respondents regarding interviewer performance and survey protocol. Each interviewer’s first or second completed interview was selected for verification. After that, cases were chosen using an algorithm to achieve 10% verification of each interviewer’s completed cases. Field supervisors followed a script (Appendix XXV) to conduct the telephone verification. Almost all (95%) of the verification calls were done with the person who completed the facility interview. The most common concern raised by respondents during these calls was that the interview was too long. However, many respondents commented that the interviewers were very professional and pleasant and made the interview experience enjoyable. Verification calls were stopped in November 2010 (the last month of data collection), because they were not revealing any new concerns and no falsifications were found. Altogether, about 15% of all completed interviews were verified.
Weekly production reports were reviewed to track various aspects of the survey and its progress, including detailed information on the following:
Screener questionnaire case status by supervisor region and strata (e.g., number and percentage of facilities out of business, ineligible, refused, eligible and complete, still pending, and other final not complete cases).
Ineligible facilities by state and reason for ineligibility.
Facility questionnaire completion status by supervisor region and strata.
Screener and facility questionnaire status for chain-affiliated and independent facilities by strata and use of gaining cooperation letters.
Final case status by strata and geographic region (i.e., ineligible; noncompletes refusal; noncompletes other; completed facility questionnaire only; completed facility questionnaire and some resident questionnaires; and completed facility questionnaire and all resident questionnaires).
Total appointed cases and completed facility questionnaires by strata.
Facility-specific information for upcoming scheduled appointments.
Screener Outreach report results by strata.
In addition to monitoring production levels and response rates across strata and region throughout the field period, these reports proved useful in determining whether the hold sample of small facilities was needed. The reports also provided information about cases requiring extra effort to complete, such as very large facilities, initial refusals, facilities requiring drive-by visits, and cancelled appointments. Biweekly updates on the status of efforts to gain chain approval were also provided in the Chain Approval Report (Appendix XXVI). These reports provided a progressive snapshot of the current status of these contacts.
The NSRCF project management team held biweekly phone conference calls throughout the field period to review field status and developments and to discuss and resolve issues that arose in the field. Detailed minutes from each meeting were used in subsequent meetings to track progress on action items and further guide discussion on project activities.
Field Memos and Calls
At three points during the field period, field memos were sent to field supervisors, recruiters, and interviewers. The memos consisted of production updates, tips for using project materials, clarifications for specific items in the survey instruments, and protocol changes. In addition, field supervisors held weekly team meetings and one-on-one calls with recruiters and interviewers in their region.