Medicare Post-Acute Care: Quality Measurement Final Report. C. Instrument Development

03/29/2001

Following the expert panel proceedings, we reviewed quality measures corresponding to the panels’ most highly rated quality indicators and developed quality assessment instruments for post-acute care. The most highly ranked indicators were in the following areas: Health Related Quality of Life (HRQL), physical function, depression, social/role function, pain, satisfaction, utilization, disease-specific symptom management, and performance of key care processes. In reviewing the panels’ final ratings, it was apparent that emphasis should be placed on global measures of quality applying to all patients served in post-acute care, but selected disease-specific measures were required for each condition.

1. Review of Existing Instruments

Complicating uniform quality measurement across post-acute care settings is the fact that all post-acute care settings use, or in the case of IRFs will use different assessment tools that are nationally mandated. The Federal Government requires that the MDS 2.0 (Minimum Data Set, Version 2.0) be used for all patients in Medicare (and Medicaid) certified nursing facilities and the OASIS be used for all skilled patients treated by Medicare HHAs. As previously mentioned, each of the instruments contains its own elements that are generally non-comparable and different assessment time points.

Clinicians and researchers who were expert panel members for this study identified as important outcomes in the measurement of the quality of post-acute care, outcomes that are not captured by existing federally required patient assessment instruments. For example, these tools do not measure whether a setting maximized a patient’s functioning, prepared him/her to return to and remain in the most independent living environment, and reintegrated him/her into prior lifestyle. Measuring such important outcomes requires input directly from patients or their proxies, as the latter two are subjective and the patient’s perception of those outcomes is, in fact, the measure of the outcome. In fact, many researchers focused on patient-centered outcomes believe that functional outcomes should derive from patients, as their perception of their function is more important than so-called objective measures of function. Patient-perceived functional capacity captures disability by integrating its relevance to the patient. If a difficulty performing a functional task is not related by a patient, it is likely of little importance or relevance to his or her life and needs. Existing patient assessment instruments use facility staff to measure outcomes, often an registered nurse or therapist, and thereby fail to capture the relevance of individual functional measures to the individual patient. Because the goal of rehabilitation is to restore patients to their previous level of function, capturing their perception of function both before and after an acute event in their terms is essential to measuring quality. Further, the expert panelists indicated that many important outcomes cannot be measured within the time frames over which post-acute services are typically provided in most settings. Given that average lengths of Medicare covered stays in the mid-1990s were approximately 25 days in SNFs and 21 days in rehabilitation hospitals,33 it is not possible to determine whether post-acute care has successfully maximized a patient’s function, allowing sustained return to the most independent living environment, and reintegration into prior activities and lifestyle. Generally, these outcomes take 60-90 days before they begin to plateau.15;19 In addition, because assessment data is not comparable across settings and whatever data is collected is only collected while the patient is treated by a particular provider type, it is not possible to measure outcomes across episodes of care that include multiple post-acute settings.

During the development of precise measures, data items, and data collection instruments, the limitations of existing data systems for measuring quality across settings were apparent. First, they did not cover many of the important domains and measures determined to be important by the expert panels (e.g., health-related quality of life; role resumption, satisfaction). Second, existing data systems do not use the individual patient as the respondent, but are based exclusively on provider reports. Third, existing instruments do not use uniform time frames that could be compared across settings. Fourth, based on our extensive review of existing patient assessment tools, the elements cannot be rendered comparable from one setting to the next because of differences in data elements, scaling and timing of completion, with the exception of basic demographic variables and an estimate of a Barthel index upon post-acute admission.

2. Provider vs. Patient-Reported Data

In addition to the concerns raised above about the need to assess patient-perceived function, concern also exists about the reliability of provider reported data.34 The primary concern, however, is over validity of provider reports. Based on literature review and expert panel members’ input, the types of measures that are relevant to post-acute care (e.g., HRQL, mental health and depression, physical and social/role function, satisfaction with care) require patient input. While patient-reported data are necessary for appropriate assessment of the above domains, the validity and reliability of this data may be affected by such factors as social desirability and acquiescent response biases.35 In addition, cognitive impairment, which is relatively common in the Medicare post-acute care populations, further complicates collection of patient-reported data. However, even among cognitively impaired subjects, proxy input on such outcome measures is more likely to reflect the needs and values of patients than provider assessment.

3. Selection of Measures

In this section, we provide the rationale for selecting the specific quality measures and data items corresponding to the quality indicators chosen by the expert panels. The selection of measures was guided by several principles. First, the burden of data collection had to be kept to a minimum; shorter versions of questionnaires were chosen whenever feasible. Second, the measures had to be relevant to patient care provided during the post-acute period and relevant to all three provider types, rather than outpatient, acute hospital or long-term care. Third, redundancy within and between measures was to be minimized. Fourth, measures needed to be responsive to the quality of the post-acute care delivered and clinically meaningful change. Finally, where possible, validated and reliable measures were chosen, particularly if they had been evaluated in older, frailer patients and in post-acute settings.

Health Related Quality of Life (HRQL)

The expert panels reached unanimous consensus with regard to the importance of HRQL as a quality measure for post-acute care. Similar to other investigators in this area,14;36;37 we struggled with how to operationalize HRQL as an outcome measure. Lack of a standardized and precise definition in the literature was a major barrier. Other terms, such as general health status and quality of life, refer to similar concepts.38;39;40 The panels discussed the utility of several commonly used and validated HRQL instruments. The three most commonly used instruments considered were the Medical Outcomes Study Short Form-36 (MOS SF-36), the Sickness Impact Profile (SIP) (as well as the SIP for nursing home residents (SIP-NH)), and the Quality of Well Being Scale (QWB).41;42;43;44

The domains encompassed by global health status or HRQL measures such as the MOS SF-36 and the SIP share a high degree of overlap with many of the important quality domains recommended by the panels for inclusion in a quality measurement instrument: physical function, depression, pain, and social/role function. Recognizing this overlap, the question arose as to whether any of these instruments were sufficient in and of themselves as measures for these domains, and whether such an HRQL instrument like the SF-36 would suffice as the entire patient self-report component of the instruments. A number of concerns, however, precluded such an approach. First, these global instruments were not designed for use in older, frailer populations, much less populations receiving post-acute care. Thus, the specific items have varying levels of relevance to the activities and goals of post-acute care. Our concerns about the psychometric performance of these instruments in older populations have been echoed in the medical literature.45;46;47 This is not a criticism of these instruments, but a recognition that we are dealing with an older, frailer population in a period immediately following an acute event that requires a unique measurement strategy.

Furthermore, the level of detail measured by any of the individual HRQL instruments in a given domain is often limited such that some areas are not given the weight that our panels recommended. For example, while these instruments examine depressive symptoms, they do not allow for adequate assessment of whether depression is present. Because depression was rated very high by the panels for three of the four conditions, ability to measure it more precisely seemed important. Similarly, while pain is examined by all of the aforementioned HRQL instruments, most instruments have only one or two pain questions. As a clinical endpoint, two pain questions may not be adequate for conditions where pain may be the dominant symptom (e.g., back and neck).

Based on these concerns, as well as comments received during the panel discussions, we chose quality measures that are specific to each indicator that the clinical panels supported. Rather than use an existing comprehensive health status instrument, we have combined individual measures and indices of physical function, depression, social/role function and pain to create a more detailed patient report instrument in the areas chosen. Collectively, we believe that our instruments capture the essential elements of HRQL described in the medical literature,14 while simultaneously providing the level of detail necessary for the patient population served by post-acute care providers.

Physical Function

Numerous physical function measures were discussed. Represented measures were: the MOS SF-36 physical function scale,41 the FIM score,48 the Barthel Index,49 the Lawton IADL Index,50 the Katz ADL Index,51 the Kane ADL Index,52 and the Basic ADL Index.53;54 Selection of the final measure was guided by the need for adaptation for use in-person or by telephone (the individual may actually be discharged from post-acute care at time of follow-up), knowledge of prior performance in post-acute populations, and most importantly, relevance of question content. Based on these guiding principles, Wolinsky's Basic ADL Index, a five-item measure that incorporates the ADLs of bathing, dressing, toileting, transferring, and walking 20 feet, was selected for inclusion.53;54 This assessment of function focuses on self-reported difficulty in ADL performance, which has direct relevance to the provision of post-acute care. It was originally derived using data from the Longitudinal Study on Aging55 and has been used extensively in large national studies of older persons and patients receiving post-acute care.15;56;57 In these post-acute care studies, serial measurement was conducted at baseline corresponding to premorbid period, three, six, nine and twelve months with both in-person and telephone administration. Similar to prior studies that used this instrument in post-acute care, the outcome measure is the number of ADLs recovered to premorbid disability levels.58

In addition to an assessment of basic ADLs, we have also included questions about recovery of IADL function50 for patients with stroke. The stroke panel strongly advocated for inclusion of IADLs as a responsive measure of the quality of care delivered to persons with stroke. While IADLs are important to the care delivered in the other conditions considered, the burden of additional data collection required limiting inclusion to only those measures identified as essential. In subsequent sections, we further describe efforts to ensure that each instrument is comprised of only those measures of most relevance and significance.

Depression

Numerous instruments have been designed for self-assessment of depression in general populations.59;60;61 However, only one instrument has been specifically developed for use in the older population, the Geriatric Depression Scale (GDS).62 The GDS places greater emphasis on affective aspects of depression over somatic symptoms. Assessing depression in older post-acute patients is particularly challenging and the GDS has been employed in prior studies in this population.15;63 A shorter version of the GDS has been validated and used successfully. With half as many items, the shorter 15-item version of the GDS poses much less responder burden with minimal loss of sensitivity and specificity.64 For the purpose of quality assessment in post-acute care, we will measure change in GDS score over time.

Social/Role Function

These two quality indicators were combined into a single indicator based on the recommendation of the expert panels. The panels believed that change in a measure of social/role function would capture important aspects of functional status not addressed by the traditional measures of function, basic ADLs and IADLs. Though the HRQL instruments noted in earlier sections include items measuring social and role function, the wording and examples in the questions are not always relevant to older post-acute patients and appear not to fully capture the concept as it applies to this patient population or as envisioned by the expert panels.

The Reintegration to Normal Living (RNL) Index65 was recommended by several expert panelists and was selected based on the relevance and wording of the items for the population of interest, the development strategy employed in choosing the items, and its brevity. In addition to acceptable psychometric properties, the RNL has been specifically tested in post-acute populations.65 The authors of the RNL solicited input from three advisory panels made up of patients with chronic illness, healthy persons, health care providers, psychologists and clergyman to ensure broad-based input and representation.66 The final instrument is comprised of eleven items, uses a visual analog scale for response and can be converted to a 100 point score. Because we foresee the need to administer follow-up interviews by telephone to measure outcomes of post-acute care, we have adapted the visual analog scale for telephone use using a five-point Likert scale. Further, several of the questions are concerned with activities of less relevance to measuring quality in the post-acute population. These questions have been removed from the instruments.

Pain

Evaluation of change in pain control was rated highly for back and neck patients. This prompted a search for a validated pain instrument that referred to a time interval relevant to post-acute care (i.e., four weeks or less). Some pain scales we reviewed placed too much emphasis on affective symptoms and thus were redundant with the depression items covered by GDS.67;68 The McGill Pain Instrument69 and the Back Pain Classification Scale69 were evaluated, but were perceived to lack face validity for the population under study. The MOS Pain Measure, which has proven to be valid and reliable in varied populations,70 was selected for these reasons and because it measures both pain severity and the effects of pain on function. In contrast, the Oswestry Low Back Pain Disability Questionnaire only examines the latter.71

Satisfaction

The relationship between patient satisfaction and quality of care continues to become more accepted and established among leading methodologists.72;73;74 The literature on patient satisfaction has been historically dominated by the inpatient hospital setting, followed by the outpatient setting as a distant second site of care. No single instrument has been developed for all three of the post-acute sites of interest for this effort. The instruments designed for satisfaction assessment in nursing homes place more emphasis on living arrangements and autonomy pertinent to a long-term care population rather than time-limited rehabilitation pertinent to post-acute patients. Similarly, questions on instruments used in home care are not easily translated to older patients who are not residing in their homes. The most promising of all satisfaction instruments reviewed was developed by researchers at the Rehabilitation Institute of Chicago.75 The main advantage of this instrument stems from the fact that the wording of the individual items allows for administration in all three sites of post-acute care. However, the length of the full instrument limits its utility in our instruments. Fortunately, the individual items have been validated and we have recommended that a representative and relevant sub-group of questions be incorporated into our post-acute care quality measurement instruments. These questions focus on the patient’s perceptions of the care delivered and how well their needs are being met with respect to returning to independent living.

Utilization Measures

Uniformly, the panels recommended inclusion of use of emergency services and rehospitalizations during the post-acute episode as indicators of quality of care. In addition, to account for the possibility that a given patient might use more than one type of post-acute care (e.g., SNF followed by home health care), total post-acute utilization was also recommended as a quality measure. Finally, the panelists recommended including, if applicable, the date of death.

Disease-Specific Measures

The number of disease-specific patient report indicators recommended varied by condition. While these indicators have intuitive appeal, few have been operationalized from the standpoint of quality measurement. In many cases, the disease-specific measures added to the patient report section have not been formally tested. For example, the back and neck panel argued that existing pain and disability instruments did not adequately address change in limb numbness and tingling for patients with back and neck conditions. These symptoms represent important target symptoms that may not be captured under a narrow definition of nociceptive pain. The back and neck instrument has been supplemented with these questions accordingly. Change in symptoms of dyspnea, fatigue and cough for patients with pneumonia were modified from formal instruments.76 For CHF, a validated disease-specific quality of life measurement instrument is included to assess the change in how CHF symptoms affect quality of life over time. The first four items of the Chronic Heart Failure Questionnaire have been included specifically to address control of symptoms.76

Additional Measures for Risk Adjustment

Additional questions were included to account for important differences in case mix when comparing quality across settings of post-acute care. Potential adjusters that could be derived from patient self-report include cognitive status, social support, and demographics. In addition to these potential self-report risk adjusters, we identify other potential risk adjusters in subsequent sections of this report. The Pfieffer Short Portable Mental Status Questionnaire (SPMSQ) has been shown to be both valid and reliable77;78 and was selected as a potential cognitive screen. The SPMSQ was developed to provide a rapid screen (approximately two minutes) for cognitive impairment in older patients and has been used extensively in both community and institutional settings.79;80 The Mini Mental Status Exam (MMSE)81 was also selected as a potential cognitive screen. The MMSE is longer and more strenuous than the SPMSQ, requiring ten minutes and thirty points. The MMSE is more commonly used as a risk adjuster in qualitative research than the SPMSQ and allows finer delineation between different levels of cognitive dysfunction. The MMSE and SPMSQ will also serve as indicators of when a proxy informant may be required for the patient self-report component of the instruments. Social supports and living arrangements will be assessed using selected items from the Older Americans Resource and Services (OARS) instrument.82 Standard demographic data will also be assessed from patient self-report for the purpose of risk adjustment. These characteristics include: age, race, gender, education and marital status.

Process Measures

The process quality measures, both global and disease-specific, were selected by the panels because of an association with high quality care and, in some cases, with defined clinical outcomes (e.g., ACE inhibitor use and mortality in CHF). However, there are some logistical and measurement considerations worth noting. Similar to the patient report symptom measures, some of the indicators rated highly by the panels do not have corresponding validated measures. Other highly rated indicators were not included because measurement was simply not possible (e.g., time to reach clinical stability following pneumonia). Thus, process measures as recommended by the expert panels were incorporated judiciously.

4. Instrument Design and Selected Measures

Patient Report

To measure most of the outcome domains specified above (physical function, social/role function, mental health, and satisfaction with care), two separate patient report instruments were created for each of the four conditions: an admission instrument and a follow-up instrument. The admission instruments were designed to be administered within ten days of post-acute care admission to assess baseline function one month prior to hospitalization. Initially, the section on physical functioning included measurement of perceived physical function at the time of the interview, in addition to the existing questions about function one month prior to the hospitalization. However, during pilot testing, we found qualitative difficulties administering the questions related to function at two distinct time periods, with subjects becoming confused. As a result, the baseline patient interview will not be used to assess function at the time of entry into post-acute care because of the difficulties encountered when trying to ask patients or proxies to describe their functional capacity at two separate time points, both one month prior to hospitalization and at the time of post-acute care admission. A tight time interval between admission and completion of the baseline instruments is not required since the questions largely refer to pre-hospitalization function and status, with the exception of mood related questions encompassed in the GDS. The follow-up instruments were designed to be administered at a pre-determined follow-up time point and to assess recovery to baseline function. For CHF and pneumonia, the follow-up time point chosen was 60 days after post-acute care admission; for stroke and back and neck conditions, the follow-up time point chosen was 90 days after post-acute care admission (see discussion in Section D1).

Provider Report

The source of provider reported data will be the medical and post-acute care record. This component is needed to assess specific process measures advocated by the expert panels, and to obtain information about comorbidity for case mix adjustment.

Risk adjustment: The medical record is a valuable source of information to create comorbidity indices for the purpose of case mix adjustment. Indices such as the Charlson Index83 have been used extensively and are relatively simple to construct. The Charlson Index is comprised of a formula that provides a weighting of chronic illnesses. These diagnoses represent major conditions that would normally be represented in an individual patient’s problem list in the medical record. Additional hospital chart review items to be collected for risk adjustment for patients with pneumonia include evaluation for delirium, blood gas results, abnormal vital signs, and lab parameters.84

In addition, the provider reported component includes information obtained from several formal patient assessment instruments, including the MDS, MDS-PAC and OASIS. In an effort to reduce redundancy, relevant items from the MDS and OASIS (and once implemented, the MDS-PAC) will be abstracted. Additional data for case mix adjustment will also be abstracted from these data sources. Data on patient function at the time of post-acute admission can be obtained from these three sources for risk adjustment. Because all three instruments are required upon post-acute care admission and all three have basic function items that allow estimation of a Barthel Index, data from these can be used sparingly to risk adjust for function at the time of post-acute admission. In particular, physical function items from these instruments can be converted to a comparable Barthel Index. Results from a forthcoming validation study conducted by our research team reveal that the MDS 2.0-based Barthel Index compares favorably to an actual independent Barthel measure. Testing and refinement of the MDS-PAC and OASIS conversions are underway. The conversion codes are provided in Appendix G.

Process measures: In addition, selected process measures recommended by the expert panels would be obtained from the medical and post-acute care records.

  • Global process measures: With regard to global process quality measures, the panels recommended that regardless of condition or post-acute setting, an evaluation by a licensed physical therapist and occupational therapist are important and essential elements of high quality post-acute care. These were the only two process measures believed to be relevant to all four conditions.

  • Disease-specific process measures: The disease-specific quality of care processes reflect a wider range of care.

    • Stroke: For patients with stroke, quality measures include a speech and swallowing evaluation, and management by a multidisciplinary care team. In addition, efforts to educate the patient’s informal caregivers need to be recorded. For patients who experience a non-hemorrhagic stroke, consideration of anticoagulation must be explicitly documented in patient records. The day the patient’s stroke occurred relative to initiation of post-acute care will be determined for the purpose of risk adjustment.

    • CHF: For patients with CHF, those patients with compromised left ventricular function should either be on an ACE inhibitor or have the reason for not using an ACE inhibitor clearly evident in their records. In addition, patients with CHF need to have the frequency of body weight assessment documented in their records as well as an attempt by post-acute care providers to evaluate their medication compliance.

    • Pneumonia: Patients with pneumonia should have documentation of advance directives discussions and immunization status for influenza and pneumococcal pneumonia. Obtaining blood cultures during the acute course of illness was recommended by the pneumonia panel out of concern that this may be overlooked in patients with pneumonia who are not hospitalized before admission to post-acute care. Distinguishing community-acquired pneumonia from nosocomial with regard to antibiotic selection was also included for its relevance to post-acute care.

    • Back and Neck Conditions: For patients with back and neck conditions, the expert panel emphasized the importance of use of prescription analgesic medications. From the perspective of quality measurement, it is not always clear to what degree use of prescription medications is associated with quality. The appropriate selection of medication and dosing is often highly tailored to the individual. Instead, a much stronger case could be made for focusing on whether or not pain was systematically assessed and that a plan of care had been devised and documented. Thus, evidence of a revised plan of care designed to optimize pain control would be considered a high quality process of care.

Utilization Data

While the data needed to measure the utilization outcomes (e.g., emergency services, rehospitalizations, and multiple post-acute care use) would likely be obtained from claims data for patients treated under the Medicare fee-for-service program, these items are included in the instruments to be certain they are collected for HMO patients.