Development of Final Set of Core and Supplemental Risk-Adjusters
The selection of the final set of core risk-adjusters was based on findings from the preliminary analyses, comments of TAG members, and examination of a small number of additional OASIS items provided by the University of Colorado following the TAG meeting. The analyses conducted after receipt of additional OASIS data included respecification of the Living Situation and Informal Support/Assistance risk-adjusters. Specifically, alternative specifications were explored utilizing the more detailed data on living arrangements (with the lives with spouse/family category in initial models separated into two categories) and the person providing assistance.
The additional data and respecification, however, did not substantially affect the contribution of the living situation and informal support/assistance measures to the explanatory power of the HHQI risk-adjustment models that already included demographic, payer and clinical measures. The one exception is the risk-adjustment model for Improvement in Medication Management. When the living arrangement and social support measures were added to a model with demographic, payer and clinical measures (i.e., added to Model 3), the R-squared statistic increased from 15.7% to 16.7%. These conceptually important measures were excluded from the alternative models because of the limited contribution to the explanatory power of the risk-adjustment models.
Table 3 lists the final set of core risk-adjusters in the alternative models along with their specification. A total of 43 OASIS items were used to construct the core risk-adjusters. The demographic and insurance measures clearly are likely to be included in electronic health records and the remaining items are all clinically relevant. The one core risk-adjuster that varies from model to model is the baseline value of the outcome indicator. The baseline value, specified as a categorical variable, tends to make a relatively large contribution to the explanatory power of risk-adjustment models. It appears to be adjusting for differences in the probability of improving (or stabilizing) related to the number of levels of the OASIS item.
Risk-adjusters specific to each outcome, other than measures of health status prior to admission, are listed in Tables 4a-4d. They are reported by domain of the outcome indicator (e.g., Table 4a lists the risk-adjusters specific to ADL outcome models). Some items are common to all risk-adjustment models within a domain. For example, obesity is included in the risk-adjustment models of all ADL outcomes. Other items are specific to a single outcome. For example, whether a patient smokes is specific to the Improvement in Dyspnea risk-adjustment model. Generally, 2-3 outcome-specific items were added to each risk-adjustment model. All of these items are clinical factors.
Tables 5a-5d list the measures of clinical status prior to home health admission that were added to the risk-adjustment models of selected OBQI outcomes. As noted above, these OASIS items were examined separately from other outcome-specific risk-adjusters because of questions about their reliability and possible elimination from the OASIS instrument. There were no directly related, conceptually important prior health status risk-adjusters used for four OBQI outcomes (i.e., Improvement in Dyspnea and the three utilization outcomes).
Comparison of Current and Alternative Models
The OBQI quality indicators are grouped into six broad domains by the University of Colorado: (1) ADLs, (2) IADLs, (3) Physiologic indicators, (4) Emotional/Behavioral measures, (5) Cognitive measures, and (6) Utilization Outcomes (see Table 1). We first present results from all models and then by domain. The models developed by the University of Colorado are referred to as the current models; the two final alternative models are referred to as the core alternative model (which includes only core risk-adjusters) and the full alternative model (i.e., Model 3 which includes outcome specific and prior OASIS items, or Model 2 where there are no relevant prior items).
The full alternative models typically have slightly lower explanatory power than the current risk-adjustment models. Specifically, the R-squared statistic for the full model tends to be within 1-2 percentage points of the R-squared statistic for the model developed by the University of Colorado. There is a similar pattern for the c statistic. While the number of OASIS items and elements is sometimes larger and sometimes smaller for the alternative models compared with current models, the overall number of OASIS items and elements employed when risk-adjusting all 31 OBQI outcome indicators is considerably smaller for the full alternative models (64 versus 88 OASIS items, and 93 versus 135 OASIS elements).
ADL and IADL Outcomes. The ADL and IADL outcomes represent 23 of the 41 OBQI quality indicators and over two-thirds of the 31 outcome indicators currently risk-adjusted by the University of Colorado. The performance (i.e., explanatory power as measured by the R-squared statistic) of the alternative and current risk-adjustment models for ADL and IADL outcomes is presented graphically in Figure 1 and Figure 2. Table 6a and Table 7a summarize the model statistics for all ADL and IADL outcome models, respectively, and Table 6b and Table 7b present the detailed regression results for the full alternative models estimated for the 23 ADL and IADL outcomes.4
As previously discussed, most of the full alternative ADL and IADL models have slightly lower explanatory power than the current models. This is not surprising since a stepwise approach was used to develop the current models. An exception is the alternative risk-adjustment model for the Improvement in Ambulation outcome where the R-squared statistic is more than six percentage points greater than the R-squared statistic for the current model. The ADL and IADL stabilization outcomes, it should be noted, are highly skewed (i.e., a very high proportion of those potentially able to stabilize do stabilize). This may explain the relatively low R-squared and relatively high c statistics for both current and alternative models.
The outcome-specific risk-adjusters generally contribute very little to the explanatory power of the ADL and IADL risk-adjustment models that already include the core risk-adjusters. In contrast, the prior OASIS items contribute substantially to the explanatory power (roughly two percentage points to the R-squared statistic) of almost all of the risk-adjustment models of improvement in ADLs and IADLs, but not stabilization in ADLs and IADLs. There is a similar pattern for c statistics.
Physiologic Outcomes. Figure 3 graphically presents the performance of the alternative and current risk-adjustment models for the five physiologic outcomes currently risk-adjusted in OBQI. Table 8a summarizes the model statistics for all physiologic outcome models and Table 8b presents the detailed regression results for the full alternative models estimated for the five physiologic outcomes that are currently risk-adjusted, and the alternative models with only core risk-adjusters for the four that are not currently risk-adjusted in OBQI.
The outcome-specific risk-adjusters tend to make a slightly greater contribution to the explanatory power of the physiologic outcome models compared to ADL and IADL outcome models. The effect of the prior OASIS items, on the other hand, is modest. Among the physiologic outcomes, the full alternative risk-adjustment model for Improvement in UTI performs considerably worse than the current UTI risk-adjustment model. The R-squared statistic for Model 3 is 5.9% compared to 12.1% for the current model, and corresponding c statistics are 0.665 and 0.740 (see Table 8a). The main reason for this difference is the exclusion of home health episode LOS from the alternative model.
Emotional/Behavioral Outcomes. None of the emotional/behavioral outcomes currently is risk-adjusted in OBQI. Only Model 1 (i.e., the model including only the core risk-adjusters) was estimated for outcomes that are not currently risk-adjusted. The model statistics for the alternative models for the three emotional/behavioral outcomes are reported in Table 9a. The detailed regression results for the final alternative models estimated for the emotional/behavioral outcomes are presented in Table 9b. The R-squared and c statistics for all three models are low.
Cognitive Outcomes. There are three cognitive outcomes in OBQI but currently only Improvement in Confusion Frequency is risk-adjusted. The right-most bar in Figure 3 graphically presents the performance of the alternative and current risk-adjustment models for Improvement in Confusion Frequency. Neither the outcome-specific nor the prior OASIS items contribute substantially to the explanatory power of the Improvement in Confusion Frequency model that already includes the core risk-adjusters. Table 10a summarizes the model statistics for all cognitive outcome models. Table 10b presents the detailed regression results for the full alternative model estimated for Improvement in Confusion Frequency as well as the alternative models with only core risk-adjusters for the two cognitive outcomes that are not currently risk-adjusted in OBQI. The R-squared and c statistics for all models are relatively low although the c statistic for the Stabilization in Cognitive Functioning risk-adjustment model that includes only the core risk-adjusters is 0.738 indicating adequate ability to predict what is a highly skewed outcome (i.e., over 90% of individuals who could stabilize did stabilize).
Utilization Outcomes. Figure 4 graphically presents the performance of the alternative and current risk-adjustment models for the three utilization outcomes (all three are risk-adjusted in OBQI). Table 11a summarizes the model statistics for all current and alternative utilization outcome models and Table 11b presents the detailed regression results for the full alternative models estimated for the utilization outcomes.
Two of the three outcome-specific variables at baseline (Dyspnea and IV/Infusion therapy) are highly statistically significant in the final, full risk-adjustment models for all three utilization outcomes (p < 0.001). Nevertheless, the outcome-specific variables as a group have only a very small effect on the explanatory power of the risk-adjustment models for the utilization outcomes. When added to models already including the core risk-adjusters, the R-squared and c statistics increase by at most roughly half a percentage point or 0.005, respectively. No prior OASIS items were included in the alternative models for these outcomes. As noted previously, the exclusion of LOS reduces the explanatory power of the alternative models for the utilization outcomes.
Comparison of Overall Number of OASIS Items and Elements Used in Risk-Adjustment
The overall number of OASIS items used in current and alternative risk-adjustment models (out of a total of 95 M0 items) is graphically presented in Figure 5. The core OASIS items in the alternative models are in the lower left-hand corner shaded in the darkest color. On the diagonal (in the next darkest shade) are the OASIS outcome specific and prior items included in the full alternative models (i.e., Model 3 for the outcomes with prior OASIS items and Model 2 where there are no relevant prior items). The OASIS items for the additional variables used in one or more of the current risk-adjustment models but not in the alternative models are in the next darkest shade. Sixty-four OASIS items were used to construct the risk-adjusters included in one or more of the full alternative models, compared to 88 for the current models developed by the University of Colorado. There are seven OASIS items that are not used in either the current or alternative models (unshaded in the upper-right-hand corner of Figure 5). The M0 items used for case-mix classification in the Medicare prospective payment system are in bold with an asterisk.
Some OASIS items include multiple elements with each element separately assessed and marked (i.e., the OASIS items with instructions to mark all categories that apply). The OASIS elements used in current and alternative risk-adjustment models are graphically presented in Figure 6 in the same manner as the OASIS items in Figure 5. There are a total of 180 OASIS elements with 93 used to construct the risk-adjusters in the full alternative models compared to 135 in the models developed by the University of Colorado. All OASIS elements in the alternative risk-adjustment models also are used in current models with two exceptions: the Current Payer elements Medicaid traditional fee-for-service (M0150_3) and Medicaid HMO/managed care (M0150_4), both of which are highlighted on the left side of Figure 6. The M0 elements used for case-mix classification in the Medicare prospective payment system are in bold with an asterisk.