The purpose of this project was to develop and test alternative risk-adjustment approaches to assessing the quality of home health care. A data-driven stepwise approach currently is used to risk-adjust OBQI quality indicators with a separate set of risk-adjusters in each outcome model. In this project, a theory and evidence-based approach was used to develop alternative risk-adjustment models for the OBQI quality indicators. Advantages of a theory and evidence-based approach include simplicity, understandability, stability of the risk-adjustment models over time, conceptual meaningfulness, and the potential for greater parsimony in data elements when a large number of outcome indicators are being risk-adjusted as is the case in the OBQI program.
The alternative models were developed within the framework of the uniform data collection system (OASIS) in place at the time of the study. A project goal was to develop alternative models that could be implemented using existing data sources and project resources limited analyses to OASIS data elements. The examination of alternative risk-adjusters developed from other data sources (e.g., Medicare claims) is an important area of future research.
Based on theory and prior empirical research, a core set of risk-adjusters was identified from among the content areas covered by OASIS. These core items were included in the risk-adjustment models for all outcomes. A small number of outcome-specific risk-adjusters then was added to each model. The outcome-specific risk-adjusters are OASIS measures of patient status on admission, as well as status prior to admission, plausibly related to a specific outcome or outcome domain.
At the time of this study, 31 of the 41 OBQI quality indicators were risk-adjusted in either OBQI or HHQI. The analysis focused on a comparison of the current and alternative models for these 31 outcomes. In particular, it focused on statistics that measure how well a model predicts an outcome, as well as the number of OASIS items and elements needed to construct the risk-adjusters. While the OBQI quality indicators represent six broad health and functional domains, 22 of the 31 risk-adjusted outcomes (over 70%) are ADL or IADL outcomes.
There are important tradeoffs and differences between the current and alternative approaches to risk-adjusting OBQI quality indicators. The first is the generally higher explanatory power of the current models versus the simplicity of the alternative models and their overall reliance on a smaller number of OASIS items and elements. That current models generally have marginally better explanatory power than the alternative models is not surprising since the stepwise approach is likely to result in models with close to the best explanatory power possible for the data set analyzed. At the same time, however, it leads to the selection of a large number of risk factors when all outcome measures are considered. In addition, because the stepwise approach fits models to the data on which they are developed, the explanatory power of these models is likely to decline when they are applied to new data sets.
A second tradeoff is between the full alternative models that include the outcome-specific risk-adjusters and alternative models with only the core set of risk-adjusters. The latter tend not to predict outcomes as well as the full models. Measures of physical functioning prior to home health admission are particularly significant in the risk-adjustment models of ADL and IADL improvement. The prior OASIS items, however, are more difficult than many other items for home health agencies to collect and are thought to be less reliable than other clinical measures. Should they be dropped from the OASIS instrument, the explanatory power of the risk-adjustment models for most ADL and IADL improvement models would be reduced roughly two percentage points.
The decision to exclude home health LOS from the alternative models, in addition, has a significant impact on the risk-adjustment models for the small but important subset of utilization outcomes. LOS was excluded because it can be affected by problems in the care process that also affect outcomes (i.e., low quality care can cause a longer stay as well as worse outcomes). If LOS is included in risk-adjustment models, conclusions about the quality of agency care could be erroneous due to quality problems being risk-adjusted away. The TAG convened to review preliminary models developed by the project team strongly supported the decision to exclude LOS from risk-adjustment models. The consequence, however, is reduced explanatory power for a small number of outcomes. A possible methodological solution, which has data burden and simplicity implications, is to collect information on the timing of all of the utilization outcomes (e.g., hospitalization) and estimate hazard models that take into account the time to the outcome of interest.
Our agency-level analysis examined how the alternative approaches to risk-adjustment of the OBQI indicators affect an agencys quality ratings as currently calculated by CMS. For most agencies and most outcomes the adjusted proportion of patients with an outcome is similar regardless of whether the current or the full alternative model is used to risk-adjust outcomes. Of greater potential concern to providers, however, is the ranking of each agency relative to others, irrespective of the size of the difference in risk-adjusted outcomes. Our analysis found that the ranking of agencies using current risk-adjustment models and the ranking using the full alternative risk-adjustment models are in close agreement for most outcomes.
The agency-level analyses were repeated using only the core risk-adjusters in the alternative risk-adjustment models. This was done in order to better understand the contribution of the outcome-specific and OASIS prior items to the finding of similar quality ratings regardless of risk-adjustment approach. The basic results hold. However, as would be expected, the quality ratings are not as close when outcome-specific and OASIS prior items are dropped from the alternative risk-adjustment models of the OBQI indicators.
One limitation of the agency analysis is that for some outcomes a relatively large share of agencies was excluded because they had too few patients with the potential to have the outcome (i.e., less than 20). Nevertheless, the results suggest that the relatively small reduction in explanatory power of most of the alternative risk-adjustment models is unlikely to have an effect on the ranking of the majority of agencies on OBQI quality indicators.
Overall, a theory and evidence-based modeling approach has the potential to simplify risk-adjustment and provide a consistent and stable basis for risk-adjustment relative to the current approach. This should make it more understandable to providers and encourage individual agencies to risk-adjust their own outcomes. The reliance on a smaller number of OASIS data elements, in addition, would contribute to the Departments efforts to streamline the OASIS instrument and potentially facilitate the identification of a parsimonious set of clinical measures appropriate for data exchange in an electronic health record environment.