Evaluation Design of the Business Case of Health Information Technology in Long-Term Care: Final Report

U.S. Department of Health and Human Services

Evaluation Design of the Business Case of Health Technology in Long-Term Care: Final Report

Executive Summary

Booz Allen Hamilton

July 13, 2006

This project summary was prepared under contract between the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Office of Disability, Aging and Long-Term Care Policy (DALTCP) and Booz Allen Hamilton. For additional information on this subject, you can visit the ASPE home page at http://aspe.hhs.gov or contact the ASPE Project Officer, Jennie Harvell, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201, Jennie.Harvell@hhs.gov.

The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.

Health information technology (HIT) is increasingly viewed as a tool that can promote quality and cost-effective care in the U.S.¹ Promoting the use of HIT is a major health initiative of the current Administration and HHS. In April 2004, the President established the Office of the National Coordinator for Health Information Technology (ONC) through an Executive Order. The Executive Order and the strategic framework developed by ONC emphasize the need for:

• Evidence on costs, benefits and outcomes associated with HIT implementation; and
• Reducing the risks that providers face in making HIT investments.

This need to establish evidence on the costs and benefits associated with HIT is driven by the limited number of systematic studies that examine these costs and benefits across care settings. The lack of a robust evidence on HIT costs and benefits is especially conspicuous in the post acute care (PAC) and long-term care (LTC) environment. A deeper, evidence-based understanding of costs and benefits is needed and is essential to inform providers contemplating purchase of HIT systems. In addition, such an understanding can provide useful and reliable information to policy makers, payors, employers, and others who seek to influence HIT adoption.

To promote this understanding, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) within the Department of Health and Human Services (HHS) engaged Booz Allen Hamilton (Booz Allen) to design an evaluation to assess the business case for HIT in PAC and LTC settings. The purpose of this project is to develop cost-effective robust study design option(s) that can greatly enhance the existing knowledge base on HIT costs and benefits in the PAC/LTC setting. In designing this evaluation Booz Allen sought to address key research questions which include:

• What types of PAC/LTC providers should be included in the evaluation design?
• What types of alternative HIT functionalities should be included?
• What are the categories of costs and benefits associated with HIT implementation?
• To whom do costs and benefits accrue?

Booz Allen developed alternative study approaches based on reviews of published literature, and stakeholder interviews. The literature review included an examination of the PAC and LTC environment, and cost-benefit studies of HIT in PAC/LTC and other settings. In consultation with a Technical Expert Panel (TEP) these approaches were narrowed to a single recommendation which is presented in this report along with the supporting evidence justifying this approach.

Overview of the PAC/LTC Environment

The PAC/LTC environment is complex representing a heterogeneous group of patients with chronic illnesses and those recovering from acute events. Patients in PAC/LTC settings can be young or old, and usually require an array of medical and nursing therapy services, as well as personal and supportive services for an extended period. PAC/LTC may be provided in a variety of settings including nursing facilities (NFs), assisted living facilities, adult day care, and home and community-based services. While there are alternative types of facilities where PAC/LTC is delivered, care is predominantly provided in nursing home (NH) and home health (HH) settings. Approximately 88% of PAC/LTC care users receive care in one of these two settings.²

Due to the variety and complexity of conditions relevant to patients in PAC/LTC, these patients experience frequent transitions from acute to PAC/LTC settings, as well as transitions between PAC/LTC settings.^{3, 4} Coleman and Berenson’s targeted review of patient transfer literature highlights the issue of transfers as a leading challenge in the delivery of healthcare, particularly in the Medicare population.⁵ These frequent transitions across the continuum of care represent “risk points” at which important clinical information may not be transmitted or may be transmitted incorrectly, creating gaps in quality and opportunities for error.⁶ Health information exchange has the potential to reduce errors and improve quality of care.

In addition to the complexities associated with a heterogeneous, high morbidity, frequently transitioning patient mix, the PAC/LTC environment faces significant staffing shortages and retention challenges due to the heavy reliance on nursing staff. HIT may help alleviate this problem through improved workflow efficiencies and improved staff satisfaction.

There are two other aspects of the PAC/LTC environment relevant to determining the accrual of HIT costs and benefits: sources of financing (i.e., who pays for the services) and payment methods (how are they reimbursed). These two aspects affect who receives the benefits and therefore how likely they are to invest in HIT. Services provided in PAC and LTC settings are primarily financed by Medicare, Medicaid, and out-of-pocket sources. An analysis of 2004 National Health Accounts Data estimates that 41% of LTC is financed by Medicaid ($65B), 23% by personal out-of-pocket ($37B), 20% by Medicare ($32B), 9% by private insurance ($14B) and the remainder by public or private sources.^{7, 8}

Reimbursement for PAC/LTC services varies depending on who pays for the services. Medicare uses the Prospective Payment System (PPS) to reimburse skilled NFs, HH agencies, inpatient rehabilitation facilities, and LTC hospitals. State Medicaid programs devise their own payment systems for NF and home and community-based service providers. Payment systems vary greatly by state and by type of provider. Many states reimburse NHs through a PPS similar to Medicare’s. Recent studies of Medicaid programs and financing trends indicate that some states have moved from cost-based systems to PPS as a cost-containment strategy.⁹ Alternative reimbursement methods can create differing incentives for HIT adoption, and understanding the distribution of benefits is important to policy makers.

HIT in PAC/LTC

Based on TEP guidance, stakeholder input and a review of the literature, we have focused our analysis on HH and NH services. As stated previously, a majority of the PAC/LTC services are provided in these settings. The NH setting has been emphasized because of the acuity of NH patients, the volume and intensity of the services provided, and the frequency of transfers. Emphasis on the HH setting is based on the continued growth in HH utilization, and the desire to facilitate further migration of care from institutional settings to the home.

Our analysis of HIT in PAC/LTC therefore focused on functionalities relevant to the NH and HH settings. Based on TEP input and stakeholder discussions, we have identified a number of HIT applications, tools, and functionalities that are particularly relevant in NH and HH environments. Exhibit 1 provides a brief description of these functionalities.

These applications may be used independently or in combination at varying levels of functionality, based on the unique needs and characteristics of the setting pursuing HIT implementation. There is also functional overlap between some of these tools, applications and functions. While anecdotal reports suggest that some PAC/LTC settings have already implemented POC and medication management tools to improve patient safety and quality of care, there is relatively little quantitative information regarding the actual penetration of these applications in PAC/LTC facilities.

In addition to stand-alone functionalities, the potential for interoperability across HIT systems holds significant promise for benefits and return on investment (ROI). For this reason, public and private groups have been working to develop standards and standard functionalities to facilitate interoperability (e.g., HL7 efforts to develop standards for an EHR Functional Model and public/private efforts to apply HIT standards to the federally required NH Minimum Data Set).

Costs and Benefits of HIT in PAC/LTC

Although there is a paucity of peer-reviewed literature quantifying the costs associated with HIT acquisition and implementation in a PAC/LTC setting, Booz Allen has used the literature on ambulatory and acute inpatient HIT costs to provide insights on the types of costs that may be incurred with an HIT implementation. Based on the HIT literature, stakeholder discussions and TEP input, we have developed a breakdown of cost categories, including:

• Selection costs;
• Acquisition costs (e.g., hardware, software, training); and
• Recurring costs (e.g., maintenance, upgrades, subscription fees).

These costs are likely to vary based on factors such as facility size, levels of functionality and connectivity, length of implementation, and extent of user training.

There are also a limited number of studies that focus on measuring benefits of HIT in a PAC/LTC setting.^{10, 11, 12} It is widely asserted that HIT can provide significant clinical benefits, in terms of safety, quality improvements, and enhanced efficiency. This is particularly relevant to the PAC/LTC setting since patients are especially susceptible to adverse drug events due to the use of multiple medications, comorbidities associated with multiple chronic conditions, and increased utilization rates of drugs.^{12, 13, 14} Evidence from studies in these settings suggests that CPOE systems and decision-support can help reduce medical errors. The effects of HIT on workflow and labor efficiency in the ambulatory and inpatient environment appear equivocal and highly dependent on usability and appropriate re-engineering of work processes. The evidence that HIT can reduce redundant testing appears to be more consistent.

Return on Investment (ROI) and Benefits Accrual

Analysis of net benefit or ROI of an HIT implementation in the PAC/LTC setting is limited. We are aware of only a single prospective study, and that analysis was conducted in a single institution over a period of about one year yielding equivocal results.¹⁰ Given the limited evidence on HIT costs and benefits in the PAC/LTC environment, we also examined peer-reviewed cost-benefit literature in the ambulatory and inpatient environment. A small number of these studies in the acute and ambulatory settings suggest that positive provider ROI may be achievable. However, positive net benefit in these studies from the provider’s perspective is dependent on a variety of factors including: quality of the implementation and workflow redesign, the level of functionality and usability, and the nature of the reimbursement environment (capitated vs. fee for service). The measurement of benefits is also highly dependent on the period of assessment. It may require up to three years (or more) for financial benefits to exceed costs and shorter studies may not capture these benefits, leading to the erroneous conclusion that a positive ROI may not be achievable.

Though PAC/LTC provider facilities bear the financial burden of HIT investments, benefits may accrue to others such as payors or patients. Whether or not a particular entity benefits from HIT investment is a function of:

• Who pays for the services;
• How the reimbursement is structured (e.g., capitated vs. fee for service); and
• The types of benefits (e.g., cost savings from generic switching or reduction of redundant tests), which are determined by the functionality of the system adopted.

Understanding the distribution of benefits is important for policy makers who seek to influence adoption of HIT. It is therefore essential that a formal evaluation of HIT costs and benefits include a thorough mapping of benefits to determine which benefits accrue to which stakeholders.

Demonstration and Evaluation Recommendations

After reviewing the literature and conducting stakeholder discussions, Booz Allen developed a range of alternatives to assess the business case for HIT in PAC/LTC settings. The options included several prospective and retrospective study designs, with or without an ASPE funded implementation. Booz Allen presented these options to the TEP for comments. The TEP response emphasized the need for a cost-effective study that could generate data-driven findings on the business case for HIT within a reasonable timeframe (2-3 years).

Based on these recommendations Booz Allen has proposed that ASPE conduct a retrospective study of 10-20 sites, focusing on NHs, HH, or both. While a prospective study would be methodologically more rigorous, the time horizon (probably in excess of five years) would be unacceptably long given the need for evidence-based guidance in the short term. Cost would be another limitation. The retrospective design would leverage existing HIT applications in PAC/LTC settings, saving both time and implementation costs. This approach would include quantitative and qualitative methods, using both administrative and interview data to inform the estimation of costs and benefits. It is realistic to expect that such a study may be completed within 18-36 months.

The purpose of this study will be to:

• Develop an improved understanding of the specific clinical and non-clinical HIT functionalities used in PAC/LTC settings;
• Estimate the costs and benefits associated with these functionalities; and
• Gain an understanding of the distribution of these costs and benefits among relevant stakeholders.

There are significant gaps in the understanding of the business case for HIT. This study will advance the state of knowledge on the costs, benefits and net benefits of an HIT implementation in PAC/LTC. In addition, it will provide an enhanced understanding of the distribution and timing of these benefits. This study could also provide insight into minimum system functionality requirements to determine potential reimbursement incentives should the Centers for Medicare and Medicaid Services (CMS) establish pay-for-performance for HIT adoption in the future.

Study Methodology

Booz Allen proposes a seven-step methodology for conducting this evaluation study. Exhibit 2 displays these seven steps.

EXHIBIT 2. Evaluation Study Steps

The evaluator will begin by identifying the specific HIT applications and functionalities used in the PAC/LTC environment. Choice of study functionalities will be influenced by stakeholder discussions and preliminary hypotheses regarding which are most likely to produce a positive ROI for the adopting provider.

After identifying the HIT functionalities of interest, the evaluator will develop a list of associated cost and benefit metrics that can be estimated as part of the study. The metrics developed must be targeted to the specific HIT applications and functionalities studied. These must be crafted so that both the baseline and outcome metrics are measurable in a retrospective analysis. Data availability and access will be a major consideration.

Site selection will occur in two steps. The evaluator will first create an initial universe of potential sites and then, in consultation with ASPE, narrow that universe to a final site list. Decisions at each Step will be based on specific criteria and a specified method of site selection. An important criterion is whether a site possesses an appropriate set of HIT functionalities as defined in Step One. Other considerations include: level of connectivity with other facilities; adopter status (e.g., early adopter, late adopter); organization size and type (e.g., chain of facilities, free standing facility, 50 beds, 200 beds); types of residents; and pre-existing information technology capabilities and experiences including legacy systems and corresponding implementation constraints.

The evaluator will develop a data collection plan that includes use of inperson site interviews and administrative data such as claims data, cost, and quality reports. When developing the data collection plan, the evaluator should be aware of the limitations of data sources and availability when gathering pre and post-implementation data retrospectively, and incorporate mitigation strategies. The data collection plan should include design and development of data collection instruments.

Upon completion of the data collection, the evaluator will conduct both qualitative and quantitative analyses and then produce a report that details the methods and findings. The evaluator will provide a discussion of those findings and make recommendations on relevant policy issues offering suggestions for future studies of HIT in PAC/LTC settings.

Challenges and Risks

We are aware that the proposed study, while cost and time efficient, possesses certain limitations. The retrospective study design limits the ability to measure the baseline, or pre-implementation state, against which to measure the impact of HIT. This may present particular challenges in assessing impact on workflow and labor efficiencies since recall bias and staff turnover, especially if the baseline state was remote in time, may cloud retrospective perceptions of the pre-implementation state. The small sample size may make it difficult to generalize to the broader population of NHs or HH agencies, particularly if there is significant variation in the types of facilities, populations, and HIT functionalities being studied. Inclusion of a comparison group would strengthen the study design and increase confidence that observed trends attributed to HIT implementation were causally related. However, cost constraints may prohibit a comparison group at this stage.

In addition, this study will not assess the adoption rates of different types of technology and functionality across the PAC/LTC sector. Such an assessment would be useful in understanding the broader baseline state of the industry. Booz Allen has recommended that ASPE consider such a survey, perhaps in partnership with another agency or organization.

There is a risk that this study may not demonstrate a benefit from HIT implementation or may even demonstrate a negative ROI. Given the paucity of existing systematic research this could have a disproportionately negative effect on PAC/LTC adoption. This would be particularly unfortunate if failure to demonstrate ROI was based on a study approach that failed to capture what, in reality, was a positive net benefit. As noted above this could relate to an inadequate assessment of the inefficiencies, and the costs related to them, in the baseline state. Failure to demonstrate a positive ROI could also result from examining a post-implementation period that is too short to allow benefits to accumulate. Studies in the ambulatory environment suggest that it may take two to three years to achieve a positive net benefit.

While these challenges and risks were considered, they were balanced with the need to develop an empirically based approach to measuring the benefits of HIT that could yield results in the relatively near future. Careful attention to study design and execution may mitigate these risks. Finally, we view this as a first step in developing a body of robust evidence on HIT costs and benefits driven by data.

Summary and Conclusion

The PAC/LTC environment is complex due to the nature of the population, the multiple care settings, the frequent transitions between care settings, and the convoluted and varied reimbursement mechanisms. While aspects of this care environment, such as frequent transitions and a population on multiple medications, suggest that HIT may produce significant qualitative and financial benefits, there is a paucity of systematic studies to support this hypothesis. Even if financial benefits are realized they may not accrue to the provider organizations making those investments.

Accordingly, ASPE has proposed to study the business case for HIT in the PAC/LTC environment so that these benefits may be quantified and mapped to the appropriate stakeholder. The outcomes of this study will be useful for PAC/LTC providers considering HIT investments, and for policy makers, payors, employers and others who may seek to influence HIT adoption in PAC/LTC settings. Booz Allen was engaged to assist ASPE in conducting the research and analysis necessary to design a cost-efficient approach to this study.

Based on review of the literature, stakeholder discussions and TEP recommendations, Booz Allen has proposed that this demonstration be performed as a retrospective analysis of HIT implementations in 10-20 PAC/LTC settings. This study will include both quantitative and qualitative data and analysis. While a larger sample size and comparison group would add to the rigor and generalizability of the study, resource constraints may preclude those options. Despite the small size of the evaluation, it would represent a significant contribution since there is a paucity of methodologically sound analyses of costs, benefits and net benefits of HIT in the PAC/LTC environment.

In addition to the recommended study design in this report, we suggest that ASPE consider other studies focused on HIT in PAC/LTC environment such as a survey of the prevalence of HIT in PAC/LTC. However, we consider the current study under consideration the most important to conduct first since it will provide much needed evidence and guidance to the broad spectrum of stakeholders in the PAC/LTC environment.

ENDNOTES

1. Rand Health. Health Information Technology: Can HIT Lower Costs and Improve Quality? 2005. Available at http://www.rand.org/pubs/research_briefs/RB913b/RAND_RB9136.pdf.

2. The Henry J. Kaiser Family Foundation (KFF). Kaiser Commission on Medicaid and the Uninsured. Medicaid and Long Term Care. May 2004. Available at http://www.kaiserfamilyfoundation.org/medicaid/upload/Medicaid-and-Long….

3. MedPAC. Report to the Congress: Issues in a Modernized Medicare Program. June 2005. Accessed at http://www.medpac.gov/publications/congressional_reports/June05_ch5.pdf.

4. Murtaugh C, Litke A. Transitions through postacute and long-term care settings; Patterns of use and outcomes for a national cohort of elders. Medical Care 2002; 40(3):277-236.

5. Coleman E, Berenson R. Lost in transition: challenges and opportunities for improving the quality of transitional care. Ann Intern Med 2004; 140:533-536.

6. Coleman E. Falling Through the cracks: Challenges and opportunities for improving transitional care for persons with continuous complex care needs. Journal of the American Geriatric Society 2003; 51:549-555.

7. National Center for Health Statistics, Health, United States. 2004.

8. KFF. Medicaid and Long-Term Care. March 2005. Accessed January 2006 at http://www.kff.org/medicaid/2186-03.cfm.

9. Coleman B, Fox-Grage W, Folkemer D. State Long-Term Care: Recent Developments and Policy Directions. National Conference of State Legislatures. July 2002. Accessed February 2006 at http://aspe.hhs.gov/daltcp/reports/stateltc.htm#preface.

10. Cherry B, Owen D. Evaluation of a Web-based Electronic Medical Record and Communication System for Long-Term Care Facility Management. Final report to the Department of Aging and Disability Services. Austin, TX: Texas Department of Aging and Disability Services. November 22, 2004.

11. Rochon P, Field T, Bates D et al. Computerized physician order entry with clinical decision support in the long-term care setting: Insights from the Baycrest Centre for Geriatric Care. J Am Geriatr Soc 2005; 53:1780-1789.

12. Gurwitz JH, Field TS, Avorn J et al. Incidence and preventability of adverse drug events in nursing homes. Am J Med 2000; 109:87-94.

13. Gurwitz JH, Field TS, Judge J et al. The incidence of adverse drug events in two large academic long-term care facilities. Am J Med 2005; 118:251-258.

14. Field TS, Gurwitz JH, Avorn J et al. Risk factors for adverse drug events among nursing home residents. Arch Intern Med 2001; 161:1629-1634.