U.S. Department of Health and Human ServicesU.S. Department of Health and Human Services
This report was prepared under contract #290-02-0003 between U.S. Department of Health and Human Services (HHS), Office of Disability, Aging and Long-Term Care Policy (DALTCP) and the RAND Corporation. Additional funding was provided by HHS's Agency for Healthcare Research and Quality. For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/_/office_specific/daltcp.cfm or contact the ASPE Project Officer, Jennie Harvell, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. Her e-mail address is: Jennie.Harvell@hhs.gov.
The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.
The United States health care system is at risk due to increasing demand, spiraling costs, inconsistent and poor quality of care, and inefficient, poorly coordinated care systems. Some evidence suggests that health information technology (HIT) can improve the efficiency, cost-effectiveness, quality, and safety of medical care delivery by making best practice guidelines and evidence databases immediately available to clinicians, and by making computerized patient records available throughout a health care network. However, much of the evidence is based on a small number of systems developed at academic medical centers, and little is known about the organizational changes, costs, and time required for community practices to successfully implement off-the-shelf systems.
An analysis of the usefulness of implementing HIT must take into consideration several factors:
The Leap Frog Group and a number of components of the U.S. Department of Health and Human Services (HHS)--the Centers for Medicare & Medicaid Services (CMS), the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of Disease Prevention and Health Promotion (ODPHP), and the Agency for Healthcare Research and Quality (AHRQ)--requested a review of the research on HIT to compile and evaluate the evidence regarding the value of discrete HIT functions and systems in various health care settings. This Evidence-based Practice Report on the costs and benefits of health information technology systems, along with an accompanying interactive database that catalogs and assesses the existing evidence was prepared by the Southern California Evidence-based Practice Center (EPC). This report systematically reviews the literature on the implementation of HIT systems in all care settings and assesses the evidence in four specific circumstances:
An electronic search of PubMed, the Cochrane Controlled Clinical Trials Register, and the Cochrane Database of Reviews of Effectiveness (DARE) was conducted for articles published from 1995 to January 2004. Additional references were obtained by reviewing the references in several major reports prepared by private industry and by RAND Health. Two reviewers, each trained in the critical analysis of scientific literature, independently reviewed each study and resolved disagreements by consensus. The principal investigator resolved any disagreements that remained unresolved after discussions between the reviewers.
Studies selected for review had to be either:
Of 855 articles screened, 256 were accepted for review. Descriptive studies of HIT implementations were identified and classified according to the categories listed below, but were not reviewed in more detail.
The contents of each selected article or report were abstracted using electronic data-abstraction forms prepared especially for this analysis. Abstracted data included the system's capabilities, interventions used, study design, implementation processes, evaluation methods, outcomes, costs, and barriers to implementation. A structured abstract was created for each report; these abstracts can be accessed in an online, interactive database created for this evidence report. (This database can be accessed at http://healthit.ahrq.gov/tools/rand.)
Overall Results
Of the 256 studies reviewed, 156 were about decision support, 84 assessed the electronic medical record, and 30 were about computerized physician order entry (CPOE). One hundred twenty-four of the studies assessed the effect of the HIT system in the outpatient or ambulatory setting, while 82 assessed its use in the hospital or inpatient setting. Ninety-seven studies used a randomized design. There were 11 controlled clinical trials, 33 studies that used a pre/post design, 20 studies that did a time series, and another 17 that were case studies with a concurrent control. Among the 211 hypothesis-testing studies, 81 contained at least some cost data.
Many of the studies concerned HIT systems developed and evaluated by academic and institutional leaders in HIT.
Studies from these institutions have contributed greatly to our knowledge about the usefulness of particular HIT functionalities (such as CPOE or computerized electronic alerts), and are examples of what can be realized by the implementation of broadly functional HIT at these specific institutions. But these studies also have limitations, in terms of their usefulness to inform decisions about the adoption of HIT elsewhere. The primary limitation is that these HIT systems were developed over the course of many years by technology champions at these institutions and, in a process of co-evolution, were adapted particularly to the working environment and culture of their respective institutions. Consequently, the "intervention" at these sites consists not only of the HIT system but also the local champions, who were often also the evaluators in published studies. Furthermore, it is challenging to calculate the cost of the development of the HIT system as a whole, since this process occurred over many years at each institution. In addition, these systems are not commercially available from a vendor--and vendors supply most HIT systems in use in the U.S.
We were able to identify only 15 studies that used a randomized or controlled clinical (RCT or CCT) design, included cost data, and assessed HIT systems that were not from one of the leading academic and institutional HIT institutions or the United Kingdom (another setting that has limited generalizability to U.S. health care institutions). When these 15 studies were examined for their HIT functionality using the classification system developed by the Institute of Medicine, 4 of them concerned only decision support and 4 assessed HIT systems with decision support and administrative processes. The remaining seven studies addressed other single functionalities or combinations of up to three functionalities. We were not able to find a single study that used a randomized or controlled clinical trial design, that did not report data from one of the leading academic or institutional HIT systems or the U.K., that reported cost outcomes and that assessed an HIT system including at least four of the eight IOM categories of functionality.
For the 103 hypothesis-testing studies that used a design other than a randomized or controlled clinical trial, 45 reported cost data. Of these 45 studies, 23 assessed systems that were not one of the leading academic or institutional HIT systems or that came from the U.K. An examination of these 23 studies for their functionalities showed, as in the studies using an RCT or CCT design, that most studies did not evaluate systems with a broad level of functionality. Five studies assessed only decision support, and three studies each assessed only administrative processes or order entry management. Three studies assessed HIT systems with two functionalities, order entry management and decision support. The remaining nine studies assessed various combinations of two or three functionalities. No study evaluated an HIT system with at least four of the eight categories of functionality.
The literature is even sparser regarding information about the organizational context of an HIT implementation. Of the hypothesis-testing studies, we identified only 3 studies that provided information about the financial context of the organization, such as the degree of managed care/capitation penetration; 6 studies with information about system penetration; 2 studies about facilitators to implementation; 1 study explicitly discussing sustainability of the HIT intervention; 12 studies reporting extrinsic factors in valuing costs and benefits such as the health care market competitiveness; and 6 and 9 studies, respectively, reporting the initial costs of the HIT system and costs of implementation.
In summary, we identified no study or collection of studies--outside of those from a handful of HIT leadership institutions--that would allow a reader to make a determination whether the study's reported benefit was generalizable. Besides these studies from HIT leaders, no other research assessed HIT systems that had comprehensive functionality while including data on costs, relevant information on organizational context and process change, and data on implementation. This limitation in generalizable knowledge is not only a matter of study design and internal validity. Even if further randomized, controlled trials are performed, the generalizability of the evidence would remain low unless additional systematic, comprehensive, and relevant descriptions and measurements are made regarding how the technology is utilized, the individuals using it, and the environment it is used in.
The Costs and Benefits of HIT in Pediatric Settings
Early evidence shows that stand-alone clinical decision-support systems (CDSS) (such as drug dosing calculators) can reduce medication dosing errors, and CPOE plus CDSS can reduce the incidence of harmful medication errors in the inpatient pediatric and neonatal intensive care settings. Other HIT systems, such as electronic medication administration records, pharmacy-based robots, smart infusion pumps/devices, and medication bar-coding, are predicted to reduce medication errors, but need further study.
The use of CPOE plus CDSS has been demonstrated, in separate studies, to (1) reduce the frequency or duration of antibiotic use for common pediatric illnesses such as pharyngitis and otitis media, and (2) improve completeness and reduce variation in clinical documentation. In the ambulatory setting, a single study showed that an appointment reminder system is cost-effective and significantly reduces missed appointments, while in the neonatal intensive care unit, another study showed that CPOE can reduce medication and radiology turnaround times. Therefore, the evidence for HIT cost-savings in pediatrics is limited, but appears optimistic.
Electronic Health Records and the Quality of Ambulatory Care
Adoption of EHR systems is widely believed to be critical to the delivery of consistent, high-quality health care, although the current use of EHRs is limited. Seven studies were identified on the use of EHR in four ambulatory care settings (three in the United States and one in the Netherlands). The findings reported in all of these studies were primarily related to the implementation processes and to changes in clinical processes.
With the exception of one study that examined the effects of incorporating HIV care guidelines and alerts on quality of care for HIV-positive patients, all the studies assessed the effects of adding various types of information related to laboratory test and prescription ordering to EHR ordering screens. In general, these studies showed that providing laboratory test guidelines and related information on test-ordering screens was associated with a decrease in orders for overused tests and an increase in orders for underused tests; provision of formulary guidance was associated with increased adherence to a formulary for at least one class of medication; and addition of HIV care guidelines and alerts was associated with improved quality of care.
The Economic Value of an EHR System
While EHR systems may be essential for improving efficiency and quality of health care, implementation of an EHR system requires substantial capital investments and organizational change. Consequently, many health care organizations are seeking evidence from previously implemented systems about the costs and benefits of EHR adoption in order to better inform decisions about the optimal timing and strategy for implementation.
Not all of the costs and benefits reported when implementing new systems or making changes to existing systems were financial. EHRs were associated with improvements in service and other resource utilization, provider productivity, care efficiency, documentation quality, clinical decisionmaking, guideline compliance, and costs of care.
Despite considerable variation among the few studies that modeled financial costs and benefits, all predicted substantial cost savings from EHR implementation. However, these studies each made a number of assumptions, and the predicted break-even points ranged from as short a time as 3 years to as long as 13 years.
HIT and Patient-Centered Care
The evidence is sparse for the ability of HIT systems to make health care more patient-centered. The best evidence of such a change is the beneficial effect on preventive care of using computerized reminders to patients. Telemedicine and consumer health informatics also have limited evidence of benefit in specific contexts. The evidence is much more limited about the health effects of more general, interactive health information technologies such as the Internet or e-mail, or the effect on patient trust and satisfaction of implementing HIT systems such as the electronic health record.
Barriers to HIT Implementation
Studies identified a large number of barriers to the implementation of HIT. These barriers can be classified as situational barriers (including time and financial concerns), cognitive and/or physical barriers (including users' physical disabilities and insufficient computer skills), liability barriers (including confidentiality concerns), and knowledge and attitudinal barriers. Cutting across all of these categories, however, may be the need for a major structural and ideological reorganization of clinical medicine as it is now practiced in the majority of settings to be able to integrate itself with and enjoy the benefits of HIT.
Limitations of the Review
General Conclusions
| The Full Report is also available from the DALTCP website (http://aspe.hhs.gov/_/office_specific/daltcp.cfm) or directly at http://aspe.hhs.gov/daltcp/reports/HITcb.htm. |