Prepared for:
U.S. Department of Health and Human Services
Office of the Secretary
Assistant Secretary for Planning and Evaluation
Prepared by:
Prashila Dullabh, Adil Moiduddin and Elizabeth Babalola
NORC at the University of Chicago
June 2009
This report was prepared under the direction of James Sorace, Office of Science and Data Policy, Caroline Taplin, Office of Health Policy and Dale C. Hitchcock, Office of Science and Data Policy.
This report may be found online at:
http://aspe.hhs.gov/sp/reports/2009/ehrutil/index.shtml.
INTRODUCTION
NORC at the University of Chicago is pleased to present this report entitled “Measurement of the Utilization of an Installed EHR” for the Assistant Secretary for Planning and Evaluation (ASPE) at the U.S. Department of Health and Human Services (HHS). In the face of rising costs and concerns about quality in the United States health care sector, an emphasis has been placed on the critical role that health information technology (IT) will play. Electronic Health Records (EHRs) have great potential to improve patient outcomes, increase patient safety, and bring about overall improvements in the quality of care delivered. As a result, the President, Congress and others have placed a great deal of attention on promoting widespread adoption of EHR technology. The 2009 American Recovery and Reinvestment Act (ARRA) authorized approximately $36 billion towards health IT, with a significant amount to promote the ‘meaningful use’ and adoption of certified EHRs.[i]
While the concept and definition of meaningful use is yet to be agreed upon, it will include provisions for e-Prescribing, Clinical Decision Support (CDS), interoperability, and quality measurement.[ii] The regulatory interpretation of meaningful use will have important implications across the medical and health IT industries. EHR vendors with less comprehensive systems will argue that their systems will be able to meet meaningful use with only a few modifications and will advocate for looser measures of meaningful use. Vendors with comprehensive EHR systems are likely to argue for more specific interpretations. Similarly, there is concern on the part of organizations that have spent significant amounts of money purchasing systems that may not meet the requirements for meaningful use. Developing a better understanding of the functions that are most commonly used in EHRs provides a basis for defining key criteria for meaningful use. For example, many practices are beginning to use electronic exchange of lab results to ensure that they receive results on a more timely basis to improve internal efficiencies and improve the quality of care delivered.
In order to define meaningful use, it is also critical to consider the intended goals and outcomes of EHR use according to key stakeholders. Ultimately the motivation to adopt enabling technologies like EHRs is largely driven by the need to improve the quality of care that is being delivered. The Institute of Medicine, in its landmark reports To Err is Human and Crossing the Quality Chasm, highlighted the EHR as a method to improve health care quality and reduce costs. EHRs can facilitate quality improvement by reducing medical errors, providing easy access to patient information, acting as a tool to engage patients in their healthcare and providing alerts and reminders that improve patient safety. Many practices have cited quality improvement as a primary goal of their adopting an EHR system.[iii] Discussions on meaningful use by the National Committee for Vital Health and Statistics (NCVHS)[iv], Markle[v] and the Healthcare Information and Management Systems Society (HIMSS)[vi] all have identified quality improvement as one of the overarching priorities when defining meaningful use.
In spite of the great interest in EHR adoption a great deal is unknown about the use of specific features of EHRs, and there is no standard set of methods that reproducibly measures their utilization. Issues that contribute to making the question of quantifying adoption challenging include the lack of a clear definition of an EHR, a lack of standards to measure usage and inconsistencies in how EHR functions are described across the myriad of vendor products that are available today. There have been numerous survey efforts to capture estimates of the level of EHR adoption in the United States. However, survey approaches have some inherent limitations related to validity, reliability, granularity and positive sampling biases. In addition, while there are many studies which measure the rate of adoption of EHRs, there is significantly less information to be found regarding actual physician use of EHR features.
To further confound matters there is no standard definition of adoption for EHRs. A Robert Wood Johnson Foundation report titled Health IT in the United States: The Information Base for Progress defined adoption as “a process that, for measurement purposes, captures the acquisition, installation and use of EHRs”[vii]. It was recommended that, in order to achieve accurate results, EHR adoption surveys should assess these three domains. Based on this definition, it is clear that adoption does not necessarily equal usage. For many practices, it often only indicates implementation: studies that merely address this dimension of adoption are assessing system availability rather than the extent to which the technology is actually being used. In measuring the level of adoption in practices, some studies have taken the approach of full versus partial adoption, where the fully functional system contained clinical support features and included more advanced computerized physician order entry (CPOE) and results management features. [viii],[ix] This, perhaps more than other methods, most accurately captures the state of EHR use in the United States.
The purpose of this white paper is to discuss some of the limitations of current methods to assess EHR adoption, to enhance our understanding of EHRs used in ambulatory care settings and to explore the feasibility of systematic approaches of tracking EHR usage over time. While our interest in EHR adoption applies broadly to the ambulatory care setting, we focus more specifically on federally supported health centers and other safety-net providers. Nationally, community health centers provide medical care to more than 16 million individuals, many of whom are uninsured and/or have chronic conditions.[x] EHRs can contribute significantly to improving healthcare and reducing health disparities for this vulnerable population. Knowledge about EHR adoption and utilization in health centers is limited as there are few nationwide studies on this topic.
Our paper covers several topics that we have researched through review of the literature as well as through a series of discussions with health center and health center network representatives and experts in the field. Topics addresses in the paper include:
- An overview of major surveys that have been used to asses adoption and utilization including a review of some of the limitations of this approach.
- An overview of the major EHR models and the core functions of EHRs.
- A review of the EHR functions that are most commonly used and the use of standards to support interoperability.
- The development of a standard set of utilization metrics (including the use of encounter notes, medical history, medication lists, allergy lists, problems lists, order entry functions largely focused around lab order entry, the viewing of laboratory results and the use of patient demographics) that a certified EHR could automatically generate on an installation and provider specific basis.
- A discussion of conclusions as they relate to ongoing federal efforts to promote EHR adoption and meaningful use and the need for additional areas of analysis to address current gaps in knowledge.
To begin with, we provide a background section that includes our current knowledge regarding adoption and the current challenges associated with measuring utilization in a consistent manner. We also elaborate on the importance of developing methods to more objectively assess EHR use and discuss some options that may be available to accomplish this.
[ Go to Contents ]
IMPORTANCE AND RELEVANCE OF THE STUDY
For the past decade, adoption of electronic health records (EHRs) has been proposed as one of the most viable approaches to improving health care.[xi] Despite the profusion of initiatives aimed towards accelerating the adoption of EHRs and the rising impetus for practices to adopt EHR systems, the health care sector is far behind other industries with respect to IT adoption. [xii] At the same time, EHR adoption in the United States lags significantly behind that of many other Western countries. [xiii] Estimates of ambulatory EHR use in Austria, Belgium and Australia are 75%, 78% and 79-90% respectively while Denmark, England, Finland, the Netherlands and New Zealand have reported rates above 90%. [xiv] In fact, a report released by Harris Interactive showed that the United States was far behind all but a few European countries in terms of EHR adoption. [xv]
Nevertheless, adoption of Electronic Health Records in the United States is slowly progressing. Figures estimate that by 2006, 17% to 24% of physician practices had implemented EHR systems[xvi]. When a distinction between a fully functional and a basic system is made, the percentage of practices using a fully functional EHR becomes significantly lower. A Robert Wood Johnson Foundation funded seminal report outlined the minimal criteria for defining EHRs. EHR systems were categorized as “fully functional” or “basic” based on their functionalities. The primary differences between a fully functional and basic EHR was the absence of certain order entry capabilities and clinical decision support in the basic EHR system.[xvii] Based on these criteria, the 2008 National Ambulatory Medical Care Survey showed that 38% of physicians indicated having an EHR, of which only 4% reported using an EHR that could be identified as fully functional (patient demographics, problem lists, clinical notes, medical history and follow-up, orders for prescriptions, orders for tests, prescription orders sent electronically, viewing laboratory and imaging results, warnings of drug interactions or contraindications, out-of-range test levels, and reminders for guideline-based interventions) while 17% used a basic system (patient demographics, problem lists, clinical notes, orders for prescription, and viewing laboratory and imaging results). [xviii]
Central to the discussion on EHR adoption is the need to have a consistent, industry accepted definition of an EHR. Developing consensus on a precise definition of an EHR has been a challenge. Many studies have been impeded by poor or vague definitions of EHRs which have resulted in limited findings. In 2008, the National Alliance for Health Information Technology (NAHIT) defined the EHR as “an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed and consulted by authorized clinicians and staff across more than one health care organization”[xix]. For the purposes of this study, we adopt this definition.
In addition to defining an EHR, another challenge has been developing a consistent set of functions that constitute an EHR. Early work was done by the IOM who developed the initial framework to identify the functionalities that constitute an effective EHR system[xx]. Health Level Seven (HL7) [xxi] , a standards development organization, has further expanded the work done by the IOM and most recently the Certification Commission for Health Information Technology (CCHIT)[xxii], an independent, voluntary, private-sector initiative, has extensively leveraged the HL7 functional model to develop their certification criteria for EHR vendors. Despite these efforts, there remains significant variability in how vendors describe different functions within the EHR which further confounds efforts to assess current EHR adoption and use in a consistent and reliable way.
It is important to recognize that the adoption of EHRs does not simply stop at installation. The dimensions of system use must also be incorporated into any study truly seeking to understand the rates of EHR adoption. EHRs are composed of many distinct features and assessing the use of these specific functionalities provides insight into how EHRs are being adopted in practice.
Finally, utilizing automated reporting mechanisms within EHRs appears to be a potential avenue for obtaining accurate data regarding EHR adoption rates. With automated reporting, aggregated data can potentially be delivered on a periodic basis directly from the EHR to a central database for analysis. With automated reporting, there would be no requirement for physicians or technical staff to be recruited for survey completion. This could overcome many of the limitations inherent in self-reporting measures. The process would be far less labor-intensive for providers and office staff and would allow for more frequent and up-to-date assessments of EHR usage. In addition automated reporting would potentially reduce inconsistencies across different sites, thereby contributing to maintaining the veracity of the data collected. Trusted third-parties can then compile this data into a user-friendly format that will provide a more accurate assessment of current adoption patterns and can be used to inform further funding approaches and research. Implementing an automated reporting mechanism can have a dramatic impact on the state of EHR adoption research.
As of 2009, while there have been no coordinated efforts to develop an automated reporting tool to report data on the use of particular EHR functions, CCHIT in its testimony to the National Committee on Vital and Health Statistics Executive Subcommittee hearing on “meaningful use” of health Information technology in April 2009, suggested the creation of an automated reporting tool as an open source project. [xxiii] In addition to assessing current EHR functionality and the current use of EHRs in the safety-net and other ambulatory care settings this report will also review what efforts are currently underway to automate the data collection process and the feasibility of implementing automating reporting on a large scale.
[ Go to Contents ]
STUDY APPROACH
In conducting this study, we sought to develop a clearer understanding of what efforts have been undertaken to assess EHR adoption at a national level, to understand the typical scope of EHR implementation in the field and specifically to explore how EHR functions were being used. We begin our overview of the study with a brief discussion of the overall approach and specific methods used to collect and analyze the information presented in this paper. NORC conducted a review of published and unpublished literature to identify previous studies of EHR adoption. We also conducted targeted Internet searches using the Google and Google Scholar search engines in order to identify government reports, unpublished articles and other relevant resources. There was specific effort made to identify studies assessing EHR adoption and use in ambulatory settings, as well as those focusing specifically on health centers and the safety-net. To further inform the study, we conducted telephone conversations with individuals representing major ambulatory EHR vendors, health center representatives, ambulatory practices and other key informants involved with quality improvement and EHR adoption.
To develop a comprehensive list of the key clinical and administrative functions within an EHR system, we reviewed the initial IOM core functionalities of an EHR system[xxiv] and expanded on this by looking at the HL7 functional model and the CCHIT criteria for ambulatory EHRs. We also reviewed the major surveys assessing EHR adoption, identifying the common EHR features that were examined in each. We organized the features into eight function-based categories: “Organize Patient Data”, “Compile Lists”, “Receive and Display Information”, “Order Entry (CPOE)”, “Decision Support”, “Communication and Connectivity”, “Administrative and Billing Support” and “Other”. Effort was made to be as accurate as possible when categorizing the EHR functionalities. As a result, there are both granular and broad functions listed in each category. For instance, some surveys mentioned the higher level function of CPOE but did not specify particular tests. In these cases, the higher-level feature “CPOE (‘Order Tests’)” was selected. The results of our analysis are illustrated in Appendix A.
In identifying which surveys to include in our analysis, we used several criteria. First, we focused our search on surveys of ambulatory settings. Second, we sought to identify surveys utilizing nationally representative samples. Finally, surveys of EHR use in health centers were specifically included. To the extent that any additional surveys were deemed to be of use in our study, we also included them in our analysis. The result is that 9 surveys were identified (Appendix B). Six of these studies were large-scale national studies that measured EHR adoption, two were surveys developed to assess health IT use in health centers and one was a statewide survey assessing EHR functionality and the level of physician use of the specific functions. This final survey, conducted by Simon et al., was used as part of their study to identify current gaps in EHR adoption and usage. It was one of the few instruments that included items assessing the availability of different EHR functions as well as the degree to which the physician used each of those functions. We were able to successfully obtain all 10 survey instruments. These instruments are described in the paragraphs below.
Our literature search identified three recurring national representative surveys that assessed EHR adoption. The National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS) both included the same sections measuring EHR use in practices. The Center for Studying Health System Change Community Tracking Study Physician Survey included a section assessing the use of IT in physician practices.
Four other key national studies emerged. The National Survey of EHR Adoption was developed by the DesRoches et al. study team and represents one of the most comprehensive studies on EHR adoption to date. Also included was the Commonwealth Fund National Survey of Physicians and Quality of Care, a 2003 survey which explored physicians’ use of IT tools. The Medical Group Management Association (MGMA) conducted a 2005 survey to assess the adoption of health IT in their medical practices.The2007 Office Systems Survey which was administered by the Centers for Medicare & Medicaid Services (CMS) as part of their Doctors Office Quality Information Technology (DOQ-IT) initiative was also included in our analysis.
Two surveys of health center adoption of health IT emerged. A 2005 survey administered by the Community Clinics Initiative to assess information management in health centers was identified. Finally, the 2006 National Association of Community Health Centers (NACHC) survey of Health Center Use of Electronic Health Information was the first national measure developed to specifically access health center adoption of HIT.
We developed an initial list of key informants based on the findings of our literature search, through discussions with ASPE and using previously-acquired contacts in the field. A classic snowballing technique was utilized in order to further identify candidates for interview. A listing of key informants and the themes and concepts covered with them is provided in Exhibit A below.
| Participant | Organizations | Discussion Topics |
|---|---|---|
| Health Center Networks and Practices |
|
|
| Informants |
|
|
| Vendors |
|
|
The health centers included in this study were not intended to be a representative sample but instead are a small subsample of early EHR adopters and their protégés. Examples of the key discussion areas covered with health centers include a background of the organization’s use of EHRs, an overview of the features and functions available in the system and evaluation of the use of the system (we made an effort to assess how and if health centers used the majority of the functionalities offered and in what percentage of patient encounters the EHR was used). Health centers were also asked if they were aware of any capabilities that their EHR system had to capture usage data and if any internal measures of EHR use were conducted. In analyzing respondents’ answers, we sought to identify any common EHR use experiences of the practices.
The main themes covered in the discussions with vendors included a background of the vendor product, CCHIT certification, an overview of the processes the vendor had instituted for collecting and reporting on usage data and questions regarding the vendors’ technical capabilities for an automatic reporting mechanism. We also made an effort to understand vendors’ business reporting policies and the technical architecture of each product.
[ Go to Contents ]
CURRENT METHODS OF ASSESSING EHR ADOPTION
Current methods of assessing EHR adoption are largely focused on survey methods, in some cases augmented by site visits and key informant discussions. Several key ongoing surveys exist. The National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS) are annual probability surveys conducted by the Center for Disease Control’s National Center for Health Statistics. The NAMCS is a survey of non-federally employed office-based physicians practicing in the 50 states or in D.C., excluding radiologists, anesthesiologists, and pathologists. The NHAMCS focuses on hospital emergency and outpatient departments, not including federal, military and Veteran’s Administration hospitals.[xxv] The Center for Studying Health System Change conducts the Community Tracking Study (CTS) Physician Survey, a nationally representative survey of non-federal physicians. Five rounds of the survey, which covers a range of topics including physicians’ IT use, have been conducted, with the latest occurring in 2008.[xxvi] The Centers for Medicare & Medicaid Services (CMS) in 2006 and 2007 conducted the Office Systems Survey, a federal survey that was administered as part of their Doctors Office Quality Information Technology (DOQ-IT) initiative. The Community Clinics Initiative (CCI), supported by the Tides Center and the California Endowment, has conducted four rounds of its Information Management Assessment Survey. The last round of the survey was administered in 2005 to the 190 CCI grantees.[xxvii]
Among the national one-time surveys of EHR adoption, several need to be mentioned. The NACHC survey was the first national survey which assessed adoption of health IT systems by community health centers. A 2005 MGMA survey and the 2003 Commonwealth Fund survey also provided valuable information regarding EHR adoption.
Research has shown that survey estimates of EHR adoption can be largely dependent on the survey method, EHR functionalities measured, timing of implementation, and on the clinical context examined (i.e., size of the practice, inpatient vs. outpatient use).[xxviii]
Limitations of Surveys to Assess EHR Adoption. While surveys have been used extensively and have provided most of the current information regarding the current state of adoption there are shortcomings to using this approach. Some of the general issues with survey methods include ensuring adequate response rates, respondents that are targeted for completion of the survey and positive self selection bias in that individuals that are more technically savvy may be more inclined to respond. Below we also highlight some additional factors that may influence the results that are obtained through surveys.
Timing of Implementation. Researchers have noted that the timing of an EHR’s implementation can influence the nature of EHR use in the facility. For systems that have been adopted more recently, more functions and advanced functionality are likely to be included in the EHR. However, the practice may be less apt to use these additional functions. An explanation that has been proposed for this phenomenon is that there is a learning curve wherein physicians with recently implemented systems tend to primarily make use of the more basic EHR features. In a study comparing the EHR use of early and later adopters, Menachemi et al. found that EHR systems used by the more recent adopters appeared to be missing key patient safety and cost control functions such as those of electronically prescribing medications, weight based dosing, patient education materials and coding advice to physicians as these had not been implemented by sites.[xxix]
EHR Functionalities and Lack of Standardized Definitions. Another subject of concern is that the general public does not yet fully understand how EHR systems are defined. A recent study by DesRoches et al. found as much as a 9% difference in reported rates of EHR adoption when a distinction was made between a full EHR system and a partial EHR system.[xxx] Another study, assessing HIT adoption in 725 Community Health Centers, found that of 177 facilities that reported having either a full or partial EHR system, 49% did not actually meet the minimum requirements for EHR functionality that had been set forth in the Robert Wood Johnson report.[xxxi] Additionally, a report from the Centers for Disease Control and Prevention indicated that almost 1 in 5 of physicians surveyed who reported that they had a “full electronic medical record” also indicated that their system was unable to maintain clinical notes or access laboratory results, both features regarded by many experts as necessary for even a basic EHR system. [xxxii] The 2005 American Academy of Family Physicians survey, which did not provide respondents with a definition of EHR, found that 46% of those surveyed reported having an EHR. This estimate is far higher than those obtained in studies that precisely defined EHRs, suggesting that the way in which the EHR system is defined does impact results.[xxxiii] With all data collection efforts, there is a fundamental need to develop a clear, succinct definition of the variable of interest. However, some surveys allow respondents to themselves determine whether their system qualifies as an EHR system. Similar issues can arise in terms of the definition of adoption used in the study. Thus, study results are likely to vary when a distinction is made between EHR implementation and actual use.
Role of Clinical Context. In terms of practice setting for instance, rates of EHR adoption and use have been found to significantly differ between large and small or solo practices. Generally, larger practices are significantly more likely to have implemented an EHR system. [xxxiv],[xxxv],[xxxvi],[xxxvii],[xxxviii] For these practices, not only is the cost per physician of system implementation and maintenance lower, but the necessary capital and resources are more readily available. Additionally, larger practices would be better able to handle the initial decrease in productivity that would occur as clinicians became adjusted to the novel workflows associated with the use of the EHR. Larger practices have also been found to be more likely to exhibit higher levels of functional use of EHR technology. Several explanations have been suggested for this. It is possible that these larger practices are better able to support and train their clinicians, that they might have implemented systems that are more user-friendly; or, by virtue of their volume of physicians, allow clinicians more time to fully learn how to use the system. [xxxix]
Variations in Survey Methodology. This significant level of variance in study methodology and in survey content results in a lack of consistency for study results, making it a challenge for any single survey approach to yield a reliable estimate of EHR adoption. Furthermore, because reported rates vary so greatly between studies, attempts to conduct meta-analyses or to rely on averages between studies are hindered.
Moreover, the installation and availability of a particular EHR feature to physicians in the practice does not indicate usage. Indeed, when questioned, some physicians have indicated that their practice has an EHR system available but that they do not personally make use of its features.[xl] Very few of the surveys identified in this literature review included measures assessing the degree of EHR use. Many merely captured the implementation aspect of adoption rather than assessing actual system use by physicians. The 2001 NAMCS was the first to include items assessing EHR adoption. However, not until the 2005 survey did the NAMCS begin including questions about EHR functionality.[xli] Similarly, in many of the studies reviewed, the measure of EHR adoption was limited to a single item asking respondents to indicate whether their practice had adopted EHR technology.[xlii] Such measures are insufficient to measure relevant EHR use and also do not capture frequency of EHR use.
The reality is that many practices that report having a fully functional EHR system in place might only be making use of some of its basic capabilities such as recording patient demographics, clinical notes, problem lists and medication lists. While EHRs’ quick physician access to patient records provide an immense advantage to paper charts, it is the higher-level functions such as Clinical Decision Support (CDS), e-Prescribing (eRx) and laboratory order entry that have been shown to have the greatest impact on improving patient outcome, reducing costs and increasing efficiency.[xliii], [xliv] Nevertheless, these more advanced EHR features are likely to remain unused. There are numerous reasons why these functions may not be used-- ranging from EHR systems design issues, provider workflow interruptions that result in reduced efficiency for the physicians, and lack of awareness of full EHR capability. For instance, it has been noted that some EHRs’ CDS systems are more sensitive than others, resulting in an inordinately high number of drug interactions being reported. When faced with such systems, physicians are likely to turn the alerts off (assuming they are given this option). Although it would be useful to better understand providers’ use of the specific EHR features, few of the past EHR studies have included such questions.
As discussed beforehand, the mere fact that practices have “adopted” EHR technology does not indicate that EHRs are being used. Although a particular facility might have EHRs available, participation might not be mandatory and individual practitioners might choose not to make use of the system. For instance, a study conducted by Simon et al. found that less than half of the physicians who had systems with clinical decision support, transmittal of electronic prescriptions, and radiology order entry actually used any of these functions most or all of the time.[xlv] Therefore, assessing adoption simply by looking at availability is an inadequate measure of the actual use of specific functions of the EHR.
[ Go to Contents ]
EHR FUNCTIONALITY REQUIREMENTS
There has been considerable work done by federally funded initiatives, standards development organizations and private organizations involved in quality improvement to develop a framework of EHR functions that includes a description of the different functions, interoperability standards and support for collecting and reporting on quality metrics. These activities have played a significant role in building industry consensus on the scope of an EHR system and have influenced the development of EHR products and certification efforts.
We begin with a brief overview of the key industry activities that have been undertaken to identify functionalities of an EHR system. In response to a request from the Department of Health and Human Services (DHHS), in 2003 the IOM developed a comprehensive list of the key care delivery-related capabilities of an EHR system.[xlvi] These features are organized into eight categories and are arranged by both time frame of implementation and site of care. The eight categories are: Health Information and Data, Results Management, Decision Support, Order Entry/Management, Electronic Communication and Connectivity, Patient Support, Administrative Processes and Reporting and Population Health Management. Additionally, IOM gives a detailed basis for the inclusion of each feature and highlights their potential benefits as well as their primary and secondary uses.
To expand the initial work done by IOM, in 2004 Health Level Seven (HL7), an international standards development organization (SDO) at the forefront of standards development for clinical and administrative data, released the initial EHR – S Functional Model (EHR – S FM), an overview of the key possible functionalities of an EHR system[xlvii]. Since 2004, the EHR-S FM has been refined and the latest version of the model consists of a list of function names and descriptions. The functions are identified in the areas of Direct Care, Information Infrastructure and Supportive functions. HL7 has also developed health IT communications protocols.
Beginning in 2005, CCHIT has leveraged the work done by HL7 and the Health Information Technology and Standards Panel (HITSP)[1] in establishing a detailed set of EHR criteria in the areas of functionality, interoperability and security. In developing these certification requirements, CCHIT sought extensive public input and considered both current vendor capabilities and the electronic functions required to provide efficient, safe and high quality patient care.[xlviii] About 450 CCHIT criteria currently exist, addressing everything from the basic functions of creating patient-specific problem lists and storing medication information to more advanced features such as the exchange of external clinical documents and e-Prescribing (eRx). Functionalities that CCHIT proposes to introduce as part of the next year’s certification criteria are identified, as are those scheduled for eventual introduction into the certification criteria. Appendix C provides an overview of the scope of EHR functions covered by the HL7 and CCHIT functionality models.
Currently, about 50% of all vendors in the market have CCHIT-certified products. [xlix] However, as healthcare organizations increasingly look for certified products, more vendors are likely to be submitting their products thorough a certification process. The 2009 ARRA also called for the adoption of ‘certified EHRs’ and, as the regulatory interpretation of ‘meaningful use’ is finalized in the coming months, this will invariably further influence the development of certification criteria for ambulatory EHRs.
The Centers for Medicare and Medicaid Services (CMS) Physician Quality Reporting Initiative (PQRI) is a general incentive payment program for physicians who care for Medicare and Medicaid patients. The 2009 PQRI consists of 186 measures, one of which is an Electronic Health Record Test developed to document whether physicians have adopted and are using health IT. Qualified EHRs must be CCHIT-certified or meet several criteria including the ability to manage problem and medication lists, to meet basic privacy and security elements, and to manually or electronically store and display laboratory results as discrete searchable elements.[l] The CMS reporting measures are run by eligible providers on a voluntary basis and this information is self reported through the claims data system. In 2009, a separate e-Prescribing reporting program was established for physicians to report on their adoption and use of qualified e-Prescribing systems.
In 2008, the National Quality Forum (NQF) developed nine structural measures aimed at assessing and encouraging clinician adoption of health IT. These measures are in the domains of e-Prescribing, interoperable EHR, care management, quality registry and Medical Home.
Our discussions with health center and network staff, vendors and other key informants indicate that, despite all these efforts, there is considerable variation in how EHR functions are described in different vendor products. For example, the terms smart forms and template forms are often used interchangeably to describe structured forms for particular disease conditions and varying levels of clinical decision support. There is also considerable variation in how a particular feature gets implemented, for example e-Prescribing may imply end–to-end electronic prescribing or simply a function that enables a provider to type the prescription using the EHR and then print the scripts to a local printer at which point they are then handed over to the patient.
[ Go to Contents ]
NATURE OF EHR USE IN AMBULATORY CARE SETTINGS
Several studies have found important variations in the features of the EHR systems implemented by different practices.Many systems included basic EHR functions such as patient demographics, clinical notes, problem lists and medication lists. However, the more advanced features like clinical decision support, and computerized physician order entry (CPOE) were significantly less available.[li] , [lii] For instance, the 2005 MGMA study found that almost all respondents’ EHRs included the basic features mentioned above. The study also found that the least available features were immunization tracking, clinical decision support in the form of clinical guidelines, and those associated with e-Prescribing such as drug formularies, drug reference information and drug interaction warnings. [liii] In addition to being less available, when available these features are also likely to be less used than other EHR features. In a 2007 study, clinical decision support and electronic prescribing with electronic transmittal to pharmacies were found to be among the least used EHR features. [liv]
We begin our discussion with a review of the functions that are commonly used by all practices, followed by a review of more advanced functions that tend to be deployed more commonly in mid-to-large practice settings. To fully understand how EHR functions are implemented and used and their capacity to exchange information with systems that are outside the practice, we also include a discussion on the different technical standards that support these functions and to what extent these standards are being used.
Our discussions with health center network representatives and ambulatory care practices showed that in general the more basic , or first tier, EHR features such as those of patient demographics, recording patient vitals, documentation of notes, entering medication and allergy information, problem lists, referrals, billing (particularly in smaller practices), medical summary and entering insurance information features were the most frequently used. These were common functionalities that were cited as having been implemented in almost all health center practices. A California Healthcare Foundation (CHCF) study of community health centers’ adoption of health IT showed that, among those that reported having an EHR, virtually all had the patient demographics feature and 83 percent had electronic clinical notes.[lv] These findings are not surprising as the use of such features is more or less necessary within the EHR. In the case of patient demographics for instance, it is difficult to conceptualize using any of the features included within the EHR without having entered patient data.
Meanwhile, features such as drug formularies and eligibility checking received lower levels of use or were not used at all. In the case of drug formularies, some practices reported that it was difficult to have a comprehensive formulary as all insurance plans may not have chosen to participate. Health plans also tended to change their formularies and formularies in the EHR may not necessarily have been updated in a timely way which resulted in providers not being very keen to use this function. Some features, for example eligibility checking, were reportedly very hard to integrate into the EHR. Instead, practices sometimes chose to make use of the eligibility checking through their existing practice management systems. Additional second-tier functions included varying degrees of clinical decision support including smart forms, alerts and reminders, drug interaction checking and clinical guidelines. Second tier functions were often not implemented when the systems were first installed and there also appeared to be variability in terms of the size of the practice; larger practices seemed to be more equipped to implement more advanced clinical decisions support features compared to smaller practices. Below we provide a more detailed description of a subset of EHR features that are most commonly used as well as those functions that appear to be rapidly increasing in use.
ELECTRONIC EXCHANGE OF LABORATORY INFORMATION
Informants for this report indicated that the electronic exchange of lab results was one of the features of the EHR system that was most commonly used. In most instances EHRs had established unidirectional interfaces with national labs (Quest or LabCorp) or local labs and sites were receiving results electronically. Similar results were found in a report developed for the Assistant Secretary for Planning and Evaluation looking at the Electronic Exchange of Clinical Laboratory Information. [lvi] Many sites reported that establishing interfaces with hospital labs was more difficult and oftentimes there was reluctance on the part of the hospital to establish a results interface with health centers and ambulatory care providers. In cases where results interfaces were established, providers routinely used the EHR to order labs that resulted in printed lab requisitions. In most cases the lab specimen was drawn at the clinic or the practice and was then sent together with the lab requisition form to a local, hospital or national lab. A few of the sites reported supporting bi-directional lab interfaces; however this was not common. In cases where lab interfaces were established with the lab, almost 100% of the results were received electronically.
Sites also reported a growing trend to support bidirectional interfaces with labs and several are making significant efforts to interface with national and local labs. In some cases the orders interface is now being implemented in sites that had previously been using a results interface only. Discussions with vendors also indicate that they are encouraging bidirectional lab interfaces at initial installation. The interface verification process is largely driven by interface implementation and is dependent on the vendor and provider requirements. In general, the time taken to validate a bi-directional interface is approximately twice that of a unidirectional interface. [lvii]
All of the informants for this study indicated that there is a growing trend to use point of care (POC) devices in providers’ offices for a variety of lab tests including HbA1c, simple blood chemistries, pregnancy tests, HIV testing and cholesterol testing. These POC devices allow for results to be quickly and more easily obtained because they eliminate the need to send away samples for laboratory testing. In addition, many of the tests performed on the POC devices fall into the category of waived tests[lviii] under the Clinical Laboratory Improvement Amendments (CLIA) requirements. Under CLIA, waived tests are defined as simple laboratory examinations and procedures. Tests that are waived by CLIA are cleared by the federal government because they employ methodologies that are simple and accurate so that erroneous results would be negligible or pose no reasonable risk of harm to the patient if the test is performed incorrectly. In cases where POC devices were being used, the extent to which they were integrated with the EHR varied. This resulted in health centers and practices supporting a variety of different workflows. In cases where the POC device had established a direct interface with the EHR, results were automatically populated into the EHR in a structured manner from the POC devices. When there was no direct interface, the technician would either scan in a copy of the printed report or manually key in the results into the EHR. Depending on the approach used, lab results from POC devices would therefore not always be available in the EHR in a format that could be used for quality reporting purposes.
E-Prescribing
According to an eHealth Initiative report on e-Prescribing, the physician adoption rate in 2007 was approximately 6% and accounted for only 2% of eligible prescriptions in the USA. [lix] Other studies also report low penetration of electronic prescribing.[lx] However, there appears to be a growing trend to make this function available, partly due to the incentives under the Medicare Improvements for Patients and Providers Act of 2008 which will, starting in 2009, provide a 2% incentive for all providers that use eRx for Medicare patients.[lxi]
Many of the health centers and ambulatory care practices we spoke to reported that a majority of the e-Prescribing done was only partly electronic and one site was yet to launch e-Prescribing. In cases where e-Prescribing is being used, the provider enters the prescription into the EHR using the e-Prescribing software. However, three different approaches are currently being employed to route the prescription to the pharmacy. We describe these approaches in detail below. Of particular note is that the CMS e-Prescribing program only pays for full e-Prescribing, when prescriptions are sent directly to the pharmacy from the point-of-care.[lxii]
- Fully electronic – Prescriptions are sent electronically to pharmacies in a paperless process, through the SureScripts-RxHub network. In this case, the prescription is electronically routed to the pharmacy information system.
- eFaxing – Prescription information is electronically faxed to pharmacies. Using this process, a fax normally prints at the pharmacy and the pharmacist manually keys in the prescription into the pharmacy information system.
- Prescription printing – A hardcopy script is printed and handed to the patient who fills the prescription at a pharmacy of choice. Among those practices capable of e-Prescribing, this approach is generally used only in instances where patients are not able to indicate which pharmacy they will print the script at or if the pharmacy does not support e-Prescribing.
In general, most of the practices we spoke to reported that they are not using fully electronic prescribing but tend to use eFaxing. Reasons cited for this included that at the time of initial implementation there were many barriers related to e-Prescribing, and that there did not appear to be any financial benefits from it. Furthermore, not all pharmacies have the capacity for e-Prescribing using the SureScripts-RxHub network. While the larger pharmacy chains are generally capable of receiving prescriptions electronically, many smaller pharmacies are not due to the high cost of implementation on the pharmacy end. Another barrier cited was the inability to use eRx for controlled substances due to current Drug Enforcement Agency (DEA) rules for eRx.[lxiii] Practices that are currently using eRx report that they have an entirely separate workflow for controlled substances. In most cases providers order controlled substances using special prescription pads. These orders are not routinely re-entered into the EHR. In very few cases, sites reported that providers can use the EHR to order a controlled substance which is then printed onto non-copyable paper using a dedicated printer. This tends to be an expensive option for many practices and therefore is not routinely implemented.
Of the practices that are using e-Prescribing, many of them also report that they have implemented clinical decision support functions which include formulary checking, drug-drug, drug-lab and drug-allergy checking. A more detailed discussion of clinical decision support follows later in this section.
Ambulatory practices recently have received a great deal of external push to perform e-Prescribing largely due to the Medicare incentives and through various quality improvement initiatives. Additionally, several discussants stated that their organizations are putting in a lot of effort into developing and customizing their e-Prescribing tools and encouraging the use of fully electronic prescribing within their practices.
Clinical Decision Support
Practices and health center networks reported having EHR systems with Clinical Decision Support (CDS) modules. These modules were capable of numerous functions including providing drug interaction alerts, clinical practice guidelines for particular chronic diseases, knowledge resources and prompts and reminders for health maintenance. In terms of level of use however, many of these CDS functions were in the second-tier, i.e. most likely implemented only once the EHR system had been in use for a while. In general, informants reported variable levels of use for CDS depending on the features in question. Providers also tended to use those functions only as it was relevant to their practice and specialty.
For practices that had implemented smart forms, which essentially are forms customized for specific disease conditions like diabetes or coronary artery disease, there appeared to be high use of this type of decision support tool. While providers appeared to favor the use of smart forms, many practices reported the need to conduct extensive work at the initial stages to ensure that the forms were customized and adequately met the needs of the providers. Given the time and effort taken to develop customized templates, health center networks and mid-to-large size practices were more likely to use this. Several informants reported that their health centers and ambulatory practices had implemented alerts and reminders in their EHR system in order to support preventive services and e-Prescribing. Although this feature has great potential to be an extremely valuable EHR tool for increasing patient safety, informants indicated that many physicians experienced problems using the drug interaction alerts component of the CDS. Some perceived these alerts to be intrusive or annoying. Others felt the information offered by the alerts was redundant and unhelpful. In both cases, the alerts often acted to interrupt and slow down physicians’ workflow. As a result, some practices allow physicians flexibility in this regard. A few sites reported that providers are allowed to selectively turn off or adjust the threshold for these warnings. Although the level of CDS alerts was sometimes set centrally at a level that all clinicians must prescribe to it, many systems allowed physicians this option (allowing EHR users the capacity to set the level of CDS interaction is a CCHIT-required functionality). Each clinician could then set their own threshold within the limits of the organization’s settings. It appeared that clinicians significantly took advantage of this option. There were also reports of physicians having the ability to adjust the level of alerts specific for each patient. For instance, a physician could adjust the threshold reminder level for mammograms for a patient whose previous mammogram returned abnormal results so that the system would request them more frequently than the standard interval.
Finally, sites reported that EHR vendors are increasingly making available knowledge resources that allows for context sensitive help from within the patient record. While the availability of this function was not commonly reported, a few sites had implemented it and regarded it as a very useful tool for providers.
Use of Other EHR Functions
In looking at more advanced EHR functions, several discussants reported that they were beginning to use their EHR for referrals and for specialty reports such as radiology reports. Generally, mid and large sized practices were more likely to be expanding current EHR use in this direction. Very few health centers and ambulatory care practices reported being able to receive radiology images. In cases where this was supported, the EHR generally received a link to the image which was hosted by an external Picture Archiving and Communication (PACS) system. Practices indicated that this was preferred as radiology images can be fairly significant in size and many small and mid-sized provider offices did not have sufficient bandwidth to support the transport and storage of large radiologic images. Respondents for this study also indicated that unless they had spent significant money in purchasing and installing high quality monitors, their ability to read and interpret the radiology images with any confidence was limited. In general, practices reported that receiving the radiology report was far more important to them than receiving the images. In cases where practices were not able to receive radiology or other reports electronically, most had at the very least implemented scanning technology that enabled them to scan the paper reports into the electronic health record. Initially, practices would use such document imaging techniques but their use generally decreased over time as practices increasingly made use of the EHR system and adopted more advanced document management strategies.
Interoperability and Standards Support
With the increasing trend towards supporting health information exchange between different primary care providers, hospitals, specialists, labs, public health authorities and the emerging minute clinics, a significant focus has been placed on the capabilities within EHRs to support interoperability in the form of messaging and content standards. In the sections above we discussed the scope of lab information exchange and e-Prescribing that is currently taking place. In this section we focus largely on the current use of standards. Interoperability requires standards for both messaging (the way information is passed between systems) and data (the content of these messages).
In general, the Health Level 7 (HL7) messaging standard is being used widely to support electronic exchange of information between provider practices and hospital, national and local labs. Sites were generally using some version of HL7 2.3 and very few sites indicated that they supported HL7 V2.51. Many of the health center networks that we spoke to used either commercially available or homegrown interface engines and reported that they spent significant amounts of time establishing interfaces with different labs and providers due to the significant variability in how the HL7 messaging standard had been implemented.
For e-Prescribing, most sites reported that the EHRs they have support the National Council for Prescription Drug Programs (NCPPDP) standards for pharmacy data and this is what is currently being used. Other than the Script standard there did not appear to be any use of National Drug Codes (NDC)[lxiv], RxNorm or Systemized Nomenclature of Medicine (SNOMED)[2]. Most practices also reported that they are using the SureScripts-RxNorm network to connect to retail pharmacies.
The Continuity of Care Document (CCD) and Continuity of Care Record (CCR) are two health record document standards that were developed to facilitate the transfer of health information among health care providers. The CCD, in addition to specifying what content should be contained within the record, also specifies the structure of the record. The sites included in this study generally supported the CCD while there were a few sites that indicated that they supported the CCR for patient summary data. Although sites reported having the capability to exchange the CCD, they had limited experience in actually exchanging patient summaries as they reported that many sites that they routinely interact with were not able to receive the CCD. Additionally, one site representative noted that although the CCD standard was quite extensive, it did not necessarily contain all the information that they needed to transmit obstetrical information and consequently it was necessary for them to modify the format to expand on the information which was being exchanged.
For sites that supported electronic exchange of lab information, there was very limited use of the Logical Observation Identifiers Names and Codes (LOINC)[lxv] standard for lab results. Many of the EHR vendors reported that they were able to support LOINC but that the labs were not sending lab results using LOINC. While it was noted that the major labs, Quest and LabCorp may support LOINC, many of the smaller labs that they had acquired were still largely using proprietary codes. In addition, none of the hospital labs were able to send results using LOINC. The approach commonly tended to be to accept whatever codes the labs were using and then map them to vendor proprietary codes in the specific EHR.
As noted above, few if any ambulatory practices reported that they had the capability to receive radiological images and launch a DICOM image viewer. A few sites were receiving links to images that were stored in a third party Picture Archiving and Communication System (PACS) and could be viewed via the web. Consequently, we found very limited use of any Digital Communication (DICOM) standard.
While there appears to be some convergence in the industry to use HL7 as the messaging standard, the great variability in how HL7 is implemented still makes exchange of information a costly and time-consuming process as practices need to work with their vendors and different data providers to ensure that the various HL7 fields are correctly mapped and that data exchange can take place. There appears to generally be very limited use of data content standards and, while CCHIT certified products may support the standards, there appears to be limited use in the field.
[ Go to Contents ]
ASSESSING EHR USAGE
In engaging with the vendors, practices and health center network representatives, we sought to identify whether any efforts were being made to measure EHR usage, to assess the extent to which the use of specific functions within the EHR was being measured, and to analyze how usage data that is collected is routinely used. Efforts to monitor EHR use varied tremendously from site to site, dependent in part on the availability of IT resources, size of the organization, availability of canned reports within the EHR system and size of practice. In general, smaller sites, or sites that relied largely on their vendor for IT support reported that they were not routinely collecting or reviewing usage data. In most cases, the vendor audit logs were a source of information to assess which providers had accessed different aspects of the EHR. This was largely done in the context of ensuring the security and privacy of patient records. Our discussions with health center networks and other large EHR implementation projects (e.g., the Primary Care Information Project (PCIP) in New York - a program that is being run by the New York Department of Public Health and Mental Hygiene to support the adoption and use of EHRs among primary care providers in NYC's underserved communities[lxvi]) indicated that there is significant effort underway to assess utilization of different EHR functions at very granular levels.
Commonly monitored EHR Functions
For sites that were tracking EHR utilization, one of the common metrics being monitored was the number of electronic lab orders that were placed using the EHR and the percentage of lab results that were received electronically. Furthermore, sites had instituted varying levels of tracking to assess how quickly lab results were viewed by providers once they had become available in the EHR and how promptly clinical notes were signed once completed. Other functions that were tracked included how many electronic notes were created and signed off on, and how many prescriptions were ordered using the e-Prescribing feature within the EHR. A few sites reported that they also tracked various other clinical decisions support functions in the context of how many Smart Forms were being used, how many drug alerts were fired, and provider responses to these alerts. A few sites reported that the EHR allows for providers to set their own threshold for alerts and reminders within the limits of the organization’s settings. These sites reported tracking how thresholds were being modified by providers at the different location.
In some cases, sites had implemented very robust capability to assess utilization of EHR functions, for example PCIP. The current reporting mechanism from eCW tracked EHR usage in five domains: provider-level, practice-level, patient portal use, use of specific tools and daily procedures. In terms of provider-level use, PCIP tracked items such as the number of patient encounters, medications viewed, amount and types of claims generated, viewing and ordering of labs, referrals and the use of structured forms by the physician. For legal reasons, physicians were required to “lock” their clinical notes by digitally signing them. PCIP tracked the number of notes left unsigned for each provider at each facility. The organization also made efforts to see if there were any patterns of clinical decision support suppressions in the system and tracked both the frequency and severity of drug and allergy alerts. The PCIP project examined whether providers accessed the e-Prescribing system and what percentage of those orders were EDI transactions, eFaxed and printed.
At the practice level, PCIP assessed the use of the system by physicians and by other clinicians, the use of immunization registries, the use of the lab interface, and examined certain aspects of eFaxing such as the proportions of failed and received faxes. They also examined medical home indicators such as the percent of patients assigned to a primary care giver and the number of active patients seen in the last year with specific diagnoses. In terms of the patient portal, PCIP tracked factors such as the number of messages from providers to patients and the number of messages sent by patients.
In general, for practices that were currently measuring EHR usage, there was a significant level of granularity in terms of the type of data collected. All sites reported that they are able to track usage both at the practice and at the physician level and assess the use of specific functions. The Alliance of Chicago, OCHIN and PCIP systems even had the ability to pool usage data by type of provider. The ability to perform such analysis is important because different patterns of EHR use can exist among different clinicians. Informants indicated that some of the usage data was easier to access particularly around basic features such as the use of templates, completing insurance information, and keeping track of the functions within the EHR system which had been disabled. However, monitoring clinician use of more advanced features (e.g., CDS) was particularly challenging. Practices had to work closely with their clinical staff to identify the measures of interest and determine how this information could be presented in a way that would be most useful to the practice.
Reporting on Quality Metrics
Unlike monitoring of usage, most of the sites that we spoke to were using EHRs to assist in reporting on different quality metrics. Many of the health center networks were reporting out on the HRSA quality indicators[lxvii], which include blood pressure control in hypertension, HbA1c in diabetics, pap smears and mammography for women, immunization for children less than 2 years, depressions screening and colorectal cancer screening. In general, sites did not use their EHR for quality reporting, but instead populated a registry or a vendor supplied reporting database with the subset of information that was needed for quality reporting. Many sites indicated that this approach was more feasible as they did not want to run reports against the production database because this could potentially slow the EHR system down. Sites also reported that many EHR systems lacked out-of-the box reporting capability for quality metrics and therefore opted to use more sophisticated tools in the form of registries or custom databases with enhanced reporting tools.
Challenges and Barriers to collecting Utilization Data
Sites reported a number of different issues related to collecting and analyzing utilization data which included not having any guidelines for deciding what they should track to assess utilization, what were the optimum metrics to use and any standard reporting templates. It was also noted that once the metrics had been decided on, a fair amount of effort was required to obtain a clean set of usage data so that it could more effectively be used for reporting.
In addition to this, some of the health center networks discovered that having a lot of data available did not necessarily mean an equivalent level of useful information could be abstracted. Rather, it was likely to complicate or impede the process of analyzing the results. Sites had to balance the need for obtaining detailed reports with placing priority on tracking items that were either feasible or practical.
Discussants highlighted the fact that most vendors did not build their system with the anticipation that clients would desire to measure detailed usage data. As a result, a significant amount of customization was needed to existing reporting capability in order to obtain granular levels of usage data. In some cases, sites that possessed in-house expertise undertook the programming and customizations that were necessary while in other cases the sites worked very closely with their vendor to build custom reports. Many sites reported that data was being collected and stored in the EHR system but accessing this information in a meaningful way involved significant effort. Given that a significant level of experience was needed both with the vendor product and with respect to developing and generating custom reports, many sites noted that this would not be something that smaller practices would be able to undertake on their own.
Sites also noted that, in most cases, focusing on system implementation was at the forefront of their priorities and assessing use of the various functions in the EHR was often only initiated once the major production issues had been dealt with and the site had transitioned into the mode of maintaining the EHR.
[ Go to Contents ]
EHR VENDOR REPORTING CAPABILITIES
One of the areas explored in particular detail with discussion participants was vendors’ capacity to internally track the use of specific EHR functionalities and the role that vendors played in practices’ assessment of their EHR usage. Vendors are compelled to adapt their standard reports to suit the data collection needs of their clients and it is in their interest to help practices examine the usage of their system. Some EHR vendors provide practices with the ability to run basic utilization reports out of the box while others work closely with the site to assist them in developing the customized reports.
Vendor capabilities for tracking EHR system usage varied greatly. While some products, such as Epic and eCW, had fairly robust reporting capabilities out of the box, other products had significantly more limited capabilities. The PCIP project selected eCW as their EHR system and has since implemented this in hundreds of small provider offices. Informants on the PCIP indicated that there were numerous highly customized reports that were developed as part of this project and these reports were subsequently made available by the vendor as part of their library of canned reports. Discussions with eCW indicate that the EHR is capable of providing both provider and facility-level usage data, as well as reporting on the activities of the same provider in different facilities.
Review of the 2008 CCHIT certification criteria for ambulatory EHRs include the need for audit trails that record, 1) the date and time of the event; 2) the component of the system (e.g. software component, hardware component) where the event occurred; 3) type of event (including data description and patient identifier where relevant); 4) subject identity and 5) outcome of the event. [lxviii] Further review of the 2008 test scripts for Security and Reliability criteria indicate that functions covered under the software component include viewing, updating, validating and exporting the patient record.[lxix]
Overall, in seeking to obtain information regarding what was currently feasible with respect to reporting and monitoring, we discovered that vendors generally had very basic reporting capability for usage data that was available in the form of canned reports. Where sites were monitoring system usage at a granular level, they were either working with the vendor IT team to create the report or had independently undertaken the task of building customized reports (this was mostly done by networks). In almost all cases, significant customization was required in order to extract the kind of information sites were interested in from the EHR. It appears that there will need to be significant changes made in existing reporting systems to ensure that they are able to report on granular use of EHR functions.
Most sites reported having established a clinical warehouse, some form of database that was updated regularly with information from the EHR, were reports were being generated directly from the databases using SQL queries. In a few cases, this feature was made available “off the shelf “by the EHR vendor. However, sites wishing to run more complicated reports generally had to work with their EHR vendors or an in-house technical team to customize their system.
[ Go to Contents ]
CONCLUSIONS
Limitations in the use of surveys as a method to assess EHR adoption. Survey methods pose many challenges because of their inherent shortcomings and published estimates of EHR adoption are of varying quality. In a report comparing existing surveys assessing EHR adoption up till the year 2008, it was found that very few were adequate to accurately capture the state of EHR use in the United States.[lxx] Limitations such as those outlined above spur the need to develop a standardized method to consistently and more reliably characterize the state of EHR adoption in the United States.
Commonly used EHR functions. Review of current EHR use in ambulatory care settings suggests that in all practices (small, mid and large) there are certain basic clinical and administrative functions that are commonly used. The clinical functions used include encounter notes, medication lists, allergy lists, problems lists, and order entry functions largely focused around lab order entry and results delivery. The use of eRX appears to be increasing dramatically but current use is still limited. Practices of all sizes also report that they support clinical decision support functions related to eRx largely in the form of drug interaction and drug-allergy checking. More comprehensive clinical decision support functions tend to be more common in larger practices and may include the use of smart forms, preventive care reminders, clinical guidelines and knowledge resources. While sites report using EHR data for quality reporting, in most cases a third party registry product or alternate database is used to generate these reports. Ambulatory EHRs have limited out-of-the box capability for quality reporting and oftentimes, due to concerns regarding system speed and response times, quality reporting is not done off the production database.
Current use of Standards Limited. Despite the availability of industry accepted standards and 2008 CCHIT requirements that certified EHRs support certain standards, current use in ambulatory care settings appears to be limited. While most sites report using HL7 for messaging many of them are not yet on HL7 V2.51. With respect to data content standards, NCPDP Script is being used for eRx. Some sites report that they generate a CCR or a CCD but have had limited experience in its use as organizations that they interact with often are not able to accept summary documents in this format. There is virtually no active use of LOINC for lab results even though lab results are one of the most commonly used functions within the EHR. Sites report that lab results from national, hospital or local labs are not LOINC coded even though the EHRs are able to receive LOINC codes. Similarly there appeared to be no use of SNOMED-CT. To promote standards use it is important to combine standards recommendations with specific implementation approaches and realistic best practices.
Tremendous variability in how EHR use is being monitored. Current practices in the tracking of EHR utilization data vary in the granularity of tracking, the features being tracked, metrics that are being used to assess utilization, the reporting database and the kinds of reports that are being generated. Furthermore, many smaller practices do not appear to have the IT resources and infrastructure to support any form of utilization tracking. Very few EHR products have canned or out-of-the box utilization reports. Larger sites and health centers appear to be tracking some utilization data but this tends to be highly customized for the site, either by in-house IT personnel or by the vendor. In order to promote a more standardized and consistent approach to utilization reporting, a standard set of measures will need to be developed and collaboration with the EHR vendors will be necessary to develop reports that include the same data elements and support a uniform format.
Vendors have limited out-of the box reporting capability but usage data is being collected for auditing purposes. Under HIPAA, many vendors provide auditing capabilities within the EHR which would include who accessed the patient record, the date and time, what aspects of the patient record were viewed or modified and what, if any, information was printed . While CCHIT includes some specifications on the granularity of audit reporting that certified systems must support, practices often selectively decide the level of granularity they will use for auditing purposes. Practices often trade-off detailed audit reporting so as not to impact the speed and responsiveness of their EHR. Given that many vendors can audit access and use of the EHR, the data collected could serve as a surrogate for utilization data. However additional work will need to be done in order to assess the gaps between auditing capability and the scope of EHR functions as well as how certain clinical decisions support functions like alerts and reminders can be tracked. Furthermore, ongoing certification efforts will likely need to introduce greater specificity regarding how different EHR functions should be audited as well as design test scripts that cover the broad range of use cases that may apply based on different interactions with the EHR.
Feasibility of an automated reporting mechanism. Today, practices are using different mechanisms of varying degrees of automation to report on quality measures. Some EHR vendors like GE and AllScripts are already providing options for their install base to upload de-identified clinical data on a monthly basis to a centralized research server. PCIP routinely collects very detailed utilization reports from the 1000+ providers that have installed eCW. Discussions with the various EHR vendors suggest that the capability of generating automated reports is technically feasible and in some cases currently exists. A careful review and comparison of the different methods that are currently being used would inform the optimal design of an automated reporting process. Additionally, developing a set of standard metrics and common definitions around specific functions across different EHR products are important next steps.
Proposed minimum set of functions to monitor EHR use. Our study indicates that there exists variability in the EHR functions that are used based on practice size, practice specialty and length of time for which the EHR has been implemented. However, there is a practical set of minimum functions that would apply to practices regardless of size or specialty that could potentially be used to monitor usage in a systematic manner. This basic set of functions includes the use of encounter notes, medical history, medication lists, allergy lists, problems lists, order entry functions largely focused around lab order entry, the viewing of laboratory results and the use of patient demographics. Additionally, EHR systems could potentially report out on number of providers in a practice that use the system and the number of patients that are entered into the system. Although the adoption of eRx in ambulatory practices is still low, once implemented, the monitoring of the eRx function appears to be very similar to that of lab orders and could be implemented with minimal effort. For most of the functions identified here, vendors appear to have the ability to currently track these functions and generate reports. In a few cases, additional EHR configuration and customization may be necessary to generate the necessary usage data.
Potential areas for future study. Findings from this study indicate that there is a common set of EHR functions that all practices are likely to have purchased from their vendor and that sites would be capable of implementing. Most EHR vendors have capability to track usage of these common functions even though they may not all support the same robust reporting capabilities. While additional work needs to be done to define the standard set of measures that could be applied to all EHR systems, undertaking a few pilot projects in health centers and other ambulatory care settings would provide a good starting point to assess the feasibility of more broad scale implementations. Pilot studies would also be likely to produce a rich set of information on issues and proposed solutions and how larger scale studies should be refined.
Finally, health centers, by virtue of their mission and the populations they serve, may also benefit more than other ambulatory providers from EHR adoption. A number of health centers have established networks to adopt health IT. Future initiatives to promote health IT use in health centers and health center networks would have much to gain by providing guidance on the EHR functions that are commonly available and are most likely to affect the quality of care delivered. Furthermore by providing tools to health centers and networks to monitor EHR utilization these organizations would be better equipped to take actions to promote EHR use and adoption proactively.
| FUNCTION CATEGORY | EHR FUNCTION | MEASURES | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NACHC 2006 | CMS DOQ-IT Office Systems Survey | Common- wealth Fund (2003) | CTS Physician Survey | MGMA Survey (2005) | National Survey of EHR Adoption | CCI 2005 | Simon (2005) | NAMCS/ NHAMCS (2007-2008) | NQF | CMS PQRI Programs | ||
| ORGANIZE PATIENT DATA | Patient Demographics | ● | ● | ● | ● | |||||||
| Clinical/Encounter Notes | ● | ● | ● | ● | ● | ● | ● | |||||
| Medical History | ● | ● | ● | ● | ● | |||||||
| Record Patient-Specific Information (Dosing, Care Plan) | ● | |||||||||||
| Patient Consent | ||||||||||||
| Generate Reports (e.g. based on patient demographics or clinical data) | ● | ● | ● | ● | ● | |||||||
| Advance Directives | ||||||||||||
| COMPILE LISTS | Medication Lists | ● | ● | ● | ● | ● | ● | ● | ● | |||
| Allergy Lists | ● | |||||||||||
| Problem/Diagnoses Lists | ● | ● | ● | ● | ● | ● | ● | ● | ||||
| RECEIVE AND DISPLAY INFO | Laboratory Test Results | ● | ● | ● | ● | ● | ● | ● | ● | ● | ||
| Radiology Results | ● | ● | ● | ● | ● | |||||||
| Radiology Imaging Results | ● | ● | ● | ● | ● | ● | ||||||
| External Clinical Documents (Capture and Store) | ||||||||||||
| ORDER ENTRY (CPOE) | Electronic Prescribing | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● |
| CPOE (“Order Tests”) | ● | ● | ● | ● | ||||||||
| Reorder Prescriptions | ● | ● | ||||||||||
| Laboratory Order Entry | ● | ● | ● | ● | ● | |||||||
| Radiology Order Entry | ● | ● | ● | ● | ● | |||||||
| DECISION SUPPORT | Clinical Decision Support | ● | ||||||||||
| Reminders for Care Activities | ● | ● | ● | ● | ● | ● | ● | ● | ● | |||
| Dosing Calculator | ● | |||||||||||
| Preventive Services | ● | ● | ● | |||||||||
| Drug Alerts (Interactions, Allergies, dosing e.t.c.) | ● | ● | ● | ● | ● | ● | ● | ● | ||||
| Disease or Chronic Care Management | ● | ● | ● | ● | ||||||||
| Knowledge Resources | ● | ● | ||||||||||
| Diagnostic Decision Support | ● | ● | ||||||||||
| Clinical Guidelines | ● | ● | ● | ● | ● | |||||||
| COMMUNICATION AND CONNECTIVITY | Electronic Referrals | ● | ● | ● | ● | ● | ● | |||||
| Clinical Messaging/ e-mail | ● | ● | ||||||||||
| Medical Devices | ||||||||||||
| ADMINISTRATIVEAND BILLING SUPPORT | Scheduling Management | ● | ● | |||||||||
| Eligibility Information | ● | ● | ||||||||||
| Electronic Billing/ Integration with Practice Billing System | ● | ● | ● | ● | ● | ● | ||||||
| Drug Formularies | ● | ● | ● | ● | ● | |||||||
| Clinical Task Assignment and Routing | ||||||||||||
| OTHER | Backup | |||||||||||
| Immunization Tracking | ● | |||||||||||
| Public Health Reporting | ● | ● | ● | |||||||||
| Patient Support (e.g. home monitoring, patient education materials, patient access to electronic records, email patients) | ● | ● | ● | ● | ● | |||||||
[ Go to Contents ]
| Survey | Source |
|---|---|
| NACHC (2006) | Alexandra E. Shields, Peter Shin, Michael G. Leu, Douglas E. Levy, Renée Marie Betancourt, Dan Hawkins, and Michelle Proser “Adoption Of Health Information Technology In Community Health Centers: Results Of A National Survey” Health Affairs, September/October 2007; 26(5): 1373-1383. |
| CMS DOQ-IT Office Systems Survey | https://secure2.mhic.org/cms/index.html |
| Commonwealth Fund (2003) | A.M. Audet, M.M. Doty, J. Peugh, J. Shamasdin, K. Zapert, S. Schoenbaum,"Information Technologies: When Will They Make it into Physicians' Black Bags?" Medscape General Medicine 6, no .4(2004):2. |
| Center for Studying System Health Change CTS Physician Survey | Grossman, Joy M. and Marie C. Reed, Clinical Information Technology Gaps Persist Among Physicians, Issue Brief No. 106, Center for Studying Health System Change, Washington, D.C. (November 2006); and Blumenthal, David, et al., Health Information Technology in the United States: The Information Base for Progress, Robert Wood Johnson Foundation, Princeton, New Jersey (2006). |
| MGMA Survey (2005) | D.N. Gans, J.E. Kralewski, T. Hammons, B. Dowd, "Medical Groups' Adoption of Electronic Health Records and Information Systems," Health Affairs 24, no. 5 (2005): 1323-1333. |
| National Survey of EHR Adoption | DesRoches, Catherine M., Campbell, Eric G., Rao, Sowmya R., Donelan, Karen, Ferris, Timothy G., Jha, Ashish, Kaushal, Rainu, Levy, Douglas E., Rosenbaum, Sara, Shields, Alexandra E., Blumenthal, David Electronic Health Records in Ambulatory Care -- A National Survey of Physicians. N Engl J Med 2008 359: 50-60 |
| CCI (2005) | CCI, "2005 Information Management Assessment Survey, Executive Director Version, "http://healthit.ahrq.gov/portal/server.pt/gateway/PTARGS_0_1248_811156_…”; and "Medical Director Version, "http://healthit.ahrq.gov/portal/server.pt/gateway/PTARGS_0_1248_811158_…” |
| Simon Survey | Steven R. Simon; Rainu Kaushal; Paul D. Cleary; Chelsea A. Jenter; Lynn A. Volk; E. John Orav; Elisabeth Burdick; Eric G. Poon; David W. Bates Physicians and Electronic Health Records: A Statewide Survey Arch Intern Med. 2007;167(5):507-512. |
| NAMCS/NHAMCS 2008 | Hsiao CJ, Burt CW, Rechtsteiner E, Hing E, Woodwell DA, Sisk JE. Preliminary estimates of electronic medical records use by office-based physicians: United States, 2008. Health E-Stat. National Center for Health Statistics. 2008. Available from: http://www.cdc.gov/nchs/products/pubs/pubd/hestats/hestats .htm |
[ Go to Contents ]
| EHR FUNCTION | HL7 | CCHIT | |
|---|---|---|---|
| Organize Patient Data | Patient Demographics | X | X |
| Clinical/Encounter Notes | X | X | |
| Medical History | X | X | |
| Record Patient-Specific Information | X | X | |
| Patient Consent | X | X | |
| Generate Reports | X | X | |
| Advance Directives | X | X | |
| Compile Lists | Medication Lists | X | X |
| Allergy Lists | X | X | |
| Problem/Diagnoses Lists | X | X | |
| Receive and Display Information | Laboratory Test Results | X | X |
| Radiology Results | X | X | |
| Radiology Imaging Results | X | X | |
| Capture External Clinical Documents | X | X | |
| Order Entry (CPOE) | Electronic Prescribing | X | X |
| Reorder Prescriptions | X | X | |
| Laboratory Order Entry | X | 2010 | |
| Radiology Order Entry | X | 2010 | |
| Decision Support | Reminders for Care Activities | X | X |
| Dosing Calculator | X | X | |
| Preventive Services | X | X | |
| Drug Alerts | X | X | |
| Disease or Chronic Care Management | X | 2010 | |
| Knowledge Resources | X | X | |
| Clinical Guidelines | X | X | |
| Communication and Connectivity | Electronic Referrals | X | X |
| Clinical Messaging/ E-mail | X | X | |
| Medical Devices | X | X | |
| Administrative and Billing Support | Scheduling Management | X | X |
| Eligibility Information | X | X | |
| Electronic Billing/ Integration with Practice Billing System | |||
| Drug Formularies | X | X | |
| Clinical Task Assignment and Routing | X | X | |
| Other | Immunization Tracking | X | X |
| Public Health Reporting | X | 2010 | |
| Patient Support | X | X | |
[ Go to Contents ]
[1] HITSP is a cooperative partnership between public and private sector stakeholders tasked with developing a broadly accepted set of standards that contributes to interoperability and health information exchange, and identifying gaps in standards development. HITSP has been tasked with harmonizing standards, developing nationwide health information network prototypes and recommending necessary changes to standardize diverse security and privacy policies. The goal of this effort is to achieve a widely accepted and useful set of standards that will enable and support widespread interoperability among healthcare software applications.
[2] SNOMED is a structured collection of clinical terms used in health and healthcare; from a lab perspective, it is used to code test results. It has been around since the late 1970s and has support from a number of the major standards initiatives including HL7, Digital Imaging and Communications in Medicine (DICOM), the Accredited Standards Committee (ACS) X12, and International Organization for Standardization (ISO). SNOMED has also been mapped to ICD 9 and there are efforts underway to map it to ICD10
[i] Rosenbaum, S., Cartwright-Smith, L., Burke, T., Borzi, P., & Goldstein, M. An overview of major health information technology, public health, Medicaid, and COBRA provisions of the American recovery and reinvestment act of 2009. George Washington University.
[ii] NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS, Hearing on "Meaningful Use" of Health Information Technology
[iii] Murchinson, J. V., Apodaca, A., Sison, C. E., Rosenberg, C., & Swafford, R. For the record: EHR adoption in the safety net. California HealthCare Foundation, 2009.
[iv] NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS, Hearing on "Meaningful Use" of Health Information Technology
[v] http://www.markle.org/downloadable_assets/20090430_meaningful_use.pdf
[vi] HIMSS Discussion Draft on the Definition of Meaningful Use of Certified EHR Technology for Physicians, April 1, 2009
[vii] Blumenthal, D., DesRoches, C., Donelan, K., Ferris, T., Jha, A., & Kaushal, R. (2006). Health information technology in the United States: The information base for progress. Robert Wood Johnson Foundation.
[viii] DesRoches, C. M., Campbell, E. G., Rao, S. R., Donelan, K., Ferris, T. G., Jha, A., et al. (2008). Electronic health records in ambulatory care -- A national survey of physicians. The New England Journal of Medicine, 359(1), 50-60.
[ix] Health information technology in the United States: Where we stand, 2008. Robert Wood Johnson Foundation, 2008.
[x] Health information technology in the United States: Where we stand, 2008. Robert Wood Johnson Foundation, 2008.
[xi] Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century National Academy Press Washington, DC.
[xii] Ford, E. W., Menachemi, N., & Phillips, M. T. (2006). Predicting the adoption of electronic health records by physicians: When will health care be paperless? Journal of the American Medical Informatics Association, 13(1), 106-112.
[xiii] Health information technology in the United States: Where we stand, 2008. Robert Wood Johnson Foundation, 2008.
[xiv] Ibid
[xv] European physicians, especially in Sweden, Netherlands and Denmark, lead U.S. in use of electronic medical records; general practitioners in Scandinavia, Netherlands and UK are using IT more than other doctors in European countries and more than primary care physicians in the U.S.(2002). Harris Interactive, 2(16)
[xvi] Blumenthal, D., DesRoches, C., Donelan, K., Ferris, T., Jha, A., & Kaushal, R. (2006). Health information technology in the United States: The information base for progress. Robert Wood Johnson Foundation. Johnson Foundation, 2006.
[xvii] DesRoches, C. M., Campbell, E. G., Rao, S. R., Donelan, K., Ferris, T. G., Jha, A., et al. (2008). Electronic health records in ambulatory care -- A national survey of physicians. The New England Journal of Medicine, 359(1), 50-60.
[xviii] Hsiao, C., Burt, C., Rechtsteiner, E., Hing, E., Woodwell, D., & Sisk, J. Preliminary estimates of electronic medical record use by office-based physicians: United states, 2008. Health E-Stat, 2008.
[xix] National Alliance for Health Information Technology. (April 2008). Defining key health information technology terms.
[xx] Institute of Medicine. (2003). Key capabilities of an Electronic Health Record system. National Academy Press Washington, DC.
[xxi] HL7 2007 EHR-S Functional Model
[xxiii] http://www.ncvhs.hhs.gov/090429p09c.pdf
[xxiv] Institute of Medicine. (2003). Key capabilities of an Electronic Health Record system. National Academy Press Washington, DC.
[xxv] http://www.cdc.gov/nchs/about/major/ahcd/nhamcsds.htm
[xxvi] http://www.hschange.org/index.cgi?data=04
[xxvii] Community Clinics Initiative. (2006). Celebrating technology advances in California’s community clinics and health centers. http://www.communityclinics.org/content/general/detail/887.
[xxviii] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[xxix] Menachemi, N., Ford, E. W., Beitsch, L. M., & Brooks, R. G. (2007). Incomplete EHR adoption: Late uptake of patient safety and cost control functions. American Journal of Medical Quality, 22(5), 319-326.
[xxx] DesRoches, C. M., Campbell, E. G., Rao, S. R., Donelan, K., Ferris, T. G., Jha, A., et al. (2008). Electronic health records in ambulatory care -- A national survey of physicians. The New England Journal of Medicine, 359(1), 50-60.
[xxxi] Shields, A. E., Shin, P., Leu, M. G., Levy, D. E., Betancourt, R. M., Hawkins, D., et al. (2007). Adoption of health information technology in community health centers: Results of A national survey. Health Affairs, 26(5), 1373-1383.
[xxxii] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[xxxiii] Blumenthal, D., DesRoches, C., Donelan, K., Ferris, T., Jha, A., & Kaushal, R. (2006). Health information technology in the United States: The information base for progress. Robert Wood Johnson Foundation.
[xxxiv] DesRoches, C. M., Campbell, E. G., Rao, S. R., Donelan, K., Ferris, T. G., Jha, A., et al. (2008). Electronic health records in ambulatory care -- A national survey of physicians. The New England Journal of Medicine, 359(1), 50-60.
[xxxv] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[xxxvi] NAMCS: Electronic Medical Record Use by Office-Based Physicians: United States, 2005
[xxxvii] Gans, D., Kralewski, J., Hammons, T., & Dowd, B. (2005). Medical groups' adoption of electronic health records and information systems. Health Affairs, 24(5), 1323-1333.
[xxxviii] Burt, C. W., & Sisk, J. E. (2005). Which physicians and practices are using electronic medical records? Health Affairs, 24(5), 1334-1343.
[xxxix] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[xl] Menachemi, N., & Brooks, R. G. EHR and other IT adoption among physicians: Results of a large-scale statewide analysis. Integration and Interoperability, 20(3), 79.
[xli] Health information technology in the United States: Where we stand, 2008. Robert Wood Johnson Foundation, 2008.
[xlii] Burt, C. W., & Sisk, J. E. (2005). Which physicians and practices are using electronic medical records? Health Affairs, 24(5), 1334-1343.
[xliii] Menachemi, N., Ford, E. W., Beitsch, L. M., & Brooks, R. G. (2007). Incomplete EHR adoption: Late uptake of patient safety and cost control functions. American Journal of Medical Quality, 22(5), 319-326.
[xliv] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[xlv] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[xlvi] Institute of Medicine. (2003). Key capabilities of an Electronic Health Record system. National Academy Press Washington, DC.
[xlvii] HL7 EHR-S Functional Model, July 2004 DSTU package
[xlviii]Certification Commission for Healthcare Information Technology. CCHIT certification – what does it require? (2008)
[xlix] http://assets1.csc.com/health_services/downloads/CSC_HITECH_Certificati…
[l] Physician Quality Reporting Initiative Measure Specifications Manual for Claims and Registry
[li] Menachemi, N., Ford, E. W., Beitsch, L. M., & Brooks, R. G. (2007). Incomplete EHR adoption: Late uptake of patient safety and cost control functions. American Journal of Medical Quality, 22(5), 319-326.
[lii] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[liii] Gans, D., Kralewski, J., Hammons, T., & Dowd, B. (2005). Medical groups' adoption of electronic health records and information systems. Health Affairs, 24(5), 1323-1333.
[liv] Simon, S. R., Kaushal, R., Cleary, P. D., Jenter, C. A., Volk, L. A., Orav, E. J., et al. (2007). Physicians and electronic health records: A statewide survey. Archives of Internal Medicine, 167(5), 507-512.
[lv] Shields, A. E., Shin, P., Leu, M. G., Levy, D. E., Betancourt, R. M., Hawkins, D., et al. (2007). Adoption of health information technology in community health centers: Results of A national survey. Health Affairs, 26(5), 1373-1383.
[lvi] http://aspe.hhs.gov/sp/reports/2009/informationissues/report.html
[lvii] Dullabh, P., & Moiduddin, A. (2008). Electronic exchange of clinical laboratory information: Issues and opportunities. Office of the Assistant Secretary for Planning and Evaluation and the U.S. Department of Health and Human Services http://aspe.hhs.gov/sp/reports/2009/informationissues/report.html>.
[lviii] American Academy of Family Physicians. CLIA waived and PPM tests defined. (2009).
[lix] Electronic prescribing: Becoming mainstream practice. EHealth Initiative (2008). http://www.ehealthinitiative.org/assets/Documents/eHI_CIMM_ePrescribing_Report_6-10-08_FINAL.pdf>.
[lx] Ibid
[lxi] GovTrack.us. H.R. 6331--110th Congress (2008): Medicare Improvements for Patients and Providers Act of 2008, GovTrack.us (database of federal legislation)
[lxii] http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=110_cong_bill…
[lxiii] Electronic Prescription of Controlled Substances: Proposed Rule
[lxiv] http://www.fda.gov/cder/ndc/
[lxvi] http://www.nyc.gov/html/doh/html/pcip/pcip.shtml
[lxvii] Health Resources and Services Administration. Quality improvement and safety net providers: Clinical quality core measures. http://www.hrsa.gov/quality/coremeasures.htm>
[lxviii] CCHIT 2008 Criteria for Ambulatory EHRs
[lxix] CCHIT Test Scripts, Security and Reliability Criteria for certification of Ambulatory EHR 2008
[lxx] Health information technology in the United States: Where we stand, 2008. Robert Wood Johnson Foundation, 2008.