Health Information Exchange in Long-Term and Post-Acute Care Settings: Final Report. Findings: Assessing the Impact of Ehie Between Ltpac Providers and Their Exchange Partners


This section seeks to answer the following research question:

  • How has the creation, transmission, and receipt of eHIE (including interoperable exchange) at times of transitions in care and during instances of shared care impacted the clinical workflow in the LTPAC settings and that of their clinical trading partners (i.e., physicians, hospitals, and pharmacies/pharmacists)?

Limited bidirectional exchange is occurring nationally and in the sites selected. As a result, several research questions around long-term impact of eHIE on patient outcomes and costs cannot be answered due to minimal interoperable exchange nationally. As described in "Implementing eHIE Between LTPAC Providers and Exchange Partners," most eHIE occurring via view-only portals; volume of this type of exchange is unknown.

Below we discuss the intermediate effect or impact to workflow on LTPAC facilities and their organizations that results from implementing eHIE as well as a comprehensive approach to assessing the quantitative impact of eHIE and the challenges to conducting a comprehensive quantitative evaluation on impact.


As mentioned before, many of the methods LTPAC providers use today to exchange information with other providers (e.g., phone, paper, and fax) are labor intensive and inefficient. While shifting to eHIE may reduce the amount of time required to collect and share necessary information,61 efforts to switch from current work processes to eHIE can be quite difficult for both post-acute and acute care providers in practice.

The challenges of adopting EHRs and implementing HIE generally are well documented for acute and primary care providers. Previous research has documented workflow barriers resulting from EHRs that lack processes for easily documenting and retrieving patient status (opt-in/opt-out) and separate, multiple log-ins (external to internal EHR log-in) to access information housed in an HIE.62 In other words, in order to access information housed in an HIE, providers often have to exit their own EHR and log into a separate solution which stores the desired health information. There was little in the literature documenting how the shift to new technology impacted the workflow of LTPAC providers specifically. Only one report from a Stratis Health study made observations on the impact of technology use in both NFs and their hospital partner,63 but there is no reason to think that the barriers encountered by acute and primary care providers do not also apply to them as well and our stakeholder interviews and site visit findings are consistent with these earlier findings.

Impact to LTPAC Provider Workflow

Overall all of the data sources confirmed that hospital portals were the primary form of data exchange, which reportedly had limited impact on the LTPAC provider's workflow. For example, respondents in our stakeholder discussions described how the prominent use of hospital portals in LTPAC facilities for gathering patient information currently minimizes workflow issues for LTPAC providers. Many suggested that as eHIE becomes more prominent, workflow breakage will likely become a more significant issue and will likely require more work-redesign attention.

However, the literature review discussed impact of eHIE on providers more broadly, and frequently did not specify the impact to LTPAC providers. The HITPAC final report (referenced above) indicated that workflow issues for both trading partners delayed implementation and eventually resulted in the discontinuation of eHIE.64 There were a few reports from some LTPAC initiatives that demonstrated that workflow issues are a top priority to LTPAC providers. Reports from early initiatives have started to identify and address the workflow challenges and potential ways to overcome them. For example, some of the ONC Challenge Grantees have been working to develop innovative and scalable solutions to improve LTPAC transitions through new workflow and clinical processes that use eHIE.65 These grants are an extension of State HIE Cooperative Agreement Programs, which funded state efforts to rapidly build capacity for data exchange. The Challenge grants are intended to encourage breakthrough innovations for eHIE that can be leveraged widely to specifically support LTPAC providers nationwide in their effort to implement eHIE.66

The majority of our findings in the area of impact to LTPAC provider workflow came from our stakeholder interviews and case study site visits and is described below.

Home Health Agencies

HHAs at each case study site were at differing levels of eHIE implementation.


In Pennsylvania, HHAs are further along in their implementation of eHIE. Many of those we spoke with have already connected to KeyHIE and have begun using it to gather patient data ahead of home visits. In addition, HHAs in Pennsylvania reportedly found great utility in Admission Discharge Transfer (ADT) alerts (described in Appendix B) and additional clinical information concerning their patients. For example, ADTs provide HHAs immediate notice when their patients have been admitted to the ER. This can prevent unnecessary home visits to patients who are under medical care elsewhere.

One HHA nurse described the process of gathering clinical information as follows: most information is gathered from KeyHIE by the intake nurse at the time of the referral. When patient information is not available in KeyHIE, the intake nurse will consult Geisinger's hospital portal, which provides valuable information about the patient's most recent acute hospital episode but not longitudinal data. The primary nurse will then review this information ahead of the home visit. After the home visit, the primary nurse will also consult KeyHIE for a history of symptoms and conditions that presented during the visit to the patient's home. Though this approach may not be as efficient as accessing needed information from a single source, as described above, home health nurses typically spend considerable time with patients and find that using these multiple methods of gathering information more efficient than gathering information via telephone and fax.

A few of the HHAs we interviewed in Pennsylvania were using Transform. They reported that transmitting information to KeyHIE using the Transform tool was also seamless; inputted OASIS data is automatically converted into a CCD and transmitted to KeyHIE. However, HHAs did express concern that the information they push is sitting in the HIE and is not being used by their trading partners because they do not know the information is there.

HHAs in Pennsylvania did not typically use DSM to communicate with trading partners due to workflow issues for both acute care and post-acute care providers. For example, home health nurses lack a consistent internet connection and are unable to share Direct mailboxes, which raises concerns that urgent messages may be missed. Trading partner resistance to eHIE is further described below.

Alerts and clinical messages from KeyHIE also provide information and insight into a patient's status that the HHA would otherwise not know or miss. For example, when a patient cared for by a HHA is admitted to an ER an ADT alert can prevent an unnecessary visit to a patient's home. Prior to the use of ADTs, a home health nurse would make an unnecessary trip and spend additional resources ascertaining the patient's status and location. However, alerts do not go to nurses in the field, but rather to the intake desk, which then notifies primary nurses of any alerts.


Whereas in Pennsylvania most LTPAC HIE was occurring in HHAs, in Minnesota there was much less HIE occurring in HHAs. HHAs were not at all involved in the Benedictine initiative, and Fairview's HHAs currently do not engage in electronic exchange.

At Fairview most patient information transfers are handled via fax; the HHA keeps a paper chart in addition to the EHR. The workflow when sending patients to trading partners involves many phone calls. One respondent said, "Our nurses spend a lot of time on the phone trying to get ahold of a doctor."

As in Pennsylvania, Fairview HHAs are able to utilize the read-only access to hospital portals (in this case Epic) to supplement information received at transfer. Reportedly, this dual process causes duplication of effort for HHAs. One Fairview nurse described the process of securing complete patient information: "the physician puts an order into Epic; the physician's agent has to call the home care clinician, who then enters the information into the local EHR (McKesson). It goes to HIM (health information management), they print it off, send it back to the physician for signature, and then they have to log it back into the record." Though Epic and McKesson both have certified EHR products, the McKesson product that HHAs use is not MU-certified. As a result, the two systems are not interoperable and cannot exchange health information.

Skilled Nursing Facilities

Like HHAs, NFs in each case study site were at differing levels of eHIE implementation. Within the two Minnesota initiatives there were also distinct approaches to eHIE implementation.


The SNFs in our case study were from smaller regional chains. All were still in the testing stage and had not yet implemented KeyHIE or Transform. Few facilities had begun considering how they would incorporate KeyHIE into their workflow, but indicated they would be doing so in the coming months. Due to a number of factors, we were unable to secure an interview with the national chain in the area, which according to other interviewees, may have been farther along in their implementation of eHIE.

Staff at NFs has limited experience using DSM to communicate with local hospitals. Though KeyHIE has provided all of its participating LTPAC facilities with Direct mailboxes (which if used would enable bidirectional HIE), LTPAC providers forego DSM use in favor of the hospital portal, which provides a complete set of information from the hospitals, but does not allow the NF to send information to hospital.

NFs in Pennsylvania currently obtain most or all of the needed health care information from the referring hospital's provider portal. There were two NFs (Maria Joseph and Presbyterian) who were reportedly trained in Transform, but after changes in leadership and EHRs, respectively, they discontinued use. This finding also highlights the need for ongoing commitment and engagement as well as greater institutionalization, otherwise progress toward eHIE between partners can be stalled or reversed with leadership turnover.


The two initiatives studied in Minnesota took distinct approaches and were at different stages in their ability exchanging health information electronically. In the case of the Benedictine-Allina initiative, the two participating facilities have modified their EHRs such that CCD exchange is enabled. When a patient is discharged from Allina owned St. Francis Hospital to Benedictine's St. Gertrude's Health and Rehabilitation Center, the hospital first calls St. Gertrude's and provides a medical record number. At that point, the admission person can enter the NF's EHR (MatrixCare) and query for a CCD. If Benedictine's MatrixCare product determines that the patient matches a record at the participating hospital, the EHR will respond with the availability of documents. At that time a person can click on it and show a CCD, which can be attached to the record as a PDF.

Though the process itself is simple, there are a few issues that disrupt workflow. In order for McKesson to show a hospital patient's CCD, the hospital must have first secured and uploaded the patient's consent. Many interviewees said the hospitals find this process cumbersome, which means that despite being able to determine that a patient has been seen at both St. Gertrude's and St. Francis, often a patient's CCD is not made available in MatrixCare because patient consent has not been received. One nurse said "It's a 10:1 ratio of 'patient match not found' versus having a CCD."

Both of these issues are mitigated by accessing the Epic portal. For example, one NF nurse mused that when privacy concerns prevented transmission from the hospital of the CCD, the nurse just logs into the Epic portal. Information from the hospital portal is also used to supplement the information provided in the CCD to get a more complete picture of a patient's condition. In total, between the CCD and Epic portal, admissions nurses spend about 15 minutes collecting all of the information needed from a patient's record. This process was common in many of the LTPAC nurses we interviewed. St. Gertrude's gets over half of its patients from St. Francis Hospital Transfers, the only hospital with which it has a server to server connection. Transfers with hospitals other than Allina are still largely done by fax; Benedictine has reduced the amount of paper transfers as a result of their effort to connect with Allina, but still receives some paper transfers from other hospitals.

The NFs participating in the Fairview-Ebenezer initiative had not yet initiated exchange, but had participated in test cases of HIE as part of their Health Information Technology for Post-Acute Care (HITPAC) project, the goal of which was to observe workflow. Prior to receiving this grant, the NFs were reportedly sending patients who were transferred to the hospital with a 100 page hard document, which they learned the hospital was not reviewing because it was too cumbersome.

The project team determined that the Interventions to Reduce Acute Care Transfers (e-INTERACT) form provided the targeted information that hospitals said they needed for an incoming patient. This form was sent via DSM to hospitals as part of the pilot. However, shortly after the pilot was completed, the hospital discontinued participation in exchange because the workflow changes were deemed problematic. Because LTPAC facilities did not have interoperable EHRS, the HITPAC project used a separate product to exchange the e-INTERACT form. This product sends DSM notifications to hospital Outlook accounts to alert of messages in their DSM account; receivers then have to exit Outlook, and log into DSM to view the message. Because of the way it was set up at the time, it did not allow distribution lists, so a single person on the mailbox would receive the notifications.

Fairview and Ebenezer anticipate similar workflow barriers for the SIM Model Testing67 activities, and said that getting hospitals and the acute care hospitals' EHR vendors on board will be critical to success.

Impact and Evaluation

Past Evaluations of eHIE Initiatives

Overall, the peer-reviewed and other literature on the potential impacts and outcomes were on eHIE generally, not eHIE with LTPAC providers more specifically. Similarly, the majority of these HIE efforts targeted for stakeholder discussions had limited to no evaluation component, as required by grant funding or that could be conducted with relatively little resources. There was great interest in evaluation research--both formative and process oriented as well as summative or outcome oriented.

In the Pennsylvania case study, KeyHIE was the subject of an evaluation through two grants, both of which are now complete (see "Funding" section for details on those grants): Geisinger's Keystone Beacon award from ONC and their HIE grant from AHRQ. Abt's evaluation targeted all providers participating in KeyHIE, not just the LTPAC providers. Abt Associates was the lead evaluator for both grants, though Geisinger staff was involved with obtaining secondary data and coordinating qualitative data collection for case studies. The Abt evaluation team was Abt and Geisinger were unable to conduct an analysis of claims data as originally planned due to challenges described in more detail below (see section "Limitations to Conducting a Comprehensive Quantitative Evaluation on Impact") but were able to conduct the following analyses:

  • A Qualitative Case Study on HHAs. A paper describing the results of those case studies is forthcoming.

  • A Patient Survey Analysis. The results of the surveys indicated that patients are relatively comfortable with giving case managers access to their health data and allowing their physician to share their information with other physicians as needed but are less comfortable with hospitals making available their health information more broadly. The surveys also revealed patients' misconceptions about the extent to which their health information is available to ERs and hospitals and general concerns about identity theft.

  • An Analysis of ICD-9 Codes. The analysis concluded that the ICD-9 codes from the problem lists in KeyHIE cannot be used as a substitute for the final diagnosis seen on a patient's bill.

In addition to contracting with Abt Associates to conduct the evaluation activities described above, KeyHIE also recently committed to conducting a new study of KeyHIE as part of a new grant from HRSA bring 50 or more LTPAC providers onto KeyHIE and expand use of KeyHIE's three tools: KeyHIE Transform, MyKeyCare and DSM. As part of that grant, KeyHIE has agreed to measure 30-day hospital readmissions and all-cause ER visits (no projections on reductions).

In the Minnesota case study, the State Health Access Data Assistance Center (SHADAC) is the state evaluator for Minnesota's SIM grant. As part of its evaluation, SHADAC is charged with evaluating the SIM-funded development and implementation grants awarded through the e-Health Grants Program, including the Fairview-Ebenezer initiative's 12-month development grant. The goals of the SHADAC evaluation include documenting what is going on, what did not go as planned, and implementation barriers and facilitators. The evaluation will focus more on coordination and transitions, and less on issues like the type of technology use, the transport, and the ability to parse. The evaluation includes both quantitative and qualitative methods. Quantitative data sources that the SHADAC evaluation team plans to use for its evaluation of the e-Health grants include a state-fielded EHR survey and a continuum of accountability matrix, which is a self-report mostly made up of process measures (e,g., "Are you able to do exchange with X-type of providers?" and "Can you use exchange for X, Y, Z functions?"). Respondents rate themselves from a beginner to more advanced. On the qualitative side, SHADAC is conducting interviews with both development and implementation e-Health grantees (e.g., Fairview-Ebenezer, Otter Tail County Public Health, Winona Health, etc.).

Prior to the SIM evaluation, Stratis Health conducted an evaluation of the HITPAC project involving Fairview and Ebenezer. The Stratis evaluation included two surveys. The first survey was an assessment of participants as they completed exchange tests and, in some cases, as they went live with exchange; the purpose of this assessment was to understand participants' experience receiving exchange. As part of this assessment HITPAC participants developed five use case scenarios (realistic examples of typical scenarios during care transitions). The purpose of the second survey in the Stratis evaluation was to understand the extent to which participants found eHIE valuable. The Stratis evaluation identified three major barriers to widespread adoption of eHIE: (1) lack of understanding about the value of interoperability; (2) lag in adoption and optimization of EHRs among SNFs; and (3) the lack of technology solutions for exchanging in eHIE.

To date, an evaluation of the Benedictine-Allina initiative is not underway or planned.

Quantitative Evaluation Plan

Unfortunately, quantitative analysis of eHIE in LTPAC is fraught with problems because of the immature state of systems in even the more advanced areas, the fragmentation of systems and technology used across types of providers, and the very local character of exchange solutions. Much of the evaluation methodology for HIE that has been discussed in the literature is hypothetical and applicable only when more robust systems are in place for public exchange--for example, envisioning use of data from HIOs for public and disease monitoring purposes.68 In this section, we discuss major limitations to a comprehensive quantitative evaluation of eHIE based on project findings.

The Urban Institute team conducted case studies and site visits to learn about the following initiatives to enable eHIE with LTPAC providers: KeyHIE in the Northcentral/Northeast region of Pennsylvania, the Fairview Health Services/Ebenezer senior services initiative in Minneapolis, and the Allina Health/BHS initiate, also in Minneapolis. Previous memoranda summarized the eHIE in these markets, particularly as those activities pertain to the involvement of LTPAC providers.69 Specifically, these memoranda summarized key findings from the site visits, efforts to prepare for and implement HIE between LTPAC providers and their partners, and any relevant evaluations underway or completed in these markets. This section summarizes findings from these analyses relating to the feasibility of conducting the comprehensive quantitative evaluation described in the prior section.

A Path Forward for Evaluating eHIE Involving LTPAC Providers

In light of what we have learned about the reality of LTPAC providers and HIE, the following describes a feasible approach to address the research questions of interest, relying on the incentives and opportunities presented by IDSNs and ACOs, and the use of interoperable HIT tools among LTPAC providers (e.g., the Transform tool).

An overarching lesson was that having a relatively advanced HIE infrastructure is a necessary, but not sufficient condition for integrating exchange with LTPAC providers into the system. At a minimum, evaluating outcomes of exchange including LTPAC requires the ability to identify locations or organizations where LTPAC providers have been integrated into exchange beginning at some identifiable event defining an "intervention" period for a pre/post design. In practice, it is relatively simple to define the event, such as initiation of a program. However, given that implementation is a process that can take substantial time to complete, the intervention period during which change may reasonably be expected can be more difficult to clearly delineate. Thus, it may important consider stages of implementation, early operation, and maturity in evaluation design. In order to implement the stronger difference-in-difference design, the challenge is finding a credible comparison group not exposed to the intervention to examine over the same time periods.

The move toward ACOs under the ACA comes from the need to contain costs in Medicare, but interest and implementation of the model extends to Medicaid programs and predates the ACA. ACOs are networks of physicians and other providers that are held accountable for the cost and quality of the full continuum of care delivered to a group of patients. The ACA authorized Medicare to contract with ACOs with the aim of achieving the "triple aim" of improving quality of care, improving population health, and reducing costs. Similar to the IDSNs of the 1990s, the premise is that ACOs will accomplish these aims by coordinating care, managing chronic disease, and aligning financial incentives for hospitals and physicians. In theory, ACOs can improve quality and lower costs using several methods, including disease management programs, improved care coordination, alignment of incentives for physicians and hospitals via shared savings, use of non-physician providers, and the formation of PCMHs.70 Over the past five years, both the number of participating ACOs and the number of participation options for them have grown dramatically, while potentially generating $400 million in savings for Medicare.71

ACOs are increasingly turning their attention to post-acute providers to better manage cost and quality across the care continuum. A recent descriptive analysis of the structural and functional provider relationships finds that ACOs are expanding their partnerships and developing relationships with LTPAC providers. For example, more than half of Pioneer ACOs have core or structural partnerships72 with HHAs, more than 40 with hospice facilities, and more than 20% with NFs.73 ACOs are also using functional relationships74 to extend the care continuum beyond what can be achieved with care partners alone, particularly for urgent care and post-acute care providers.75

An evaluation of eHIE among ACOS/IDSNs and partnership/acquired LTPAC providers would aim to address the following research questions:

  • Prior to forming a partnership with an ACO/IDSN, what type of EHR systems were LTPAC providers using? Were they electronically exchanging health information with other providers? What type(s) of information were they exchanging and how?

  • How did LTPAC provider's EHR system and technology change after forming a partnership?

  • Once the partnership started, what type of information was exchanged within the system? Outside the system? What technology is being used to exchange this information?

  • How did patient outcomes, utilization, and costs change after the partnership was formed?

  • There are a number of advantages to evaluating HIE between LTPAC providers and their exchange partners within an ACO, or similar model of care (e.g., integrated delivery network [IDN]) setting. First, as previously mentioned, this is priority policy area in the Medicare program and findings from this evaluation would complement prior and ongoing evaluations of the ACO model. Second, it would likely be easier to obtain data by partnering with a single ACO or IDN as opposed to partnering with an HIE that represents multiple organizations. As shown with the prior Abt evaluation, providers participating in KeyHIE declined to provide access to their data for research purposes out of concern that they would be handing over key information to a major competitor. It might also be easier to access CMS claims data because an ACO-focused evaluation would directly benefit the Medicare program. Finally, our case studies and prior research76 indicate that a key advantage of private HIEs within IDNs and ACOs is that eHIE, particularly with LTPAC providers, is more robust within these private organization than in state-sponsored HIEs.

  • The major drawbacks of this approach, however, are that this type of evaluation would be limited and not provide an overall assessment of HIE within a region or market. The organizations that ACOs or IDNs connect are sometimes restricted based on strategic and proprietary interests. For example, hospitals may choose to connect with the ambulatory care and post-acute care providers with whom they would like to more closely affiliate, regardless of existing referral patterns in the market. This complicates overall participation in HIE, data re-use, and ultimately care coordination.

Given what was learned in Pennsylvania and Minnesota, a first step in an evaluation would be an evaluability analysis of candidate sites, using interviews with relevant informants within proposed ACOs/IDNs and focusing on such critical issues as willingness to participate in an evaluation, data availability and access, existence of comparisons, and volume of exchange with LTPAC providers occurring. Having identified the most promising site or sites, we would use a mixed methods approach to address the research questions listed above. We would conduct a survey of LTPAC providers within the selected "treatment" ACO/IDN, and ideally, comparison group providers. This survey would assess the technology used, the regularity and frequency of use, the primary objectives of use, the motivations to engage in exchange, implementation challenges, and the benefits realized. To complement the survey and fill in any potential gaps in understanding of the exchange environment in which the LTPAC providers are operating, we would conduct additional targeted case study interviews with key decision makers within the ACO/IDN across the care continuum. Finally, we would conduct quantitative data analysis with the best available data, which would depend on the location and organizations selected.

Analyses could draw on claims data, EHR and other clinical data, and measures developed from the survey data. Claims data could provide direct measures of patient encounters (e.g., readmission rates) and some treatments and medications. Claims data are accessible from government entities (states, CMS) and from private payers, and increasingly, states and other stakeholders are working to establish all-payer claims databases (APCDs). Based on the experience of a recent ACO evaluation, as a federal contractor, we anticipate that CMS would be willing to approve a data use request for research identifiable Medicare claims and enrollment data from the Chronic Conditions Warehouse (CCW) and ACO-specific data that contains identifying information for participating providers and aligned beneficiaries and their corresponding ACOs.77 A critical issue for the value of the latter information is the ability to identify ACOs that have integrated LTPAC facilities into their networks.

While claims data is currently the main data source used to calculate outcome measures, it might be feasible to use clinical data from EHRs. Much of the information in claims data is now being captured by EHRs and is available at the system level. A notable limitation of EHR data, in contrast to claims, is that comparable data may not be available for potential comparison groups. In addition, the possibility of data sharing arrangements would need to be explored early on.

We would attempt to find a comparison group that consists of similar fee-for-service (FFS) Medicare beneficiaries in markets not served by an ACO and who do not receive care from an ACO/IDN. Alternatively, comparisons might be feasible between IDNs or ACOs in locations where there is a distinct difference in LTPAC participation across networks.

Examples of Potential Settings


The Geisinger Health System is one example of an ACO-like model that has incorporated LTPAC providers and continues to do so. In contrast to Abt's evaluation of KeyHIE as a whole, we would only assess eHIE among LTPAC providers and their partners within the Geisinger system.

Results from our site visit suggest that in a departure from its traditional business strategy, Geisinger is increasingly becoming interested in purchasing LTPAC providers. Initially, Geisinger focused mostly on acquiring HHAs. For example, in 2014, Geisinger acquired Sun Home Health and Hospice.78 Several respondents indicated that after completing the acquisition of these HHA sites, Geisinger has focused on the NF sector. One interviewee indicated that Geisinger is trying to develop a "SNFist model" where providers can make decisions at the NF site instead of taking the patients back to the hospital.

From an evaluation and policy perspective, a unique aspect of Geisinger is their development of the Transform tool. Geisinger's 2010 Beacon Community grant provided funding for LTPAC provider outreach and the development of the Transform tool. The KeyHIE Transform tool takes MDS and OASIS data and converts the clinically meaningful information to a CCD. This CCD can be exchanged using KeyHIE so that the all participating providers could access the CCD. The Transform tool is inexpensive relative to the cost of interfacing with an exchange, which appeals to LTPAC providers who may otherwise not be willing to participate in information exchange. The Transform tool was launched in 2013 and provides a unique opportunity for a quasi-experimental design evaluation, with the "pre" period being before 2013 and the "post" period including 2013 and later years. A key question to address in an evaluation would be whether LTPAC providers acquired by Geisinger use the Transform tool and/or whether their EHRs were integrated into Geisinger's system. Another key issue to address is the extent to which Geisinger is working with LTPAC providers that they did not acquire, and the extent to which these providers use Transform.


The Benedictine-Allina project also represents an example of an ACO-like model that has incorporated LTPAC providers. Allina Health is a non-profit health care system based in Minneapolis that owns or operates 14 hospitals and more than 90 clinics throughout Minnesota and Western Wisconsin. Allina Health is participating in the Medicare ACO program. The BHS is one of the largest senior care organizations in the United States, with 36 NFs, 25 ALFs, and one HHA.

We would propose to evaluate the March 2013 e-Health Connectivity Grant as a policy intervention. In 2013, Benedictine received $375,000 from the State of Minnesota to develop MatrixCare software so that it can exchange CCDs with Allina's Epic system peer-to-peer. This new software was launched in December 2013, creating a "post-intervention" period of 2014 and later.


Colorado represents a number of potential evaluation opportunities, from the perspective of delivery system reforms involving both Medicare and Medicaid, HIE infrastructure, and data. Colorado also still is largely a FFS state, although its SIM plan includes transitioning to capitation over the next several years.

Colorado's Medicaid Accountable Care Collaborative (ACC), launched in 2011, draws on seven Regional Care Collaborative Organizations (RCCOs) state-wide that develop networks of providers. The RCCOs are responsible for connecting beneficiaries with needed clinical and other services and fostering communications between providers to improve care coordination. The ACC did not initially enroll dually eligible beneficiaries, but it expanded membership to include them in 2014 under the state's Financial Alignment Initiative demonstration.79 The focus will be on improving chronic disease management and transitions between hospitals, rehabilitation hospitals, NFs and community residence.

Physician Health Partners, a medical management company based in Denver, became a Medicare Pioneer ACO in 2012 in partnership with the Primary Physician Partners and South Metro Primary Care. The ACO serves about 30,000 Medicare beneficiaries in the seven-county metro area, and in 2014 began participating in the Medicare Shared Savings Program.

The state has a large and well-established regional HIO, the CORHIO, which in 2011 received a challenge grant from the ONC to increase connections with LTPAC facilities including post-acute rehabilitation hospitals, NHs, assisted living centers, home health care agencies and hospice. As of June, the CORHIO network included 48 hospitals, more than 2,600 providers, 131 long-term care facilities, 39 behavioral health centers, four large medical laboratories, EMS providers, the Colorado Springs Military Health System, and the state health department.

CORHIO provides bidirectional exchange with provider EHRs, but most LTPAC providers are using secure, web-based query access to a community health record system from which they can have real-time access to patient information and the ability to generate CCDs, regardless of whether they have an interoperable EHR. In 2015, CORHIO received a new ONC grant to support implementation of the Transform tool, which would allow LTPAC providers with or without EHRs to translate information from MDS and OASIS assessments and share them through the HIO. Thus, two possible evaluation points are defined by the initial 2011 challenge grant to increase connections with LTPAC providers and the 2015 grant to implement Transform.

The state also has an APCD, administered by the non-profit Center for Improving Value in Health Care (CIVHC). The APCD was established by the legislature in 2010, and as of January 2015, its data warehouse reported health insurance claims from Medicare, Medicaid, and the 20 largest health plans for individual, large group fully-insured, small group and some self-insured lives, as well as Medicaid and Medicare. The claims represent more than 3.5 million unique covered lives and 65% of the insured population in Colorado. Medicare claims for 2009-2011 and 2013 data for commercial payers and Medicaid is currently available through the Data Release Review Process and will be available on the data website in 2015.80 Unlike Minnesota, Colorado allows release of APCD data at varying levels of detail and specificity for research under a CMS-like review process requiring "that the intended use supports reaching the Colorado Triple Aim of better health, better care, and lower costs."

Setting Strengths and Limitations

It is important to consider several factors while conducting an evaluability assessment of the proposed sites. Table 2 uses the three settings described above to illustrate the type of questions to be addressed in selecting an evaluation site or sites. This table provides cross-setting information on several factors, including existing contacts, the availability specific settings and interventions, and the relative ease of access to quantitative data.

Each site has a specific setting and intervention to evaluate. We would evaluate the launch of the KeyHIE Transform Tool in 2013 in Pennsylvania, the 2013 e-Health Connectivity grant in Minnesota, and the 2011 and/or 2015 HITECH grants in Colorado. However, the magnitude of these interventions is likely to vary across settings. For example, the Transform Tool has a more global focus, with the ability to be used by more providers, relative to the smaller e-Health Connectivity grant intervention. Similarly, Colorado's new grant to implement Transform has a broader application than the earlier grant. In contrast to Pennsylvania, however, it might be easier to find a valid comparison group in Minnesota and Colorado, where there is a relatively high prevalence of similar health care systems in the region, compared with Pennsylvania, where Geisinger is one of the most unique and advanced IDNs in the nation. Colorado is likely the best site in terms of claims data access due to the availability of APCD data to researchers.

TABLE 2. Cross-Setting Comparisons
  Pennsylvania Minnesota Colorado
Contacts from site visits? Yes Yes No
Specific setting to evaluate? Geisinger BHS-Allina Health Physician Health Partners and other potential options
Intervention for evaluation? 2010 Beacon Community grant to develop its Transform tool to convert MDS and OASIS data into a CCD (launched in 2013) March 2013 e-Health Connectivity grant for exchange and use of CCDs between BHS (long-term care system) and Allina (hospital system), via MatrixCare and EpicCare software (launched December 2013). 2011 HITECH grant to expand LTPAC access

2015 HITECH grant to implement Transform tool

Comparison group feasibility? Challenging, due to Geisinger's uniqueness Relatively easy, due to high prevalence of IDNs in Minneapolis region Relatively easy, due to multiple regional networks in operation.
Medicare ACO program? [could improve likelihood of CMS approval for claims data] Medicare Shared Savings Plan ACO (Keystone ACO) Pioneer ACO (Allina) Pioneer ACO (Physician Health Partners)
APCD? No Yes, but not accessible to evaluation except for state contractors Yes

General Limitations

There are some general limitations that apply to all settings as well. First, small sample sizes could hinder evaluation efforts at each of the potential sites, especially given the limited post-implementation period of the interventions considered and the relatively low prevalence of NH residence, hospitalizations and post-acute care. About 20% of all Medicare enrollees use hospitals in a year; about 5-6% use SNFs, Part A Home Health, and Part B Home Health, respectively, not adjusting for enrollees using more than one type of post-acute care; and 3% of those age 65 or older reside in NHs.81 Similarly, the interventions to be evaluated are not discrete, that is, implementation was likely phased-in over a relatively long period of time. Second, sample selection could bias any potential estimates since these interventions were not randomly assigned, and each site could also suffer from omitted variable biases as multiple policy interventions and changes to the health care were occurring during the same analysis period. Third, research organizations in any of these settings will likely need to obtain multiple IRB and data use agreement approvals, thus creating substantial time costs in obtaining data. Finally, across all settings, it will likely be very challenging, if not impossible, to directly obtain data from providers (e.g., EHR data) due to privacy and security concerns. However, researchers could potentially obtain aggregated EHR data for sites that cooperate.

View full report


"LTPACsetting.pdf" (pdf, 1.22Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®