An Environmental Scan of Pay for Performance in the Hospital Setting: Final Report. V. Summary of Findings from Environmental Scan


Mounting cost pressures and substantial deficits in the quality of care within the U.S. health care system have led policy makers to consider options for system reform to drive improvements.  Value-based purchasing is one reform option being examined and tested by payers in the public and private sectors, and it includes both financial (e.g., P4P) and non-financial (e.g., transparency of performance scores) incentives designed to change the behavior of providers.

The Deficit Reduction Act of 2005 (Public Law 109-171, Section 5001(b)) created a statutory mandate for the Secretary to develop a VBP plan for Medicare hospital services commencing FY 2009.  This mandate was delegated to the CMS Hospital VBP Workgroup. This environmental scan was conducted to inform the development of the VBP plan for Medicare hospital services. Our scan comprised a review of the literature and key informant discussions with a wide array of individuals who could provide a picture of the current state-of-the-art in hospital pay for performance, including 27 program sponsors, 28 hospitals, 7 hospital associations, 5 data support vendors, and a number of individuals with expertise in rural hospital issues.  As part of our discussions, we also examined the experiences of hospitals participating in the Medicare RHQDAPU pay-for-reporting program.

Among the key findings of this review is that hospital P4P has been implemented by more than 40 sponsors, in some cases for more than three-to-five years. Little empirical evidence has emerged, however, from these initiatives to gauge the impact of hospital P4P in meeting a program sponsor’s objectives.  This is primarily a function of the absence of formal evaluation occurring in most P4P programs and the challenges of conducting evaluation in real-world applications that lack comparison groups to assess the impact of the P4P intervention.  The strongest evidence on the impact of hospital P4P to date has been shown through the Premier evaluation of the Premier Hospital Quality Incentive Demonstration (PHQID) and the Lindenauer study of the impact of PHQID relative to the Medicare pay-for-reporting program. These studies suggest the additional effects of P4P are somewhat modest relative to public reporting and other quality interventions that are occurring simultaneously. The literature suggests, however, that multifaceted interventions will be most effective at producing sustained improvements in patient care (Grol et al 2002; Grol and Grimshaw 2003).Drawing from the theoretical literature on the use of incentives, it appears that incentives can be effective in changing behavior, and that how the incentives are structured will determine the type and magnitude of the behavioral response.

In our hospital and P4P program sponsor discussions, there was an expressed desire to allow experimentation to create models where learning could occur, which could help inform design structures. The discussants anticipate that the results of P4P and specific design options may differ as a function of the varying structure of local health care markets. 

Given that P4P is a newly emerging reform tool and that little information is currently available about the impact of P4P or the influence of various design structures on P4P outcomes, P4P programs should incorporate evaluation and ongoing monitoring into their design as a means of building a knowledge base.  The collection and broad dissemination of this type of information will be critical to future efforts to construct P4P programs so that they can meet their programmatic objectives.  Funding will be necessary to support program evaluation, and the evaluation work needs to be sustained over multiple years to fully assess impact and monitor for unintended consequences.

The key design and implementation lessons that emerged from our discussions with program sponsors, hospitals, and data vendors included: 

  • Measures—Hospitals expressed concerns about growing data collection and reporting burdens across the various P4P programs and reporting initiatives being developed by an array of sponsors, whose efforts are not fully aligned.  Hospitals expressed a strong desire for measures to be aligned, for reporting efforts to be coordinated, and for use of evidence-based standardized measures to minimize physician pushback. While P4P program sponsors desire to expand the number and types of performance measures to ensure a more comprehensive picture of hospital quality, hospitals stated a desire for a more limited set of measures on which they could focus quality improvement efforts. Given the limitations in the number and type of measures currently available for use in pay for performance and public reporting, resources will be required to support additional measure development and testing as well as the development of methods to create composites.
  • Payment structures—There is consensus among hospitals that payment structures should use absolute thresholds and reward all good performers, rather than providing incentives on a relative-performance basis, for example only to the top 10 or 20 percent of hospitals participating in a P4P program.  This was seen as critical when the measures of performance used have scores that “top out,” reflecting little meaningful difference in the performance across hospitals, as has occurred for several process-of-care measures (e.g. for care of acute myocardial infarction).  Another approach that could avoid the payment issues associated with topped out measures is to use the appropriate care composite, which reduces the ceiling effect, as the basis for payment rather than individual measures.  Programs sponsors felt strongly that performance improvement as well as attainment of specific benchmarks should be included as a component of the payment structure, at least in the early years of the program, in order to engage all hospitals in the P4P program. Hospitals also noted the difficulty of getting physicians to change their behavior absent aligned incentives on the physician side, and called for program sponsors to create parallel physician incentives focused on inpatient care for the same conditions used in hospital programs.  Physicians would also be more likely to support P4P programs that did not place an additional burden on physicians in terms of data collection or documentation. 
  • Data infrastructure—Current validation efforts are weak, and program sponsors and hospitals acknowledged the need to strengthen validation as more money is put at risk in P4P programs. Hospitals indicated the need for technical support to comply with P4P program requirements, citing the important role played by QIOs and data vendors in this regard.  Health information systems require modification moving forward to capture the data elements used to produce performance measures, and absent this investment, hospitals will continue to have to extract
  • Public reporting—Hospitals indicate they do pay attention to how their institution looks publicly and that public reporting has forced their boards to more closely monitor quality and provide resources for quality improvement.  Both program sponsors and hospitals cited a need for simplification of performance information presented on consumer websites, such as the CMS Hospital Compare website, to facilitate consumer understanding and use of the information.
  • Engagement strategies—Program sponsors noted the importance of engaging hospitals in the planning and execution of P4P programs to encourage a more collaborative versus payer-driven approach to implementing this payment reform.  Engagement strategies included involving providers in the measures selection process and program design more broadly, and in ongoing planning as the program evolves over time.

Our discussions also uncovered a number of program implementation challenges that merit consideration during program design and implementation.  One challenge that affects a sizeable number of hospitals is the problem of having only a small number of events or cases to report for one or more measures; a small number of events to score leads to unstable estimates of performance to use in performance-based incentive payments.  While this is a more acute problem for small and rural hospitals with a small number of patients per year, the problem can also occur for medium- and large-size hospitals depending on their service mix, details of measure specifications, and the use of sampling during data collection.  Use of all-payer data, collecting data over extended periods of time, use of composite measures, and identifying measures relevant to smaller providers are approaches that can help to mitigate the small numbers problem.

The data collection burden, which affects how many measures a P4P program can reasonably require a hospital to collect and report, creates challenges for efforts to comprehensively assess the performance of hospitals.  The more comprehensive the measure set used, the greater the burden on hospitals, given existing information technologies. Current information systems are not equipped to capture and easily retrieve the clinical information used to create performance measures, nor are they structured to enable routine monitoring of quality of care.  Until health information systems are upgraded to capture this information, program sponsors will be constrained in the number and breadth of measures they can expect hospitals to collect and report.  P4P programs are also challenged with an acute need to ensure the integrity of the data used to score hospitals and make differential payments, which requires resources for data validation.  Allocating sufficient resources to validation work is critical for program credibility, and today only limited resources are being used for data validation within P4P programs.  Most hospitals stated that the current level of validation is insufficient, given the potential to shift large sums of money within the system. 

P4P programs have the potential to drive system improvements. The success of these programs in meeting improvement goals will be affected by their design, implementation, and allocating sufficient resources to engage in the necessary day-to-day operations, program monitoring and impact evaluating, and ongoing modification. Given the limited knowledge base, it is critical that P4P programs include evaluation in their design to generate the knowledge to support smart program design and efficient use of resources.

Hospitals understand that P4P is likely to be part of their future and generally seem supportive of the concept. They face a number of challenges to their ability to successfully participate in these programs, including lack of physician engagement, inadequate information infrastructure that necessitates the manual collection of data from charts, and potentially conflicting signals from various organizations measuring hospital performance. These implementation challenges should be carefully considered in the design of any hospital P4P program.

View full report


"PayPerform07.pdf" (pdf, 1.22Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®