Dissemination and Adoption of Comparative Effectiveness Research Findings When Findings Challenge Current Practices. Key Barriers to Clinical-Practice Change


Study Design and Conduct

A number of aspects of the trial’s design were criticized by experts:

1.     Some interventionists perceived that the trial was designed with the objective of disproving that PCI was beneficial. This may have polarized opinions even before the results were released.

2.     There was controversy about whether or not COURAGE enrolled a population with lower risk than the typical population of patients with stable CAD. In the view of some experts (mainly interventionists), the timing of randomization—following angiography—guaranteed that high-risk patients would be triaged immediately to PCI rather than being enrolled in the trial, leaving a population of lower-risk patients to be enrolled. These experts cited the cohort’s average ejection fraction (60.8 percent) and low cardiac death rate (0.45 percent/year) (Bangalore and Messerli, 2007; Tommaso, 2008) as evidence of the cohort’s low risk. Others argued that COURAGE enrollees represented a relatively high-risk population, noting that their five-year MI rates were similar to those of patients with ACS—a condition associated with a high mortality rate (Diamond and Kaul, 2007). Some compared the risk profile of COURAGE enrollees to the average risk of patients undergoing elective PCI in the NCDR and found them to be comparable (Peterson and Rumsfeld, 2008).

3.     Most critically, because randomization occurred after angiographic findings were known, the trial appeared to be designed to inform decisionmaking only after angiography. In practice, however, diagnostic angiography and PCI are often performed in tandem without an opportunity for shared decisionmaking between the two procedures. The trial did not explicitly address decisions that may occur before angiography. Experts sug-gested that trials designed to explicitly address the upstream decisionmaking process (prior to angiography) would be more likely to have an impact. Trials using this design are currently under way and are considered to be “more promising.”

4.     The trial may have been underpowered. The intent was to detect a survival difference of 22 percent—the expected benefit of PCI in patients with ACS—between groups (Diamond and Kaul, 2007; Tommaso, 2008). However, some interventionists felt that detecting this difference in a population with stable CAD was unrealistic, because of the low severity of illness of COURAGE enrollees.

5.     Crossover was significant; 33 percent of patients randomized to the OMT group crossed over and received PCI. The relatively high rate of crossover meant that the apparent equivalence in primary outcomes could be due to the effectiveness of OMT or to the added benefit of PCI among those patients who crossed over. Thus, this may have affected stakeholders’ interpretations of the trial’s results. Crossover rates are frequently higher in CER trials involving surgery. In the second Medical, Angioplasty, or Surgery Study (MASS-II), which compared CABG surgery, PCI, and OMT, the crossover rate was nearly 24 percent (Diamond and Kaul, 2007).

6.     Some criticized the poor quality of PCI procedures performed during the study. Experts pointed to the high revascularization rate in the PCI group (21 percent) and the low rate of multiple-stent use. Critics noted that only 36 percent of patients received more than one stent, even though 70 percent had two-vessel disease (Kereiakes, Teirstein, et al., 2007). Others said that this frequency is comparable to the experience of patients elsewhere, including those within the New York state angioplasty registry (Diamond and Kaul, 2007). Critics also noted that PCI was successful in only 89 percent of cases whereas a 95-percent success rate better reflects current practice (Prasad, Rihal, et al., 2008).

7.     Other aspects of the trial raised questions about the validity and/or generalizability of its findings. The strict enrollment criteria of COURAGE might also have reduced the generalizability of the results. The small number of patients enrolled (2,287, only 6.4 percent of the 35,539 patients screened) suggested that the results had limited generalizability (Kereiakes, Teirstein, et al., 2007), and some criticized the absence of any data characterizing the non-enrolled patients. However, similar trials, including MASS-II, also had low enrollment rates (2.9 percent of 20,769 screened patients). The fact that drug-eluting stents were not used in the trial was also cited as a design flaw. While many experts agree that their inclusion would not have changed the frequency of primary trial end points (Bhatt, 2007; Boden, 2007), drug-eluting stents might have improved the performance of PCI with respect to angina symptoms and quality-of-life outcomes (Diamond and Kaul, 2007). A sensitivity analysis conducted as part of the cost-effectiveness analysis showed no impact of the use of drug-eluting stents (although that study was performed on a limited sample) (Kirtane and Cohen, 2008).


Financial Incentives

Financial incentives are widely believed to be a potent driver of PCI use. While hospital and physician fees for elective PCI may average $20,000 per case, managing drug therapy and delivering lifestyle counseling are poorly reimbursed. One CER researcher commented that “[doctors] are paid a lot to do an angioplasty and nothing to talk about treatment options.” There appear to be few countervailing incentives that would reduce the use of PCI for patients with stable CAD. Interventional cardiologists may recommend PCI because they believe that other cardiologists and primary-care physicians expect them to do so and that advising against the procedure might jeopardize future referrals. Most experts indicated that few patients fail to receive PCI once they are referred to an interventionist. Competition for patients between interventionists and cardiac surgeons may reinforce these patterns. Elective PCI is also a key revenue source for hospitals. The emergence of for-profit heart hospitals has been driven, in part, by the high margins from elective PCI procedures. To date, few payers have imposed significant barriers to the use of PCI.

Psychological Issues Driving PCI Use

The strong urge to open all significant coronary lesions amenable to PCI once they have been detected through angiography has been referred to as the “oculostentotic reflex” (Lin, Dudley, et al., 2007), which suggests that psychological factors may drive the use of PCI. The published literature and expert accounts describe at least three specific drivers. First, the procedure-based solution to the problem of CAD addresses a physician’s desire to “do something,” while OMT is perceived as inaction. Eliot Freidson made the observation many years ago that physicians often prefer action to inaction even when there is little chance of success (Freidson, 1970). In many cases, the desire to act is a joint sentiment. Patients often desire an active treatment approach; to some, chest pain may provoke significant anxiety even if it has a stable pattern (Lin, Dudley, et al., 2007). Finally, bad outcomes leave a strong impression on physicians, especially if an action was not taken. One or two negative personal experiences may contribute to a physician pursuing more-aggressive management of CAD (Lin, Dudley, et al., 2007).

Bias Against OMT

Historically, the value of PCI relative to OMT was perceived to be high, because OMT had limited efficacy. However, our discussants described OMT today as being “far superior” to past therapy. Some speculated that the low event rates in trials might be due to the efficacy of modern OMT (O’Gara 2010). While physicians may be aware of the benefits of modern OMT, patients may be less convinced because they are difficult to observe directly. Physicians are also aware that adherence to OMT is relatively poor (Kereiakes, Teirstein, et al., 2007), because of cost barriers and other factors (O’Gara, 2010). Thus, many may view the level of OMT attained in the COURAGE trial as unrealistic and not achievable in practice.

Lack of Change to Practice Guidelines

In the years preceding COURAGE, guidelines published by the American College of Cardiology (ACC), the American Heart Association (AHA), and the Society for Coronary Angiography and Interventions (SCAI) consistently recommended OMT as the favored first approach for patients with stable CAD (Boden, 2008). While COURAGE might have potentially strengthened that recommendation, no major changes were made to practice guidelines in the four years since the findings were released, although an updated set of guidelines has been in development for more than three years. Despite the rigorous guideline development process at the ACC, experts feel that the societies do a poor job of disseminating and promoting adherence to cardiology guidelines. While provocative, the nuclear substudy findings from COURAGE were not seen as definitive and therefore did not lead to updated guidelines.

Duration of the Trial

Enrollment of patients for COURAGE began in June 1999, and the results were published in April 2007, a span of nearly eight years. Over the course of the trial, standards of practice changed dramatically. In particular, drug-eluting stents became the standard of care, despite the fact that few enrollees received them. As mentioned earlier, some experts therefore questioned the generalizability of the trial, while others showed empirically that this limitation did not affect the results. One discussant, noting that PCI technology had evolved during the past decade, bemoaned the fact that the use of outdated procedures is “always the first critique of studies that fail to show a benefit for a procedure.” A recently proposed trial to validate the nuclear substudy findings in COURAGE is projected to take between seven and eight years to complete and may pose similar challenges. Some argue that the increasing use of international clinical trials may help to shorten the time required to release results and will thus improve their relevance. The cardiologists we spoke with emphasized that the appropriate role of observational CER studies in this area of cardiology has not been clearly articulated.

View full report


"rpt_RANDFinal.pdf" (pdf, 2.01Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®