Dissemination and Adoption of Comparative Effectiveness Research Findings When Findings Challenge Current Practices. Policy Options



Create a transparent governance mechanism for oversight of the CER translation process. Financial incentives play a major role in many or all of the phases of CER translation into practice. Such incentives are unavoidable given the structure of the U.S. economy, and financial conflicts of interest may also be unavoidable, but the consequences of those conflicts of interest are not inevitable. While much of the funding for CER comes from public sources, much of the support for translation activities comes from private sources. Transitioning all CER translation activities to public funding is unrealistic, although some of these activities, including the development of guidelines, quality measures, and the nation’s health IT infrastructure, have been publicly funded to a degree.

To avoid the potential for financial incentives to derail CER translation, especially when new practices challenge vested interests, a governance mechanism that monitors the translation process and documents activities could maximize the incorporation of CER results into clinical practice in a timely way. This governance mechanism could take many forms but would most likely serve as an advisory committee. Its stakeholder membership and its relationship to federal agencies, PCORI, and other organizations would have to be determined. For instance, it might work closely with PCORI and be modeled along the lines of the PCORI methodology committee. Going one step further, this entity could develop specific policy recommendations that would address the root causes of failure of CER translation. Given the diversity of involved stakeholders, it seems unlikely that this body would have the authority to enact policies, but it could play a convening role, bringing together key stakeholders who wield the authority to enact policies and document their activities. By representing key stakeholder groups and ensuring a transparent consensus-development process, this body could have substantial influence.

The case studies suggest some especially important opportunities for this governance structure that may help to mitigate financial conflicts of interest. The following opportunities would increase transparency.

1.      Include patient or consumer representatives. As the recipients of healthcare, patients are the key beneficiaries of the clinical practices guided by CER results. Ultimately, members of the public pay for healthcare, as well as the investment in CER. Patient and consumer representatives should be involved early in decisionmaking concerning CER (e.g., the questions to be answered by CER, the types of patients to be included in CER studies, and appropriate outcome measures), and they should also be involved in guiding the other phases of translation, including interpretation of results, formalization, dissemination, and implementation. Such engagement may require a public investment, and patient and consumer representatives may require training to enable their full and active participation in governance committees. The goal of such engagement would be to ensure that the CER translation process is well informed by the end-user perspective and to provide a counterbalance to other stakeholder interests—particularly those of stakeholders with opportunities for financial gain.

2.      Include public and private payer and purchaser representatives. As stewards of the financing of healthcare and representatives of their member or customer interests, payers and purchasers have a strong interest in the translation of CER evidence into high-value clinical practices. Payers tend to focus on specific clinical topics and face common challenges in using CER evidence. Because they ultimately have a central role in formalization, dissemination, and implementation of policies based on CER results, it is crucial to include their perspectives in the governance process for CER translation. Inclusion of public payers is especially important, as they serve distinct populations with higher health risks and vulnerability. The interests of public and private payers often differ in the current U.S. healthcare system. Payers and purchasers may be able to identify the specific opportunities for high-value care that would be especially amenable to CER and for which modification of payment or coverage policies could be especially effective in optimizing clinical practice.

3.      Enable and support public-comment opportunities. Vigorous solicitation of public comment with verification and full disclosure of potential conflicts of interest (both financial and intellectual) can enhance the credibility of CER. An entity and mechanisms to oversee compliance with disclosure policies would need to be established. Use of electronic media to enable structured rating and voting exercises (with appropriate safeguards to prevent manipulation) could create a transparent system to aggregate input from a broad range of stakeholders on CER priorities and concerns about the use of this information. These data might also help in arriving at consensus interpretations of CER evidence and its implications for practice.

4.      Institute strong policies on disclosure and management of potential conflicts of interest. A perception of CER evidence as biased by financial interests can undermine its credibility and impede the take-up of new practices. Conflicts of interest cannot be eliminated, but the IOM experience suggests that they can be disclosed and managed effectively. The governance of the financing of CER projects will undoubtedly be subject to conflict-of-interest policies. Our case studies suggest that the governance and oversight of all the phases of the CER translation process could benefit from such policies, which might be patterned on the recent IOM recommendations for achieving trustworthy guidelines. While most current approaches focus primarily on financial conflicts of interest, disclosure could also include potential institutional and intellectual conflicts of interest. The optimal strategy for monitoring and enforcing disclosure could be specified by a body such as PCORI.

5.      Use the governance mechanism to generate a prospective public record of stakeholder expectations. Documenting the positions of relevant stakeholders at the outset of CER studies with respect to the objectives of the study and the parameters around which the results should be interpreted would create a public record of expectations of each stakeholder in the CER process. While it would be non-binding, public documentation can create both a record of expectations that can be drawn upon during the interpretation phase and an institutional memory in organizations that have relatively high rates of executive turnover. Such a record could discourage post-hoc efforts to undermine the credibility of studies that produce results contrary to the interests of specific stakeholders.


Support and enhance creation of standards for CER generation and translation. Standards can enhance transparency and produce consistency of interpretation. Efforts to create standards for the design and conduct of CER are already in progress through activities of federal agencies and PCORI. However, standardization, including guidelines and quality measurement, can be helpful at every phase of the CER translation process. At least two other opportunities for standardization, described below, hold promise.

1.      Incorporate data elements that are critical to translation activities into the CER registry. A registry of federally funded CER studies is currently being developed by ASPE. Similar to the government-sponsored clinical-trials registry, the CER registry will create a common format for the presentation of key characteristics and results of each study that will enable the public to easily retrieve information about proposed, in-progress, and completed CER. This registry will include a variety of important facts about CER studies that could assist in consistency of interpretation of results:

·    Patterns of use of interventions relevant to the CER studies at the time they are initiated

·    Anticipatory commentary, including prioritized lists of the methodological strengths and weaknesses of proposed CER studies

·    Inclusion and exclusion criteria for patients in a CER study

·    Prespecified definitions of equivalence and nonequivalence of treatments

·    Prespecified thresholds for clinical action based on CER results (e.g., effectiveness, cost, and safety) and the specific actions that would be recommended under different result scenarios.

Creating explicit standards for CER study objectives, design, sampling, and causes of heterogeneity in sampled populations may be useful to guide the forming of a consensus interpretation of CER study results. Predefining thresholds for action may reduce post-hoc reframing of questions and implications by stakeholders to serve their own interests.

2.     Encourage development of standardized electronic clinical data systems (clinical registries). CER studies can be expensive and time-consuming. Standardized electronic clinical registries are increasingly providing opportunities to conduct low-cost, rapid CER. Apart from generating new research, registries can provide data for longitudinal tracking systems to evaluate the impact of CER translation activities on clinical-practice patterns. Engaging clinicians and patients in the use of clinical-data registries may also increase the credibility of CER results by enhancing the perceived trustworthiness of the data for end users, overcoming some of the traditional skepticism about administrative data and claims data generated for billing purposes. A common finding from our case studies was that feedback to individual physicians on utilization patterns was often a potent motivator to change practice—particularly when the data were of high quality.

Registries can support and enhance the creation of standards, because the design decisions necessary to build a clinical registry (e.g., specification of clinical concepts, data elements, measurement approaches) usually lead directly to standardization. Furthermore, generating consensus on design decisions frequently raises questions about priorities and methods, and it identifies additional opportunities for CER. Development of registries and provider participation in them may be stimulated through the CMS EHR Incentive Program and the clinical-data needs of ACOs. However, additional incentives may be required to advance these efforts.


Encourage public financing of CER translation and promote the use of CER evidence in payment programs. The case studies suggest two policy options related to CER financing.

1.     Provide direct and indirect public support for formalization of CER evidence and the dissemination of CER-based clinical practices. In line with the emphasis on public, patient, and consumer representation in the governance of CER, public financing could be provided for translation of high-profile CER evidence into guidelines, quality measures, and clinical decision support tools. Forcing guideline and measure development to rely upon industry funding tends to skew the formalization process toward generation of CER with profit-making potential for industry rather than serving the public’s interest in promoting high-value healthcare. Public financing need not replace the selective and targeted funding of such activities by industry or professional interests but would play a complementary role.

Likewise, public financing for dissemination activities and quality improvement initiatives that are based on high-quality evidence (e.g., counterdetailing and other efforts) could supplement the substantial private support for activities. This public financing could focus on counteracting marketing approaches of industry interests that might otherwise undermine CER evidence or selectively promote less-effective or more-costly practices. The public activities need not be broad-based but might use empirical analysis to identify utilization patterns that are at odds with the CER evidence to enable targeted counterdetailing programs.

2.     Promote the use of CER-based clinical practices through payment policy and incentive programs directed toward providers and patients (e.g., value-based purchasing). New payment models could reduce perverse fee-for-service incentives that may drive the use of procedures or services that have weak evidence of effectiveness. Through reliance on guidelines, quality measures, and clinical decision support, these models can incorporate CER evidence. Public payers have a special role given their accountability to the public. Encouraging payers such as Medicaid and Medicare to use prudent purchasing strategies based on CER evidence can assure that the financial incentives they (and patients) face are well aligned to support evidence-based clinical practices and discourage practices that are not evidence-based. Value-based insurance design, an approach that may be gaining favor among insurers, could begin simply with clinical areas in which the CER evidence is strongest and could then evolve into other areas.


Supporting professional consensus across the phases of CER translation. Two policy options could encourage development of consensus.

1.     Foster and support a broad vision of professionalism in the governance of CER translation. With the increasing subspecialization of medicine and surgery, subspecialty professional societies have proliferated. Conflicting professional-society guidelines are one manifestation of this trend. Differences in the interpretation, formalization, and dissemination of CER evidence and lobbying by subspecialty societies may impede effective changes in clinical practice.

Broadly constituted professional committees may be capable of producing balanced, consensus interpretations of CER results, resolving differences of opinion about their interpretation in a transparent manner. To date, organizations involved with guideline and performance-measure development have only slowly begun to diversify their expert panels to accommodate a more diverse mix of specialties; some have resisted this process. To achieve a broad vision of professionalism, steering committees and advisory committees could include generalist representatives in addition to subspecialists, and the balance between the numbers of generalist and subspecialist professionals could be set so that no single perspective could dominate the policymaking of these groups. Assuring disclosure of conflicts of interest and managing these conflicts effectively could neutralize external influences.

Other steps might reverse the tendency for conflict among professionals. As noted above, encouraging the participation of professionals and patients in clinical registries might enhance professional and public trust in CER results produced from such registries. Engaging subspecialty societies in the construction of shared clinical registries and health-information exchange might counteract the tendency to focus on narrowly defined subspecialty interests.

2.     Provide training for professionals on the role of cognitive biases in diagnostic and treatment decisionmaking. As described in Chapter Seven, cognitive biases may contribute to the failure of CER to affect clinical practice. Cognitive biases may be difficult or impossible to change, however, since they are “hard-wired” to some extent. However, it is possible to compensate for some cognitive biases by altering the decisionmaking context. For example, the framing of the description of expected gains and losses related to an intervention can affect the likelihood that the intervention will be recommended and accepted. If CER evidence informs this framing, implementation of a CER-based clinical practice may be more likely to succeed.

Enabling professionals to recognize the circumstances under which decisions and clinical recommendations are prone to cognitive biases may reduce the influence of factors other than CER evidence. Helping professionals identify these circumstances could increase the alignment of clinical recommendations and decisions with CER evidence. Training in the nature, role, and impact of cognitive biases could be incor-porated into the curricula of professional schools and clinical training programs. Such curricula might also include education on specific methods for modifying the decisionmaking context to compensate for cognitive biases.

Education and Marketing

Promote demand for CER-based clinical services through public education and marketing. Marketing campaigns, including detailing to clinicians and direct-to-consumer advertising, are generally aimed at exploiting cognitive biases, and this may impede the take-up of CER-based clinical services. Recent initiatives by some states and organizations have been undertaken to reduce the influence of detailing of physicians. Regulation of the claims permitted in direct-to-consumer advertising may also play an important role in mitigating bias in decisionmaking. Two options are suggested.

1.     Promote patient demand for CER-based clinical services through shared decisionmaking using formal patient decision aids. Stimulating patient demand can be a powerful adjunct to other approaches for increasing the use of CER-based clinical practices. CER results and CER-based clinical practices are disseminated through a variety of channels, including clinicians’ recommendations, mass-media publications and broadcasts, direct-to-consumer advertising, and the Internet. The challenge for patients (and for clinicians) is to separate recommendations grounded in CER from non–evidence-based claims.

Shared decisionmaking involving the use of formal decision aids could ensure that patients receive the best available evidence about alternative tests and treatments in a usable form, but decision aids have not been incorporated routinely into clinical practice. Many demonstration projects and studies are under way to identify the optimal methods and for timing the delivery of high-quality information to patients. Federal and state legislation is beginning to encourage the use of patient decision aids through quality-measurement and payment-incentive programs. Efforts to assure that CER evidence informs decision aids and that decision aids are used routinely in practice could provide the needed impetus to the uptake of CER-based clinical practices. Decision aids could also counter the messages that promote suboptimal clinical practices.

2.     Support “social marketing” for high-profile CER results to counteract the effects of industry-sponsored detailing and direct-to-consumer advertising. Patient demand for services can undermine the implementation of CER-based clinical practices. Social marketing—the application of marketing techniques to promote behavioral change—has the potential to increase awareness of and demand for evidence-based healthcare services. Social marketing campaigns following the release of new CER findings could stimulate patients to engage their providers in discussions about the meaning of the results for their own care, leading to the use of treatments that may be better aligned with the CER evidence. In particular, social marketing may counteract the unintended consequences of direct-to-consumer marketing and complement academic detailing programs that may be costly in certain practice contexts.

Social media, such as Facebook and Twitter, may be effective tools for enhancing social marketing. In the same way that they have been used increasingly to enhance recruitment of volunteers for clinical trials, social media could play an important role in facilitating translation of CER into practice.

View full report


"rpt_RANDFinal.pdf" (pdf, 2.01Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®