Dissemination and Adoption of Comparative Effectiveness Research Findings When Findings Challenge Current Practices. Framework for Translation of CER into Practice


On the basis of a review of existing frameworks that have evolved from the science of diffusion, dissemination, and implementation of innovations; discussions with our expert panel; and discussions with other experts, we developed a simplified framework to guide both our collection of case-study data and our subsequent analysis of the barriers to and enablers of CER translation into practice. As we conducted the case studies, we refined the framework to better reflect what we observed and to enable us to synthesize and organize the key themes that emerged (summarized in Chapter Seven).

Our conceptual framework posits that the CER translation process follows five key phases, shown in Figure 1.1. While Figure 1.1 suggests a linear temporal process, the phases are concurrent to some degree, and there appear to be multiple interactions between stakeholders at different phases. The phases are described in Table 1.1.

The first phase is generation, which includes the design and execution of the CER study. It involves primarily scientists, funders, and the public but is shaped by the needs of many other key stakeholders. In particular, patients or their advocates and clinical professionals may play an important role in helping to ensure that outcomes of CER studies are relevant to them. CER design elements decided during the generation process may strongly influence the relevance of the findings and thus the degree to which they are accepted and lead to changes in clinical practice.

The completion of a study and the publication of its findings initiate the interpretation phase, in which individual stakeholders assign a specific meaning to the results. Interpretation is a complex process shaped by the strength of the CER evidence, applicability of the evidence to the potential adopter’s practice setting, each adopter’s personal experience, and prior expectations of the benefits and harms of each treatment. For example, patients may assume that more costly treatments are more effective. Researchers who conduct systematic reviews also interpret the evidence, albeit with the use of protocols. Interpretation is influenced by messages from other key stakeholders as well, including professional societies, industry, the media, and opinion leaders. Stakeholders whose interpretations of CER results play a large role include

·        CER researchers, whose initial presentation of the evidence may play a critical role in shaping interpretations by others

·        Professional societies, which may produce consensus statements about the CER

Figure 1.1

Conceptual Framework for Translation of CER into Clinical Practice

This figure is an illustration of our conceptual framework. It depicts the five phases of the process by which CER evidence is translated into clinical practice. Each phase is depicted by a box with a label that identifies the phase. The first box is labeled “CER generation,” which includes both the design and conduct of the CER. This step produces a CER result depicted as a circle (because it is not a phase but a product of a phase). An arrow connects the CER generation box to the circle. A second box represents the second phase of the process and is labeled “CER interpretation.” An arrow extends from the CER result circle to the CER Interpretation box. The next three boxes represent phases three through five and are labeled “CER formalization,” “CER dissemination,” and “CER implementation.” Each is arrayed linearly, with an arrow extending from one pointing to the next. A large box surrounds the CER result circle and four of the five boxes (phases of CER)—excluding the first phase (CER generation). This large box signifies the fact that these phases occur within a unique market context, regulatory context, and professional context.</p>
<p>One arrow extends from the CER formalization box and points “backward” to the prior box (CER interpretation). This “loopback” arrow means that the CER formalization phase can also impact interpretation of the evidence by other stakeholders. Similarly, arrows extend from the CER dissemination box and loop backward to both the CER formalization and CER interpretation boxes, indicating that dissemination can impact both of these phases.</p>
<p>Finally, there is an arrow that extends from the large “context” box that loops back to the CER generation box. A label superimposed on this arrow indicates that scientists, the public, and policymakers mediate the closing of the loop. It indicates that these stakeholders have a role in ensuring that the translation process informs the generation of new CER topics.<br />

Table 1.1

Phases of the Translation of CER into Clinical Practice




Generation includes the design and conduct of the CER study; it involves primarily funders and CER researchers, but research priorities are influenced by the needs of multiple stakeholders, including scientists, the public, and policymakers.


Stakeholders ascribe meaning to CER results based on a number of factors, including the strength of evidence, applicability of the evidence to the potential adopter’s practice setting, personal experience, and messages received by other stakeholders (e.g., professional societies, industry, media, and opinion leaders).


Formalization is the process by which the interpretations of CER results are converted into guidance instruments such as clinical-practice guidelines, performance measures, and quality improvement tools. Multiple stakeholders may play roles in formalization through participation in guidelines committees, regulatory committees, and performance-measure development and endorsement processes.


Dissemination is the process by which CER information and/or associated tools designed to influence practice is actively transmitted to stakeholders. It typically promotes (or discourages) implementation of a new practice but may also have the goal of promoting a particular interpretation of the CER results.


Implementation is the adoption of new clinical practices based on CER results. Implementation decisions may depend on a wide range of factors, including the dissemination of messages and the successful embedding of CER-related clinical guidance into tools that facilitate practice change, as well as the local market, regulatory, and professional context that may promote or impede changes. The implementation phase takes place primarily in local practice contexts.


·        Industry, whose marketing divisions play a key role in developing and disseminating messages

·        Funders, who may support and shape the communications strategies for CER results

·        Advocacy organizations, which may interpret the evidence on behalf of their constituents

·        Systematic reviewers, who interpret CER results in the context of the larger body of evidence and draw conclusions about the meaning of the results.

To produce a change in practice, CER results and their interpretations may pass through a formalization phase, in which the clinical practice that will be altered based on the CER evidence is identified and different interpretations are reconciled to produce some form of consensus practice recommendation. In our model, formalization occurs through the activities of national committees that generate or modify clinical-practice guidelines, create performance measures, determine rules for clinical decision support applications, or specify quality improvement strategies. Another type of formalization is the development of “knowledge-summarizing applications” that physicians and patients often use as online reference tools to identify the current recommendations regarding a particular clinical situation. Payers may also formalize CER results through the definition of coverage policy. Because these tools may be derived directly from systematic reviews and clinical-practice guidelines, there may be a lag between the publication of CER evidence and the tools that result from formalization.


Key stakeholders involved in formalization include


·        Professional societies, mainly through the development of guidelines, but also through the development of performance measures and quality improvement tools

·        Performance-measure developers, who create performance measures based largely on guidelines and other evidence

·        Pharmaceutical and device-industry companies that produce detailing materials (e.g., pocket cards) and may support the development of other quality improvement tools

·        Clinical decision support (CDS) developers, who may develop alerts, reminders, or clinical-pathway support tools to promote using care in line with the CER evidence

·        Payers/purchasers, who develop coverage and reimbursement policies based on the CER evidence

·        Policymakers, through the development and/or implementation of performance measures

·        Decision-aid developers, who integrate CER evidence into patient decisionmaking tools.

To reach stakeholders, the formalization tools must be part of an active dissemination phase. Interpretation, formalization, and dissemination of messages and tools are unlikely to follow a linear process. Rather, as indicated in Figure 1.1, new evidence that evolves and is disseminated through the activities and messages of stakeholders reshapes the context in which interpretation and formalization occur. These processes are likely to be concurrent and may reinforce one another. While a range of stakeholders may play a role in dissemination, key stakeholders include

·        Professional societies, which have multiple formal channels for reaching their members, including professional meetings, websites, and journals, through which they disseminate guidelines, consensus statements, and the CER research itself

·        Performance-measure developers and CDS developers, who have channels for alerting end users to new tools based on CER evidence

·        Payers and purchasers, who may implement new coverage or reimbursement policies or promote new delivery-system interventions consistent with CER evidence

·        Advocacy organizations and drug or device manufacturers, who often produce messages in response to new CER evidence to influence other stakeholders

·        Medical publishers and mass media, which can reach narrow and large audiences, respectively, to disseminate new CER evidence

·        Specialty boards, which may disseminate new findings through certification requirements that require knowledge of relevant CER and demonstration of competency for quality improvement.

Finally, implementation occurs when and if stakeholders agree with the formalization of the CER findings sufficiently to adopt the new practice:

·        Clinicians’ recommendations of CER-informed practices are likely to be highly determined by reimbursement, availability of decision support, and sufficient capacity to deliver the intervention.

·        Patients may vary in their treatment preferences and may face out-of-pocket costs associated with the intervention.

·        Delivery organizations may have a wide array of decision supports and referral processes in place that facilitate adoption of CER-based clinical practices.

·        Payers and purchasers influence implementation through reimbursement levels and financial and nonfinancial incentives.

All of the phases are highly dependent on a range of contextual factors, including the financial incentives defined by opportunities in the marketplace, practice expectations shaped by the professional environment, and the regulatory environment that defines what is possible within a legal or regulatory framework. Different factors may play a role in facilitating or impeding the adoption of new clinical practices during one or multiple phases, and individual factors may interact or offset one another.

Our conceptual framework contains a feedback loop that reflects the tendency of CER studies, through the process of translating research into practice, to generate related questions that were not addressed by the original study and whose answers have value for one or more stakeholders. While many stakeholders may advocate for new CER studies, in general, scientists, policymakers, and the general public have the largest role in the determination of new topics for CER.


View full report


"rpt_RANDFinal.pdf" (pdf, 2.01Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®