Best Intentions are Not Enough: Techniques for Using Research and Data to Develop New Evidence-Informed Prevention Programs. Executive Summary

04/01/2013

Despite increased attention to the value of implementing evidence-based programs, there are many issues for which no programs have been proven effective. Evidence-informed prevention strategies can be used to develop effective programs to fill these gaps. This brief describes ways research and data may be used to inform the steps involved in developing new or modified prevention programs. At each step, the goal is to draw on data, research, and evaluation findings, as well as theory and the experience of the practice community, to inform the development of strategies that improve outcomes.

Defining the Problem. At this initial stage, data can be used to assess trends and identify the outcome(s) the program will address, as well as the communities or the social, economic or demographic groups where the issue is most prevalent.

Identifying Relevant Risk, Protective and Promotive Factors. Once the target problem has been established, the varied factors that affect this outcome can be identified. These include behaviors, knowledge, values, goals, or attitudes that affect the likelihood that the target outcome will occur. Meta-analysis represents a useful way to identify factors that affect the targeted outcome(s). Developmental theory, the knowledge of experienced practitioners and community members, and data from longitudinal studies can also help identify relevant factors.

Selecting Strategies Most Likely to Influence Targeted Risk, Protective and Promotive Factors. Several types of evidence can also inform the choice of strategies to influence risk, protective, and promotive factors. Meta-analysis is highlighted at this stage as well, but identification of research-based kernels can also identify ways to modify these factors. These kernels (Embry & Biglan, 2008) are proven small units of behavioral influence that can be used to create new solutions to persistent or novel problems of human wellbeing or to construct adaptations of existing proven programs. Additional ways to identify factors for interventions include analysis of data from longitudinal studies and consultation with practitioners, clients, and other stakeholders. Strategies that come to the fore based on varied types of evidence warrant particular attention. It is necessary, though, to identify factors that are malleable, that have large enough effects to bring about the desired change, and that are cost-effective.

Assembling Your Intervention Using a Logic Model. While varied approaches to organizing information are feasible, logic models are an important tool to increase clarity about how the program developer expects targeted outcomes to be achieved. The logic model is developed by first specifying the targeted outcome(s); then the risk, protective and promotive factors that affect the outcome are identified; and then the strategies, approaches and activities that affect those factors can be depicted. Developing a logic model can force program designers and stakeholders to be explicit about their assumptions and confirm that the elements of the model are likely to produce the desired change in the target outcome. It illustrates how the intervention is hypothesized to produce the intended results. However, the elements of the model still need to be tested.

Testing the Elements of Your Evidence-Informed Program. Once a programmatic approach is developed, data from performance management systems, observations, implementation evaluation, and behavior analysis can be examined to assure that the intervention(s) can be implemented as designed and achieve their intended results. Sufficient time needs to be allocated to this step, because iterative efforts will be required to examine the evidence to see whether the strategies lead to the desired outcomes.

It is possible to improve the likelihood of a program's success by building more consistently on several types of existing knowledge bases and combining effective components in thoughtful ways to address new problems or new populations. Triangulating across information from research and evaluation, including meta-analysis and research-based kernels, as well as developmental theory, longitudinal and other research, and the wisdom of experienced practitioners, can inform the development of programs that are more effective at achieving the outcomes desired for children, youth, and families. This is not a quick or easy endeavor; but investing the time and effort necessary to develop evidence-informed interventions should result in more effective programs and thus better outcomes for children and youth.

This Research Brief is one of four presenting material developed under a research project titled Emphasizing Evidence-Based Programs for Children and Youth: An Examination of Policy Issues and Practice Dilemmas Across Federal Initiatives. Others in the series include:

  • Key Implementation Considerations for Executing Evidence-Based Programs
  • Core Intervention Components: Identifying and Operationalizing What Makes Programs Work
  • The Importance of Implementation for Research, Practice and Policy

View full report

Preview
Download

"rb_bestintention.pdf" (pdf, 488.63Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®