Best Intentions are Not Enough: Techniques for Using Research and Data to Develop New Evidence-Informed Prevention Programs. Selecting Strategies Most Likely to Influence Targeted Risk, Protective and Promotive Factors

04/01/2013

Having identified risk, promotive, and/or protective factors as potential targets for intervention, it is then necessary to identify ways to modify those factors and determine if doing so in fact reduces the adverse outcome and increases the positive outcome at issue. Here, also, meta-analysis may be helpful by providing a summary of the available research on the effectiveness of strategies that target different risk and need areas for preventing or reducing that adverse outcome. Figure 1, for instance, identified a number of predictive variables that are often addressed by various forms of counseling, e.g., externalizing behavior, general behavioral problems, family functioning, school participation, and the like. Similarly, Lipsey (2009) conducted a meta-analysis of 540 studies of interventions intended to reduce recidivism among juvenile offenders, examining the effectiveness of various forms of counseling programs along with other intervention approaches. Figure 2 summarizes the findings of the counseling studies. As shown, all of the different counseling approaches showed positive effects on delinquent behavior as indicated by reduced recidivism rates. An especially clear example of targeting an identified risk factor with resulting effects on the ultimate outcome at issue can be seen for general family counseling and family crisis counseling. These interventions target family functioning and related aspects of parent-child interactions, factors that Figure 1 above showed to be modestly predictive of subsequent antisocial behavior. Figure 2 below then shows, in turn, that counseling that addresses those issues does, in fact, result in reductions in delinquent behavior.

This kind of meta-analysis can be a useful starting point for developing or adapting programmatic strategies. A meta-analysis, of course, is not a "how-to-guide" for the specifics of what makes an evidence-informed strategy tick on the ground. For that, one must look carefully at specific experimental studies and variations to discern what strategy might work best for a particular adaptation or innovation. For example, some types of counseling can be harmful; some can help a bit; some can help a lot. Demonstrating the negative, rigorous studies by Dishion and colleagues have shown that aggregating delinquent youth in groups can trigger a cascade of accidental or covert reinforcements from peers for deviant behavior (Dishion & Patterson, 2006; Dishion , Spracklen et al., 1996), which in turn predicts much more serious criminal offenses in the future (Dishion, Ha & Veronneau, 2012; Fosco, Frank & Dishion, 2012).

 

Figure 2. Mean Effects for Counseling Interventions with Juvenile Offenders

 

Source: Lipsey, M. W. (2011). Using Research Synthesis to Develop "Evidence-Informed" Interventions. ASPE Forum: Emphasizing Evidence-Based Programs for Children and Youth. Washington DC.

Another consideration is the balance between the level of effort required for implementation and the payoff in terms of expected outcomes. Some counseling might be easy to do, with modest but consistent effects, like brief motivational interviews (Grenard, et al., 2007; Reinke, 2006). Other evidence-based strategies can require equivalent time to learn and implement, yet have very large impacts in both the short and longer term (Bach, Hayes, & Gallop, 2011). Yet another type of counseling might have larger effects, but be very difficult to learn or implement (Durlak, 2013). This in depth analysis goes beyond consulting various lists of evidence-based programs to consult the underlying studies that make up the evidence base.

To illustrate, a common reason for being re-arrested or experiencing revocation of probation involves drug use by adolescents and young adults. A careful analysis of the components used in counseling programs shows that contingency management protocols (reinforcements) for being drug-free are far superior to psychotherapy alone (Dutra, Stathopoulou et al., 2008). Thus, saying we have a counseling program in place is not sufficient to have an effective program that prevents recidivism. Understanding the active ingredients that comprise the broader categories of effective programs is required (Blase & Fixsen, 2013). These additional considerations weigh on designing an adaptation or innovation.

It is not always feasible to do a meta-analysis, however, if developers lack resources or capacity. Most importantly, there may be too few studies available for this technique to be useful. For example, to develop a program for preventing pregnancy among Latina adolescents, there may be only a few relevant studies. Therefore, additional strategies are needed that are based in research and can aid in program development. Evidence-based kernels provide another approach to identifying effective program practices, components, and active ingredients that are linked to specific behaviors. Evidence-based kernels (Embry & Biglan, 2008) are proven small units of behavioral influence (some of which are based on meta-analyses), that can be used to create new solutions to persistent or novel problems of human wellbeing or to construct adaptations of existing proven programs.

The concept of kernels arose in response to many of the challenges inherent in implementing evidence-based programs. Specifically, efficacy trials of evidence-based programs may demonstrate effectiveness; however, when taken to scale in real-world settings, it may be difficult to effectively replicate or sustain programs. Alternatively, challenges may arise that are outside of the scope of the specific intervention (Embry & Biglan, 2008). For instance, unanticipated events outside the implementing agency's purview may affect the ability of staff to carry out the intervention. Additionally, though many strategies have been used to identify evidence-based programs, evidence supporting effective program diffusion and dissemination is often modest. These challenges indicate that there is value in understanding the specific components of programs that operate as key ingredients and that are essential to program success and can help supplement or strengthen programs.

Kernels are supported by experimental studies that demonstrate their effectiveness and are commonly used strategies in prevention research. Kernels include strategies such as providing praise in the classroom, peer-assisted learning, using self-regulation techniques such as deep breathing or self-monitoring, or sending a note home from school to a child's parents. The essential characteristics of an evidence-based kernel can be summarized as follows.

Kernels are:

  • The smallest unit of scientifically proven behavioral influence.
  • Indivisible, that is, removing any part makes it inactive.
  • Produces quick, easily measured change that can grow much bigger over time.
  • Can be used either alone or in combination to create new programs, strategies or policies.
  • Are the active ingredients of most evidence-based programs.
  • Can be spread by word-of-mouth, by modeling, by non-professionals.
  • Can address historic disparities without stigma, in part because they are also found in cultural wisdom.

Embry and Biglan (2008) have identified in the research literature 52 discrete kernels, most of which can be used across the lifespan and in many different program settings. While this is not an exhaustive list of such tested behavioral interventions, those identified do provide an array of program elements known to work. Embry and Biglan characterize these kernels[2] in four types:

  1. Antecedent Kernels are elements that happen before the behavior you are trying to influence. One example is a warm, pleasant, and personal greeting that welcomes children and youth. Also, instead of raising his or her voice, a teacher or leader can use a less negative approach, such as raising a hand or using a musical cue to attract attention without causing stress.
  2. Reinforcement Kernels happen after the behavior you are trying to influence, like a written thank you note for someone who did a good deed, or a regular public posting to provide recognition for accomplishments and challenges for further achievement.
  3. Relational Frame Kernels are predictable words or phrases that increase or decrease behaviors, such as motivational interviews, or "soft team" competition. In the latter, teams are created to compete toward a visible and positive public purpose. There are no winners or losers, but social commendation is provided for accomplishments that further a greater good.
  4. Physiological Kernels directly affect the probability of behaviors by affecting brain functions or processes. For example, physical activity can increase cognitive performance, vitamin D can reduce multiple mental illnesses, and Omega-3 can reduce aggressive behaviors.

When adapting an existing strategy or developing a new strategy for prevention, intervention, treatment or recovery, it is wise to examine and combine kernels from all four domains. Incorporating such diverse combinations of kernels increases the robustness and reliability of the strategy (Embry and Biglan, 2008). The new or adapted strategy would be evaluated for whether it can be implemented successfully and, ultimately, for its ability to achieve the intended impacts.

Kernels are robust program elements, but they are far from the only resource for selecting program strategies. Longitudinal studies represent another valuable resource for identifying strategies that can change risk, protective, and promotive factors. For example, longitudinal studies that follow individuals over many years, like the Dunedin Child Development study (Silva, 1990) or various studies of twins, can provide useful information on how to change those risk, protective, and promotive factors that predict an outcome. For example, reading by third grade might promote school engagement and success (Annie E. Casey Foundation, 2010), which are regularly found to predict delinquency, teen parenthood, and school drop out. Thus, early reading might be a focus for intervention. Similarly, numerous studies have found that low educational aspirations and poor school performance are strongly associated with risky sexual behavior and teen childbearing, over and above the influence of other risk factors (Kirby, 2001). This research suggests that helping students improve their academic performance and goals represents a good target strategy for preventing teen parenthood. Such research studies often seek to structure a developmental timeline to identify what input predicts what outcome sequentially. This kind of causal thinking posits that because particular variables precede, they likely affect the outcome.

In addition to consulting the scientific knowledge base, it is important to consult with experienced practitioners, youth, community, and tribal stakeholders as well. Almost all of the very best prevention strategies in the 2009 Institute of Medicine Report on Prevention of Mental, Emotional, and Behavioral Disorders (O'Connell, Boat & Warner, 2009) have deep roots in direct observation of people in their natural settings, such as home, school, and community.

View full report

Preview
Download

"rb_bestintention.pdf" (pdf, 488.63Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®