Core Components Approaches to Building Evidence of Program Effectiveness

08/25/2020

Recent focus on evidence-based decision-making has led to multiple federal efforts to determine the evidence behind various programs, particularly those seeking federal funding. This has led to the development of multiple evidence-based registries, most of which are based on “model programs” – a particular branded program with an implementation manual that has been rigorously evaluated and found to have at least one positive impact on an outcome of interest. While this work represents significant advancement toward evidence-based decision-making, it has not always led to widespread uptake of these model programs in the field. Branded programs are often expensive to implement, difficult to adapt to populations different from those with which they were evaluated, or do not meet the specific needs of a community.

This reality has led numerous researchers, evaluators and policymakers to begin considering a complementary approach to building evidence of effectiveness: a core components approach. Core components are the parts, features, attributes, or characteristics of a program that a range of research techniques show influence its success when implemented effectively. These core components can serve as the unit of analysis that researchers use to determine “what works,” and they become the areas practitioners and policymakers seek to replicate within and across a range of related programs and systems in order to improve outcomes. Research techniques such as meta-analysis can shed light on which components make programs successful across a range of programs and contexts, and help researchers identify with greater precision what works, in which contexts, and for which populations.

Products on this page represent ASPE’s current work on core components approaches to building evidence of program effectiveness.

  • Improving Programs for Children and Youth that Address Behavioral Problems: Recommendations for Aligning Programs with Evidence on Core Components – Most young people in the U.S. are healthy, progress through school, and successfully transition to adulthood, but some children and youth have behavioral problems that can present challenges for overall well-being. Community and family factors may hinder the development of cognitive and interpersonal skills, which in turn may lead to behavioral problems in school and with peers. To address these issues, many communities and schools run programs to reduce aggression, bullying, and other disruptive behaviors, referred to in this guide as externalizing behavior. Research offers insight into which programs are more effective than others at reducing externalizing behaviors – and which core components of these programs are associated with program effectiveness. This guide takes that research and translates it into a set of recommendations intended to help practitioners make evidence-based decisions about ways to improve programs aimed at preventing or reducing externalizing behavior.

  • Improving Social Competence Programs for Children and Youth: Recommendations for Aligning Programs with Evidence on Core Components – Most young people in the U.S. are healthy, progress through school, and have successful relationships with family, peers, and teachers. However, some youth struggle to learn the skills needed for successful social interactions – they need support developing their social competence. Social competence includes a range of skills related to developing and maintaining relationships with other people, and it is related to positive long-term outcomes in academic performance, social and emotional well-being, and behavior. Because of its critical role in positive youth development, prevention programs that target social competence are available in many communities and schools. Research offers insight into which types of youth programs are more effective than others at improving social competence – and which core components of these programs are associated with program effectiveness. This guide takes that research and translates it into a set of recommendations intended to help practitioners make evidence-based decisions about ways to improve programs aimed at improving social competence.

  • Developing Evidence-based Practice Guidelines for Youth Programs: Technical Report on the Core Components of Interventions that Address Social Competence – This report describes and illustrates an approach to using evidence to improve the effectiveness of youth programs, both in terms of how they are designed and how they are implemented. The approach capitalizes on the fact that across the many program environments that offer youth programs (e.g., community, mental health, public health, child welfare settings, schools), there is a great deal of well-controlled research available. There is also considerable variability in effectiveness across programs that can be reliably predicted from information reported in the research. Our goal is to better understand the sources of that variability so that we can uncover the characteristics of effective programs and share guidelines about how to make them more effective with those who design, support, and implement such programs. Findings reported here were used to inform evidence-based guidelines for improving practice, Improving Social Competence Programs for Children and Youth: Recommendations for Aligning Programs with Evidence on Core Components

  • Developing Evidence-Based Practice Guidelines for Youth Programs: Technical Report on Effectiveness Factors for Interventions that Address Externalizing Behavior Problems – This report describes a core components approach to using evidence to improve the effectiveness of youth programs. Across the many program environments that offer youth programs (e.g., community, mental health, public health, child welfare settings, schools), there is a great deal of well-controlled research available. Meta-analysis was used to uncover the characteristics of programs effective in reducing externalizing behavior problems, which will be translated into practice guidelines for those who design, support, and implement such programs.

  • Advancing the Use of Core Components of Effective Programs: Suggestions for Researchers Publishing Evaluation Results – Core components are the parts, features, attributes, or characteristics of a program that a range of research techniques show influence its success when implemented effectively. These core components can serve as the unit of analysis that researchers use to determine “what works,” and they become the areas practitioners and policymakers seek to replicate within and across a range of related programs and systems in order to improve outcomes. Research techniques such as meta-analysis can shed light on which components make programs successful across a range of programs and contexts, and help researchers identify with greater precision what works, in which contexts, and for which populations. This brief explains why it is important to collect and report on a wide range of program characteristics and the kinds of characteristics that should routinely be collected and reported in order to facilitate future meta-analyses that can help the field identify core components of effective programs.

  • Core Intervention Components: Identifying and Operationalizing What Makes Programs Work – This brief is part of a series that explores key implementation considerations important to consider when replicating evidence-based programs for children and youth. It focuses on the importance of identifying, operationalizing, and implementing the "core components" of evidence-based and evidence-informed interventions that likely are critical to producing positive outcomes. The brief offers a definition of "core components," discusses challenges and processes related to identifying and validating them, highlights rationales for the importance of operationalizing core components, and explores implications for selecting, funding, implementing, scaling up, and evaluating programs.