Performance Improvement 2013-2014. How Can Successful Replication of Evidence-Based Program Models and the Development of Evidence-Informed Models Be Ensured?

01/01/2014

Significant investments have been made in the replication of evidence-based program models (EBPs) and testing evidence-informed and innovative approaches in the areas of home visitation, teen pregnancy prevention, and the reduction of long-term foster care. This project examined the challenges confronting stakeholders involved in EBP replication and scale-up, the issues around implementing evidence-informed strategies, and lessons from research and practice to address these challenges.

The first study found that despite the increasing availability of information that identifies evidence-based programs, a large gap remains between what the research has shown with regard to program outcomes and the key mechanisms that can facilitate or inhibit program implementation. Themes emerged during a forum held in April 2011 during which experts discussed challenges encountered when selecting and replicating evidence-based programs (EBPs) and approaches were identified for developing evidence-informed programs when EBPs are not available or applicable for a given population. 

The second study detailed the fundamentals of quality program implementation that have been identified through research and practice. It highlighted the importance of a high quality implementation and identified factors that affect implementation, which range from societal, community, program, practitioners, and organizational influences, as well as the implementation process itself. Researchers explained how implementation should focus on core components, allowing adaptation of other aspects to suit the population and setting.

The third study found that strategies for a well-operationalized evidence-based program include a clear description of: the context of the program; the core components; the active ingredients to operationally define the core components so they can be taught and learned and can be implemented in typical settings; and a practical strategy for assessing the behaviors and practices that reflect the program's values and principles, as well as the program's active ingredients and activities. When outcomes are not achieved, an understanding of core components and whether they were implemented correctly is essential to understanding whether a program is ineffective, or alternatively, whether it was not implemented well. Researchers suggested that program funders consider including requirements to specify the core components of interventions as deliverables at the end of a demonstration or pilot phase to facilitate replication and scalability.

The fourth study identified and described key strategies for using research evidence and data to inform the development and testing of new evidence-informed interventions. These included the use of meta-analysis and “kernels” to identify research-based components and practices to incorporate into new programs, and logic models to organize information on how activities are expected to lead to desired outcomes and to guide program development, testing, and revision. Meta-analysis is a technique to synthesize the results of many studies on a topic, while kernels are program elements or practices that have been shown in research to have behavioral impacts and that can be re-combined in the development of new interventions.

Report Title 1: The Importance of Implementation for Research, Practice, and Policy
http://aspe.hhs.gov/hsp/13/KeyIssuesforChildrenYouth/KeyImplementation/rb_KeyImplement.cfm
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning and Evaluation
Federal Contact: Diana Tyson, 202-401-6670
Performer: Child Trends
Record ID: 9532 (April 30, 2013)

Report Title 2: The Importance of Quality Implementation for Research, Practice and Policy
http://aspe.hhs.gov/hsp/13/KeyIssuesforChildrenYouth/ImportanceofQuality/rb_QualityImp.cfm
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning and Evaluation
Federal Contact: Laura Radel, 202-690-5938
Performer: Child Trends
Record ID: 9532.1 (February 22, 2013)

Report Title 3: Core Intervention Components: Identifying and Operationalizing What Makes Programs Work
http://aspe.hhs.gov/hsp/13/KeyIssuesforChildrenYouth/CoreIntervention/rb_CoreIntervention.cfm
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning and Evaluation
Federal Contact: Laura Radel, 202-690-5938
Performer: Child Trends
Record ID: 9532.2 (February 22, 2013)

Report Title 4: Best Intentions Are Not Enough: Techniques for Using Research and Data To Develop New Evidence Informed Prevention Programs?

http://aspe.hhs.gov/hsp/13/KeyIssuesforChildrenYouth/BestIntentions/rb_bestintentions.cfm
Agency Sponsor: OASPE, Office of the Assistant Secretary for Planning and Evaluation
Federal Contact: Laura Radel, 202-690-5938
Performer: Child Trends
Record ID: 9532.3 (April 19, 2013)

View full report

Preview
Download

"PerformanceImprovement2014.pdf" (pdf, 671.65Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®