Evidence for the importance of high quality implementation has been obtained in multiple areas including education, mental health, health care, technology, industry, and management (Durlak & Dupre, 2008; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). Moreover, implementation is important regardless of characteristics of the target population, the type of program, and specific program goals.
Research clearly indicates that the quality of program implementation is one critical factor associated with youth outcomes. For example, one review of school-based prevention programs found that implementation quality was the most important program feature associated with reducing aggressive behavior (Wilson, Lipsey, & Derzon, 2003). In many cases, programs have failed to achieve their intended outcomes for youth when implementation was poor whereas, in other cases, program impact was much higher when there were reports of more effective implementation (Durlak & Dupre, 2008). In other words, participants may receive more benefits as a result of better program implementation, or they may receive no significant benefit if program implementation is poor.
Additional research findings indicate the importance of high quality implementation. In reviews of bullying prevention programs (Smith, Schneider, Smith, & Ananidou, 2004) and youth mentoring programs (DuBois, Holloway, Valentine, & Cooper, 2002), authors have compared outcomes for youth who had participated in programs that varied in the quality of their implementation. Compared to participants in programs that were poorly implemented, youth who had been in programs that had been implemented with higher quality demonstrated two or three times as much benefit on outcomes such as increased social competence and lower levels of bullying.
Still another example illustrates the importance of quality implementation in affecting critical youth outcomes. In a large-scale review of school-based programs involving over 200 studies and over a quarter of a million youth, the benefits demonstrated by students receiving programs associated with higher quality implementation were compared to those participating in programs that were implemented with poorer quality (Durlak, Weissberg, Dymnicki, Taylor & Schellinger, 2010). The former students showed gains in academic performance that were twice as high as the latter group; furthermore, the students in the better implemented programs also showed a reduction in emotional distress (e.g., depression and anxiety) that was more than double the reduction shown by the latter group and a reduction in levels of conduct problems that was nearly double that of the latter group. In other words, effective implementation can lead to larger gains for youth in several important domains of adjustment. With poor implementation, you may get no or just a small amount of change; with effective/high quality implementation, you may get changes of larger magnitude. The above data indicate it is clearly worthwhile to strive for high quality implementation.
Table 1. Twenty-three Factors that Affect Implementation
Community-wide or societal factors
- Scientific theory and research
- Political Pressures and Influences
- Availability of funding
- Local, State or Federal Policies
- Perceived need for the program
- Perceived benefits of the program
- Skill proficiency
Characteristics of the program
- Compatibility or fit with the local setting
Factors related to the organization hosting the program
- Positive work climate
- Openness to change and innovation
- Integration of new programming
- Shared vision and consensus about the program
- Shared decision-making
- Coordination with other agencies
- Openness and clarity of communication among staff and supervisors
- Formulation of tasks (workgroups, teams, etc.)
- Effective leadership
- Program champion (internal advocate)
- Managerial/supervisory/administrative support
Factors specific to the implementation process
- Successful training
- On-going technical assistance
The importance of implementation quality is widely recognized in the medical field, and drug treatment for medical conditions offers a useful analogy: The correct drug must be given and in sufficient dosage to obtain the desired effect. Moreover, there is always a need to monitor drug use because many patients do not follow the prescribed drug regimen. When drug monitoring occurs, changes can be quickly made so the effect of the drug can be accurately assessed. Otherwise, the physician cannot determine if the use of a particular drug is having the intended effect.
The same goes for any evidence-based program in the area of human services. It is important to ensure an evidence-based program is implemented with high quality in order to achieve the intended effects. This means we must periodically monitor program implementation so we can make adjustments as needed to help ensure high-quality implementation. For example, an evidence-based program may be unsuccessful in one setting due to poor implementation, but the same program may be successful in another setting when it is implemented with quality.
In sum, implementation quality is important throughout the entire range and nature of child and youth services, whether the goal is to treat children with adjustment problems, prevent later problems, promote young people's personal and social development, increase students' academic performance, promote infant health, or prevent teenage pregnancy.
Of course, success is never guaranteed; if it were, then we would always know what results would occur in every situation. The point is that quality implementation is necessary to increase the chances of being successful. In other words, "when it comes to implementation, what is worth doing, is worth doing well."