Positive Youth Development in the United States: Research Findings on Evaluations of Positive Youth Development Programs. Program Implementation and Assurance of Implementation Quality

11/13/1998

Issues of program implementation have recently emerged as some of the most important topics in the positive youth development field. Based on the evidence of many of these evaluations, attention to implementation quality, management and measurement has steadily increased. Among multi-year, well-funded studies, separate evaluations of implementation, in addition to outcomes evaluations, are becoming more common. The science of studying implementation has taken investigators in many different directions, some evaluations offering supplemental statistical analyses of outcomes based on perceived level of implementation quality (e.g., Gottfredson, et al. 1993; Battistich, 1996). In a major, multi-year evaluation such as Midwestern Prevention Project (Pentz et al., 1990), operational definitions of implementation have been offered, organizing types of implementation by adherence, exposure, reinvention. The term "fidelity" is associated with implementation quality, with evaluators of multi-year evaluations such as Life Skills in 56 New York Public Schools (Botvin et al., 1995; Botvin et al., 1990) reporting outcomes based on analyses of high versus average fidelity of program implementation.

Our analysis showed that the effective positive youth development programs consistently attended to the quality and consistency of program implementation. Twenty-four (96%) evaluations in some way addressed and/or measured how well and how reliably the program implementers delivered the intervention.

Although not as high as the well-evaluated programs, the percentage of programs in the excluded category that addressed or assured implementation quality was fairly high (28, or 70%, n=40).