In addition to these program criteria, the program's evaluation had to meet the following criteria in order to be included in this review.
Adequate Study Design and Outcome Measures
To be included, an evaluation was required to be sufficiently well designed to rule out threats to the internal validity and to allow conclusions to be drawn regarding program effectiveness. Therefore, an experimental or quasi-experimental design with an equivalent, structured comparison group and behavioral outcomes was required.
Adequate Description of the Research Methodologies
Several additional criteria were used to determine whether the study used adequate research methodology. Any attrition that occurred from pre-test measurement needed to be accounted for and analyzed through the statistical modeling of attrition effects, including effects of differential attrition from conditions. Statistical methods had to be appropriate for analyzing the outcome data. Other methodological aspects of the study were evaluated to determine that studies used the appropriate unit of analysis, had adequate power to detect program effects, and had reasonably equivalent experimental and comparison groups at baseline.
Description of the Population Served
The evaluation had to specify the demographic and other relevant characteristics of the groups with whom the intervention was shown to be effective. Descriptions were expected to include information on gender, age, cultural group, socioeconomic status, and family and community factors of the groups served. A description of the recruitment and consent processes was also expected.
Description of the Intervention
It was expected, though not required for inclusion, that the evaluation describe how the intervention components were expected to affect behavior. A criterion for inclusion was a description of how the intervention promoted positive youth development constructs. Adequate descriptions of the intervention included program goals and content, intervention delivery protocols, personnel delivering the intervention, site(s) of intervention, and duration and extent of program.
Description of Implementation
While not required for inclusion, the quality of program implementation was assessed if presented in the evaluation. Descriptions of how consistently the intervention was actually executed at all stages of delivery can help to distinguish implementation problems from problems in the design or conceptualization of the program. The ability to assess program implementation was enhanced in studies that included process measures collected by observers, or detailed documentation of contacts with participants.
Effects Demonstrated on Behavioral Outcomes
For inclusion in the review, evidence of significant behavioral outcomes was required. These included evidence that positive outcomes were enhanced or negative outcomes were reduced, or that both occurred. Programs were included if they demonstrated behavioral outcomes at any point, even if these results decayed over time. Programs were also included if they demonstrated effects on part of the population studied. Outcomes associated with changes in knowledge and attitudes were also examined, if behavioral outcomes were presented, because these provided evidence as to whether youth development constructs were strengthened.