For the full sample, the statistical power of the study to detect impacts is high. Based on the observed explanatory power of the regression models, the study sample supports detection of true overall program impacts of roughly 0.08 standard deviations. (This is based on standard assumptions of 80 percent statistical power and 90 percent statistical confidence, two-tailed.) For a proportional outcome with a mean of 50 percent, this reflects an estimated impact of roughly 4 percentage points. Program impacts that are smaller in size may also be detected from the study sample, but the likelihood of doing so is below the 80 percent probability (power level) that is commonly preferred.
For the individual program sites, statistical power is naturally lower. This is particularly true in the two sites that experienced program nonparticipation, ReCapturing the Vision and FUPTP. For example, in the absence of nonparticipation, the size and allocation of the study sample would support detection of true site-specific impacts on the order of 0.16 standard deviations or larger for ReCapturing the Vision and 0.18 standard deviations for FUPTP. However, in light of the existing nonparticipation, the impacts on participants would need to be considerably larger about 0.25 standard deviations for ReCapturing the Vision and 0.32 standard deviations for FUPTP given equivalent levels of statistical power and confidence. This means the available samples in these two sites provide a high likelihood of detecting (that is, stating as statistically significant) true participant impacts only if they are fairly large; for example, for a proportional outcome with a mean of 50 percent, the minimum detectable impacts for participants are about 13 and 16 percentage points in the two respective sites. For the remaining two sites, My Choice, My Future! and Teens in Control, detectable impacts (at 80 percent power) are better roughly 0.17 and 0.13 standard deviations, respectively.