To date, the evaluation team has successfully implemented a scientifically rigorous impact study design in the five targeted program sites. Study enrollment is completed, and longitudinal tracking of youth is ongoing. Because of the importance of having large sample sizes and sufficient followup of program and control youth, no impact estimates are available for inclusion in this report. This report does, however, provide detail on the key features of the study design, the characteristics of the youth served by these programs, and plans for the impact analysis and reporting.
As discussed in greater detail later, critical features of the impact analysis study design are:
- The Impact Study Uses Experimental Designs in All Sites. In each site, program effectiveness will be measured by comparing outcomes of eligible youth who were randomly assigned to the program or to a control group.
- Sample Sizes Are Between 400 and 700 Youth per Site. Large sample sizes protect against the possibility of failing to detect true program impacts simply because the study lacks statistical power. To achieve adequate sample sizes, youth were enrolled in the study from fall 1999 through fall 2001.
- Followup of the Sample Will Continue for Up to Three Years After Study Enrollment. The first wave of data collection occurred at “baseline” as participants were first enrolled in the program or the control group (fall 1999 through fall 2001), the second wave occurs 6 to 12 months later (fall 2000 through spring 2002), and the third wave will occur 18 to 36 months after initial sample enrollment (spring 2002 through fall 2003).
- Data Collection Procedures Respect the Rights of Students and Parents and Protect the Privacy of Respondents. The evaluation uses an active parental consent process whereby the parent or guardian must provide written consent before a student is eligible to participate. The evaluation also uses confidentiality protections designed to ensure that no one from local schools — including teachers, administrators, and program staff — has access to the student surveys.
- Student Surveys and Data Collection Procedures Are Designed to Maximize the Accuracy and Reliability of Student Responses. Research indicates that youth may underreport socially undesirable behaviors, such as sexual activity and other risk-taking activity. Furthermore, abstinence means different things to different people. The surveys for the study measure specific, clearly defined behaviors. Students provide their answers on self-administered forms under the supervision of trained, independent interviewers.
The first report on short-term program impacts will be released early in 2003, once the second wave of survey data is available for the full study sample. The final impact analysis report will be released in summer 2005.