The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report . Site Selection

04/01/2002

The first step of the evaluation entailed selecting abstinence programs for study.  The evaluation team first called and met with numerous state officials and experts across the country to identify promising programs for inclusion in the evaluation.  Grant applications and program documents then provided additional detail on program goals, target population, activities, size, and curricula.  The evaluation team visited and observed 28 abstinence education programs across the nation.  After extensive communication with abstinence experts and DHHS staff, 11 programs were invited and agreed to participate in the evaluation (Table 1).  Although not a representative set of Section 510 abstinence education programs, these 11 programs are judged to offer a rich range of program strategies and implementation settings for study.

Five of the programs included in the evaluation are referred to as “targeted” programs, as they target their services to specific, identifiable groups of youth.  In addition to providing valuable information about program implementation, each of these programs offers the potential to provide rigorous evidence on the effectiveness of abstinence education program strategies in reducing teen sexual activity and other risk behavior.  The other six programs represent a range of multifaceted, community-wide initiatives that aim to alter youth behavior through stimulating systemic change.  These programs are using their abstinence education funding in a variety of ways to increase public awareness of the problems of teen sexual activity, to change community norms and attitudes, to engage parents and encourage stronger parent-child communications, and to engage youth in abstinence education and support services.  Including these community-wide initiatives in the evaluation adds breadth to an understanding of strategies for changing youth behavior.

For several reasons, though, rigorous impact studies of these community-wide abstinence program initiatives are not possible.  First, these programs often use their abstinence funding to form or become part of a larger network of services for teens, making it impossible to separate the effects of the abstinence program from those of other programs or providers.  Second, the target population often is not easily identifiable, since, by intent, these programs aim to change the norms and behavior of an entire community.  Finally, systemic change is a long-range goal, and the time frame for the impact analysis and federally funded evaluation is too short to allow such change to be measured.

In selecting programs for the evaluation, the goal was to maximize the overall knowledge that would be generated, including providing operational lessons and impact estimates based on a range of program approaches and implementation settings.  Moreover, all 11 programs met two additional criteria:  (1) each conformed to, and in some cases was based on, a theoretical framework that links its services to changes in youth knowledge, attitudes, intentions, and behavior; and (2) each appeared to be reasonably well implemented, which included having its core services operational, committed staff in place, and key partnerships established.

Table 1: Abstinence Education Programs Participating in the Evaluation
Location and
Sponsoring Agency
Principal Program Components Target
Population
Targeted Programs (Impact, Implementation, and Process Analysis)
FL (Miami)
Youth service agency
Elective class offered daily, all year to girls in middle schools.  Urban setting; diverse student population.  Curriculum:  ReCapturing the Vision and Vessels of Honor Grades 6–8
MS (Clarksdale)
Community health agency
Mandatory weekly year-long curriculum.  Rural community; extremely poor population.  Curriculum:  Revised Postponing Sexual Involvement and Sex Can Wait Grades 5–6
SC (Edgefield)
Youth service agency
Five-session mandatory curriculum with voluntary enrollment in weekly or biweekly character clubs.  Middle-income community.  Curriculum:  Heritage Keepers Grades 6–9
VA (Powhatan)
County health department
36-session mandatory curriculum.  Middle-income community.  Curriculum:  Reasonable Reasons to Wait; The Art of Loving Well; and Choosing the Best Grades 8 and 10, with 9th and 11th grade boosters
WI (Milwaukee)
Social service agency
Voluntary after-school program; two hours daily all year for multiple years.  Seven-week summer program.  Poor, inner-city community.  Curriculum:  Families United to Prevent Teen Pregnancy Grades 3–8 

Community-Wide Initiatives (Implementation and Process Analysis)

IA (Cedar Rapids)
Not-for-profit coalition
Abstinence curriculum; community resource library; classroom presentations; workshops for parents and educators; Baby Think It Over dolls; speakers; mentoring program; teen panels Emphasis on middle school youth
NY (Monroe County)
County health department
Not Me Not Now media campaign; abstinence curriculum; parent outreach program; interactive web site; mentoring program Emphasis on 9–14 year olds
SC (statewide)
Youth service agency
Abstinence curriculum; character clubs; school assemblies; training/information for medical providers, faith workers, parents, and media personnel; numerous collaborations and partnerships Middle and high school youth
TX (Fort Bend County)
Community-based organization
Abstinence curriculum; separate youth development programs for girls and boys; peer education program; school assemblies; community training; parent education programs; parent resource center; community events; medical provider involvement 9–17 year olds, with a heavy focus on middle school youth
TX (McLennan County)
Community-based organization
Abstinence curriculum; school assemblies; character education in elementary schools; mentors; media spots; medical provider training; faith-based partners; resource library Emphasis on 10–14 year olds
UT (Tooele County)
County health department
Abstinence curriculum; parenting class; self-esteem days for 5th–8th graders; self-esteem classes for high-risk youth; Baby Think It Over dolls; peer educators; school fairs; billboards and newsletters; merchant involvement; faith-based linkages 9–14 year olds

For the five programs targeting services on particular groups of youth, three additional site selection criteria were applied in order to ensure the feasibility of conducting a rigorous impact analysis of each program:

  • The program’s services and activities differ from what participants otherwise would receive.  The evaluation contrasts outcomes of youth receiving program services with those of control group youth who do not.  If participants would receive similar services without the program, the evaluation would essentially compare a program to itself, leading to negligible estimated impacts.
  • The program can readily adapt to evaluation procedures.  It was important that a program be able and willing to adapt to critical evaluation procedures without adversely affecting the basic program services.  It was essential that the evaluation be able to address programmatic concerns while not compromising the evaluation’s credibility.
  • Both the programs and the target population need to be large enough to support the sample size requirements for a rigorous study.  It was important that a program be able and willing to adapt to critical evaluation procedures without adversely affecting the basic program services.  It was essential that the evaluation be able to address programmatic concerns while not compromising the evaluation’s credibility.

While the abstinence education programs selected are considered interesting and well-implemented programs, they are not necessarily better than or representative of the more than 700 abstinence education programs funded under Section 510 and operating nationwide.  Many of the other Section 510 abstinence education programs are being examined in evaluations that states and localities have funded themselves, and these other studies will provide rich and important detail on the range of abstinence strategies and their effects (Mathematica Policy Research, Inc. 2001).