Impacts of Four Title V, Section 510 Abstinence Education Programs

By: Christopher Trenholm, Barbara Devaney, Ken Fortson, Lisa Quay, Justin Wheeler, and Melissa Clark Mathematica Policy Research, Inc.

Project Director: Christopher Trenholm

Contract No.: HHS 100-98-0010 MPR Reference No.: 8549-110

Submitted to:U.S. Department of Health and Human ServicesOffice of the Assistant Secretary for Planning and Evaluation

Project Officer: Meredith Kelsey

"

Acknowledgments

Many individuals contributed in important ways to this report. First, we thank Rebecca Maynard, former project director and co-principal investigator, whose hard work, creative thinking, and strong leadership during the evaluations first several years made this study and report possible. We also thank Amy Johnson, who provided critical support throughout this study in numerous capacities, including her role as Deputy Project Director and Survey Director.

We also thank the Title V, Section 510 Abstinence Education Program grantees who have generously allowed us to visit their programs, meet with staff, and observe their operations. Among these grantees, we are especially grateful to those who created and/or are directing the programs that are the focus of the study: Jacqueline Del Rosario with ReCapturing the Vision in Miami, Florida; Vicki Hearns Moses with Teens in Control in Clarksdale, Mississippi; Gale Grant, Ginelle Ampy, and Kathy Douglas with My Choice, My Future! in Powhatan, Virginia; and Dawn Groshek, Marty Kerrigan, Rosemary Fisher, and Lyn Hildenbrand with Families United to Prevent Teen Pregnancy in Milwaukee, Wisconsin. We also would like to thank the many school district staff in these communities who have supported the evaluation.

We have consistently received outstanding guidance and support from members of the projects technical workgroup. Current members include Marilyn Benoit, Sarah Brown, Ron Haskins, Jim Jaccard, Joe McIlhaney, Robert Michael, Kristin Moore, Susan Philliber, Robert Rector, David Rowberry, Freya Sonenstein, John Vessey, and Brian Wilcox. Many members of the research and policy community  especially Christine Bachrach, Cassie Bevan, Nancye Campbell, Stan Koutstaal, Susan Newcomer, David Siegel, Shepherd Smith, Matthew Stagner, Amy Stephens, and Pat Funderburk Ware  have been generous in answering questions and offering advice. Gary Burtless, Judy Gueron, Rob Hollister, and David Myers provided an extremely valuable review of the study design.

Carolyn Miller, John Homrighausen, Anne Self, Linda Mendenko, Jason Markesich, Milena Rosenblum, and Linda Bandeh shared various responsibilities for the design and oversight of the survey and school records data collection. Peter Schochet and Alan Hershey provided many useful comments on drafts of the documents. William Garrett expertly produced this report.

We are deeply grateful for the longstanding support of the staff at the U.S. Department of Health and Human Services who oversee this project, especially to Meredith Kelsey, the project officer for this study, Martha Moorehouse, Barbara Broman, Lisa Trivits, and Nicole Gardner-Neblett. They have accompanied us on site visits, worked with us to stay abreast of and respond quickly to emerging issues and concerns related to the study, and provided consistent support and guidance for the project.

Although we gratefully acknowledge the input of these and many other individuals, we alone are responsible for any errors or omissions in the report. Any opinions expressed in this report are those of the authors and do not necessarily reflect the views of the U.S. Department of Health and Human Services or others.

Christopher Trenholm, Project Director (ctrenholm@mathematica-mpr.com)
Barbara Devaney, Co-Investigator (bdevaney@mathematica-mpr.com)

Introduction

Authorized under the Social Security Act of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA), the Title V, Section 510 Abstinence Education Program was one of the legislative centerpieces that increased both the funding and visibility of abstinence education programs. Since fiscal year 1998, the Title V, Section 510 Abstinence Education Program has allocated $50 million annually for programs that teach abstinence from sexual activity outside of marriage as the expected standard for school-age children. Under the matching block grant program administered by the U.S. Department of Health and Human Services (DHHS), states must match this federal funding at 75 percent, resulting in a total of up to $87.5 million annually for Title V, Section 510 abstinence education programs.

In the Balanced Budget Act of 1997, Congress authorized a scientific evaluation of Title V, Section 510 abstinence education programs. The resulting multi-year evaluation included two major components. The first was an implementation and process analysis that documented the experiences of the organizations and communities that applied for and received the block grants authorized under Title V, Section 510. The second was a rigorous, experimentally based impact evaluation designed to estimate the effects of selected Title V, Section 510 abstinence education programs on teen sexual abstinence and related outcomes.

This report presents the behavioral impact findings of four selected programs that received funding through the Title V, Section 510 grants:

1. My Choice, My Future! in Powhatan, Virginia;
2. ReCapturing the Vision in Miami, Florida;
3. Families United to Prevent Teen Pregnancy (FUPTP) in Milwaukee, Wisconsin; and
4. Teens in Control in Clarksdale, Mississippi.

Like most programs supported by the Title V, Section 510 grants, these four programs all served youth in school settings, usually in the upper elementary or middle school grades. All programs offered more than 50 contact hours and lasted for one or more school years, making them relatively intense among programs funded by the Title V, Section 510 grant. One of the programs, FUPTP, served youth on a voluntary basis in an after-school setting. The other three programs served youth in classrooms during the school day much like any other course, although ReCapturing the Vision augmented these classroom-based services with a number of extracurricular offerings.

This report examines the impact of these programs on teens' sexual abstinence, their risks of pregnancy and sexually transmitted diseases, and other behavioral outcomes. The report is based on survey data collected in 2005 and early 2006В  four to six years after study enrollmentВ  from more than 2,000 teens who had been randomly assigned to either a program group that was eligible to participate in one of the four programs or a control group that was not.

Background

By the time Congress enacted PRWORA and authorized funding for abstinence education programs under Title V, Section 510, there was growing concern over the dramatic rise in teen pregnancy and childbirth rates during the late 1980s and early 1990s. By 1991, teen pregnancy and childbirth rates had reached highs of 116.5 and 62.1 per 1,000 women 15 to 19 years of age, respectively. Rates have dropped since that time; for example, by 2004, the teenage birthrate had fallen to 41.1 births per 1,000 women 15 to 19 years of age. However, concerns over the high incidence of births to unwed teen mothers, as well as the broader risks of teen sexual activity, have persisted (Centers for Disease Control and Prevention 2006; Weinstock et al. 2004; Chesson et al. 2004).

Title V, Section 510 Funding

Beginning in fiscal year 1998, the Title V, Section 510 funding provided $50 million of annual federal support for abstinence education programs that teach abstinence from sexual activity outside of marriage as the expected standard for school-age children. In order to receive these grants, states must match $3 of every $4 contributed by the federal government, which results in a total of up to $87.5 million available annually. Upon receipt of federal funding, states have discretion over which programs to fund and at what level. However, all funded programs are required to be consistent with the "A-H" definition of abstinence education prescribed in the Social Security Act (Table I.1).

Originally administered by the Maternal and Child Health Bureau (MCHB) of the Health Resources and Services Administration within DHHS, the Title V, Section 510 funding is currently distributed to states by the Administration of Children and Families (ACF) in the form of grants. These grants are based on a formula that compares the proportion of low-income children in the state to the total number of low-income children in all states. Just two years after the start of the Title V, Section 510 funding, states had funded over 700 programs nationwide. Among the groups that received funding were community-based organizations, school boards, local health departments, faith-based organizations, universities, local coalitions and advocacy groups, consultants, research firms, health care organizations, and non-profit organizations (MCHB 2000). Congress reauthorized the Title V, Section 510 funding in 2002.

Other Major Federal Abstinence Funding

In 2000, Congress increased funding of abstinence education through a federal earmark known as Community Based Abstinence Education (CBAE). Like the Title V, Section 510 programs, the CBAE-funded programs must be consistent with all eight of the "A-H" criteria. However, CBAE differs from Title V, Section 510 in the way that the funding is distributed. Under Title V, Section 510, funding passes through the states before reaching the abstinence education programs; CBAE funding, by contrast, is provided directly from the federal government to community-based programs. Also in contrast to Title V, Section 510, all programs funded by CBAE must specifically target youth between 12 and 18 years of age. Initially administered by MCHB, administration of CBAE was re-assigned to ACF in 2005.

Prior to the enactment of Title V, Section 510, federal funding for abstinence education had been provided mainly though the Adolescent Family Life Act (AFLA) or Title XX of the Public Health Services Act of 1981. Funding through AFLA is modest relative to Title V, Section 510. In fiscal year 2005, the Office of Adolescent Pregnancy Programs awarded about $13 million in Title XX grants to 58 public and private community organizations for projects that specifically promote several abstinence programs for adolescents (DHHS 2006). All programs funded by AFLA must be consistent with the same eight "A-H" criteria spelled out for the Title V, Section 510 funding.

Evaluation of Title V, Section 510 Abstinence Education Programs

Guiding the evaluation of the Title V, Section 510 programs is a logic model describing how the funded programs aim to reduce teen sexual activity and related risk behaviors (Figure I.1). Beginning in Box A, the logic model assumes that adolescent decision-making is influenced by numerous antecedents, including their own backgrounds and experiences and the characteristics of their schools and communities. Youth decision-making may also be influenced by the formal education services that they receive (Box B). As one of these possible services, the Title V, Section 510 abstinence programs aim to change the health, family-life, and sex education that youth normally receive (Box C). This change, in turn, is hypothesized to have favorable impacts on several intermediate outcomes that may serve as mediators of behaviors (Box D). For example, youth participating in the programs might develop more positive views towards abstinence and marriage or improve their knowledge of sexual activity risks. Through these and other changes, programs are ultimately hypothesized to affect longer-term behavioral outcomes (Box E). Among these outcomes are the rate of sexual abstinence and the potential consequences of sexual activity, such as STDs and pregnancy.

Figure I.1.
Logic Model for Evaluating the Impact of Title V, Section 510 Programs

A series of evaluation reports has studied the pre-behavioral components of the logic model, spanning Boxes A through D. In an initial DHHS study report, Devaney et al. (2001) examined Boxes A and B of the logic model  describing the populations served by the programs and the characteristics and implementation experiences of programs funded through Title V, Section 510. In two subsequent DHHS study reports, Maynard et al. (2005) and Clark and Devaney (2006) examined Boxes C and D of the logic model  measuring the first-year impacts of five selected Title V, Section 510 programs on the services youth received and on selected intermediate outcomes that may influence risk behavior.

Building on these earlier findings, the current report focuses mainly on the behavioral outcomes of youth, summarized in Box E of the logic model. The report addresses three questions:

1. What impacts do programs have on behavioral outcomes? Do the four selected Title V, Section 510 abstinence education programs affect behavioral outcomes summarized in Box E  rates of sexual abstinence and sexual activity and risks of STDs and pregnancy?
2. What impacts do programs have on possible mediators of behavior? Do the four programs improve knowledge of pregnancy and STD risks, knowledge of the health consequences of STDs, and other possible mediators of behavior, such as views toward abstinence and relations with peers, which were a focus of earlier DHHS study reports as well?
3. What are the links between possible mediators and behavior? How well do the potential mediators (Box D), measured after the first program year, predict the rates of sexual abstinence and sexual activity three to five years later? This analysis provides valuable insight into whether the intermediate outcomes that programs seek to affect (such as self-esteem and skill building) are in fact associated with future behavior.

The next chapter (Chapter II) describes the four programs that are the focus of this report, highlighting their common features and key differences. This is followed, in Chapter III, by a description of the research design and analytic methods used to measure the programs' impacts. Chapters IV through VI present the report findings, addressing respectively each of the three research questions listed above. Finally, Chapter VII summarizes the main study findings and considers the implications of these findings for future policy and research.

Focal Programs

Four Title V, Section 510 programs are the focus of this report:

1. My Choice, My Future! in Powhatan, Virginia;
2. ReCapturing the Vision in Miami, Florida;
3. Families United to Prevent Teen Pregnancy (FUPTP) in Milwaukee, Wisconsin; and
4. Teens in Control in Clarksdale, Mississippi.

This chapter provides a brief overview of these programs, how they were selected, and the features that distinguish them and the communities they serve. It also briefly describes how certain program features have influenced the design of the study and the interpretation of findings.

Overview of the Programs

One of the earliest stages of the evaluation entailed selecting Title V, Section 510 abstinence education programs for the study. The evaluation team first called and met with numerous state officials and experts across the country to identify promising programs for inclusion in the evaluation. Grant applications and program documents provided additional detail on program goals, target population, curriculum used, and funding levels. The evaluation team visited and observed 28 abstinence education programs across the nation. Eleven of these, representing a range of program models and serving different target populations, were invited and agreed to participate in the evaluation.

This report focuses on 4 of these 11 programs. These four programs are called impact sites because they had program features and staff capable of supporting a rigorous, experimental-design impact evaluation. (A fifth program  Heritage Keepers® in South Carolina  is also an impact site but is not included in this report because it has a different research design.⁽¹⁾ The remaining six programs were community-wide, systemic-change initiatives that aimed to increase public awareness of the problems of teen sexual activity, change community norms and attitudes, involve parents and encourage stronger parent-child communication, and engage youth in abstinence education and youth development services.

While these community-wide initiatives broaden our understanding of strategies for changing youth behavior, by design they are less able to support a rigorous impact study of program effectiveness.

The four programs are the following:

1. My Choice, My Future!  A three-year, mandatory, classroom-based program, My Choice, My Future! served students, beginning in the eighth grade, who attended Powhatan, Virginia County Schools.
2. ReCapturing the Vision.  A one-year, elective, classroom-based program, ReCapturing the Vision served mainly seventh and eighth grade girls attending selected middle schools in Miami, Florida.
3. Families United to Prevent Teen Pregnancy (FUPTP).  An elective, afterschool program available on a voluntary basis to students between the ages of 8 and 13, FUPTP served students attending selected elementary and middle schools in Milwaukee, Wisconsin.
4. Teens in Control.  A two-year, mandatory, classroom-based program, Teens in Control served students, beginning in the fifth grade, who attended selected elementary schools in the Clarksdale, Mississippi area.

Each of the four programs had qualities commonly found in programs supported by the Title V, Section 510 funding. Each program complied with the A-H guidelines, delivered its services in school settings, and focused on upper elementary and middle school youth. The four programs curricula also shared a similar focus and had many specific topic areas in common (Table II.1). For example, all four programs taught physical development and reproduction, promoted risk awareness, taught goal-setting and good decision-making, provided instruction about healthy relationships, and helped develop interpersonal and riskavoidance skills.

Distinguishing Program Features

Despite the programs similarities, each program had several other distinguishing features, including community characteristics, existing services, duration and intensity of services, and curriculum used.

Community Socio-Demographic Characteristics

Two of the programs, My Choice, My Future! and Teens in Control, operated in rural communities; however, the communities differed markedly in their socio-demographic makeup (see Table II.2, upper panel). My Choice, My Future! served youth attending Powhatan County Public Schools. The county is about 40 miles west of Richmond, and many of the newer residents commute to the city for work. The median income of the county is above the national average, and a majority of the youth attending the school system are white, non-Hispanic and live in two-parent families. Teens in Control served youth attending selected schools in three county school districts near Clarksdale, Mississippi  Coahoma, West Tallahatchie, and Tunica. In contrast to Powhatan, there is no urban center near these counties and, despite some recent growth in Tunica, the median income remains well below the national average. Most of the youth attending the three school districts are African American and live in single parent households.

The two other programs, FUPTP and ReCapturing the Vision, operated in two large urban settings  Milwaukee, Wisconsin, and Miami-Dade County, Florida, respectively. In both cities, the schools served by the programs were located in low-income neighborhoods characterized by program staff as having high rates of poverty, teen pregnancy, crime, and deteriorating housing. Youth in these schools are predominantly African American, and most live in low-income households with a single parent.

Existing Health, Family Life, and Sex Education Services

Both rural school districts-the Powhatan school district and the districts in Mississippi  offered only a modest degree of health, family-life, and sex education (Table II.2, lower panel). In Powhatan, all eighth grade youth not enrolled in My Choice, My Future! participated in a nine-week health and physical education class. This class covered alcohol, drugs, tobacco, personal safety, communicable and non-communicable diseases, consumerism, mental health, nutrition, and fitness. However, the class did not cover sex education, STDs, contraceptive use, abstinence from sexual activity, or marriage. In ninth grade, these students were enrolled in a health course that covered similar health topics. While the ninth grade course included material on abstinence, it did not cover sex education or contraceptive use. Teens in Control operated in schools that had an even more limited, district-wide health, family-life, and sex education curriculum for elementary and middle school youth. Usual services consisted of occasional presentations by outside organizations that generally consisted of a few sessions over a period of weeks.

Compared to the rural districts, both urban school districts offered a fairly significant set of health, family-life, and sex education services. The Miami Dade County Public Schools, served by ReCapturing the Vision, had a mandated health and sex education curriculum for youth in grades six through eight, which included a week long unit each year on human growth and development taught as part of the science class. The curriculum covered the stages of reproduction and human development and included discussions of contraceptive use. The sixth grade curriculum also covered drug and alcohol prevention, peer pressure, STDs, and the benefits of abstinence from sexual activity. The Milwaukee Public Schools, served by FUPTP, already had a mandatory family life curriculum for kindergarten through grade 12, a curriculum that both program and control group youth experienced. This curriculum included what was described as grade appropriate coverage of comprehensive health education; sexuality and HIV/AIDS; drugs, alcohol, and tobacco; and violence prevention. Abstinence and contraceptive use were covered beginning in fifth grade.

Program Delivery

All four programs began serving youth in elementary and middle school, when few among the target population had become sexually active. Two of the programs  My Choice, My Future! and ReCapturing the Vision  served youth beginning in seventh and eighth grade, when they were, on average, about 13 years of age. The other two programs  Teens in Control and FUPTP  served youth beginning in fourth and fifth grade, when they were, on average, 10 to 11 years of age. ReCapturing the Vision only served girls; the other three programs served both boys and girls.

The four programs also differed by whether they were elective or non-elective. My Choice, My Future! and Teens in Control were both non-elective programs that met during the school day much like any other class. Students could only "opt out" of consideration for the class if their parents gave permission. ReCapturing the Vision was an elective program whereby girls who were identified by school and program staff as potential candidates had the choice of participating or not. After students chose to participate, however, the program had required attendance like any other class. FUPTP was also an elective program, but youth could attend on a voluntary basis.

Program Structure: Duration, Intensity, and Curricula

The four programs differed significantly in duration and intensity and featured a variety of curricula (Figure II.1). My Choice, My Future! served youth for three years, though at a modest level of intensity; it included 30 classroom sessions in the first year, 8 in the second year, and 14 in the third year. Teens in Control was somewhat similar; it served youth for two years and met once a week during the school day. ReCapturing the Vision served youth for only one school year, but the program met daily, making it more intense than the other two classroom-based programs. Finally, FUPTP was an after-school program that met for two and one-half hours daily throughout the school year. The program was available to students as long as they attended the program school, which could have been up to four years in some cases. This made the program both relatively long and intense, assuming youth chose to attend.

 

My Choice, My Future! used a different curriculum for each of the three years that youth were enrolled in the program. The eighth grade curriculum, Reasonable Reasons to Wait: The Keys to Character, focused on character development, reasons to wait to engage in sex, peer influence, dating, avoiding STDs, relationship skills, and the benefits and ingredients of a strong marriage (Duran 1997). The ninth grade curriculum, the Art of Loving Well: A Character Education Curriculum for Today's Teenagers, featured short stories, poetry, classic fairy tales, and myths that taught about healthy and loving relationships (Boston University 1993). During the final year of the program, tenth graders received the WAIT Training curriculum, which focused on relationship skills and risk avoidance. The tenth grade program also featured slide show materials from the Medical Institute for Sexual Health (MISH), which provided information on STDs and instructed students that abstinence is the only sure way to avoid contracting them.

ReCapturing the Vision used two curricula  ReCapturing the Vision and Vessels of Honor  during the one year program. The ReCapturing the Vision curriculum centered on identifying personal strengths and resources, developing strategies for fulfilling personal and career goals, and building critical skills that would help youth achieve positive goals and resist negative influences (Del Rosario 2003). The complementary Vessels of Honor curriculum included six key areas of focus: (1) honorable behavior, (2) effective communication for resisting pressure to engage in sex and other high-risk behaviors, (3) development of good relationships and satisfying social needs and emotional feelings through friendships rather than sex, (4) physical development and its implications for changing pressures, (5) sexual abuse and date rape and how to avoid both, and (6) strategies for choosing a mate and the benefits of a committed marital relationship (Del Rosario 1999). In addition to the class-based lessons and activities, the ReCapturing the Vision program provided a number of complementary services, including home visits by social workers, referrals to local services, after-school tutoring, community service projects, cultural events, a family retreat, an annual Teen Abstinence Rally, and an annual Teen Talk Symposium with celebrity panelists.

FUPTP's abstinence curriculum, A Life Options Model Curriculum for Youth, was delivered as a key component of its after-school activities. This curriculum covered 10 topic areas, nearly all of which have abstinence as a central focus: (1) group-building, (2) self-esteem, (3) values and goal-setting, (4) decision-making skills, (5) risk-taking behavior, (6) communication skills, (7) relationships and sexuality, (8) adolescent development and anatomy, (9) sexually transmitted diseases, and (10) social skills (Rosalie Manor, undated). The unit on relationships and sexuality addressed marriage in addition to abstinence; however, because of the young age of many FUPTP participants, marriage received relatively less attention. In addition to the in-school services, youth and parents in FUPTP could participate in other programming that Rosalie Manor made available; these included periodic parent workshops, a Saturday teen mentoring program, and a seven-week summer program with teen mentors.

Teens in Control used a different curriculum for each of the two years that youth participated in the program. In fifth grade, program group youth received the Postponing Sexual Involvement curriculum (Howard and Mitchell 1990), which was designed to increase the awareness of the risks and pressures associated with early sexual involvement and to develop skills that could help youth remain abstinent. The five topic areas covered in this curriculum focused on the risks of early sexual involvement and the benefits of abstaining from sex until marriage, social and peer pressures to have sex, and the development of specific skills for resisting peer pressure using extensive practice sessions and reinforcement. The sixth graders in the program received the Sex Can Wait curriculum, which covered several key areas: self-concept and self-esteem; the psychological and physical changes during puberty; values; communication skills; information on the risks of STDs; skills for resisting social and peer pressures; and the formulation of career goals, planning how to achieve them, and how sexual abstinence is important for achieving these personal goals.

Study Implications

Two program features had notable implications for the study design and for this report. The first is the targeting of age groups in the upper elementary or middle school grades. While this is a common program feature of Title V, Section 510 programs, it required the evaluation to include an extended follow-up period so that program impacts on sexual abstinence and activity could be measured. At the time of the final follow-up survey, administered four to six years after youth enrolled in the study sample, the age of the study youth ranged from 12 to 20 years with a mean and modal age of 16 years (Figure II.2). The upper end of this age distribution includes youth primarily from the two programs that served middle school students, ReCapturing the Vision and My Choice, My Future!, while the lower end of this age distribution includes youth primarily from the two programs that served elementary school youth, FUPTP and Teens in Control. Youth in the former two programs averaged 18 years of age at the time of the final follow-up survey; youth in the latter two programs averaged 15 years of age.

Figure II.2.
Age Distribution of the Study Sample at the Time of the Final Follow-Up Survey

Given this age distribution, substantial variation was expected in the rates of sexual abstinence and sexual activity across the study sites at the time of the final follow-up survey (the survey on which findings in this report are based). Youth in the FUPTP and Teens in Control samples were expected to report relatively low rates of sexual activity compared to youth in the other two program sites.

Even at these fairly young ages, however, rates of sexual activity were expected to be at levels such that true program impacts could be detected. For example, according to the CDC's Youth Risk Behavior Survey, 51 percent of teens in Milwaukee, Wisconsin, report having had sex by ninth grade-the grade of the typical youth in the FUPTP study sample. While comparable data are not available for the other program that targeted elementary school youth (Teens in Control), its service area in Mississippi has among the highest rates of teen pregnancy in the state, making it likely that a large fraction of youth in the study sample would be sexually active by the time of the final follow-up survey.

The second program feature with important study implications is the elective versus non-elective nature of the programs, which leads to differences across the programs in both program participation and attendance. As shown in Figure II.3, both My Choice, My Future! and Teens in Control were non-elective school-based programs, and, as with any typical course offered in school, attendance was mandatory among those assigned to the program. In contrast, both ReCapturing the Vision and FUPTP were elective programs, meaning that eligible youth could choose whether or not to participate.

Figure II.3.
Program Participation and Attendance

In the case of ReCapturing the Vision, program staff identified a set of high-risk girls in the spring of seventh grade and invited them to apply to the program. Once girls applied and were randomly assigned to the program, they could have chosen not to participate in the fall of eighth grade because they either faced scheduling conflicts (such as a required math class) or may have changed their minds and decided to take another elective. This happened for 35 percent of the girls assigned to the program group. Attendance was mandatory, however, for all girls who chose to participate.

In the case of FUPTP, the program was not only elective but attendance was also completely voluntary, meaning that youth could attend as many or as few times as they chose. In practice, many students did not participate at all, though this was often for involuntary reasons such as transportation problems or other constraints. As a result, many youth assigned to the program group, 43 percent, did not participate in any FUPTP classes.⁽²⁾ In addition, among those who did participate, only a fraction attended most or all of the classes that were available. Specifically, among the 57 percent of program group youth with any participation, only 11 percent attended more than 80 percent of program services in the first year and 45 percent attended more than half. Even for those youth with low attendance rates, however, the total contact hours were still high because of the program's high intensity (a session available every school day for 150 minutes). Indeed, the average program group youth who participated in FUPTP received an estimated 146 hours of program services in the first year  more than the total annual contact hours for either My Choice, My Future! or Teens in Control.

The substantial nonparticipation among program group youth in ReCapturing the Vision and FUPTP reflects the reality of many abstinence (and other) programs that serve youth on an elective basis, making it an important program feature to include in this study. Consistent with standard research practices, the analysis of program impacts is conducted in two ways. The first presents impacts for all youth that the program intended to serve  that is, those randomly assigned to the program group. The second presents impacts for those who actually participated in the programs. As discussed in the next chapter, while the estimated impacts differ between these two approaches, their associated statistical significance is roughly equal. Thus, the main conclusions from this study differ little when based on one measure or the other.

Endnotes

1.  Specifically, for Heritage Keepers®, the evaluation was designed to measure the impact of adding an abstinence-focused character club to a classroom-based abstinence curriculum rather than to measure the impact of an overall abstinence program versus services as usual. A separate report on the impact of the Heritage Keepers® program is forthcoming [Now available, see link].

2.  This 43 percent rate of nonparticipation reflects an upper bound because the program did not have available attendance records for youth who attended fewer than 25 percent of the classes they had the opportunity to attend. The actual rate of true nonparticipation is therefore lower than 43 percent.

Design and Methods for the Final Impact Evaluation

Central to the evaluation of Title V, Section 510 abstinence education programs is a rigorous analysis of the programs' impact on teen sexual abstinence and teen sexual activity. To this end, the impact analysis for the evaluation relies on an experimental design. Under the experimental design, youth in the study sample are assigned to either a program group that receives the services provided by a selected group of Title V, Section 510 programs or a control group that receives only the usual services available in the absence of these programs. When coupled with sufficiently large sample sizes, longitudinal surveys conducted by independent data collectors, and appropriate statistical methods, this design is able to produce highly credible estimates of the impacts of the focal programs.

Impact Study Design

Obtaining an unbiased impact estimate requires determining what the outcomes of program group youth would be in the absence of that program (known as the counterfactual). Because the counterfactual cannot be observed directly, it must be estimated. Randomly assigning members of the study sample to either a program group or a control group is considered the most valid approach for estimating the counterfactual. Because of random assignment, program and control groups are similar in all respects except their access to the program services, making the control group a highly credible counterfactual. As a result, unbiased estimates of program impacts can be generated by simply comparing mean values of outcome measures (such as sexual abstinence) for the program group with those for the control group. The evaluation of Title V, Section 510 program uses this type of experimental design.

Sample Intake and Random Assignment

Sample intake took place near the beginning of three school years-1999-2000, 2000-2001, and 2001-2002. In each of these years, either program or school staff identified those youth who were eligible to participate in the programs. In the two non-elective programs  My Choice, My Future! and Teens in Control  eligible youth included all those in the targeted grade level (8th and 5th grades, respectively). In the elective programs  ReCapturing the Vision and FUPTP  eligible youth included all those in the targeted grade levels who had been identified by school or program staff as potential candidates. Once identified, eligible students were given a study consent form that notified parents of their child's eligibility for the program, explained the program and the evaluation, and described how selection for the program would take place through a lottery (random assignment).

In order for a student to be eligible both for the lottery and to participate in the evaluation, parents had to provide signed, active consent. In the two non-elective programs, parent consent rates were high, in excess of 90 percent (youth with parents not agreeing to participate were automatically placed in the classes that control group youth were to take). In the two other programs, parental consent rates could not be estimated because of their elective nature; however, there were few known cases of parents who wanted their child to participate in the program but opted out of the lottery because of the requirement to participate in the study.

In order to conduct the random assignment, lists of the eligible students who had active parental consent were sent to the evaluation team near the beginning of each school year and a random number generator was used to order the applicant pool. Once programs informed the evaluation team of the number of program slots available, the evaluation team released the names of students with that rank order or less in the assignment hierarchy. For example, in a program with 200 applicants and a capacity to serve 100 youth, the evaluation team released the names of the first 100 youth in the randomly ordered list of eligible applicants for inclusion in the program group. All students not selected for the program group formed an ordered "waitlist" and control group. In cases where it was necessary to maintain a minimum program enrollment, program vacancies were filled by releasing youth on the waitlist in the order of their original random number. Along with the original set of youth selected for the programs, all students selected from the waitlist to fill program vacancies became members of the study's program group regardless of whether they actually participated. All remaining students on the ordered waitlist formed the control group for the study.

In some instances, lists of eligible students were sent to the evaluation team in batches, leading to multiple rounds of random assignment within a school year. These multiple rounds of assignment, coupled with sample enrollment taking place over multiple school years, led to modest variation in the likelihood of students being selected for the program or control group. This variation in the selection probability was addressed in the analysis by using sample weights, as described below.

Sample and Data Collection

 The resulting study sample includes 2,501 youth, enrolled over a three-year period from fall 1999 through fall 2001 (Table III.1; top panel). Within each program site, sample sizes ranged from 504 for FUPTP to 849 for Teens in Control. Just less than 60 percent of the study sample was assigned to the program group (1,461); the remainder was assigned to the control group (1,040).

Data were collected from the study sample through a series of four surveys.⁽¹⁾ They included a baseline survey, administered near the time that youth began participating in the study, and three follow-up surveys. The surveys were administered either in school using a pen-and-paper instrument or by phone.

The impact findings presented in this report are based on data collected from the final follow-up survey, which was administered to study youth between spring 2005 and winter 2006. This reflects a follow-up period of roughly 42 to 78 months after youth began participating in the study, depending on the year in which they began participating and the exact timing of the survey. The response rate on this survey ranged from 80 to 84 percent across the four study sites, leading to an 82 percent rate overall (Table III.1, middle panel).

In each site, the sample size available for this report is given by the product of the number of youth in the study sample (upper panel of Table III.1) and their corresponding response rate on the final follow-up survey (middle panel of Table III.1). As seen in the lower panel of Table III.1, the resulting sample size for this report ranges from 414 youth for FUPTP to 715 for Teens in Control. The total sample size across the four sites totals 2,057 youth.

Evidence suggests that the program and control groups are well matched, as would be expected given an experimental design. Across a wide range of baseline measures, only a minimal number of differences between the program and control groups were statistically significant-no more than expected by random chance. For example, of over 40 measures based on baseline data (see Appendix Table A.1), no more than seven in each site show a statistically significant difference between the program group and control group.

Sample Characteristics

The sample characteristics of youth in each site reflected both the targeting of the programs and the communities in which youth lived (Table III.2). In the two program sites serving middle schoolers, My Choice, My Future! and ReCapturing the Vision, sample youth averaged over 18 years of age by the time of the final follow-up survey. This is considerably older than the two program sites serving upper elementary school youth, FUPTP and Teens in Control, in which sample youth averaged only 15.5 years of age. While the ReCapturing the Vision sample included only girls, reflecting the targeting of the program, the gender mix in the other three sites was fairly close to even. The race/ethnicity of youth in the study samples largely reflected their communities' composition. More than 80 percent of the youth in the My Choice, My Future! sample were white, non-Hispanic, while high proportions of youth in the other three sites were African American or Hispanic.

Youth in the study sample come from backgrounds that put them at relatively high risk of having sexual intercourse at an early age. With the exception of My Choice, My Future!, one third or fewer of the sample youth in each site reported at baseline having parents who were married. They also reported relatively high rates of life stressors, such as sisters getting pregnant or siblings dropping out of school. Moreover, although almost all youth reported that they had a mother figure (95 percent), only four out of every five youth in the Recapturing the Vision and FUPTP samples reported having a father figure.

Outcome Variables

All outcome measures were based on data from the final follow-up survey. These measures fall into two categories (Table III.3):

1. Measures of Risk Behavior and Behavioral Consequences. These include the measures that are most central to the evaluation, including whether youth remained abstinent, expected to abstain in the future, and engaged in unprotected sex (Table III.3; top panel). They also include measures of potential consequences of teen sexual activity, such as pregnancy or reported STDs, and important behavioral correlates of teen sexual activity, such as alcohol and drug use.
2. Measures of Potential Mediators (Knowledge and Perceptions). Several potentially important mediators of teen sexual behavior, shown in the lower panel of Table III.3, were not available until the final follow-up survey. Therefore, along with the measures of risk behavior and behavioral consequences, they are also a main focus of this final impacts report. This new set of measures spans two broad categories: knowledge and perceptions. Measures of knowledge include the ability of youth to identify STDs, their understanding of potential risks of unprotected sex, and their knowledge of potential health consequences of STDs. Measures of perceptions focus on whether youth believe condoms or birth control pills are effective for preventing pregnancy and STDs.

Analytic Methods

For each outcome measure, program impacts were estimated as the difference in regression adjusted mean values between the program and control groups. These impacts were estimated both overall and for each site individually. The overall estimate was obtained simply by averaging the estimated impacts for each of the four individual sites. This approach was preferred to weighting each site according to the size of its sample, which would have arbitrarily given some sites (most notably Teens in Control) more importance when computing a pooled estimate.

Multivariate Estimation

The regression analysis used weighted least squares models and pooled data across all four sites. Each regression model included a series of binary variables reflecting the interaction between program site and program status (program or control group). The site-specific estimate is obtained from the regression simply from the difference between the binary variables corresponding to that site's program and control groups. The pooled impact estimate for a given outcome is obtained from the average of these four program-control differences. The weights used in the regressions accounted for the variability in the probability of selection to the program or control groups as well as for youth who did not complete the final follow-up survey.⁽²⁾ Standard errors from the models were calculated taking into account the variability associated with these weights.

In addition to these variables, the regression models included a large number of variables to control for individual demographic and background characteristics measured from the baseline survey (Table III.4). For the small fraction of the sample who did not complete a baseline survey (fewer than 5 percent), a supplemental survey was administered at the next survey to collect key demographic information such as age, gender, and race/ethnicity. For other covariates, missing data were imputed using the mean for the sample in a given program site.

Along with site-level results, the report presents estimated impacts on behavioral outcomes for several subgroups of potential interest.⁽³⁾ Among these are subgroups defined by gender and several measures that might be linked to eventual behavior, such as baseline support for abstinence, religiosity, marital status of parents, and television viewing. All of these subgroups were defined from survey data collected at baseline, prior to any potential influence of the programs. A final subgroup, enrollment cohort, is also investigated because of important variation found across cohorts in an earlier DHHS study report (Maynard et al. 2005). The first of these subgroups includes youth enrolled in the 1999-2000 or the 2000-2001 cohorts; the second includes youth enrolled in the final, 2001-2002 cohort.

Impacts were estimated for one subgroup at time, following nearly the same methods as described above for the full sample. The only difference with these methods is that explanatory terms were added to the regression models reflecting the interaction between a given subgroup of interest (for example, gender) and each of the site dummies and the "site by treatment" interaction terms. Estimates for a given subgroup were then computed using the coefficients on these terms, following the same procedure described above.

Missing Outcomes Data

Although non-response on the individual survey questions was generally very low, typically just one or two percent, for certain outcomes it could still result in slightly biased estimates of outcome measures if left unaddressed. The first set of these questions pertain to knowledge questions  for example, "can you get pregnant if you have sexual intercourse only once?"  where there is a single correct answer. For these questions, it is likely that youth who completed most of the survey section on knowledge, but skipped an individual question or two, did so because they did not know the correct answer. Thus, in order not to understate the proportion of youth who were unsure of a correct answer, the response on individually-skipped knowledge questions was categorized as "don't know/unsure." In contrast, youth who skipped an entire section are excluded from the analysis for that set of outcomes.

A more serious form of missing data pertains to conditional questions, meaning that they are answered by youth only if they provide a particular response on a prior question or questions. For example, in order to answer the question on the number of sexual partners, the respondent must first indicate on the survey that s/he has had sexual intercourse. Since youth who have not had sexual intercourse can correctly be assigned a value of zero partners, this conditional wording means that all missing values for the question will pertain to youth who have had sexual intercourse. In turn, unless there are no missing data, the reported mean value for the full sample will be incorrect  in this case understating the mean number of sexual partners. To correct for this conditional item non-response, missing values were imputed following a commonly used "hotdeck procedure." This procedure assigns a value on the item that was missed based on the reported values of youth with characteristics similar to those of the item non-respondents. Through this method, the estimates for the program and control groups preserve the natural variability of the sample.

Nonparticipation and Crossover

As noted in Chapter II, a sizeable proportion of youth assigned to the program group in the two sites with elective programs, ReCapturing the Vision and FUPTP, did not participate in any program classes or other services (35 percent and 43 percent, respectively). To address this program nonparticipation, impact estimates are presented two ways in the report. The first is for the full program group. This estimate reflects the average effect of having the opportunity to participate in the program, whether or not the youth actually chose to participate. These estimates are featured throughout the report since it generalizes to the youth who were made eligible for the programs. The second is for only those youth in the program group who actually participated. These estimates are derived following the procedure developed by Bloom (1984), which divides the full-sample estimate by the participation rate. Because the standard errors and significance levels associated with the participant-only estimates are roughly similar to those for the full program group, impact estimates found not statistically significant for the full program group are typically not statistically significant for the participants either. As a result, the conclusions from the study do not differ substantively when based on one set of measures or the other.

Crossover of control group youth into the program group was rare, including at most 5 percent of the sample. For this reason, the report does not present estimates that account for crossover. To the extent that youth who did cross over experienced positive benefits from participating in the programs, the impact estimates reported are understated slightly.

Statistical Power

For the full sample, the statistical power of the study to detect impacts is high. Based on the observed explanatory power of the regression models, the study sample supports detection of true overall program impacts of roughly 0.08 standard deviations. (This is based on standard assumptions of 80 percent statistical power and 90 percent statistical confidence, two-tailed.) For a proportional outcome with a mean of 50 percent, this reflects an estimated impact of roughly 4 percentage points. Program impacts that are smaller in size may also be detected from the study sample, but the likelihood of doing so is below the 80 percent probability (power level) that is commonly preferred.

For the individual program sites, statistical power is naturally lower. This is particularly true in the two sites that experienced program nonparticipation, ReCapturing the Vision and FUPTP. For example, in the absence of nonparticipation, the size and allocation of the study sample would support detection of true site-specific impacts on the order of 0.16 standard deviations or larger for ReCapturing the Vision and 0.18 standard deviations for FUPTP. However, in light of the existing nonparticipation, the impacts on participants would need to be considerably larger  about 0.25 standard deviations for ReCapturing the Vision and 0.32 standard deviations for FUPTP  given equivalent levels of statistical power and confidence. This means the available samples in these two sites provide a high likelihood of detecting (that is, stating as statistically significant) true participant impacts only if they are fairly large; for example, for a proportional outcome with a mean of 50 percent, the minimum detectable impacts for participants are about 13 and 16 percentage points in the two respective sites. For the remaining two sites, My Choice, My Future! and Teens in Control, detectable impacts (at 80 percent power) are better  roughly 0.17 and 0.13 standard deviations, respectively.

Hypothesis and Sensitivity Testing

For each impact estimate, a two tailed t statistic tests the null hypothesis that there is no difference between the regression adjusted means for the program and control groups. The associated p value, which reflects the probability of obtaining the observed impact estimate when the null hypothesis of no effect is true, is used to judge the likelihood that a program had a measurable (statistically significant) impact. For categorical outcome variables, a t-test is conducted on the mean (proportion) for each response. In addition, an F-statistic tests the null hypothesis that there is no difference between the distributions of responses for the two experimental groups. This statistic is computed from a site-specific multinomial logistic regression of the categorical outcome variable on an indicator for program status and the covariates listed in Table III.4. The findings based on the F-statistics are consistent with those based on the individual t-test statistics.

Impact estimates with p-values less than 0.10, on two-tailed tests, are denoted in the report by asterisks and referred to in the text as statistically significant (Table III.5). While researchers sometimes use a lower p-value, 0.05 or less, to determine significance, this higher threshold allows a careful assessment of the findings across the range of outcomes being examined. The adoption of this threshold, however, does raise the likelihood of detecting significant impacts that have resulted merely by chance. Therefore, when interpreting the findings, attention is paid to whether significant impact estimates are isolated or whether they are part of a pattern of significant estimates that would point more strongly to a true program effect.

Additional analyses were conducted to examine the robustness of the impact estimates presented in the report. These included estimating impacts through logistic regression models (for binary outcomes) rather than linear probability models, and estimating impacts dropping various combinations of regression adjustment, data imputation, and sample weights. Across all these alternative estimates, findings were consistent with those presented in the report.

Endnotes

1. Copies of these surveys are available online at [http://www.mathematica-mpr.com].

2. Selection weights were calculated as the inverse probability of selection to the group of assignment. Non-response weights were calculated using standard modeling techniques to estimate the probability of survey non-response as a function of baseline covariates.

3. Subgroups defined by race/ethnicity could not be investigated because of the very high correlation between program site and a given racial/ethnic group.

Impacts on Sexual Abstinence and Teen Risk Behaviors

An earlier DHHS study report (Maynard et al. 2005) examined the impact of the four focal programs near the end of the first school year that youth were enrolled in the study.В  At that time, youth in the study averaged only 12 years of age, far too young for researchers to assess the impact of the programs on sexual abstinence and activity.В  The earlier report therefore focused on whether the programs had impacts on any of several potential mediators of these behaviors, such as support for abstinence, communication with parents, and refusal skills.В  Findings indicated that programs did achieve short-term success on some but not all of these potential mediators; for example, program group youth were significantly more likely than control group youth to report views more supportive of abstinence and less supportive of teen sex, but they displayed no statistically significant differences in their refusal skills or communication with parents.

Using data from a final follow-up survey, collected an average of five years after youth enrolled in the study sample, this chapter examines whether the near-term gains achieved by the programs translated into longer-term impacts on behavior.В  Key among these are whether the programs increased the likelihood that youth abstained from sexual intercourse, reduced the extent of sexual activity among youth, and increased their expectations to abstain from sex in the future.В  In addition, the chapter examines the impact of the programs on potential consequences of teen sex, such as pregnancy, and risk behaviors that are correlated with teen sex, such as drug and alcohol use.

Findings indicate that, despite the effects seen after the first year, programs had no statistically significant impact on eventual behavior.В  Based on data from the final follow-up survey, youth in the program group were no more likely to abstain from sex than their control group counterparts; among those who reported having had sex, program and control group youth had similar numbers of sexual partners and had initiated sex at the same mean age.В  Youth in the program group, however, were no more likely to have engaged in unprotected sex than their control group counterparts.В  Finally, there were no differences in potential consequences of teen sex, including pregnancies, births, and reported STDs.

Impacts on Abstinence and Sexual Behavior

• Program and control group youth reported similar rates of sexual abstinence.

As seen in Table IV.1, about half of both control and program group youth reported remaining sexually abstinent, and a slightly higher proportion reported having been abstinent during the 12 months prior to the survey (55 percent of control group youth versus 56 percent of program group youth).  This small difference was not statistically significant.

None of the individual programs had statistically significant impacts on the rate of sexual abstinence, whether measured as either always remaining abstinent or being abstinent during the last 12 months.  Across the four sites, differences between the program and control groups were modest (five points or less) and not consistent in direction.  On both measures, ReCapturing the Vision displayed the largest positive difference between the groups, but neither difference was statistically significant.  Teens in Control and FUPTP displayed negative, but not statistically significant, differences on both measures.

Program and control group youth also did not differ in the number of partners with whom they had sex.  Comparing the program and control groups overall, the distributions of the number of reported sexual partners are nearly identical (Figure IV.1).  About one-quarter of all youth in both groups had sex with three or more partners and about one in six had sex with four or more partners.  Distributions for each of the four sites, shown in Appendix Table A.4, likewise show no statistically significant differences between the program and control groups.

Figure IV.1.
Estimated Impacts on Reported Number of Sexual Partners

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Table A.4.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

Programs did not affect the age at which sexually experienced youth first engaged in sexual intercourse (data not shown).  Based on a question asking non-abstinent youth the age at which they first had sex, the reported mean age at first intercourse is identical between the program and control groups, 14.9 years.Table IV.2).  This pattern is similar for the other two measures  expectations to abstain from sex through high school and as a teenager (until age 20).  On each measure, program group youth had slightly higher expectations than control group youth, but the differences are not statistically significant.

Looking at the individual programs, one program, FUPTP, does display a large and statistically significant impact on expectations to abstain until marriage.  Specifically, 43 percent of youth in the program group for FUPTP reported that they expect to abstain from sex until marriage compared with 33 percent of corresponding control group youth  a statistically significant difference of ten percentage points (p-value = 0.04).  However, on the two other expectations measures, through high school and as teenagers, FUPTP displays no statistically significant impacts.  In fact, the estimated impacts are not positive (-4 and 0, respectively).  Findings for a second program, ReCapturing the Vision, are not statistically significant but the estimated impacts are somewhat large.  On the marriage measure, for example, 41 percent of program group in ReCapturing the Vision reported that they would abstain until marriage compared to 34 percent of control group youth.  The difference, seven percentage points, is not statistically significant (p-value = 0.13).

• Program group youth were no more likely than control group youth to have unprotected sex.

Eight percent of all control group youth and seven percent of all program group youth reported having had sexual intercourse and not using a condom the first time (Figure IV.2).  There are similarly no differences when measured over the last 12 months  17 percent of youth in both groups reported having had sex in the last 12 months and using a condom only sometimes, and 4 percent reported having had sex in the last 12 months and never using a condom. (Figure IV.3).  For all youth, this latter result is equivalent to about half of recently sexually active youth not always using a condom in the last 12 months.  Across the individual programs, estimated impacts on unprotected sex, measured either at first intercourse or in the last 12 months, were likewise small and statistically insignificant (see Appendix Tables A.5 and A.6, respectively).

Figure IV.2.
Estimated Impacts on Unprotected Sex at First Intercourse

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Table A.5.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

Figure IV.3.
Estimated Impacts on Unprotected Sex, Last 12 Months

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Table A.6.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

Programs likewise did not increase rates of unprotected sex when considering other forms of birth control (including those that only protect against pregnancy), such as birth control pills or Depo-Provera.  For example, in both groups, slightly more than half of youth had remained abstinent in the last 12 months (as reported above) and an additional 29 percent of youth reported that they had had sexual intercourse and always used a form of birth control (Figure IV.4).  This leaves only about one in six youth in both groups  15 percent in the program group and 16 percent in the control group  who reported that they had had sexual intercourse and had not always used a form of birth control.  Across the individual programs, these distributions varied but displayed no statistically significant program impacts (see Appendix Tables A.7 and A.8 for findings at first intercourse and last 12 months, respectively).

Figure IV.4.
Estimated Impacts on Birth Control Use, Last 12 Months

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Table A.8.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

• Programs had no impact on reported pregnancies, births, or STDs.

Very few youth in the study sample reported ever having been pregnant or ever having had an STD, and there were no statistically significant differences between the program and control groups on these measures (Table IV.3).  Ten percent of youth in both the program and control groups reported having been pregnant or gotten someone pregnant, and roughly half of them (five percent overall) reported that they had had a baby.  With respect to STDs, only a small fraction of youth in both groups, about five percent overall, reported being told by a doctor that they had an STD.  (Equal numbers of youth also reported being tested; not shown.)  Across the individual program sites, rates of all these outcomes varied modestly and displayed no statistically significant program impacts.

Impacts on Other Risky Behavior

• Program and control group youth reported no differences in their drinking or marijuana use.

As shown in Table IV.4, 16 percent of program group youth and 19 percent of control group youth reported smoking cigarettes in the last month, a difference that is statistically significant (p-value = 0.07).  However, with respect to alcohol and marijuana use, behaviors more closely associated with risk behavior, there are no statistically significant differences between the program and control group youth.  Overall, about one in four youth in both groups reported drinking once a month or more, while 30 percent reported ever having used marijuana.

Alcohol and drug use varied considerably across the program sites, and this is due at least in part to the variation in the ages of the study youth.  Within each site, however, program and control group youth reported no statistically significant differences in either behavior.  Differences between the two groups also varied in direction.  For example, program group youth in ReCapturing the Vision reported rates of alcohol use six percentage points lower than corresponding control group youth, while program group youth in FUPTP reported rates five points higher than corresponding control group youth.  Neither difference is statistically significant (p-values = 0.16 and 0.11, respectively).

Subgroup Impacts

• For several subgroups examined, programs show no consistent evidence of impacts on sexual abstinence or other behavioral measures.

Estimated impacts across a series of subgroups, summarized in Appendix D, display few statistically significant impacts on any of the behavioral outcomes examined above for the full sample.  Take, for example, the subgroup defined by whether youth have relatively high or low support for abstinence at baseline.  As discussed later in Chapter VI, this measure proves to be an important predictor of sexual abstinence, and it might also be expected to affect the way that youth respond to program messages.  However, as seen in Table IV.5, this is not the case.  Differences between program and control group youth are small in both subgroups defined by the measure and are not statistically significant.  A similar pattern of results is evident for other subgroups, including gender, religiosity, television viewing, and enrollment cohort, with a mix of positive and negative impact estimates that are small and rarely statistically significant (all shown in Appendix D).

Impacts on Participants Only

As discussed in Chapter II, participation was elective in two sites (ReCapturing the Vision and FUPTP), leading some youth not to participate despite being randomly chosen for the program group.  The implication of this nonparticipation is that the estimates of program impacts for the subsample of participants will be larger than the estimates for the entire sample in these two sites.  However, because there is a corresponding loss of statistical power when estimating impacts for the smaller, participant-only sample, the statistical significance associated with these participant-only impacts is roughly equal to those for the full sample.  Thus, the benefit of examining impacts for the participants-only sample is merely in identifying any notable program-control group differences, regardless of significance, that might have been less evident for the full sample.

As highlighted by Table IV.6, the estimated impacts on a few of the behavioral measures are somewhat notable in size for the participant-only sample, although they are not statistically significant.  (For complete results of the participant-only impact analysis, see Appendix Table A.9.)  With regard to sexual abstinence, for example, participants in ReCapturing the Vision had rates 7 percentage points higher than their control group counterparts.  In contrast, program group youth in FUPTP reported rates of sexual abstinence that were 4 percentage points lower than their control group counterparts and 3 points lower when measured as abstinent in the last 12 months.

Estimated impacts on expectations are also more notable in size when focusing on participants only, though they are rarely statistically significant.  For ReCapturing the Vision, program participants were 9 percentage points more likely to expect to abstain from sex through high school and 11 percentage points more likely to expect to abstain until marriage than their control group counterparts.  However, neither difference is statistically significant (respective p-values = 0.34 and 0.13; not shown).  For FUPTP, participants were 18 percentage points more likely than their control group counterparts to expect to abstain until marriage, a difference that is statistically significant (p-value = 0.04; not shown).  Participants were also six percentage points less likely to expect to abstain through high school; however, this difference is not statistically significant.

Endnote

[1]  This measure of the mean age at first intercourse is based on the subsample of program and control group youth that reported having had sex.  An alternative to this measure is the proportion of youth who report having had sex by a particular age (by age 14, for example).  Regardless of the age cutoff examined, the findings indicate no statistically significant difference between program and control group youth.

Knowledge and Perceptions of Risks Associated with Teen Sex

While findings from the prior chapter show no evidence that programs affected behavior, results from an earlier DHHS study report had indicated that programs had statistically significantly impacts on the health, family, and sex education services that youth received (Maynard et al. 2005).В  Perhaps most notable among these changes was a reported increase in the value of these services for understanding pregnancy and STD risks.В  Using data from the final follow-up survey, this chapter examines whether these changes in services resulted in sustained impacts on knowledge of STDs and the potential risks associated with sexual activity.В  In addition, the chapter examines whether programs affected youth perceptions about the effectiveness of condoms or birth control pills for preventing pregnancy and STDs.^[1]

Findings indicate that both program and control group youth had a good understanding of their risks for pregnancy but a less clear understanding of STDs, particularly with respect to their health consequences.В  Programs display some modest gains on measures of these outcomes.В  On a measure of STD identification, program group youth reported significantly higher average levels of knowledge than their control group counterparts.В  One program, My Choice, My Future!, is largely responsible for this result.В  My Choice, My Future! also displayed a significant impact on two knowledge scales associated with pregnancy and STD risks.

Additional findings indicate program and control group youth had similar perceptions of condom effectiveness for preventing pregnancy, but program group youth were less likely than control group youth to perceive condoms as effective at preventing STDs.В  The same pattern is evident for perceptions of birth control pills.В  While program and control group youth had similar perceptions of whether birth control pills are effective for preventing pregnancy, program group youth were less likely than control group youth to perceive them as effective at preventing STDs.В  As with the knowledge findings, My Choice, My Future! displays the most consistent evidence of affecting these perceptions.

Knowledge of STD and Pregnancy Risks

• Study youth correctly identified STDs only about two-thirds of the time.  Programs increased this proportion by a modest amount.

On the final follow-up survey, youth were given a list of 13 diseases and asked whether or not each was a sexually transmitted disease; of these diseases, nine were actual STDs and four were not STDs (see Appendix C for the exact questions).  Youth in the program group identified an average of 69 percent of these diseases correctly, as STDs or not, while youth in the control group identified an average of 67 percent correct.  The difference is statistically significant (Table V.1).  Comparing the four programs on this measure, My Choice, My Future! displays the largest difference by far.  It raised the rate of STD identification by an estimated eight points, from a mean of 75 percent for the control group to a mean of 83 percent for the program group.

Findings remain consistent when examining impacts separately for diseases that are STDs and those that are not.  Overall, program group youth correctly identified a higher percentage of diseases of both types, though only the impact on true STDs remained statistically significant (Table V.1).  This consistency suggests that programs did not simply raise the likelihood that youth believed any disease was transmitted sexually; rather, they had a beneficial long-term impact on STD identification.

• Many study youth understood the risks of pregnancy and STDs from unprotected sex, but they often lacked an understanding of the potential health risks from STDs.  Program and control group youth had similar levels of knowledge.

On a two-item scale measuring their understanding of unprotected sex risks, youth in both the program and control groups had high scores (0.88) (Table V.2).  Program and control group youth likewise reported similar levels of knowledge on a three-item scale measuring their understanding of potential health risks of STDs.  However, their respective mean values on this scale were relatively low, 0.52 and 0.51, and corresponded to a typical youth answering only about half the items of the scale correctly.  (See Appendix C for a list of the questions and coding of responses for these scales).

Despite the lack of an impact on these scales across the four programs, one program, My Choice, My Future!, shows consistent evidence of raising youths knowledge.  On both scales shown in Table V.2, the mean among program group youth in My Choice, My Future! was significantly higher than among their control group counterparts, reflecting a gain in knowledge attributable to the program.  Other programs also displayed some statistically significant differences between program and control group youth on the two scales, but these differences are less consistent.  For example, on the measure of knowledge of STD consequences, program group youth in FUPTP reported a mean score that is five points higher than their control group counterparts, a difference that is statistically significant.  However, on the measure of unprotected sex risks, the difference in mean scores is only two points and not statistically significant.  Teens in Control displays a similar pattern between these two scales, except that the differences between the program and control groups are in the opposite (negative) direction.

Perceptions of Condom and Birth Control Pill Effectiveness

• Most study youth reported that condoms were at least sometimes effective at preventing pregnancy.  Programs had no impact on this perception.

About half of program and control group youth responded that, when used correctly, condoms usually prevent pregnancy (Figure V.1).  Most of the remainder, 38 percent, reported that condoms sometimes prevent pregnancy.  Only three percent of youth thought that condoms never prevent pregnancy, while seven percent reported being unsure.

Figure V.1.
Estimated Impacts on Perceived Effectiveness of Condoms for Preventing Pregnancy

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Table A.10.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

• Many study youth reported being unsure about whether condoms prevent STDs.  Overall, program group youth were less likely than control group youth to perceive condoms as effective at preventing STDs.

Roughly one-quarter of youth in both the program and control groups reported being unsure about how effective condoms are at preventing chlamydia and gonorrhea or at preventing herpes and HPV (Figure V.2).  In addition, a sizeable fraction in both groups, about one in seven, reported being unsure about condoms effectiveness for preventing HIV.  These findings are in sharp contrast to those for pregnancy (above), for which very few youth in either group reported being unsure about their effectiveness.

Youth in the program group were significantly less likely to report that condoms usually prevent STDs than those in the control group.  And, for each STD type, this difference was offset by a significantly higher proportion of program group youth reporting that condoms are never effective at prevention.^[2]  Specifically, programs raised the proportion of youth who reported that condoms never prevent HIV from an estimated 17 to 21 percent; the proportion who reported that condoms never prevent chlamydia and gonorrhea from an estimated 14 to 20 percent; and the proportion who reported that condoms never prevent herpes and HPV from an estimated 15 to 23 percent.

Findings at the site level, detailed in Appendix Tables A.11 through A.13, indicate that two programs, My Choice, My Future! and Teens in Control, are largely responsible for the impacts seen overall.  The My Choice, My Future! findings mirror the overall results most closely  for all STDs examined, youth in the program group were significantly less likely than those in the control group to report that condoms are usually preventive, and they were significantly more likely to report that condoms are never preventive.  For Teens in Control, the same pattern of results holds, though the differences are less often statistically significant.  FUPTP also displayed some statistically significant differences between program and control group youth.  Most notably, program group youth in this site were more likely to report that condoms usually prevent HIV while also more likely to report that condoms never prevent herpes and HPV.

Figure V.2.
Estimated Impacts on Perceived Effectiveness of Condoms for Preventing Sexually Transmitted Diseases:
Prevention of Chlamydia and Gonorrhea

Prevention of HIV

Prevention of Herpes and HPV

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Tables A.11 through A.13.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

• Most study youth reported that birth control pills were usually or sometimes effective at preventing pregnancy.  Program and control group youth shared similar perceptions.

Just over half of the youth in both the program and control groups reported that, when used properly, birth control pills usually prevent pregnancy (Figure V.3).  Only three percent of youth in each group reported that birth control pills never prevent pregnancy, and seven percent were unsure about their effectiveness.  At the site level, shown in Appendix Table A.14, program group youth in FUPTP were significantly more likely than control group youth to report birth control pills usually prevent pregnancy.  This difference is not evident in the overall findings because it is offset by small, negative differences in the other three program sites (none are statistically significant).

Figure V.3.
Estimated Impacts on Perceived Effectiveness of Birth Control Pills for Preventing Pregnancy

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Table A.14.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

 

• Program group youth were less likely than control group youth to perceive birth control pills as effective at preventing STDs.

More than two out of three study participants reported, correctly, that birth control pills do not prevent STDs.  For each STD investigated, a significantly higher proportion of youth in the program group than the control group reported this was the case (Figure V.4).  For example, 73 percent of program group youth correctly reported that birth control pills never prevent HIV compared to 69 percent of control group youth, a statistically significant difference of four percentage points.

As with several previous measures, My Choice, My Future! is the main source for the difference seen overall in these perceptions (see Appendix Tables A.15 through A.17).  For each STD type, the proportion of program group youth in My Choice, My Future! who reported that birth control pills never prevent STDs was significantly higher than that of the control group.  Differences ranged from 8 to 11 percentage points.  In contrast, the other three program sites display no statistically significant differences between the two groups for any of the STDs examined.

Figure V.4.
Estimated Impacts on Perceived Effectiveness of Birth Control Pills for Preventing Sexually Transmitted Diseases:
Prevention of HIV

Prevention of Chlamydia and Gonorrhea

Prevention of Herpes and HPV

Source:  Wave 4 Survey of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2005), administered to youth 42 to 78 months after enrolling in the Title V, Section 510 Abstinence Education Program study sample.

Note:  All estimates are based on weighted regression models.  For details on the covariates, see Appendix Table A.1.  Sample sizes and R-square statistics are in Appendix Tables A.2 and A.3, respectively.  Findings by site, as well as F-tests of the difference in the distribution of the outcome measure between control and program groups, are in Appendix Tables A.15 through A.17.

***p-value (of program-control difference) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

Endnotes

[1] These potential mediators of sexual abstinence (knowledge and perceptions) were not measured until the final follow-up survey.  Therefore, this is the first report to examine them.  For updated impact findings on other potential mediators of sexual abstinence  all of which were examined in a prior DHHS study report by Maynard et al. (2005)  see Appendix E.

[2] For each STD category, youth who reported in the never effective category were significantly more likely to have remained abstinent than those who reported into one of the other categories.  This difference merely reflects an association, not evidence of a causal relationship.  Indeed, it is evident among both program and control group youth, suggesting it is related to factors other than program participation.

Predictors of Sexual Abstinence

The national evaluation of Title V, Section 510 abstinence education programs collected survey data on study youth over a four to six year period, depending on the year that they began to participate.В  At the time of enrollment in the study, youth were of middle school age or youngerВ  in most cases, too young to be sexually active.В  Over the course of the evaluation, they aged into mid-to-later adolescence, when many youth are making decisions about their own sexual activity.В  To gain insight into the unfolding of these decisions over time and the effect of program participation on these decisions, the evaluation has included analyses of both short-term and longer-term program impacts.

A previous DHHS study report found that the four focal programs had an impact on several of their intended short-term outcomes, which were hypothesized to lower rates of teen sexual activity (Maynard et al. 2005).В В  Most notably, relative to their peers in the control group, youth in the program group reported views more supportive of abstinence and less supportive of teen sex, and they demonstrated a heightened awareness of the possible negative consequences of teen sex.В  Program group youth were also significantly more likely than youth in the control group to make formal pledges to abstain from sex until marriage.

This chapter explores two potential explanations for the apparent inconsistency between short-term impacts on outcomes believed to be predictive of abstinence and the lack of longer-term impacts on abstinence:В  (1) these outcomes failed to affect, or mediate, sexual abstinence as hypothesized, and (2) the short-term impacts on these outcomes were simply too small or did not persist for long enough to have an impact on eventual sexual activity.

Notably, while the chapter provides insight into the links between potential mediators and sexual abstinence, it cannot establish causality.В  An observed relationship between a mediator and sexual abstinence might reflect the effect of unobserved factors correlated with that mediator rather than the causal impact of the mediator itself.В  For example, peer pressure could have a causal effect on sexual abstinence, in which case peer pressure and sexual abstinence would be correlated.В  But an observed correlation between peer pressure and sexual abstinence could also arise from youth with an unobserved propensity to engage in sexual activity selecting into peer groups in which peer pressure is high.В  The analytic approach, presented below, cannot disentangle these two explanations for any correlations between potential mediators and sexual abstinence.

Methods

The evaluation draws on a rich longitudinal data set that includes multiple measures of attitudes and other possible mediators of youth behavior as well as of their behavioral outcomes.  These data not only allow the analysis of program impacts over time, but also enable us to examine the pathways through which programs might have affected behaviors.

The logic model for the evaluation, presented in Chapter I and reproduced below (Figure VI.1), presents the pathways through which program effects were hypothesized to occur.  Programs aimed to alter the level and nature of services youth received in ways that would influence potential mediators of teen sexual activity.  Examples of these potential mediators include youth views toward abstinence, their relations with peers, and their perceived consequences of teen sex.  The first year impact report examined program impacts on receipt of services (Box C) as well as on several potential mediators (Box D).  The earlier chapters of this report estimate program impacts on long-term behavioral outcomes (Box E).  This chapter focuses on the potential links between selected mediators (Box D) and teen sexual behavior (Box E).

Figure VI.1.
Logic Model for Evaluating the Impact of Title V, Section 510 Programs

The relationships between mediators and sexual abstinence are estimated using a multiple regression model.  The outcome of interest  whether youth have remained abstinent (measured at the time of the final follow-up survey)  is regressed against a set of covariates that measure several potential mediators of abstinence.  These mediators are based on data from an initial follow-up survey, conducted six to nine months after youth enrolled in the study.  Findings from the regression thus provide an estimate of whether a potential mediator of behavior, such as relations with peers, does in fact predict whether youth have abstained from sex three to five years later (the period between the initial and final follow-up surveys).

The analysis focuses on five groups of potential mediators, all measured from the initial follow-up survey data.  They include (1) youth views toward abstinence, sex, and marriage; (2) peer influences and relations; (3) self-concept, refusal skills, and communication with parents; (4) perceived consequences of teen sex; and (5) pledging to abstain from sex.[1]  Ideally, the analysis would be expanded to include the other potential mediators of teen sex shown in Figure VI.1, such as knowledge of STD and pregnancy risks.  However, measures of these potential mediators were not collected from the initial follow-up survey.

Findings

Two potential mediators from the initial follow-up survey  views supportive of abstinence and friends support for abstinence  were significantly predictive of reported sexual abstinence on the final follow-up survey (Table VI.1).  Specifically, youth reporting views more supportive of abstinence were more likely to report abstaining from sexual intercourse on the later survey. The magnitude is large; a one-unit increase in the measure is associated with an eight percentage point increase in the likelihood of remaining abstinent (p-value = 0.01).  Likewise, having a network of close friends who are supportive of abstinence was strongly associated with increased sexual abstinence.  A one-unit increase in support for abstinence among friends is associated with a five percentage point increase in the likelihood of remaining abstinent.

Of the remaining potential mediators, none is associated with sexual abstinence in the direction hypothesized in the logic model (Table VI.1).   One measure, support for marriage, has a negative association with sexual abstinence, which is inconsistent with the logic model.  The remaining measures  including self-concept, refusal skills, and communication with parents; perceived consequences of teen sex; and pledging  all bear no statistically significant association with later sexual abstinence.  Perhaps the most surprising of these findings concerns the pledge, which two previous studies (Bearman and Bruckner 2001; Rector et al. 2004) found to be associated with delayed sexual initiation but this study finds to have no statistically significant association with later sexual abstinence.

Given that support for abstinence by youth and peer support for abstinence are the only significant long-term predictors of sexual abstinence found in this study, the remainder of this chapter focuses on these two measures (defined in Table VI.2) and how they changed over time.  For findings on the other measures shown in Table VI.1, see Appendix E.

Changes in Youth Support for Abstinence Over Time

Support for abstinence among sample youth declined notably between the initial and final follow-up surveys, regardless of whether they were in the control or program group (Figure VI.2).  Among youth in the control group, the mean score on a scale measuring views toward abstinence fell from 1.80 to 1.60.  This change is equivalent to one-fifth of control group youth moving one unit on this scale measure; for example, from responding that they agree with the series of statements supportive of sexual abstinence (shown in Table VI.2) to responding that they disagree.

Figure VI.2.
Youth Support for Abstinence Over Time

Sources:  Wave 2 and Wave 4 Surveys of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2000, 2005) administered to youth 6 to 12 months and 42 to 78 months, respectively, after enrollment in the Title V, Section 510 Abstinence Education Program study sample.

***p-value (of change shown) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

Among youth in the program group, the decline in support for abstinence was even greater than among control group youth, leading program impacts on this measure to disappear over time.  At the time of the initial follow-up survey, program youth held views significantly more supportive of abstinence than youth in the control group (Figure VI.2; top panel).  However, by the time of the final follow-up survey (three to five years later), views among program group youth had fallen from a mean of 1.89 on the scale to a mean of 1.62  a larger drop than the one seen for the control group youth.  The result is that the difference in support for abstinence between the two experimental groups seen at the time of the initial follow-up survey is not statistically significant by the time of the final follow-up survey.

Changes in Friends Support for Abstinence Over Time

Support for abstinence among friends also fell substantially between the two follow-up surveys (Figure VI.3).  Based on a scale measure ranging from 0 to 5, youth in the control group averaged a score of 3.50 at the time of the initial follow-up survey.  This is equivalent to youth reporting that three to four of their five closest friends had attitudes or behaviors supportive of abstinence.  By the time of the final follow-up survey, this measure fell sharply to a mean score of 1.96, indicating that on average fewer than half of their friends held attitudes supportive of abstinence.

Figure VI.3.
Peer Support for Abstinence Over Time

Sources:  Wave 2 and Wave 4 Surveys of Teen Activities and Attitudes (Mathematica Policy Research, Inc., 2000, 2005) administered to youth 6 to 12 months and 42 to 78 months, respectively, after enrollment in the Title V, Section 510 Abstinence Education Program study sample.

***p-value (of change shown) < 0.01; **p-value < 0.05; *p-value < 0.10, two-tailed test.

Youth in the program group reported a similar decline in friends support for abstinence, and the program had no impact on this measure, either in the short or longer term.  At the time of the initial follow-up survey, youth in the program group reported slightly higher peer support for abstinence than those in the control group  a mean of 3.58 compared to 3.50  but the difference was not significant at conventional levels (p-value = 0.15).  By the time of the final follow-up survey, friends support had fallen slightly more among program group youth than control group youth, resulting in nearly identical levels of support between the two groups.

Summary

Several potential mediators of teen sexual abstinence commonly addressed by Title V, Section 510 program curricula are found to have no association with sexual abstinence three to five years later.  Notable among these are self-concept, refusal skills, and communication with parents; perceptions of negative consequences from teen sex; and pledging to abstain from sex.  Two other potential mediators are found to be significantly associated with future sexual abstinence: youth support for abstinence and their friends support for abstinence.  Although the analysis cannot determine whether either of these associations is causal, findings suggest that promoting support for abstinence, both among youth and their friends, should be an important feature of future abstinence programs.

The programs evaluated in this report had at most a small impact on support for abstinence in the short term, and they had no impact on support for abstinence in the longer term.  However, levels of support among both program and control group youth did change significantly over time.  For example, at the end of their first school year in the study sample (the time of the first follow-up survey), most program group youth reported having a majority of friends supportive of abstinence.  But by the time of the final follow-up survey  when most program youth had entered middle to late adolescence and all youth had completed the programs  only a small fraction had maintained this high level of peer support.

Endnote

[1] See Appendix Table E.1 for definitions of these measures.  Pledging to abstain is shown in Box C of the logic model (services received) because it is often a component of programs curricula; however, the act of pledging may act as a mediator of future behavior, making it relevant for this analysis.

Conclusions

This report examines the impacts of four selected Title V, Section 510 abstinence education programs on adolescent sexual activity and related knowledge and behavioral outcomes. The impact findings show no overall impact on teen sexual activity, no differences in rates of unprotected sex, and some impacts on knowledge of STDs and perceived effectiveness of condoms and birth control pills. This chapter summarizes these main impact results, considers some important lessons learned from the evaluation, and ends with a look ahead.

Summary of Impact Results

The main objective of Title V, Section 510 abstinence education programs is to teach abstinence from sexual activity outside of marriage. The impact results from the four selected programs show no impacts on rates of sexual abstinence. About half of all study youth had remained abstinent at the time of the final follow-up survey, and program and control group youth had similar rates of sexual abstinence. Moreover, the average age at first sexual intercourse and the number of sexual partners were almost identical for program and control youth.

Some policymakers and health educators have questioned the Title V, Section 510 abstinence education programs, believing that the focus on abstinence may put teens at risk of having unprotected sex. The evaluation findings suggest that this is not the case. Program and control group youth did not differ in their rates of unprotected sex, either at first intercourse or over the last 12 months. Less than 10 percent of all study youth (8 percent of control group youth and 7 percent of program group youth) reported having unprotected sex at first intercourse. Over the last 12 months, 21 percent of both program and control group youth reported having sex and not always using a condom.

Findings on behavioral outcomes for each of the four individual sites likewise indicate few statistically significant differences between program and control group youth. In each site, most differences between youth in the program and control groups were small and inconsistent in direction. ReCapturing the Vision displayed the largest positive differences with respect to abstinence from sex; 48 percent of program youth in this site reported being abstinent in the last 12 months compared with 43 percent of control group youth.

ReCapturing the Vision also displayed a positive difference of seven points in the proportion of youth who reported expecting to abstain from sex until marriage. Neither of these differences is statistically significant. Given the smaller sample sizes available for estimating impacts at the site level, however, the study cannot rule out modest site-specific impacts on these outcomes.

Many Title V, Section 510 abstinence education programs focus on the risks of STDs, and the evaluation results show some improvements in knowledge of STDs. Program group youth correctly identified a significantly higher proportion of STDs than control group youth, and program group youth were significantly more likely than control group youth to report (correctly) that birth control pills are never effective at preventing STDs (including HIV, chlamydia and gonorrhea, and herpes and human papillomavirus [HPV]). For both outcomes, My Choice, My Future! is the main source of the differences seen overall.

Program group youth, however, were less likely than control group youth to perceive condoms as effective at preventing STDs. Compared with control group youth, program group youth were less likely to report that condoms are usually effective at preventing HIV, chlamydia and gonorrhea, and herpes and HPV. Furthermore, program group youth were more likely than control group youth to report that condoms are never effective at preventing these STDs. As above, My Choice, My Future! is a main source of these overall impacts.

Lessons Learned

The national evaluation of Title V, Section 510 abstinence education programs has been conducted over a period of nine years. It started just after the funding authorization in 1998 and focused on the first generation of the A-H abstinence education programs. The evaluation has included both implementation and impact analyses, with multiple site visits to observe the programs and longitudinal follow-up of study youth over a period of four to six years. Several prior DHHS study reports document the implementation experiences of the schools and communities operating the programs and first year impacts of the programs on potential mediating outcomes (Devaney et al. 2002; Maynard et al. 2005; Clark and Devaney 2006). These reports, together with this final impact report, highlight several important considerations for addressing persistent concerns associated with teen sexual activity.

Teens Have Important Gaps in Knowledge of STDs

Program and control group youth appear better informed about the risks of pregnancy than about the risks or consequences of contracting STDs. Although a high proportion of youth reported that having unprotected sex just once could result in an STD, 47 percent of sexually active youth had unprotected sex in the previous 12 months. Moreover, on a scale measuring their understanding of the health consequences of STDs, youth on average got only about half of the answers correct; on a scale measuring STD identification, youth were correct only about two-thirds of the time. In summary, there is a lack of knowledge of the consequences of STDs among both groups.

Targeting Youth at Young Ages May Not Be Sufficient

As with the four programs in this study, most Title V, Section 510 abstinence education programs have been implemented in upper elementary and middle schools. In addition, most Title V, Section 510 programs are completed before youth enter high school, when rates of sexual activity increase and many teens are either contemplating or having sex. Findings from this study provide no evidence that abstinence programs implemented in upper elementary and middle schools are effective at reducing the rate of teen sexual activity several years later. However, the findings provide no information on the effects programs might have if they were implemented for high school youth or began at earlier ages but served youth through high school.

Peer Support for Abstinence Erodes as Youth Move Through Adolescence

At the time when most Title V, Section 510 abstinence education programs are completed and youth enter their adolescent years, support for abstinence among their friends falls dramatically. For example, survey data from the start of the impact study show that nearly all youth had friends who exhibited attitudes and behaviors supportive of abstinence. Four years later, however, the typical youth in the study reported that only two of his or her five closest friends remained supportive of abstinence.

Youth who participate in Title V, Section 510 programs may also find themselves unable to maintain their peer networks as they advance from elementary to middle school or from middle school up through high school. In some urban settings, for example, the parent(s) of a child attending a particular middle school might have the option of sending that child to potentially dozens of high schools in the school district. Alternatively, in many other communities, children from several elementary (or middle) schools might feed into a single middle (or high) school. To the extent that the Title V, Section 510 abstinence programs aim to influence peer networks, this dispersal or dilution of peer networks after youth complete the programs presents a significant challenge to sustaining positive change.

Looking Ahead

This evaluation highlights the challenges faced by programs aiming to reduce adolescent sexual activity. Nationally, about half of all high school youth report having had sex, and more than one in five students report having had four or more sexual partners by the time they complete high school. One-quarter of sexually active adolescents nationwide have an STD, and many STDs are lifelong viral infections with no cure. Findings from this study speak to the continued need for rigorous research on how to combat the high rate of teen sexual activity and its negative consequences.

References

Bearman, Peter S., and Hannah Bruckner. "Promising the Future: Virginity Pledges and First Intercourse." American Journal of Sociology, vol. 106, no. 4, January 2001, pp. 859-912.

Bloom, Howard. "Accounting for No-Shows in Experimental Designs." Evaluation Review, vol. 8, no. 2, 1984, pp. 225-246.

Boston University, College of Communication and School of Education. The Art of Loving Well: A Character Education Curriculum for Today's Teenagers. Boston, MA: The Loving Well Project, 1993.

Centers for Disease Control and Prevention. "National Youth Risk Behavior Survey, 1991 2005: Trends in the Prevalence of Sexual Behaviors." Available at [http://www.cdc.gov/HealthyYouth/YRBS/pdf/trends/2005_YRBS_Sexual_Behaviors.pdf]. Accessed August 1, 2006.

Chesson, Harrell W., John M. Blandford, Thomas L. Gift, Guoyu Tao, and Kathleen L. Irwin. "The Estimated Direct Medical Cost of Sexually Transmitted Diseases Among American Youth, 2000." Perspectives on Sexual and Reproductive Health, vol. 36, no. 1, 2004, pp. 11-19.

Clark, Melissa A., and Barbara Devaney. "First-Year Impacts of the Heritage Keepers® Life Skills Education Component." Princeton, NJ: Mathematica Policy Research, Inc., August 2006.

Del Rosario, Jacqueline. ReCapturing the Vision. Miami, FL: Empowerment Concepts, Inc., 2003.

Del Rosario, Jacqueline. Vessels of Honor. Miami, FL: Empowerment Concepts, Inc., 1999.

Devaney, Barbara, Amy Johnson, Rebecca Maynard, and Chris Trenholm. "The Evaluation of Abstinence Education Programs Funded Under Title V Section 510: Interim Report." Princeton, NJ: Mathematica Policy Research, Inc., April 2002.

Duran, M.G. Reasonable Reasons to Wait: The Keys to Character. Chantilly, VA: A Better Choice in Education, 1997.

Howard, M., and M. Mitchell. Postponing Sexual Involvement: An Educational Series for Young Teens. Atlanta, GA: Adolescent Reproductive Health Center, 1990.

Maternal and Child Health Bureau. "1999 Annual Summary for the Abstinence Education Provision of the 1996 Welfare Law, P.L. 104-193, Section 510 of Title V of the Social Security Act." Rockville, MD: Maternal and Child Health Bureau, 2000.

Mathematica Policy Research, Inc. Wave 4 Survey of Teen Activities and Attitudes. Princeton, NJ: MPR, 2005.

Mathematica Policy Research, Inc. Wave 2 Survey of Teen Activities and Attitudes. Princeton, NJ: MPR, 2000.

Mathematica Policy Research, Inc. Wave 1 Survey of Teen Activities and Attitudes. Princeton, NJ: MPR, 1999.

Maynard, Rebecca A., Christopher Trenholm, Barbara Devaney, Amy Johnson, Melissa A. Clark, John Homrighausen, and Ece Kalay. "First-Year Impacts of Four Title V, Section 510 Abstinence Education Programs." Princeton, NJ: Mathematica Policy Research, Inc., June 2005.

Rector, Robert, Kirk A. Johnson, and Jennifer A. Marshall. "Teens Who Make Virginity Pledges Have Substantially Improved Life Outcomes." Washington, DC: The Heritage Foundation, September 2004.

Rosalie Manor. Families United to Prevent Teen Pregnancy. A Life Options Mode Curriculum for Youth. Milwaukee, WI: Rosalie Manor, Inc., undated.

U.S. Department of Health and Human Services. "Abstinence Education Projects: FY 2005/2006." Available at [http://opa.osophs.dhhs.gov/titlexx/ae-project-descriptions-2006.pdf]. Accessed August 2, 2006.

Weinstock, Hillard, Stuart Berman, and Willard Cates, Jr. "Sexually Transmitted Diseases Among American Youth: Incidence and Prevalence Estimates, 2000." Perspectives on Sexual and Reproductive Health, vol. 36, no.1, 2004, pp. 6-10.