Positive Youth Development in the United States: Research Findings on Evaluations of Positive Youth Development Programs. Programs in Three Social Domains: Family + School + Community

11/13/1998

Across Ages

LoSciuto, Rajala, Townsend, and Taylor (1996) evaluated Across Ages, an intervention that addressed 11 positive youth development constructs, including social, emotional, cognitive, and behavioral competencies, bonding, resiliency, self-efficacy, recognition for positive behavior, positive identity, opportunities for prosocial involvement, and prosocial norms.

Program Description

The program's theoretical foundation integrated positive youth development, youth identity development, social problem-solving, and the social development model: strengthening protective factors in the individual, the family, the school, the peer group, and the community/neighborhood was expected to increase resiliency in children.  The goal of Across Ages was to demonstrate the impact of an intergenerational mentoring approach to drug prevention for high risk sixth grade students.  Program components included mentoring for at least two hours twice a week for the school year (bonding, opportunities for prosocial involvement) by adults 55 years old or older; one hour every two weeks of community service activities with the mentor (opportunities for positive involvement); 26 sessions of exposure to the Social Problem-Solving model (competencies) as used by Weissberg and Caplan (1988) in the Positive Youth Development Curriculum; parental involvement and strengthening parent-child bonds by coaching parents in more effective parenting styles (opportunities for prosocial involvement, recognition for positive behavior, bonding, competencies) during Saturday workshops.  The program also focused on developing positive relationships between mentors and parents.

Implementation

Workshops were held for teachers on the use of the Social Problem Solving Model.  Teachers rated students on how much they participated in the positive youth development curriculum and community service sessions, and project staff rated mentors

on the level of involvement with each student.  Parental participation was not assessed due to its being "sporadic" and not providing sufficient information for analysis.

Research Design

The study used an experimental pretest, posttest research design.  At the time of evaluation, data had been collected for three years and there were two years remaining in the project.  Experimental and control group classes were selected randomly from among sixth grade teachers in three schools who had indicated a willingness to participate.  Three classes in each school were selected randomly from the remaining pool of sixth grade classes and assigned to one of three groups, two intervention conditions and a control condition.  One intervention group (PS) received the Positive Youth Development Curriculum, community service and parent workshop components, and the other intervention group (MPS), received those components and mentoring from older adults.  Mentors were carefully recruited, screened, trained and matched with the youth.  The program was successful at retaining mentors, with two-thirds of the mentors at evaluation having been in the program since its inception four years earlier.  Data were collected for the 1991-92, 1992-93 and 1993-94 academic years and combined in this evaluation.  Pre-intervention group equivalence was established.  Attrition rates were similar for all groups (Group C, 23%; Group PS, 22%; and Group MPS, 25%).  For the three evaluation years, a total of 729 students completed the pretest.  The final sample use in the evaluation consisted of 562 students who completed both the pretest and the posttest (77% of those originally pretested).  The sample of 562 children were sixth grade students attending three public middle schools in Philadelphia neighborhoods.  The target population included African American (52.2%), Asian (9.1%), Latino (9.0%) and Caucasian (15.8%) students, 180 of whom were served by Across Ages each year for three years.  The neighborhoods were characterized by poverty, a high incidence of substance abuse and drug related crime, and a significant number of abandoned houses.  In each school, student achievement was low and attendance poor.  Many children were living with grandparents or other relatives (percentages not specified).  Fifty-three percent of students who completed both the pretest and posttest were female.  Approximately equal numbers of students completed both the pretest and posttest in each of the three experimental groups (189 in the control group, 193 in Group PS and 180 in Group MPS).

Results of the Intervention

The evaluation reported significant outcomes using ANCOVAs that compared two groups at a time.  The authors hypothesized that one of the two intervention conditions, the combined mentoring-problem solving (MPS) group, would show more positive outcomes compared with the control group, and also when compared with the other intervention condition.  The reported results showed significant effects for the combined group condition when compared with the control group on a number of measures.  The combined condition also showed several significant improvements compared with the other intervention.  The significant outcomes for the combined MPS group compared with the control included increased positive attitudes (F(1,316) = 4.34, P=.038) on four dimensions (school, the future, elders, and older people); increased knowledge about older people (F(1,313) = 7.04, p=.008); improved reactions to situations involving drug use (F(1,271) = 4.17, p=.042); and higher levels of community service (F(1,208) = 5.10, p=.025).  The combined mentoring-problem solving condition also reported significantly better results compared to the other intervention condition for attitudes toward older people (F(1,316) = 6.36, p=.012).  Having a mentor and participating in the MPS intervention significantly improved school attendance for youth in that intervention condition, compared with both control and the other intervention group (F(2,447) = 4.58, p =.01).  The problem-solving condition significantly improved their knowledge about older people compared with controls (F(1,368) = 5.32, p =.022).  Within-group analyses of the mentoring group on mentor/student bonding measures showed that level of mentor involvement was positively associated with improved school attendance (F(2,138) = 25.03, p = .000).

The Adolescent Transitions Project

Andrews, Soberman and Dishion (1995) evaluated the Adolescent Transitions Project, a program that addressed eight positive youth development constructs including social, cognitive, and behavioral competencies, bonding, self-efficacy, recognition for positive behaviors, opportunities for prosocial involvement, and prosocial norms.

Program Description

The intervention used a parent and youth skills training model designed for substance use and problem behavior prevention.  The evaluation reviewed two deliveries of the program, one in a community mental health setting and one in a school setting, but presented outcome data on only the first delivery.  The main intervention components in both versions were the Parent Focus and Teen Focus conditions.  Intervention groups targeted predictors of problem behavior, including family management practices, communication, limit setting, problem solving, goal setting, and dealing with negative peer influences. Parent consultants were used to assist group leaders.  The Parent Focus curriculum combined three sets of family management skills, including prosocial fostering, limit-setting, and problem-solving.  The Teen Focus component was developed from a Botvin & Wills cognitive-behavioral curriculum created in the mid 1980s.  The curriculum emphasized behavior modeling by employing a peer counselor who had successfully completed the modeling component.  Adolescents learned self-regulation skills including realistic and incremental goal-setting and problem-solving.  The program included 12 sessions of curriculumwith 18 hours of contact total.  The evaluation contained comprehensive implementation data on participation, program retention, parent and teen engagement, and satisfaction with the program.

Research Design

The evaluation used an experimental, pretest posttest design.  A cluster sampling approach was used first and followed by random assignment to conditions. Parents were recruited through newspapers, community flyers, school counselors, and other community professionals.  Following parent inquiry, a telephone screening was conducted using a 10 question instrument based on risk factor research (Bry, McKeon & Pandina, 1982).  If the child was assessed as at risk on at least four dimensions, the family was randomly assigned to one of four intervention conditions: Parent Focus, Teen Focus, self-directed change, and a control group.  Interventions were conducted over two years, with four cohorts of approximately 30 families per cohort.  Each intervention group consisted of approximately seven to eight families.  Boys and girls received assignments separately to assure equal distribution of gender across conditions.  The attrition analysis showed program retention was 90%; this meant 143 of 158 families were represented in the termination assessment. The data revealed no significant differences between families who stayed and those who left.  The sample of 143 families resulted in a youth sample of 83 boys and 75 girls between ages 10 and 14, enrolled in middle schools containing sixth through eighth grades, with the mean grade level for the sample being seventh grade.  Ninety-five percent of the sample was European-American.  Other demographic information was not provided in the evaluation.  Approximately one fourth of the families had an annual income of less than $10,000 and more than half the families received governmental financial assistance.  Approximately 50% of mothers and 45% of fathers had some college education.

Results of the Intervention

Results were assessed through child behavior inventories, audiotaped problem-solving scenarios and videotaped family problem-solving sessions.  Outcomes were reported for participant engagement and satisfaction, skills acquisition, family interaction patterns, and changes in family conflict.  Both the Parent and Teen components were shown to be effective in engaging both parents and teens, teaching the targeted skills, and reducing parent-child conflict.  The Parent component had a short-term effect on reducing teen problem behavior in school.  The Teen Focus condition did not demonstrate a significant effect on the students' problem behavior in school, but youth in the Teen Focus condition demonstrated higher social learning scores (F(1,140) = 5.76, p<.05).  Youths' negative engagement was reduced significantly when parents participated in the Parent Focus intervention, while control group youth increased in negative engagement (F(1,107) = 5.27, p<.01).  The same pattern of effects occurred in the Teen Focus intervention, with a reduction in their negative engagement with parents compared to the inactive intervention conditions (F(1,114) = 4.89, p.  < .05).  Fewer than one half of fathers participated in the videotaped family interaction, and analyses were restricted to mothers' behavior.  Mothers who were exposed to both the Teen (F(1,113) = 6.51, p<.01) and Parent (F(1,106) = 7.1, p<.01) Focus conditions showed less negative engagement in the family interaction, compared to mothers in the control condition who tended to increase their negativity.  Those in the Parent Focus condition reduced conflict levels (F(1,101) = 10.95, p<.001)from pretest to posttest.  There was a decrease in family negative events for families in this intervention condition while negative events increased for those in other conditions.  The Teen Focus group also yielded significant reductions in family conflict (F(1,118) = 8.1, p<.005) compared with the control group, and significantly fewer family negative events (F(1,107) = 6.54, p<.01).  Teacher ratings of child behavior revealed effects specific to the Parent Focus condition.  Teens with parents in this program condition reduced their rate of aggression, while teens in the inactive intervention conditions were rated as more aggressive at posttest (F(1,106) = 5.33, p<.05).

The Midwestern Prevention Project - Project STAR - Kansas (MPP)

Pentz, Dwyer, Johnson, Flay, Hansen, MacKinnon, Chou, Rohrbach and Montgomery (1994), Pentz, et al.  (1989), and Pentz, Trebow, et al.  (1990) evaluated the implementation and five-year follow-up results of the Midwestern Prevention Project - Project STAR - Kansas (MPP).  The multi-year intervention addressed seven positive youth development constructs including social, cognitive, and behavioral competencies, self-efficacy, recognition for positive behaviors, bonding, and prosocial norms.

Program Description

In the 1984-1985 school year, the MPP was initiated in 50 public middle/junior high schools and 15 communities.  The components of this comprehensive intervention included mass media programming, a school-based educational program for students, parent education and organization, community organization and health policy.  The STAR Program consisted of five components implemented over five years at the average rate of one per year while mass media program was used in all years of the intervention.  The school program involved 20 hours of direct contact with students and their parents in the first and second years (10-session school program for resisting and counteracting drug use influences and 10 hours prevention practice homework activities with parents).  Community organization activities created metropolitan task forces against drug use in the third through the fifth years.  Mass media coverage included 16 television, 10 radio and 30 print media events throughout the program delivery period.  Two hours of training were provided to television station managers.  An average of 31 mass media programs per year occurred in the first through the third years, decreasing to 10 per year in the fourth and fifth years.  This entailed news clips, commercials, talk shows, press conferences and articles covering baseline drug use and STAR goals in Kansas City, introduction of each program component, skills demonstration, and public recognition of participating students.  Tobacco and alcohol policy changes occurred in the fourth and fifth years.

Research Design

The evaluation used a quasi-experimental, partial randomized control trial that varied the intervention condition (prevention program or delayed program control with health education as usual) and the school grade of the children's initial intervention (grade six or seven, depending on which represented the transition year from middle or junior high school).  The MPP measurement design consisted of annual assessments of several different youth samples.  The universal population of students was initially selected from 50 schools in 15 communities, but the study design led to only eight out of 50 being randomly assigned (schools were assigned to conditions based on school administrator scheduling flexibility after the start of the school year).  The rest of the assignments used demographic matching and relied on the flexibility of schools to implement the intervention.  Twenty-four were matched and assigned non-randomly to program schools and 18 were matched and assigned non-randomly to control schools.  For measuring longitudinal effects of the intervention on the entire population, two of these samples were merged: a panel consisting of all students from the 1984-1985 entering grade cohort in eight schools who were tracked individually over time (n = 1607) and a 25% sample of students from the 1984-1985 cohort in the remaining 42 schools, who were cross-sectionally and randomly sampled by classroom each year (n = 3771 in 1984-85).  The middle or junior high school was the unit of assignment for the initial intervention.  Of an average of 4664 students targeted for assessment, 94% received annual parental consent for participation.  The final sample at posttest had an ethnic group representation that was 76.6% white, 19.2% African-American, 49.3% female, 60.9% low SES, 22.6% in sixth grade, and 60.8% urban.  At two years into the intervention 63.6% of the sample were enrolled in schools different from the schools of origin.

Implementation

The Midwestern Prevention Project conducted extensive implementation methods and measures.  Teachers were selected to implement the intervention based upon whether they had primary responsibility for teaching health education courses or, in schools where formal health education was not offered, responsibility for teaching courses in which health education matter could be most easily assimilated; and course loads which reached all students in the target grade.  Program teachers received an initial three-day workshop and a one-day refresher workshop on school and booster program implementation, including drug prevention skills and program delivery.  Program teachers provided training to two to four peer leaders per class in a one-hour training session. Training was followed by program staff making monthly phone calls to teachers and periodic meetings with principals.  The parent program was implemented as a series of planning meetings chaired by the principal and attended by two to four parent representatives and two STAR program students.  An annual Parent Skills Night was offered for all parents of STAR program students. Principals, parent representatives and student representatives received an annual one-day workshop on parent program implementation.  In addition to training parents in parent-child communication and prevention practice support skills, multiple methods were used to incorporate parents at an organizational, system-wide level.  Parents were mobilized to change school policies about institutionalizing drug prevention curricula and restricting drug use in and around schools.

Results of the Implementation Analysis

Implementation analyses reported data from parent phone calls and meetings of STAR program staff with principals; observations of program sessions and content analyses of media events by an MPP project archivist; program implementation evaluations completed by teachers (self-report surveys) and parent group members (phone survey interviews); and a consensus rating of overall program implementation quality by three STAR program staff members.  The investigators used three operational definitions for implementation:  adherence (was the intervention only well implemented in the experimental and not the control conditions?), exposure (what was the amount of the intervention that was delivered?), and reinvention (how much does the implementation in question differ from the program standard that is being tested?).  Sixty-five teachers from 27 program schools were trained to implement the program, and in questionnaires administered immediately post training, all teachers reported that they had been "very adequately" or "moderately adequately" prepared to implement the program.  Program implementation effects were estimated for prevalence rates of drug use, with school as the unit of analysis.  To generate school level data on implementation, individual teachers' ratings were averaged within each school.  Teacher reports showed that all twenty-seven schools assigned to the program condition implemented the program during the school year as planned.  Staff reports confirmed implementation by all program schools, and also confirmed that the twenty-three control schools adhered to the control condition design.  The number of sessions implemented by program schools ranged from three to 10 with a mean of 8.76 sessions (2.06 SD).  Average length of time per session was forty minutes. Program exposure ranged from 2.75 to 9.17 hours with a mean of 6.47 (1.74 SD).  Of the teachers, 100% responded that they had not deviated substantially from implementation as designed (32% deviated "not at all" and 68% deviated "slightly").

Results of the Intervention

The program was evaluated as an entire intervention package; however, it should be noted that one of its components, the community organization and mass media coverage, were available to both program and control students and therefore limits the external validity of the program.  Outcome data came from self-report survey of substance use, biochemical measure of smoking, and school records.  Results showed the intervention significantly reduced monthly, weekly, and heavy cigarette, marijuana, and alcohol use through three year follow-up (p<.05).  There was some decay of effect at four year follow-up which corresponded to a decrease in control group use.  By the five year follow-up the increase in the proportion of students reporting use continued to be significantly higher in the control group than the program condition on all of the monthly drug use measures as well as for weekly cigarette use. The exceptions were the effects on daily cigarette use and heavy marijuana use.  Results assessing the impact of varying implementation rates on substance use were highly significant.  Schools with a high level of implementation had little or no increase in rates of weekly use of substances and a decrease in the use of cigarettes in the last month (compared to increases for low implementation and control groups).  Six years after MPP began, the increase in substance use prevalence rates for the 1984 cohort of intervention school students continued to be lower than control schools, with average reductions of 8.4% in monthly use, 5.7% in weekly use, and 4.9% in heavy use for cigarettes, alcohol, and marijuana.

Project Northland

Perry, Williams, Veblen-Mortenson, Toomey, Komro, Anstine, McGovern, Finnegan, Forster, Wagenaar and Wolfson (1996) evaluated Project Northland, a family, community and school-based intervention that addressed eight positive youth development constructs including social, emotional, cognitive and behavioral competencies, bonding, self-efficacy, opportunities for prosocial involvement, and prosocial norms.

Program Description

The intervention was conducted over a three year period, beginning in the sixth grade in fall of 1991 and continuing through seventh and eighth grade.  Although the program's focus was on preventing substance abuse, particularly seeking to influence children's choices about using alcohol and cigarettes, Project Northland used many strategies that simultaneously promoted positive youth development.  Strategies were designed to influence psychosocial factors such as peer influence, self-efficacy, child-parent communication, and perceived ease of access to substances.  The study used a multi-level, multi-component, community-wide approach.  Students received many forms of skills training intended to enhance their competence in dealing with their parents, and with peer pressure and normative expectations about alcohol.  In addition to specific skills development, the intervention addressed community-level changes in alcohol-related programs and policies.  The school component used a social-behavioral curriculum, homework, peer leadership training, parental involvement/education, and community-wide task force activities ("Slick Tracy," "Fun Night," "Amazing Alternatives," "PowerLines").

Research Design

The study employed an experimental, delayed control group design.  Twenty combined school districts were blocked by size (small, medium, large, very large) and randomized to either an intervention or delayed condition (N = 10 in each group).  This was made possible by combining four of an original 24 districts with other districts to generate an adequate sample size in each unit to be randomized.  Annual surveys were used to measure alcohol use, tobacco use and psychosocial factors.  Comparison group schools were allowed to use other programs such as DARE or Project Quest until 1994 when they implemented Project Northland.  In a 1992 survey, over 90% of these students reported having taken part in DARE (40% in intervention districts) and 21% had taken part in Quest (compared to 2% in intervention districts).  The study appropriately matched unit of analysis and assignment (school district).  Tests were performed to assess group equivalence and showed pretest equivalency on a number of key variables.  Group differences showed that, at baseline, more students in the intervention districts reported alcohol use, were slightly older (0.1 years older), and had fewer white students than in the reference districts. Of the 2,351 students measured at baseline, 93% (N = 2,191), 88% (N = 2,060) and 81% (N = 1,901) were surveyed at the end of the sixth, seventh and eighth grades respectively.  The attrition analysis showed that of 450 (19%) lost to follow-up at the end of the eighth grade, 231 (51.3%) were in the intervention condition and 219 (48.7%) were in the control condition.  No significant differences were noted in baseline alcohol use between those who were lost to follow-up in the intervention vs the control conditions, or between those who were lost to follow-up and those who remained..  Of the 450, 278 (62%) had moved, 31 (7%) were absent, 42 (9%) had moved to a Project Northland school in a different treatment condition, 87 (19%) refused or were not allowed by their parents to participate and 12 (3%) were deleted from the analyses due to three or more inconsistent responses.  The final sample consisted of 1901 students in grades six to eight from 20 schools in northeastern Minnesota. Students belonged to two ethnic groups, with 94% Caucasian and 4.5% Native American.

Implementation

Implementation was extensively addressed and measured.  Teachers and peer leaders received training before the program was implemented.  Implementation was measured through exposure and participation indices completed by parents and teachers.

Results of the Intervention

The primary effects of the intervention were measured with self report questionnaires given to students and parents.  Other forms of measurement included observations of alcohol purchase attempts by underage buyers, telephone surveys of alcohol merchants, and interviews with community leaders.  However, only the self-report questionnaires were included in the evaluation.  The analyses were based on comparisons among all students, and among baseline non-users and baseline users.

For all students in the intervention districts, there were significantly lower scores on the alcohol scale by the end of eighth grade than control district students (16.0 vs 17.5, p<.05, on a 8-48 scale where 8 was no tendency to use, and 48 was high levels of use).  The scale score was also significantly lower among baseline non-users in the intervention districts compared with the control districts (13.8 vs 15.3, p<.01).  For all students, the percentages who had lower alcohol use in the past month (23.6% vs 29.2%, p<.05) and past week (10.5% vs 14.8%, p<.05) were significantly better in the intervention districts compared with the control districts.  In the intervention districts, there were lower onset rates for baseline non-users in the past month (15.3% vs 21.2%, p<.05), and the past week (5.3% vs. 9.8%, p<.01) by end of eighth grade, and lower past year use for baseline non-users at the end of seventh (21.1% vs 29.1%, p<.05) and eighth (30.4% vs 41.6%, p<.006) grades.  Non-users had lower rates of marijuana use (3.1% vs 6.2%, p<.05) and cigarette use (15.5% vs 20.7%, p<.05) in the intervention districts.  There were no significant differences between conditions for all students in cigarette use, smokeless tobacco, or marijuana use.  Positive changes for alcohol-related knowledge and attitudes were linked to resisting peer influence.  Among all students, those in the intervention districts had significantly lower scores on peer influence by the end of eighth grade (24.6% vs 27.0%, p<.05).  Intervention students were significantly more likely to report being able to resist alcohol in social settings even though the self-efficacy scale showed no significant differences between groups.  Among baseline non-users, students in intervention districts had significantly lower scores by eighth grade on peer influence (22.8% vs. 25.4%, p<.05) and higher scores on self-efficacy (21.6% vs 20.4%, p<.05).  By the end of sixth grade, intervention district students were significantly more likely to report that their parents talked with them about drinking related problems.  Among all remaining psychosocial variables for all students, only one finding was significant, the greater likelihood of being disciplined by school for consequences of driving after drinking (2.5% vs 2.2%, p<.001).  Baseline non-users in intervention districts at the end of eighth grade were more likely to say they had influence in their communities on alcohol-related issues than baseline non-users in the control condition.

The Richmond Youth Against Violence Project/Responding in Peaceful and Positive Ways (R.I.P.P.)

Two evaluations of The Richmond Youth Against Violence Project (Farrell & Meyer, 1998, 1997) assessed the effectiveness of a multiple-domain, school-based intervention called "Responding in Peaceful and Positive Ways" (R.I.P.P.).  The intervention addressed 10 positive youth development constructs, including social, emotional, cognitive, behavioral and moral competencies, bonding, prosocial norms, self-efficacy, opportunities for prosocial involvement, and recognition for positive behavior.

Program Description

This review analyzed two evaluations of the R.I.P.P.  curriculum, one using 1993-94 data and the other, 1995-96 data.  The intervention described in the evaluation of 1995-96 outcome data represented an expanded 25-session version of an earlier 16-18 session program originally developed from Prothrow-Stith's (1987) violence prevention model and concepts from "The Friendly Classroom for a Small Planet" (Children's Creative Response to Conflict, 1988).  R.I.P.P. is an ongoing, multi-year intervention that the program authors described as a "developmentally-anchored health promotion model" (1996:13).  The initiative of which R.I.P.P.  is part began in 1991 as a broad collaborative effort between the Richmond Public Schools, the Richmond Behavioral Health Authority, and Virginia Commonwealth University.  In 1993 this collaboration was expanded through a cooperative agreement with the National Center for Injury Prevention and Control within the Centers for Disease Control and Prevention.

The expanded 25-session curriculum was based on the results of an evaluation of the earlier 18-session model.  The expanded curriculum used standardized manuals and sought to address gender differences at the level of curriculum development and personnel training; to increase intensity in order to produce a stronger effect; to have clearly stated objectives; to have a firm base in research and theory about adolescent violence; to be sufficiently standardized in order to minimize differences in interpretation of the program's content; and to emphasize how students can use the skills they learn in the program outside of school and throughout their lives.  The components included adult role modeling (prosocial norms, bonding), peer mediation (opportunities, bonding and competence), team-building activities (bonding, competence, and opportunities), relaxation techniques (emotional competence), small group work (bonding and opportunities), role plays (opportunities and competence), and cognitive restructuring methods such as mental rehearsal.  The program also provided staff development and parent training in non-violence and conflict resolution methods.  Students worked with a trained prevention specialist who implemented the R.I.P.P.  curriculum once a week, modeled and reinforced appropriate non-violent behavior in the schools, promoted a caring community of students and adults, and supported prosocial norms and expectations.  The specialist also implemented a school-wide peer mediation program available to all students at each school.

Research Design

The intervention used an experimental design conducted at three middle schools in Richmond during the 1995/1996 school year.  The intervention was implemented with half the sixth grade students at each participating school and in each school the classes were randomly assigned to either the intervention or control groups.  The initial sample included 295 intervention and 307 control group members.  Data from 23 intervention students were removed when they were determined to have missed more than a third of the sessions.  Analyses showed that these students had significantly higher rates of suspensions, lower grade point averages and attendance, and higher frequencies of violent behavior. The final sample of 579 sixth grade students at three urban middle schools consisted of even numbers of boys and girls in the intervention (n=135 boys, n=137 girls) and control (n=154 boys, n=153 girls) groups.  Students ranged in age from 10 to 15, and 96% were African-American.  There were no significant differences between the intervention and control groups on ethnicity, gender or age.  Complete data were ultimately available on 455 students, and analyses compared the 124 students with incomplete data.  Although there were no gender or ethnicity differences, students with incomplete data were significantly older, had lower grade point averages and school attendance, had more violations for fighting and weapons, and more suspensions.

Short Term Results of the Intervention

Data included school code violations for fighting, assaults, weapons in school, and school suspensions, and self-report measures of self-restraint, nonviolent responses, violent behavior, and attitudes supporting violence.  All weapons violations except one occurred in one school, so this school alone was used in that analysis.  Results at immediate posttests showed that program participants had significantly fewer disciplinary code violations in the last quarter of the year for fighting (2.2% vs 5.2%, p<.05) and carrying weapons (1.9% vs 7.4%, p<.04), and lower rates of in-school suspensions (1.5% vs 5.5%, p<.01) compared to the control group.  Program participants also improved their knowledge of the intervention material (F(1,299 = 56.66, p<.001), used their school's mediation program more frequently (42% vs 30%, p<.05), and reported significant reductions in fight-related injuries relative to the control group (4% vs 9%).

Preliminary Six-Month Follow-Up Results

Unpublished data (Farrell, 1998) for all three time points (pretest, posttest and follow-up) were available for 353 students (R.I.P.P.  n=169; control n=184). Significant effects for posttest to 6-month follow-up changes were found for the knowledge test, the violent behavior frequency scale, the suppression of anger scale, and the frequency of threatening a teacher.  Several gender-specific effects were also found.  For boys there were significant effects for improved impulse control, frequency of drug use and an item that asked students how frequently they skipped school due to concerns for their safety.  For girls there was a significant positive effect on the problem situation inventory.  School disciplinary data at follow-up also showed significant sustained effects, including fewer in-school (4.8% vs 11.7%) and out-of-school suspensions (16.5% vs 21.3%) compared to the control group.

The Valued Youth Partnership Program

Cardenas, Montecel, Supik, and Harris (1992) evaluated the impact of the Valued Youth Partnership Program, which addressed 11 positive youth development constructs, including social, emotional, cognitive and behavioral competencies, bonding, self-efficacy, recognition for positive behavior, positive identity, opportunities for prosocial involvement, belief in the future, and prosocial norms.

Program Description

Comprehensive positive youth development was at the core of the program philosophy, which emphasized the integration of individual and social environmental strategies to help transform a child's self-concept (positive identity).  The components of the program included bilingual instruction for limited-English proficient students; a cross-age tutoring component (bonding, competence); classroom enhancement activities; school-business partnerships (belief in the future); increased student recognition of accomplishments and talents (recognition, positive identity); parental involvement in school activities; staff development; leadership models; and self-paced and individualized instruction curriculum.  Tutors received classes once a week to develop their tutoring skills, engaged in tutoring at least four hours per week, participated in at least two annual field trips, and attended various presentations by role models.  Tutoring groups were generally conducted at a one to three (tutor to learner) ratio.  Although specific implementation varied by site, critical elements were identified that all sites adhered to, including weekly classes for tutors with a minimum of 30 sessions per school year, a minimum age and grade difference of three years between tutor and tutee, provision of a stipend, and a flexible curriculum based on students' tutoring and academic needs.  The specific goals of the program were to reduce dropout rates, enhance students' basic academic skills, strengthen students' positive perceptions of self and school, decrease student truancy, reduce student disciplinary referrals and form school-home-community partnerships to increase the level of social and emotional support.

Research Design

The evaluation used a quasi-experimental design in which a pool of students was identified based upon being limited-English-proficient as defined by the State of Texas guidelines and reading below grade level on a standardized achievement test.  The tutoring group was selected from a pool first based upon scheduling and availability, and then the comparison group was randomly selected from the remaining pool of at-risk students.  The sample of 194 participants was drawn from a largely Hispanic population of at-risk limited-English-proficient middle school students on four campuses in two public school districts in San Antonio, Texas.  A total of 101 secondary school tutors and 93 comparison group students, average age 12 years old, participated in the program.  The ethnic group composition reported for tutors was 61% Hispanic and 2% Caucasian (remaining percentages and ethnicities not reported); for the control group, 69% Hispanic and 0% Caucasian (remaining percentages and ethnicities not reported).  While 33% of the tutors reported having been retained a grade at baseline, 42% of the control group had.  Baseline data showed no significant differences between tutors and comparison group on age, average grade in reading, quality of school life and self-concept scores, ethnicity or retention.  The only statistically significant difference between the two groups at baseline was eligibility for school-lunch program with the tutor group having lower socioeconomic status than the comparison group. At posttest, only 63 of 101 tutors and 70 of 93 control students had data for reading, self-concept and quality of school life.

Implementation

Implementation teams were given clear roles and customized guidelines for each implementation team member.  The evaluation component of the program had a structured implementation component to measure program operations and develop corrective action as needed.

Results of the Intervention

Posttest data were collected on student grades, disciplinary action referrals, absenteeism, self-concept, and quality of school life.  The Valued Youth Partnership study showed significant impacts for tutors, particularly in reducing dropout and improving reading grades.  Results of the reading grades data were analyzed only for those students who had data for reading, self-concept and quality of school life (63 tutors and 70 comparison group students). Being in the tutor group led to significantly higher reading grades after the first year, which continued in the second year: the tutors scored on average nearly three points higher than the comparison group (p<.05).  Tutors' self-concept (p<.05) and attitudes toward school (p<.01) also improved, but their gains were seen only in the first year, leveling off during the second year.  The drop-out rate decreased, with one tutor out of 101 (1%) of the tutors having dropped out by the end of the second year of the program compared to 11 of the 93 comparison group students (12%) (p value not provided).

Reported Cost Factors

The study included information on costs: the general budget for a program of 25 tutors and 75 tutored students was approximately $25,000 plus transportation and per diem, or $250 per student served.  Half of this budget went to tutor stipends and the rest to evaluation ($3000-5000) and training/technical assistance ($5000).

The Woodrock Youth Development Project (WYDP)

LoSciuto, Freeman, Harrington, Altman and Lanphear (1997) evaluated the Woodrock Youth Development Project (WYDP), an intervention which addressed eleven positive youth development constructs, including social, emotional, cognitive and behavioral competencies, bonding, resiliency, self-efficacy, recognition for positive behaviors, prosocial norms, positive identity, and opportunities for prosocial involvement.

Program Description

The program is based on Problem Behavior Theory and Social Inoculation Theory, and emphasizes life skills and social competence training while also promoting an anti-drug message and providing broad systems support across all three domains.  Intervention components included human relations classes, peer mentoring, extracurricular school activities, and structured interactions between students and teachers, and children and parents.  Human relations classes were designed to promote positive self-perceptions (self-efficacy, positive identity) , to raise awareness of the dangers of alcohol, tobacco, and drug use and promote healthy attitudes about not using substances (prosocial norms), and to aid in the development of an appreciation of other ethnic and cultural traditions (positive identity).  Peer mentoring involved high school students as mentors who tutored younger students and actively engaged them in individualized projects designed to help develop talents and critical thinking skills (competencies, bonding, opportunities for prosocial involvement, recognition for positive behavior).  Extracurricular activities included clubs and weekend retreats designed to improve academic performance, provide creative outlets, widen horizons and provide a "space for students to develop positive interpersonal relations" (opportunities for prosocial involvement, competencies, bonding).  The school domain involved having youth advocates meet regularly with teachers to help monitor progress and develop goals (opportunities for involvement, recognition for positive behavior).  The family domain involved support through home visits designed to build communication and establish parental trust in the program; and parenting classes covering such topics as communication, parent and school relations, and planning summer activities for children (opportunities for prosocial involvement, recognition for positive behavior, competencies).  Although the community domain component was not extensively discussed in the evaluation, program materials documented the strong anti-drug norms (prosocial norms) that the Woodrock program seeks to promote in the Kensington community of Philadelphia.

Research Design

The experimental design was a randomized pretest, posttest control-group design.  Classrooms within four Philadelphia schools were randomly assigned to program or control conditions.  Questionnaires were administered in November, 1994, and again in June, 1995, to students from ages six through nine (n=170); and students from ages 10 through 14 (n=197).  Attrition analyses indicated that there were no gender differences or differences in baseline levels of outcome measures between those who dropped out and those who remained.  There was a significant difference in age, with older students being significantly more likely to leave.  Dropouts were most likely to be in the program group for the older sample and in the control group for the younger sample.  The sample of 367 students (130 experimental; 237 control) remained after 19% of the original sample of 453 students from ages six through 14 (161 experimental; 292 control) was lost to attrition.  Participants in the final measurement sample were 46.9% female, 44.4% Latino, 19.9% Caucasian, 11.4% African American, 11.2% Asian, 9.3% mixed or "other" ethnic identity and 1.9% Native American.

Results of the Intervention

The evaluation reported outcomes for two subsamples of different age participants in the Woodrock program.  More outcome data have been collected on the total sample and these will be published soon.  Our analysis only had the published report on the two subsamples available.  In the published evaluation, the authors reported significant positive differences between the intervention and control groups for the younger sample, and mixed results for the older sample, with one statistically significant (non-behavioral) outcome in the wrong direction.

Younger Group.  These children were ages six to nine.  For three of the five dependent variables (outcomes as a function of the intervention) the results showed statistically significant improvements for intervention group compared to the control group.  The intervention group showed a significant change on two drug related measures: use in the last year (F(1,165) = 4.75, p<.05) and in the last month (F(1,166.) = 11.70, p<.001).  There was also significant positive change for younger students' relationships with and perceptions of students of races other than their own (F(1,166) = 4.02, p<.05).

Older Group.  For the older group, there was one positive, statistically significant behavior change for the intervention group compared with the control group: reduced levels of drug use in the last month (F(1,193) = 8.86, p<.003).  There were no significant differences for drug use in the last year, self esteem or attitudes about race relations.  The finding in the wrong direction was on a scale measuring attitudes toward drug use, which for the intervention group participants of the older subsample significantly worsened compared with the control group (F(1,193) = 10.12, p=.002).