by Julia H. Littell and John R. Schuerman
Westat, Inc., in association with James Bell Associates, and the Chapin Hall Center for Children at the University of Chicago.
A part of the National Evaluation of Family Preservation Servcies
For the Office of the Assistant Secretary for Planning and Evaluation
Department of Health and Human Services
Prevention of Subsequent Child Maltreatment
Program Effects on Child and Family Functioning
Results of Controlled Studies
Correlates of Reunification and Reentry
TABLES 1, 2, 3
TABLE 1: SUMMARY OF NON-EXPERIMENTAL STUDIES OF FAMILY PRESERVATION PROGRAMS
TABLE 2: SUMMARY OF CONTROLLED STUDIES OF FAMILY PRESERVATION PROGRAMS
TABLE 3: SUMMARY OF CONTROLLED STUDIES OF FAMILY REUNIFICATION PROGRAMS
How effective are current efforts to preserve and reunify families in child welfare? In this paper we review research on programs aimed at preventing out-of-home placement of children, broader family preservation programs, and programs designed to reunify families with children in foster care.<1> We examine what is known about the outcomes of these programs, relationships between service characteristics and outcomes, and the response of subgroups of clients to services.
Claims that family preservation programs result in substantial reductions in the placement of children are based largely on non-experimental studies. Such studies do not provide solid evidence of program effects. Evidence from controlled studies of placement prevention effects is much weaker. The results of controlled studies suggest that difficulties in targeting services to families at risk of placement contribute to the lack of effects on likelihood of placement. The small amount of evidence on outcomes other than placement suggests that these programs have little effect on the recurrence of child maltreatment, although they may produce modest, short term improvements in some aspects of child and family functioning.
Research on family reunification programs is in its infancy and there are very few controlled studies in this area. Available evidence is mixed. While some studies suggest that intensive, in- home services can speed the process of family reunification, the long-term effects of these programs are largely unknown. In particular, it is not clear whether intensive service programs increase the rates at which children return home, reduce the risk of foster care reentry, or lessen the chance of subsequent child maltreatment.
We conclude our review with a discussion of directions for further research in this area.
As indicated in the companion paper on current family preservation programs, one of the most striking features of these efforts is their diversity. They vary on a number of dimensions, including the extent to which the focus is on placement prevention versus other goals, such as the improvement of family functioning.<2> There is also variation in the intensity and duration of services provided to families and in adherence to various "models" of family preservation.<3> One criticism of the research in this area is that it has not adequately encompassed this diversity. In this section, we review research on the effects of intensive, in-home services programs in which placement prevention was either the primary goal or one of several objectives. Since the central concern of family preservation programs has been the prevention of placement, this has been a major focus of evaluations. We begin with an examination of what is known about the placement prevention effects of programs.<4>
Many early evaluations of programs designed to prevent placement used non-experimental designs in which groups receiving these services were followed without comparing them to other groups or in which nonequivalent comparison groups were used.<5> The studies of only groups receiving services appear to have had implicit "phantom" nontreatment control groups in which it was assumed that nearly all children would be taken into custody.<6> Such an assumption has been proven false in subsequent research.
The results of studies without control groups suggested that most families remain intact during and shortly after family preservation services. An early study of the Homebuilders' model (Kinney, Madsen, Fleming, and Haapala 1977) found that 97 percent of 80 families remained intact three months after the intervention had ended. Since then, evaluations of the Homebuilders program have found that 73 to 91 percent of families were intact at 12 months after referral for service (Kinney, Haapala, and Booth 1991). Studies of other programs have found that at least two-thirds of families remain together within a year after the end of services. For example, 66 percent of 747 families who received family preservation services in Iowa remained intact one year after termination (Thieman, Fuqua, and Linnan 1990). A study of family preservation services in Connecticut found that 69 percent of 591 families remained intact one year after services and 82 percent of the 1,588 children in these families were not placed during this period (Wheeler, Reuter, Struckman-Johnson, and Yuan 1993). Eighty-eight percent of 367 families in the In-home Family Care Program in northern California were intact one year after services ended (Berry 1992). Table 1 provides a summary of some other recent nonexperimental studies of placement prevention efforts.<7>
The Families First program in Michigan has received a great deal of attention because of its claimed success in preventing placement. An evaluation (Bergquist, Szwejda, and Pope 1993) compared 225 children referred to the program (thought to be at "imminent risk of placement") with a matched group of 225 children who had recently exited foster care.<8> It was found that 76 percent of the children in the Families First group remained in their homes at 12 months after the intervention while 65 percent of children in the comparison group remained in their homes for 12 months after they had returned from foster care. However, these groups cannot be considered to be comparable. Children in families entering a family preservation program and those recently discharged from foster care cannot be assumed to be similar in their likelihood of future placement.<9>
Additional claims for the effectiveness of the Michigan program, made by the state, are based on a decrease in the number of children placed in foster care in 1992, four years after the initiation of the Families First program.<10> However, changes in foster care rates over a few years do not provide evidence of the effects of family preservation programs because such rates are affected by many other factors. In many jurisdictions, foster care caseloads have increased despite the presence of family preservation services; it is possible that these increases would have been greater in the absence of family preservation efforts. Alternatively, intensive in-home services may actually contribute to the rise in foster care rates because these services involve more extensive scrutiny of child rearing practices than occurs in their absence (Schuerman, Rzepnicki, and Littell 1994).
These findings have been used to suggest that family preservation programs reduce the need for out-of-home placement of children.<11> However, nonexperimental studies such as these do not provide convincing evidence of program effects, since it is not clear that families would have experienced placement of children in the absence of these services. Claims that children were at "imminent risk of placement" at the time of referral have not been supported by evidence. Referring workers may assert that placement is imminent in order to obtain intensive services for families.<12>
Overflow designs, in which a comparison group is composed of cases not served because programs are full, provide information about effects that is somewhat better than single group or non- comparable group designs. We review four such studies here.
FamiliesFirst in Davis, California, was an intensive, in- home service program based on the Homebuilders model (Wood, Barton, and Schroeder 1988). Families were referred to the project by child protective services staff. Eligible families had children who had been abused or neglected and were thought to be at risk of having at least one child placed out of the home. An overflow comparison study was conducted. One year after intake, 25 percent (15) of the 59 children in the in-home services group were placed compared with 53 percent (26) of 49 children in the comparison group (a statistically significant difference).
Family Preservation Services in Hennepin County, consisted of intensive home-based services delivered by eight "specially trained social workers" (Schwartz and AuClaire 1989; Schwartz, AuClaire, and Harris 1991). The service was intended to last for four weeks. The evaluation of this program involved a non-random comparison group. There were 58 cases in each group, selected during the period August through December 1985. Three of the experimental group cases were in placement during the entire follow-up period and were excluded from outcome analyses. Follow-up extended until December 31, 1986. Placement occurred in 56% of 55 experimental cases and 91% of the 58 comparison cases (a significant difference). Fifty-five percent of cases in the family preservation group and 64 percent of those in the comparison group experienced multiple placements.
The Bronx Homebuilders Program, modeled after Homebuilders, began accepting clients in May 1987 (Mitchell, Tovar, and Knitzer 1989). Cases were referred from two sources, the city Child Welfare Administration (CWA) and the Pius XII Court Designated Assessment Service (Pius). The average length of service was 35 days. A one year follow-up of 45 families referred in the first year was conducted. An overflow comparison group of 12 families was available for the Pius group, one of which was not followed up. Families in the overflow group had relatively fewer placements than those in the service group. At three months, 19 percent (4 of 21) CWA, 23 percent (5 of 22) Pius treatment, and 9 percent (1 of 11) Pius comparison families had experienced a placement. At 12 months, 24 percent (5) of the CWA, 27 percent (6) of the Pius treatment, and 18 percent (2) of the Pius comparison families had experienced placement. Apparently, all children who were placed were still in placement at the end of the follow-up period.
The Family-Based Intensive Treatment (FIT) Study concerned intensive home-based services based on the Homebuilders model in Utah and Washington State (Pecora, Fraser, and Haapala 1992). In Utah a 60-day service model was provided in two sites by the state child welfare department while in Washington a 30-day service was provided in four sites by Homebuilders (under contract with the state agency). The criteria for referral were risk of imminent placement, safety of the child with service, and willingness of at least one parent to cooperate with service. At termination, 9 percent of 172 Utah children and 6 percent of 409 Washington children in the treatment groups had been placed. At a 12 month follow-up, 41 percent of 97 Utah children and 30 percent of 245 Washington children had been placed. In an overflow comparison group of 27 Utah children, 85 percent were placed during the 12 month follow-up period.<13>
Unfortunately, 54 percent of the cases served in the Washington project during the study period did not participate in the study.<14> In addition, 32 percent of the cases in the overflow comparison group were not tracked. The researchers attempted to deal with this problem by matching a subsample of program cases with cases in the overflow comparison group; however, the variables used in the matching design were only weakly related to placement and the number of matched pairs was quite small.
Missing data for the overflow comparison group seriously compromises the interpretation of differences. If few of the unstudied overflow cases were placed, between-group differences in placement rates would have been much smaller than the differences observed.<15>
In the studies reviewed so far, rates of placement in the groups provided family preservation services were quite low. However, we cannot conclude from these results that the services were the cause of the low rates of placement. The reason for this is that we cannot be sure what would have happened to these cases in the absence of services. To determine this, we need comparison groups that are as similar as possible to the groups provided services. While the overflow studies did incorporate comparison groups, the overflow groups were often quite small and we cannot be certain that they were similar to treatment groups at the outset. The best assurance of initially equivalent comparison groups is to randomly assign cases that are referred for services to treatment and control groups.
Below we review the results of controlled studies of family preservation programs; these studies are described in greater detail in Table 2.<16> Early studies (those conducted in the late 1970s and early 1980s) involved smaller groups of clients than more recent evaluations.
The New York State Preventive Services Demonstration Project, conducted in the mid-1970s, foreshadowed later family preservation programs. It provided intensive services to families over approximately 14 months (Jones, Neuman, and Shyne 1976). During the Spring and Summer of 1974, the project served cases in which placement was thought to be imminent, families with children in placement, and those in which children had recently been returned home. The goals of the project were to prevent placement, reunify families, and prevent reentry into foster care. Here we focus on the subgroup of families in which children were living at home at the time of referral (the effort to reunify families with children in foster care is discussed below). Families of 525 children were randomly assigned to the program or a control group. At the end of treatment, placement rates were significantly lower in the experimental group than in the control group (7% versus 18%). Six months after the termination of services 8 percent of children in the program group and 23 percent of those in the control group had been placed (Jones, Neuman, and Shyne 1976). A follow-up study of a subsample of 243 children in the experiment was conducted five years after the project ended. At that time, 34 percent of the children in the experimental group and 46 percent of those in the control group had been placed in foster care, a statistically significant difference (Jones 1985). Thus, the program appears to have had beneficial effects on placement, although the differences between the experimental and control groups were not large, and determination of long term effects is quite problematic because of sample loss at the time of the five-year follow-up (less than 50% were followed).
Special Services for Children, a public agency in New York City, provided intensive services to families with children "at risk of placement." A randomized experiment involved 120 families with 282 children (Halper and Jones 1981, reviewed in Stein 1985). Four percent (6) of the 156 children in the experimental group and 17 percent (22) of 126 in the control group were placed in substitute care during the project (a statistically significant difference).
The Hudson County (New Jersey) Special Service Project in the late 1970s served families whose children were thought to be at "risk of placement within the next two years" (Magura 1981, Stein 1985). Ninety families were randomly assigned to program and control groups. At the end of the three-year demonstration project, 24 percent (11) of families in the program and 18 percent (8) of those in the control group experienced placement of a child (a non-significant difference) (Willems and DeRubeis 1981).
Nebraska Intensive Services to Families at Risk served families at risk of placement because of actual or suspected child maltreatment (Nebraska Department of Public Welfare 1981, reviewed in Stein 1985). One hundred and fifty-three families were randomly assigned to experimental or control groups. Experimental cases were more likely to be placed with relatives and friends than control cases which required more public foster care. Although the exact number of children placed is not known,<17> available data indicate that 4 percent (3) of 80 families in the experimental group and 11 percent (8) of 73 families in the control group had one or more children placed in out-of-home care (Stein 1985), a non-significant difference.
The Home Based Services Demonstration Project of the Ramsey County, Minnesota (St. Paul) child protective services department (Lyle and Nelson 1983) involved random assignment of 74 families to an experimental, family-centered, home-based unit or one of three traditional child protection units (Frankel 1988). Three months after services ended, 33 percent of families in the experimental group had experienced placement of one or more children, compared with 55 percent of families in the control group. Of the children who were placed, those in the experimental group spent significantly less time in substitute care (Frankel 1988).
The Family Study Project in Hennepin County, Minnesota (Minneapolis) involved random assignment of 138 cases to experimental and control units of the county agency (Hennepin County Community Services Department 1980, reviewed in Stein 1985). The families served had children under age 15 who "were at risk of placement, but who were judged by intake workers not to be at imminent risk of abuse or neglect" (Stein 1985, p. 116). The experimental group had a higher number of children placed in foster care (123 versus 84 children in the control group); however, the total number of children in each group was not reported (Stein 1985). Of those placed, children in the experimental group spent slightly fewer days in placement (mean of 199 days) than those in the control group (mean of 208 days).
A Social Learning Treatment Program in Oregon, involved parents with children between the ages of 3 and 12 who were considered at risk of placement because of child abuse and neglect. A randomized experiment compared the experimental services with regular child protective services (Szykula and Fleischman (1985). The experiment included families of 48 children.<18> Cases were identified as more or less difficult by workers, based on numbers of prior abuse reports and types of family problems.<19> Cases within each difficulty group were randomly assigned to program or control services. The experimental program appeared to reduce the risk of placement among less difficult cases: 8 percent (1 of 13) of the children in the less difficult experimental group and 38 percent (5 of 13) of those in the comparable control group were placed. However, there was no significant difference between program and control groups in placement rates for more difficult cases: 64 percent (7 of 11) of children in the more difficult experiment group versus 45 percent (5 of 11) in the control group. The overall effect of the program (for both groups) was not significant.
The results of early experimental studies of family preservation programs were mixed: some found little or no effects on placement while others found that the programs achieved slight reductions in placement. However, in all studies, relatively few control group families experienced placement. This means that services were generally not delivered to the target group of families at risk of placement.
California's AB 1562 In-home Care Demonstration Project, in operation in eight counties from 1986 to 1989, was an intensive, in-home services program. Cases thought to involve "imminent risk of placement" due to abuse or neglect were referred by county child protective services offices.<20> Families were served for an average of 7 weeks in programs conducted in eight sites by seven private agencies and one public mental health agency. Data were collected on 709 (96%) of the 741 families served by these programs over a three year period (Yuan, McDonald, Wheeler, Struckman-Johnson, and Rivest 1990).
A sub-study in five of the eight counties involved the random assignment of cases to in-home services or to regular services of the county child welfare agencies. There were 152 families in each group. Cases were followed for 8 months after random assignment. Outcome data were available for 293 (96%) of the cases in the randomized experiment. In 20 percent of the control group families and 25 percent of the experimental group families a placement occurred between two and 8 months after referral--a non-significant difference.<21> (A similar proportion of the entire group of 709 families experienced placement in the study period.) There were no substantial differences in lengths of time in placement and costs of placement.<22>
New Jersey Family Preservation Services (FPS), modeled after Homebuilders, provided services for a median of 6 weeks. Private agencies in five counties served families referred by local child welfare offices, county family court or crisis intervention units, and regional community mental health centers. The FPS programs served "several waves" of families before a randomized experiment was instituted by the state. Data are available on 117 experimental and 97 control cases that were randomly assigned in four of New Jersey's 21 counties (Feldman 1991). Another 33 families were "turned back" after random assignment to the experimental services (because they did not meet selection criteria, the caretaker refused to participate in the program, or the children were deemed at imminent risk of harm and were removed from the home); these cases were not included in the analysis. The exclusion of 22 percent of the cases assigned to the experimental group seriously compromises comparisons of the experimental and control groups. These cases are clearly different from those that remained in the experiment and comparable cases were not excluded from the control group. Since these cases are likely to have experienced placement, the observed placement rate in the experimental group is probably understated.
During the intervention period (approximately 6 weeks) 6 percent of families in the experimental group and 17 percent of the families in the control group experienced placement of at least one target child. At 6 months post-termination, 27 percent of families in the experimental group and 50 percent of control group families had experienced at least one placement.<23> At one year post-termination 43 percent of those in the experimental group and 57 percent of families in the control group had experienced placement. (Differences between groups were statistically significant at each point in time.)<24> There is some evidence that the program delayed placement but the magnitude of this effect dissipated over time. For the first target child to enter placement in each family, there were no significant differences between the experimental and control groups in types of placements,<25> numbers of placements, or duration of time in placement. We report below findings on measures of family functioning.
In the Family Support Project in Los Angeles families were referred by the county Department of Children's Services to two private child welfare agencies for in-home family support services. Referrals were based on "caseworker judgment about need for the services" and were not limited to cases in which children were thought to be at imminent risk of placement (Meezan and McCroskey 1993).<26> An evaluation involved random assignment of 240 families to in-home services or regular child protective services. Data on placements were available for 231 families. At the beginning of the project 37 (34%) of the 108 families in the in-home services group and 30 (24%) of 123 families in the control group had one or more children in placement. During the project, 19 (6%) of the 335 children in the experimental group were placed, compared with 34 (8%) of 424 children in the comparison group. At the end of the project (12 months after services ended), families in the experimental group had more children in out-of-home placements than those in the comparison group (38% versus 24%) (McCroskey and Meezan 1993). Below we report the study's findings regarding program effects on family functioning.
The Illinois Family First Experiment is the largest randomized experiment conducted in this area to date (see Schuerman, Rzepnicki, and Littell 1994). The primary goal of the Family First program was to prevent placements among families in which a child had been abused or neglected; other goals included reducing the risk of subsequent child maltreatment, improving child and family functioning, linking families to other community services, and closing cases in the child welfare system. Initially, families referred to the project were thought to be at imminent risk of placement. Family preservation services were provided by sixty private agencies under contract with the state. Data were collected on 6,522 families referred to the program between December 1988 and December 1992. Between April 1990 and April 1992, families in six sites (containing 18 Family First programs) were randomly assigned to intensive family preservation services or regular child welfare services.<27> (A seventh site was dropped from the experiment because 20% of the case assignments in that site were violated.) A total of 1,564 families participated in the experiment (995 were assigned to Family First and 569 to the control group). These cases were followed through March 1993.
Family First workers carried caseloads of 5 families on average, compared to average caseloads of 50 for workers who provided services to families in the control group. Overall, cases in the Family First program received far more intensive contact than those in the control group. Family First cases were much more likely to receive counseling, crisis intervention, advocacy, parent education, referrals for medical and specialized services, and an array of concrete services including transportation, material aid, and cash assistance. One-fifth of the cases in the control group were never opened for services in the state child welfare agency and 51 percent of those that were opened received no services of any kind during the first 90 days after random assignment. Interviews with a subsample of 278 parents in the program and control groups in three experimental sites were conducted to obtain longitudinal data on child and family functioning, parents' views of the services they received, major life events, social support, and further service utilization. These interview data support the conclusion that FPS cases received much more extensive help than cases in the control group.
Overall, the Family First program appeared to result in a slight increase in the risk of placement. At one year after random assignment, placement had occurred in approximately 27 percent of Family First cases and 21 percent of control cases. In the two experimental sites in the Chicago area, increases in the risk of placement for children in the Family First group were statistically significant (there were no sites in which the program produced a significant reduction in placement rates). Differences between experimental and control groups in placement rates were not significant once variations in case characteristics were taken into account. There were no significant differences between groups in the duration or types of placements (Schuerman, Rzepnicki, and Littell 1994).
The risk of placement among cases in this experiment was very low at the time of referral. Placement rates in the control group were approximately 7 percent at one month after random assignment, 17 percent at six months, 21 percent at one year, and 27 percent at two years. Since the program served few cases that would have experienced placement in the absence of family preservation services, we can conclude that Family First did not reach its target population of cases at "imminent risk of placement."
The Family First program had a net-widening effect in that it provided services to families that would not ordinarily have received services in the child welfare system (as noted above, 20% of the control cases were never opened for services). This effect was particularly striking in several sites. However, the program had no long-term effect on the duration of time families spent in the child welfare system.
The effects of the Family First program on subsequent maltreatment and on measures of child and family functioning are discussed below.
Relationships between Case Characteristics and Placement Rates
Several studies have reported results of analyses of the characteristics of families that are likely to experience placement during or soon after family preservation services. For example, in a review of eleven programs in six states, Nelson, Emlen, Landsman, and Hutchinson (1988) found that the risk of placement was higher for children with prior group or institutional placements, families with more severe problems, families with problems related to adolescence, and families who were not motivated to receive services. Yuan et al. (1990) found that placement rates were higher for families on public assistance, families with a disabled caretaker, and families who had subsequent investigations of abuse or neglect or children at high risk of neglect. Compared with other children, the risk of placement was greater among younger children, disabled children, children who had been placed previously, and children who were court dependents (Yuan et al. 1990). Fraser, Pecora, and Haapala (1991) reported that placement rates were higher when parents requested placement, were openly hostile to their children, or had poor verbal discipline skills, and when children had intensive intervention histories, drug involvement, truancy, delinquency, oppositional behaviors, or mental illness. Feldman (1991) found that placements were more likely among minorities, families with poor parenting skills, and children with behavioral or emotional problems. In Iowa, placement was more likely among families with "multiple functioning problems," low incomes, and children with delinquency problems (Thieman, Fuqua, and Linnan 1990). Haapala (1983) and others have found that younger children are more likely to be placed than older children. Reid et al. (1988) compared 31 families with a child in placement with a matched sample of 55 intact families. The placed cases included a higher proportion of children whose problems were numerous and serious, adolescents (who were placed because their behavioral problems were more numerous and were seen as a threat to the community), and families with fewer resources who used services less, made less progress, and were less satisfied with the agency's efforts on their behalf. Parents in the placed group were more likely to see the child as the problem and were reluctant to acknowledge family problems, compared with intact families.<28>
These findings have sometimes been used to describe the kinds of cases in which family preservation services are more or less likely to be "successful." However, analysis of relationships between case characteristics and outcomes within groups receiving intensive services does not provide information about the relative effects of particular services for various subgroups. This is because the "base rates" of outcomes, in the absence of these services, vary across subgroups. For example, it may be that cases with drug involvement do not do as well in family preservation services as those without drug involvement. But cases with drug involvement may also do worse without services. To identify subgroups that benefit most from a particular service, it is necessary to look within subgroups, comparing cases that received family preservation services with those that did not. Thus, we should compare outcomes in drug cases with services to those of drug cases without services, with the assignment having been made randomly.
This type of analysis has been conducted in three studies (although two of these studies are based on small samples). The study by Szykula and Fleischman (1985) described above suggested that efforts to prevent placement may be more successful for families in which child abuse and neglect are not chronic and other family problems are relatively less severe. Feldman (1991) found that family preservation services appeared to result in reduced risk of placement for single-parent families. Approximately two-thirds (68%) of the single-parent families in the control group experienced placement, compared with 49 percent of the single-parent families who received family preservation services.<29>
The Illinois experiment examined the effects of intensive family preservation services for sixteen subgroups of cases.<30> Results showed that the program increased the risk of placement for households headed by single adults; at one year after random assignment, placement had occurred in approximately 29% of single-parent families in the program group compared with 20% of those in the control group. The Illinois Family First program had no significant effect on placements for any of the other subgroups examined (Schuerman, Rzepnicki, and Littell 1994).
Relationships between Service Characteristics and Placement
Several studies have examined correlations between service characteristics and placement outcomes. For example, Yuan et al. (1990) found that placement was more likely among families who received less intensive family preservation services. Nelson et al. (1988) reported that placement rates were lower in programs that offered more focused, shorter-term, office-based services to families with fewer risk factors (versus more comprehensive, in- home services for families with more risk factors). Nelson and Landsman (1992) found that placement was less likely when caretakers participated in most or all treatment sessions. In addition, the provision of paraprofessional services was correlated with reduced placement rates among child neglect cases, while reduced placement rates were related to the receipt of marital counseling in cases of physical abuse (Nelson and Landsman 1992).
Associations between characteristics of services and outcomes in these studies do not provide convincing evidence for the effects of these service characteristics. Characteristics of cases (such as family problems and their severity) determine the kinds of services provided and these characteristics are also likely to be related to outcomes, so that case characteristics become an alternative explanation for associations between services and outcomes. Again, the best way to determine the effects of service characteristics is to randomly assign cases to varying treatments, thereby eliminating the association between case characteristics and services provided. A weaker alternative approach is to model the relationship between case characteristics and service provision, using such models to control for the effects of case characteristics.
In the Illinois experiment, relationships between service characteristics and placement outcomes were analyzed using statistical controls for the effects of case characteristics on features of service provision and case outcomes (see Schuerman, Rzepnicki, and Littell 1994). There were no overall relationships between the duration of family preservation services, amounts of contact with caseworkers or parent aides in the first 90 days of services, or the number of concrete services provided and the likelihood of placement once the effects of case characteristics were taken into account. In a related study, separate analyses of relationships between service characteristics and outcomes were examined for specific subgroups of cases (Littell, Schuerman, and Chak 1994). The subgroups were: families with cocaine problems, families with inadequate housing problems, cases in which a parent had a serious emotional problem, and families with serious child care skill deficits. Once the effects of other case characteristics on service provision and case outcomes were controlled, it appeared that the duration, amount of contact with workers, number of concrete services, and specific types of services provided were not related to the risk of placement.<31>
Although many non-experimental studies have suggested that high percentages of families remain intact after intensive family preservation services, the results of randomized experiments provide more convincing tests of the extent to which "placement prevention rates" can be attributed to the effects of these programs. The findings of the controlled studies we reviewed are mixed: Seven of the eleven randomized experiments (Hennepin County Community Services Department 1981; Nebraska Department of Public Welfare 1981; Willems and DeRubeis 1981; Szykula and Fleischman 1985; Yuan et al. 1990; Meezan and McCroskey 1993; Schuerman, Rzepnicki, and Littell 1994) and one overflow group study (Mitchell, Tovar, and Knitzer 1989) found that the programs did not produce significant overall reductions in placement. Four randomized experiments (Jones, Neuman, and Shyne 1976; Halper and Jones 1981; Lyle and Nelson 1983; Feldman 1991) and three overflow comparison studies (Wood, Barton, and Schroeder 1988; Schwartz, AuClaire, and Harris 1991; Pecora, Fraser, and Haapala 1992) found significant reductions in placement in favor of the experimental groups.
In studies that found significant reductions in placement, differences between groups were relatively small. For example, in New Jersey, the difference between groups in the proportion of cases in placement at one year after treatment ended was 14 percent (Feldman 1991). Although larger differences were found in the overflow studies, questions about the comparability of groups in these studies remain and sample sizes were generally small. Small sample sizes are also a concern in the earlier experimental projects.
The fact that placement occurred within a short period of time after group assignment in less than half of the control or comparison cases in most studies suggests that these programs were generally not delivered to families with children at risk of placement. (The placement rate in a control group is an estimate of the risk of placement for both groups in the absence of experimental services.) When the risk of placement among family preservation clients is low, it is unlikely that a program will demonstrate significant reductions in placement. It is not meaningful to talk about preventing an event if the event wouldn't have happened anyway.
Finally, available evidence sheds little light on whether family preservation programs have differential effects on placement for different kinds of families or on the relative effectiveness of different approaches to placement prevention. Evidence of the effects of family preservation programs for specific subgroups of clients is scant and the results of available studies are somewhat contradictory. Although it is correlational in nature, the best available evidence suggests that features of services that are often considered among the hallmarks of family preservation programs--brevity and intensity of services and the provision of an array of concrete and specialized services--may not be critical.
The hope in family preservation programs is to prevent the placement of children without subsequent maltreatment. Few studies have examined the effects of family preservation programs on the recurrence of child maltreatment. Obviously, it is impossible to detect all maltreatment of children, so researchers have generally depended on reported incidents. In the five-year follow-up study of the New York Preventive Services Demonstration, Jones (1985) found that 21 percent of 98 families in the experimental group and 25 percent of 44 control group families had experienced one or more indicated reports of child maltreatment. The difference between groups was not statistically significant. Yuan et al. (1990) reported that approximately one- quarter of families in both the program and control groups experienced an investigation of child abuse or neglect within 8 months after referral. In the Illinois experiment, children in the family preservation program were somewhat more likely to be identified as victims of subsequent maltreatment than children in the control group; although statistically significant, the difference between the groups was small (Schuerman, Rzepnicki, and Littell 1994).<32>
As with placement, the rates of maltreatment in both the experimental and control groups in these studies were fairly low. Had placement been prevented, the results could be taken as indicating that this benefit was attained without increased harm to most children. However, most children in both groups remained in their homes, and the results indicate that the experimental services did not reduce an already low rate of subsequent harm.
Several studies examined effects of services on measures of family functioning.<33> Some studies have simply examined change in these measures over time, but change can often be explained by maturation, statistical regression, and a host of external influences. Hence, we are interested in comparing the status of the treatment group following services and the status that group would have attained at the same point in time in the absence of the intervention. Alternatively, we might compare amounts of change over the same period of time. Randomized experiments provide the best estimates of such effects.
In the New Jersey study, both the treatment and control groups made gains on the Moos Family Environment Scale, Interpersonal Support Evaluation List, and Child Well-Being Scales, but there were few statistically significant differences between groups in the amount of change (Feldman 1991).<34>
In Meezan and McCroskey's (1993) study, six scales were used to measure family functioning: parent-child interactions, living conditions of the family, interactions between caregivers, supports available to parents, financial conditions of the family, and developmental stimulation of children. Families in both in-home services and regular services generally reported that they did not have significant problems in family functioning at case opening and did not see significant change in these areas at case closing. However, families in the in-home services group reported more improvements in living conditions and financial conditions at one year after termination, compared to families who received regular child protective services. Parents in the program group also reported more improvements in their children's behavior between referral and case closing, although there were no differences between groups one year after services had ended. In contrast to parents' views of family functioning, workers who provided home-based services reported that the families had significant problems in all areas of family functioning at case opening and made significant improvements in four of six domains at case closing. (The four areas in which improvements were noted were: parent-child interactions, living conditions, supports available to families, and developmental stimulation given to children.) The validity of workers' ratings of change in cases in which they are invested is open to question. Further, since caseworkers' reports were not available for the control group, we cannot be sure that changes reported by workers were due to the services provided.
Measures of child and family functioning in eight domains were obtained through interviews with a subsample of parents in three of the six sites in the Illinois experiment (Schuerman, Rzepnicki, and Littell 1994). The domains were: housing conditions, economic conditions, physical child care, discipline and emotional care of children, children's academic adjustment, children's conduct, children's symptomatic behavior, and parental coping skills. A few statistically significant differences between groups were found (most were in favor of the program group), but these gains were modest and did not last over time.<35>
Family preservation programs have been promoted as a way to save costs on foster care. Claims of cost savings based on non- experimental studies must assume that most of the families who receive intensive, home-based services would have required placement in the absence of these services. The costs of intensive services are then compared with estimated costs of placements.<36> As we have shown, the assumption that placement would have occurred in the absence of services is not supportable.
Few controlled studies have examined costs in treatment and control groups. In an overflow comparison group study, Wood, Barton, and Schroeder (1988) reported that the cost of 4 to 6 weeks of in-home services for 26 FamiliesFirst cases plus the cost of placements that occurred in these cases over a one-year period totaled $124,783, compared with $176,015 in placement costs alone for 24 cases in the comparison group. Information on the costs of other services provided to program and comparison cases was not available.
Only one randomized experiment has examined costs in both treatment and control groups. Yuan et al. (1990) found that the placement costs for in-home services and control cases were similar ($141,375 versus $145,388) for the 152 families in each group. In addition, the average cost of providing intensive, home-based services was $4,767 per family served, over $700,000 in total (Yuan et al. 1990). Unfortunately, data on the costs of non-placement services provided to the control group were not available, but it is reasonable to assume that these were considerably lower than the cost of intensive, in-home services. Thus, it is evident that the total costs for cases in the family preservation program exceeded the costs of services to control cases.
On balance, evidence that family preservation programs save money is scant and the results of available studies are mixed. Obviously, if evaluations do not show that programs avert placement, they cannot show that costs are reduced.
In addition to establishing the objective of preventing placements, the Adoption Assistance and Child Welfare Act of 1980 called for the reunification of children in foster care with their biological parents whenever possible, within the larger context of efforts to achieve permanent and safe living conditions for children. Yet, the issue of family reunification continues to be overshadowed by burgeoning attention to efforts to prevent placements. While most states have developed programs aimed at preserving intact families, there are relatively fewer programs designed to reunify families (Ahart, Bruer, Rutsch, Schmidt, and Zaro 1992). Furthermore, Barth and Berry (1987) suggest that children who are reunified with their parents are the group that is least well-served; they called attention to the need for more and longer-lasting services for these children and their families to prevent reabuse and foster care reentry. Here we review what is known about the outcomes of intensive services designed to facilitate the reunification of families with children in foster care.<38>
In assessing the impact of programs aimed at family reunification, we are interested in program effects on the rates at which children are returned home. Since reunification occurs during the normal course of child welfare services (most children in foster care are returned to their homes within a two year period),<39> we need to know whether intensive services actually improve the chances of family reunification and shorten the time to reunification. As in the case of placement prevention programs, the best way to determine this is through randomized experiments, in which cases eligible for reunification services are assigned randomly to treatment and control groups.
The achievement of reunification is only one of the goals of these programs. As indicated above, an overarching objective is to find safe and permanent living situations for children.<40> Yet children who are reunified with their families are at greater risk of subsequent maltreatment than children in out-of-home placements (Barth and Berry 1987). Regarding permanence, there is some indication that reentry rates may increase as a result of efforts to speed reunification (Wulczyn 1991).<41> Thus, intensive family reunification programs try to resolve the conditions that led to placement and improve the chances that children will remain in their homes without further maltreatment. To gauge the success of these efforts, we need information about what happens after children are returned to their homes and whether their families provide stable and safe living conditions for them. Relevant outcomes include rates of subsequent maltreatment among children who are returned to their families, other indicators of child and family functioning after reunification, and the rates at which children reenter foster care or move into other types of living arrangements.
Since there have been few intensive reunification efforts, evaluations of the outcomes of these programs are scant. Most of the studies that do exist are based on small samples, most have used non-experimental designs or non-equivalent comparison groups, and few have obtained information on outcomes other than reunification rates.
For example, Boyd (1979) found that children in the Temporary Foster Care program in Michigan spent less time in foster care and were more likely to be returned home than children who were in foster care prior to the implementation of the program. Lahti (1982) reported results of an evaluation of the three-year Oregon Permanency Planning Project. Children in this project received intensive services aimed at removing barriers to reunification. Three years and four months after the project began, the placement status of 259 children served was compared with that of 253 children who had received regular child welfare services. There were no significant differences between groups in the proportion of children that were reunified with their families (26% of the children in the project and 24% of those in the comparison group had been returned to their parents).<42> At a 15-month follow-up period, there were no significant differences between groups in the stability of placements.
More recent studies include a Homebuilders pilot project which reported that 13 of 14 "hard to serve" adolescents were quickly reunified with relatives (the average time between intake and reunification was 8 days) and 12 of the 14 adolescents were not in out-of-home care at a one-year follow-up (Haapala, Johnston, and McDade (1990). Similarly, Brown and Little (1990) reported that all 50 families involved in a study of the Full Circle program in California had been reunified after three months, 80% remained intact after six months, and 74% after one year. However, Lerner (1990, cited in Maluccio, Fein, and Davis 1994) reported a reunification rate of only 25% in a privately- funded program in a public housing project in Brooklyn, New York. Walter McDonald and Associates (1992) found that 57% (20) of 35 children served in a Milford, Connecticut reunification program were returned to their homes at the end of services.
Fein and Staff (1993) reported that 38 percent (26) of 68 children served in the first two years of the Casey Family Services reunification program<43> were reunited with their families. Of those reunited, 19 children were still at home at the end of the second year (13 were still receiving program services) and 7 (10%) had been returned to foster care.
In a recent report on the results of the family reunification initiative in Illinois, Schuerman, Rzepnicki, and Johnson (1994) reported that 40% of the children in placement at the time of referral to the project had been reunified with their families within 6 months after referral, 48% were reunited within 9 months, and 53% within one year.<44> Reunification rates varied widely across the agencies that participated in this project.<45> Comparisons were made to other children who entered foster care during the time period of the study and met other criteria for referral to the program.<46> Children in families in the intensive reunification program had shorter stays in placement than those in the comparison group. Differences were greatest in Chicago, where reunification rates for the comparison group were relatively lower. The program did not appear to affect the likelihood of foster care reentry. Approximately 12% of the children who had been reunited with their families returned to out-of-home care within six months, at 12 months the figure was around 20%, at 18 months 24%. These rates were similar to those in the general population of children who return home from foster care.
In sum, reunification rates have varied from 25 percent to 100 percent across several studies of programs aimed at reunifying families. There is some evidence that families in intensive reunification programs are reunified more quickly than other families with children in foster care, but as with evaluations of placement prevention programs, the results of non- experimental studies of intensive reunification efforts are difficult to interpret in the absence of clear evidence about what the rates of reunification and foster care reentry would have been in the absence of these services. Since there are considerable variations in the reunification and reentry rates for different subgroups of children, across geographic locations, and over time (Maluccio, Fein, and Davis 1994), comparisons between program participants and other foster care cases do not provide convincing evidence of program effects. Cases referred for intensive reunification services may be those which workers believe are good candidates for reunification; thus, in the absence of intensive services, the likelihood of reunification may be greater in program populations than in the larger population of families with children in foster care. Controlled studies are needed to provide information on the effects of a program on reunification and reentry rates.
We know of only two well-controlled studies of special services designed to reunify children in foster care with their biological families. One was conducted before the passage of Adoption Assistance and Child Welfare Act of 1980 and one after that Act. These two experiments are described below and in Table 3.<47>
The New York State Preventive Services Demonstration Project, described above, provided both placement prevention and reunification services to families in the mid 1970s (Jones Neuman, and Shyne 1976; Jones 1985). Here, we focus on the subgroup of 314 children (in 195 families) who were in foster care (81%) or had recently been discharged from care (19%). Cases were randomly assigned to experimental and control groups. Intensive services were provided to families by seven private and two public agencies. Workers carried caseloads of 10 and the duration of services averaged 8.5 months. At the end of the demonstration, 47% of 205 children in the experimental group and 38% of 109 children in the control group had been reunified with their families. At a six month follow-up, 62% of the children in the experimental group and 43% of those in the control group had returned home. At a five-year follow-up, Jones (1985) found no significant differences between groups in the proportion who had been discharged from foster care.
The Utah Family Reunification Services project. The most recent experiment in this area, reported by Walton and her colleagues, assessed the effects of an intensive, in-home family reunification project implemented in 1989 in four social service districts in Utah (see Table 3). Services were limited to 90 days and involved at least three visits per week with each family. Caseworkers carried caseloads of no more than six families at a time and spent an average of 3.1 hours per week with each family; contacts during the first two weeks of services were somewhat more intensive (an average of 5.4 hours per week). The program provided concrete services (e.g., financial assistance, transportation, clothing, food) and training in communication skills, parenting skills, and anger management. Follow-up services were arranged for all of the families in the treatment group. A total of 110 families with children in substitute care were randomly assigned to treatment and control groups (Walton, Fraser, Lewis, Pecora, and Walton 1993).<48> Most (76%) of the children were in foster care at the time of their inclusion in the study (Walton et al. 1993); others were in shelters, group homes, residential placements or inpatient psychiatric care (Walton 1991). Families in the control group received services from regular foster care workers, who had average caseloads of 22 and were expected to visit families at least once a month.
At the end of the 90-day treatment period 93 percent (53) of 57 children in the treatment group had been returned to their homes, compared with 28 percent (15) of 53 control children--a significant difference (Walton et al. 1993). Six months after termination 70 percent (40) of the children in the program group and 42 percent (22) of those in the control group were at home. At the one-year follow-up, 75 percent (43) of the children in the program group and 49 percent (26) of those in the control group were living at home. Differences between the groups were statistically significant at each point in time but decreased over time. Fifty-six children (all but one) in the treatment group were returned home at some point during the 15 month study period; seventeen (30%) of these children reentered out-of-home care, although 5 of the 17 were returned to their homes again before the end of the study.<49> In contrast, of the 30 children in the control group who returned home, 5 (17%) reentered care. Children in the treatment group spent significantly more time (days) at home during the study period than those in the control group (Walton et al. 1993). Walton (1991) found no significant differences between groups on several measures of individual and family functioning at the end of the 90-day period. Thus, the intensive services program appeared to facilitate reunification, although some of these children did not remain at home.
Several researchers have examined characteristics that are associated with reunification. Some studies have focused on clients in programs aimed at facilitating reunification, although most have examined factors associated with "natural" reunifications (that is, the return home from regular foster care). These studies do not support causal inferences, but we report the findings for their heuristic value. Hess and Folaron (1991, 1992) found that parents' ambivalence about parenting was a substantial obstacle to reunification. Turner (1984a) found that reunification was less likely in cases in which parents had multiple problems and when parents had requested the initial placement. Schuerman, Rzepnicki, and Johnson (1994) found that children who had been abused were more likely to return home than those who had been neglected; in addition, families with housing problems, substance abuse, emotional problems of parents, and mental illness were less likely to be reunited, as were families headed by single parents. Cases in which children displayed problems in school or in relationships with peers or siblings were more likely to be reunited than other cases. Courtney (1994) showed that the likelihood that children would return home from regular foster care was lower for children with health problems or disabilities, families that were eligible for Aid to Families with Dependent Children (AFDC), African American infants, and African American children over 12 years of age; cases of sexual abuse were reunited more quickly than those involving child neglect. Among cases of physical abuse, Barth et al. (1986) found that families with less severe abuse, those whose children had few school problems, and families of higher socioeconomic status were more likely to experience reunification than other families.
Several studies have examined relationships between service characteristics and family reunification. For example, Schuerman, Rzepnicki, and Johnson (1994) found that the amount of contact between families and workers and the duration of time in reunification programs were positively related to reunification (this may reflect the fact that cases that were likely to be reunified were provided with more help and remained in the program longer than those that were not considered good candidates for reunification). In a study of "natural" reunification processes Goerge (1990) showed that the probability of reunification decreased as the length of time in foster care increased. Reduced length of stay in foster care has also been related to the number of contacts between family members and child welfare workers and number of contacts initiated by family members (Gibson, Tracy, and DeBord 1984), intensity of contacts between family members and workers (Barth et al. 1986), and the frequency of parent-child contacts (Gibson, Tracy, and 1984; Lawder, Poulin, and Andrews 1986). Barth et al. (1986) found that the provision of in-home services was not related to whether a child returned home and Courtney (1994) found that pre-placement services did not affect the chance that a child would return home from regular foster care.
As noted above, "successful" reunification is often thought to involve the resolution of problems that led to placement, stabilization of the child in the family home, and avoidance of foster care reentry. Lack of resolution of the problems or behaviors of the parent that led to placement, child neglect, poor parenting skills, and limited support from extended family members, friends, and neighbors have been associated with foster care reentry (Hess and Folaron 1991, 1992; Hess, Folaron, and Jefferson 1992; Davis, English, and Landsverk 1993; Festinger 1994). Longer stays in foster care and the duration of case management services both before and after the child's return home have been related to reduced likelihood of foster care recidivism (Wulczyn 1991, Turner 1984b).
There is little solid evidence that programs aimed at preventing out-of-home placements or reunifying families with children in foster care have the intended effects. Results of nonexperimental studies have been misleading and the findings of controlled studies in these areas are mixed. Some studies involved samples that were so small that it would have been quite difficult to detect significant program effects. Further, information about the nature of interventions was often incomplete. In response to these problems, recent evaluations have used larger samples and increasingly more sophisticated methods--including the use of comparison or control groups; systematic collection of data on family problems, services, and outcomes; and attempts to understand factors related to outcomes for families. Yet, there are few large, well-controlled studies of family preservation and reunification programs. Problems of sample size and questions about the nature of services provided and the comparability of groups remain, even in recent experiments (Bath and Haapala 1994, Littell 1994).
As to the effects of intensive family preservation services on placement and maltreatment, many of the programs studied did not focus on populations that had high rates of placement or maltreatment and, thus, these rates in both experimental and control groups were low. Hence, the possibility of detecting effects on placement or maltreatment was low. It is not surprising, then, that few studies have demonstrated program effects in these areas and that, in the studies that have found such effects, they tend to be small and short-lived. Targeting problems are apparent in reunification projects as well (reunification rates in some projects have been quite low), but these are not as well documented as in placement prevention programs.<50> Efforts to improve targeting in these areas are needed and these should be the subject of further study.
Our review suggests that family preservation programs have very modest effects on family and child functioning. Researchers have found few significant differences between program and comparison groups in levels of child and family functioning after services have been provided and the results of available studies are conflicting. We suggest that it is not realistic to expect dramatic results in this area, given the number and magnitude of the problems faced by many child welfare clients and the short- term nature of family preservation services. It should be noted that the approaches that have been tried tend to focus on the parent or the family and often ignore conditions in the community or larger social environment that may contribute to child maltreatment and other problems in family functioning.
Evaluations of family reunification programs are in a nascent stage. A few studies have reported reunification rates that are encouraging. There is a need for greater clarification of the goals and expected outcomes of these programs (Ahart et al. 1992) and better understanding of phases in the reunification process (Maluccio, Fein, and Davis 1994). Information on child and family functioning, subsequent maltreatment, and foster care reentry is needed to gauge how well children fare after they return home.
This review provides many lessons for further research on family preservation and particularly for the National Evaluation of Family Preservation Services. To begin, it is evident that evaluations must use the most rigorous methods whenever possible, that is, randomized experimental designs. The story of family preservation research shows that early uncontrolled studies were quite misleading, when viewed in light of later more rigorous studies. It is also evident that if the objective of placement prevention is to be seriously addressed, both programmatically and evaluatively, the problem of targeting must be solved. Whether targeting of these programs can in fact be substantially improved remains an open question, but further efforts should be made before giving up on this issue.
In the view of many, the most promising approach to family preservation is the Homebuilders model. Unfortunately, this approach has not been subjected to large scale, well controlled evaluation, so this should have high priority in planning future evaluations. Beyond attention to this one approach, we need further exploration of the differential effects of various models and the question "do models matter?" needs to be addressed. We also need to get into the details of work with families, to explore the effects of differences in dose, types of services, and other activities with clients.
We know little about the differential impact of services such as these on various subgroups of families. A crucial step in pursuing this issue is the specification of groups to examine. We have no clearly delineated diagnostic system to rely upon here. Do we define groups in terms of presenting problems, family structure, history of involvement with public systems, ages of children and parents, or some other characteristic? Beyond this, there is the overriding question of the interaction between family characteristics and services: what works best for whom?
It seems likely that evaluations will continue to use placement of children as a principal outcome measure, but clearly they should also measure other outcomes, in order to detect potential benefits of these programs. These other outcomes should include maltreatment subsequent to referral, as well as various measures of family and child functioning. It is also clear that families should be followed for some time after the completion of service, to attempt to determine the persistence of effects. How long the follow up period should be is a matter of some debate, but it seems reasonable to follow families for at least a year and possibly two. In this connection, the effects of after care services have rarely been considered in evaluations and should be examined in future work.
Finally, we need to pay more attention to the effects of contextual factors, including community characteristics and availability of community services. It is likely that future evaluations will take place in the context of major changes in the public welfare and child welfare systems in this country, and evaluations must find ways to account for the effects of these changes.
[The paper version includes tables summarizing non-experimental studies of family preservation programs, controlled studies of family preservation programs, controlled studies of family reunification programs, and a lengthy list of references.]
<1> Studies of efforts to preserve families served in the juvenile justice (e.g., Henggeler, Melton, and Smith 1992; Collier and Hill 1993) and mental health systems are not included here.
<2> For example, the prevention of placement was the primary objective in programs studied by Yuan, McDonald, Wheeler, Struckman-Johnson, and Rivest (1990); Feldman (1991); Fraser, Pecora, and Haapala (1991); Schwartz, AuClaire, and Harris (1991); and Schuerman, Rzepnicki, and Littell (1994). The program described by Jones, Neuman, and Shyne (1976) emphasized placement prevention and other goals. Placement prevention was not a primary goal in the Family Support Project in Los Angeles (Meezan and McCroskey 1993).
<3> For descriptions of various models, see Nelson, Landsman, and Deutelman (1990) and Cimmarusti (1992).
<4> Much of the material in this section is a revision of material in Schuerman, Rzepnicki, and Littell (1994).
<5> Studies that did not employ comparison or control groups include Kinney, Madsen, Fleming, and Haapala (1977); Florida Office of the Inspector General (1982); Leeds (1984); Landsman (1985); Hinckley and Ellis (1985); Van Meter (1986); Bribitzer and Verdieck (1988); Fondacaro and Tighe (1990); Thieman, Fuqua, and Linnan (1990); Kinney, Haapala and Booth (1991); Smith (1991); Berry (1992); Wheeler, Reuter, Struckman-Johnson, and Yuan (1993); Bartsch and Kawamura (1993); and Scannapieco (1994). Studies which employed non-equivalent comparison group designs include Pearson and King (1987); Reid, Kagan, and Schlosberg (1988); Bergquist, Szwejda, and Pope (1993); Landsman, Richardson, Clem, Harper, Schuldt, and Nelson (1993); Schafer and Erickson (1993); Arizona Department of Economic Security, Division of Social Services, Administration for Children, Youth and Families (1994); Hoecker (1994); North Carolina Department of Human Resources, Division of Family Development (1994); Showell, Hartley, and Allen (N.D.); and Thieman and Dail (1993).
Previous reviews of this literature have been provided by Schuerman, Rzepnicki, and Littell (1994); Kaye and Bell (1992); Nelson and Landsman (1992); Wheeler (1992); Fraser, Pecora, and Haapala (1991); Rossi (1991); Davis (1988); Frankel (1988); Jones (1985); Stein (1985); and Magura (1981). See also Bath and Haapala (1994), Littell (1995), and Bath and Haapala (1995).
Studies of programs designed to prevent placement of status offenders (Nugent, Carpenter, and Parks 1993) or delinquent and emotionally disturbed children (Cunningham, Homer, Bass, and Brown 1993) have also relied on non-experimental designs.
<6> The idea of phantom control groups is due to Rossi and Freeman (1993).
<7> Table 1 was constructed by Larry Cohen.
<8> To create the comparison groups, one child who was designated "at imminent risk of placement" within each Families First case was matched with a child who had exited foster care within 90 days of the date the Families First case was initiated. The pairs of children were also matched on age, county of residence, type of referral, and prior involvement with protective services.
<9> It is not easy to sort out all of the problems here. The assumption appears to have been made that cases referred to the program would have been likely to have been placed in the absence of the program. The comparison group of children discharged from care presumably was composed of those deemed unlikely to need further care in the near future, otherwise they would not have been discharged. Hence, the groups could not have been considered comparable in placement propensities at the outset. However, a further complication is that the assumption that referrals to family preservation programs consist of imminent risk of placement cases has been proven incorrect in controlled studies, as we shall see below.
<10> The Michigan Families First program began in 17 counties in 1988 and was quickly expanded to the rest of the state. According to data from the Michigan Department of Social Services (1993), the number of new foster care placements increased steadily from 6,490 in 1988 to 8,299 in 1991, followed by a decrease to 7,632 new placements in 1992. The foster care caseload in Michigan grew from 15,878 in 1988 to 17,124 in 1992. These data are somewhat at variance with data in the Multistate Foster Care Archive of the Chapin Hall Center for Children. Archive data indicate that there were 6,368 new admissions in 1988, increasing to 7,188 in 1991, with a decrease to 6,603 in 1992. Archive data on the foster care census in Michigan show a total of 10,901 at the end of 1988, increasing to 12,671 by the end of 1991, decreasing to 12,265 in 1992. During this period discharges from care steadily increased.
<11> See Kinney, Haapala, and Booth 1991; Berry 1992; Pecora, Fraser, and Haapala 1992; Hartman 1993.
<12> See Wilson, 1994. Interviews with child protective services workers in Illinois also suggest that this practice is viewed as advocacy on behalf of the client.
<13> The overflow group consisted of 26 of the 38 families that were referred to the family preservation program but not served because program staff had full caseloads. They received traditional child welfare or mental health services. Twelve of the 38 families were referred to the program early on and could not be traced. The remaining 26 cases were tracked for one year or until a child at risk was placed, whichever came first (Pecora, Fraser, and Haapala 1991).
<14> Of the cases that did not participate, slightly more than half (51%) were asked not to participate by their worker (for reasons that are not entirely clear), 24 percent refused to participate, 20 percent did not have the opportunity to participate because of research administration problems, and 5 percent were excluded for treatment reasons (Pecora, Fraser, and Haapala 1991).
<15> Placed children in the overflow group may have been easier to track than unplaced children. If this was the case, the observed placement rate in the overflow group would have been biased upward.
<16> This table and our review of controlled studies on family preservation programs are adapted from Schuerman, Rzepnicki, and Littell (1994). Other reviews of this literature have been provided by Jones (1985); Stein (1985); Frankel (1988); Fraser, Pecora, and Haapala (1991); Rossi (1991); Wells and Biegel (1991); Nelson and Landsman (1992); and Kaye and Bell (1992).
<17> Data on informal placements with relatives and friends and on placements outside the project county were not available.
<18> The authors describe another study, involving an A-B-A reversal design that focused on the numbers of substitute care placements in Jackson County, Oregon before, during, and after installation of a social learning treatment program. Although the authors suggest that placements declined during the nine month period in which the program was in operation, the results are not convincing since placement was a fairly low-incident event in this county (only 58 placements were recorded during the entire 49-month study period).
<19> The "less difficult" group included families with fewer than three reports of abuse, no serious housing or transportation problems, and children with conduct problems. Those in the "more difficult" group had three or more prior reports; serious problems with employment, transportation, and housing; and "major problems outside of their relationship with their child" (Szykula and Fleischman 1985, p. 281).
<20> During the second year of the study, "imminent risk" was defined as the expectation (based on statements from the referral source) that action would be taken to remove the child(ren) within two weeks unless intensive services were provided. The researchers reported that many caseworkers found this definition too stringent and confusing.
<21> Placements that terminated within 8 weeks of random assignment were not included in analyses of placement rates; in these cases, children were considered to be reunified with their parents during the intensive service period. A child-level analysis showed that 18 percent of children in the project group and 17 percent of children in the control group were placed between 2 and 8 months after random assignment.
<22> Control group children tended to be placed more quickly than those who received intensive in-home services. Rossi (1991) has termed this the "moratorium effect" of family preservation programs in delaying, but not necessarily preventing, placement.
<23> For control group cases, termination was defined as "6 weeks after referral to FPS or actual termination of community services, whichever came first" (Feldman 1991, p. 69).
<24> Differences between groups were computed at termination and at 1, 2, 3, 6, 9, and 12 months post-termination.
<25> Types of placements included homes of relatives, foster homes, emergency and runaway shelters, residential centers, detention, independent living, mental health in-patient facilities, and teaching family homes.
<26> The project also accepted some referrals from schools, hospitals, mental health clinics, and other community agencies. Compared with families referred by DCS, cases that were referred by other sources were seen by the in-home services workers as having less severe problems at referral (Meezan 1993).
<27> The probability of assignment to family preservation services was .6. Thus, about 60% of the families were assigned to these services, the remaining 40% to regular services.
<28> See also Nelson and Landsman (1992), Nelson (1991), and Yuan and Struckman-Johnson (1991).
<29> A reanalysis of Feldman's data indicates that this difference is significant at p < .05.
<30> The subgroups examined included: families that were new to the child welfare system; those in which a child had been physically injured prior to referral; cases of chronic neglect; families with housing problems; families with severe income or resource deficits; families with cocaine problems; those with alcohol problems; cases with other substance abuse problems; families in which a parent had a chronic mental illness or serious emotional problem; cases with marital or adult relationship problems; cases with serious child care skill deficits; families in which a child had a health, development, or learning problem; cases in which the primary caregiver was a teenager; households headed by a single adult; households with extended family members; and cases in which protective custody of a child had been taken within one year prior to referral. These categories were not mutually exclusive. Of course, in the examination of a number of categories, one must be aware of the possibility of Type I error, some results may appear significant simply by chance.
<31> This study examined the effects on placement of the following types of services for specific subgroups: provision of substance abuse treatment in cocaine cases, assistance in moving to new housing for families with housing problems, individual counseling in cases with serious emotional problems, and parent education and homemaking services in cases with serious child care skill deficits.
<32> The large number of children in this study made it relatively easy to detect statistically significant differences. There were no sites in which significant reductions in the recurrence of maltreatment were found, nor did the program affect the risk of subsequent maltreatment for any of the subgroups of cases examined.
<33> See Wells and Whittington 1993; Berry 1992; Fraser, Pecora, and Haapala 1991; Yuan et al. 1990; Mitchell, Tovar, and Knitzer 1989; and Jones 1985. Other studies have compared measures of functioning for clients who received home-based services and those with children in foster care (e.g., Wald, Carlsmith, and Leiderman 1988).
<34> Measurements on social support and the Family Environment Scale were taken at three points in time, at the beginning of service, at termination (six weeks after referral for the control group), and three months post termination. Child well being was measured only at Time 1 and Time 2. On only one of four social support measures (availability of people to do things with) was there a difference between experimentals and controls in time 2 to time 3 change, favoring the experimental group. On child well being there was no difference in change scores overall, one of three subscales (parental role performance) did show a significant difference in change, favoring the experimental group. On the Family Environment Scale there were no differences in change between time 1 and time 2 on any of 10 subscales. There were time 1 to time 3 differences in change on two scales, family cohesion and personal growth, again favoring the experimental group.
<35> In each of the three waves of interviews in Chicago, differences were found favoring the family preservation group in one of the eight domains of functioning, but these improvements were not stable over time. Family preservation clients reported fewer problems in housing at the first interview, fewer problems in physical child care at the second interview, and fewer problems in children's academic adjustment in the third interview. An analysis of change over time indicated that the proportion of problems reported in children's academic adjustment in Chicago tended to decrease over time in Family First cases, while it increased among regular services cases. In the other two areas, differences over time favoring the Family First group were found in one or two of the domains, but these effects were modest and short-lived. Overall, the program had no significant impact on parents' feelings of self-efficacy or on the availability of informal social support. Improvements in the receipt of informal support were quite limited and disappeared over time. The program had no lasting effects on the use of formal services.
<36> For examples of these types of cost estimates, see Florida Office of the Inspector General (1982); Hinckley and Ellis (1985); Citizens for Missouri's Children (1989); Kinney, Haapala, and Booth (1991); Bartsch and Kawamura (1993); and Bergquist, Szwejda, and Pope (1993).
<37> This section is based in part on an unpublished review of the research on family reunification programs compiled by Tom Lawless for the Chapin Hall Center for Children. Other relevant reviews have been provided by Rzepnicki and Stein (1985), Barth and Berry (1987), Rzepnicki (1987), Frankel (1988), and Maluccio, Fein, and Davis (1994).
<38> Studies of efforts to reunify families with children in residential treatment are provided by Carlo (1985, 1993) and others.
<39> See Wulczyn, Goerge, and Harden 1993.
<40> Ahart et al. (1992) observed that most family reunification programs have very broad definitions of successful outcomes. They report that most of these programs focus on permanency planning and few view family reunification as an appropriate goal for all clients.
<41> Foster care recidivism generally occurs within three years for approximately 30% of the children who are returned to their homes (Wulczyn and Goerge 1992).
<42> Forty percent of children in the program and 21% of those in a comparison group were adopted at the end of the project--a statistically significant difference. However, to be eligible for the project, children had to be considered (by their caseworkers) adoptable and unlikely to return home, while this criteria was not applied in the selection of comparison cases.
<43> This 3-year demonstration program was instituted in Hartford, Connecticut, Portland, Maine, and White River Junction, Vermont in 1989.
<44> Within six months of referral, partial reunification (that is, the return home of at least one child in placement) had occurred in 45% of the families served; within nine months, 54% of the families had been reunited with at least one of their children; within one year, the figure was 59%.
<45> Across 23 agencies, nine-month reunification rates ranged from 8% to 73% of the children in placement at the time of referral. Reunification rates in the Chicago area were generally lower than in other parts of the state.
<46> Like cases in the reunification program, children in the comparison group were under 12 years of age and had been the subject of fewer than four substantiated investigations of child maltreatment. To increase comparability with cases in the reunification program, the comparison group was limited to children who had been in non-relative foster homes for at least 7 days.
<47> In addition to the studies described here, Stein, Gambrill, and Wiltse (1978) reported results of a controlled study of the Alameda County project, which provided intensive services to biological parents of children in out-of-home care. Services were aimed at increasing parents' participation in decisions about future living arrangements for their children. The goal of the project was to increase continuity of care for children in out-of-home placement--thus the project was not aimed at reunification per se. Yet, 48% of children in the experimental group were returned to their homes, compared with 30% of those in the control group.
<48> In an earlier report on this project, Walton (1991) stated that after random assignment, 7 treatment cases and 14 control cases were considered inappropriate for the project and excluded from the study. Then an unspecified number of cases was recruited for the control group (only) to make up for this loss. Walton (1991) reports data on total of 120 cases in the experiment. We assume that 10 cases recruited for the control group outside of random assignment were dropped from the study reported by Walton et al. (1993).
<49> This adds to 44 children at home at the end of the study, compared to the 43 cited above.
<50> The detection of targeting problems in reunification programs is a somewhat different matter than in placement prevention programs. Presumably, all cases referred to reunification programs have a child in placement at the time of referral, so the objective of reunification may be considered to be relevant (if not appropriate) for all cases. Targeting problems could arise in two ways: the referral of cases in which reunification would take place in the absence of the program and referral of cases in which reunification is not possible within the time limits of the program. The first of these problems would be revealed by high reunification rates in randomized control groups while the second problem would be seen in low reunification rates in the experimental group. Of course, low rates of reunification in the group receiving reunification services may be seen as either a targeting problem or as an indication of ineffectiveness of the service.