Many early evaluations of programs designed to prevent placement used non-experimental designs in which groups receiving these services were followed without comparing them to other groups or in which nonequivalent comparison groups were used.5 The studies of only groups receiving services appear to have had implicit "phantom" nontreatment control groups in which it was assumed that nearly all children would be taken into custody.6 Such an assumption has been proven false in subsequent research.
The results of studies without control groups suggested that most families remain intact during and shortly after family preservation services. An early study of the Homebuilders' model (Kinney, Madsen, Fleming, and Haapala 1977) found that 97 percent of 80 families remained intact three months after the intervention had ended. Since then, evaluations of the Homebuilders program have found that 73 to 91 percent of families were intact at 12 months after referral for service (Kinney, Haapala, and Booth 1991). Studies of other programs have found that at least two-thirds of families remain together within a year after the end of services. For example, 66 percent of 747 families who received family preservation services in Iowa remained intact one year after termination (Thieman, Fuqua, and Linnan 1990). A study of family preservation services in Connecticut found that 69 percent of 591 families remained intact one year after services and 82 percent of the 1,588 children in these families were not placed during this period (Wheeler, Reuter, Struckman-Johnson, and Yuan 1993). Eighty-eight percent of 367 families in the In-home Family Care Program in northern California were intact one year after services ended (Berry 1992). Table 1 provides a summary of some other recent nonexperimental studies of placement prevention efforts.7
The Families First program in Michigan has received a great deal of attention because of its claimed success in preventing placement. An evaluation (Bergquist, Szwejda, and Pope 1993) compared 225 children referred to the program (thought to be at "imminent risk of placement") with a matched group of 225 children who had recently exited foster care.8 It was found that 76 percent of the children in the Families First group remained in their homes at 12 months after the intervention while 65 percent of children in the comparison group remained in their homes for 12 months after they had returned from foster care. However, these groups cannot be considered to be comparable. Children in families entering a family preservation program and those recently discharged from foster care cannot be assumed to be similar in their likelihood of future placement.9
Additional claims for the effectiveness of the Michigan program, made by the state, are based on a decrease in the number of children placed in foster care in 1992, four years after the initiation of the Families First program.10 However, changes in foster care rates over a few years do not provide evidence of the effects of family preservation programs because such rates are affected by many other factors. In many jurisdictions, foster care caseloads have increased despite the presence of family preservation services; it is possible that these increases would have been greater in the absence of family preservation efforts. Alternatively, intensive in-home services may actually contribute to the rise in foster care rates because these services involve more extensive scrutiny of child rearing practices than occurs in their absence (Schuerman, Rzepnicki, and Littell 1994).
These findings have been used to suggest that family preservation programs reduce the need for out-of-home placement of children.11 However, nonexperimental studies such as these do not provide convincing evidence of program effects, since it is not clear that families would have experienced placement of children in the absence of these services. Claims that children were at "imminent risk of placement" at the time of referral have not been supported by evidence. Referring workers may assert that placement is imminent in order to obtain intensive services for families.12
5 Studies that did not employ comparison or control groups include Kinney, Madsen, Fleming, and Haapala (1977); Florida Office of the Inspector General (1982); Leeds (1984); Landsman (1985); Hinckley and Ellis (1985); Van Meter (1986); Bribitzer and Verdieck (1988); Fondacaro and Tighe (1990); Thieman, Fuqua, and Linnan (1990); Kinney, Haapala and Booth (1991); Smith (1991); Berry (1992); Wheeler, Reuter, Struckman-Johnson, and Yuan (1993); Bartsch and Kawamura (1993); and Scannapieco (1994). Studies which employed non-equivalent comparison group designs include Pearson and King (1987); Reid, Kagan, and Schlosberg (1988); Bergquist, Szwejda, and Pope (1993); Landsman, Richardson, Clem, Harper, Schuldt, and Nelson (1993); Schafer and Erickson (1993); Arizona Department of Economic Security, Division of Social Services, Administration for Children, Youth and Families (1994); Hoecker (1994); North Carolina Department of Human Resources, Division of Family Development (1994); Showell, Hartley, and Allen (N.D.); and Thieman and Dail (1993).
Previous reviews of this literature have been provided by Schuerman, Rzepnicki, and Littell (1994); Kaye and Bell (1992); Nelson and Landsman (1992); Wheeler (1992); Fraser, Pecora, and Haapala (1991); Rossi (1991); Davis (1988); Frankel (1988); Jones (1985); Stein (1985); and Magura (1981). See also Bath and Haapala (1994), Littell (1995), and Bath and Haapala (1995).
Studies of programs designed to prevent placement of status offenders (Nugent, Carpenter, and Parks 1993) or delinquent and emotionally disturbed children (Cunningham, Homer, Bass, and Brown 1993) have also relied on non-experimental designs.
6 The idea of phantom control groups is due to Rossi and Freeman (1993).
7 Table 1 was constructed by Larry Cohen.
8 To create the comparison groups, one child who was designated "at imminent risk of placement" within each Families First case was matched with a child who had exited foster care within 90 days of the date the Families First case was initiated. The pairs of children were also matched on age, county of residence, type of referral, and prior involvement with protective services.
9 It is not easy to sort out all of the problems here. The assumption appears to have been made that cases referred to the program would have been likely to have been placed in the absence of the program. The comparison group of children discharged from care presumably was composed of those deemed unlikely to need further care in the near future, otherwise they would not have been discharged. Hence, the groups could not have been considered comparable in placement propensities at the outset. However, a further complication is that the assumption that referrals to family preservation programs consist of imminent risk of placement cases has been proven incorrect in controlled studies, as we shall see below.
10 The Michigan Families First program began in 17 counties in 1988 and was quickly expanded to the rest of the state. According to data from the Michigan Department of Social Services (1993), the number of new foster care placements increased steadily from 6,490 in 1988 to 8,299 in 1991, followed by a decrease to 7,632 new placements in 1992. The foster care caseload in Michigan grew from 15,878 in 1988 to 17,124 in 1992. These data are somewhat at variance with data in the Multistate Foster Care Archive of the Chapin Hall Center for Children. Archive data indicate that there were 6,368 new admissions in 1988, increasing to 7,188 in 1991, with a decrease to 6,603 in 1992. Archive data on the foster care census in Michigan show a total of 10,901 at the end of 1988, increasing to 12,671 by the end of 1991, decreasing to 12,265 in 1992. During this period discharges from care steadily increased.
11 See Kinney, Haapala, and Booth 1991; Berry 1992; Pecora, Fraser, and Haapala 1992; Hartman 1993.
12 See Wilson, 1994. Interviews with child protective services workers in Illinois also suggest that this practice is viewed as advocacy on behalf of the client.