There is little solid evidence that programs aimed at preventing out-of-home placements or reunifying families with children in foster care have the intended effects. Results of nonexperimental studies have been misleading and the findings of controlled studies in these areas are mixed. Some studies involved samples that were so small that it would have been quite difficult to detect significant program effects. Further, information about the nature of interventions was often incomplete. In response to these problems, recent evaluations have used larger samples and increasingly more sophisticated methods--including the use of comparison or control groups; systematic collection of data on family problems, services, and outcomes; and attempts to understand factors related to outcomes for families. Yet, there are few large, well-controlled studies of family preservation and reunification programs. Problems of sample size and questions about the nature of services provided and the comparability of groups remain, even in recent experiments (Bath and Haapala 1994, Littell 1994).
As to the effects of intensive family preservation services on placement and maltreatment, many of the programs studied did not focus on populations that had high rates of placement or maltreatment and, thus, these rates in both experimental and control groups were low. Hence, the possibility of detecting effects on placement or maltreatment was low. It is not surprising, then, that few studies have demonstrated program effects in these areas and that, in the studies that have found such effects, they tend to be small and short-lived. Targeting problems are apparent in reunification projects as well (reunification rates in some projects have been quite low), but these are not as well documented as in placement prevention programs.50 Efforts to improve targeting in these areas are needed and these should be the subject of further study.
Our review suggests that family preservation programs have very modest effects on family and child functioning. Researchers have found few significant differences between program and comparison groups in levels of child and family functioning after services have been provided and the results of available studies are conflicting. We suggest that it is not realistic to expect dramatic results in this area, given the number and magnitude of the problems faced by many child welfare clients and the short- term nature of family preservation services. It should be noted that the approaches that have been tried tend to focus on the parent or the family and often ignore conditions in the community or larger social environment that may contribute to child maltreatment and other problems in family functioning.
Evaluations of family reunification programs are in a nascent stage. A few studies have reported reunification rates that are encouraging. There is a need for greater clarification of the goals and expected outcomes of these programs (Ahart et al. 1992) and better understanding of phases in the reunification process (Maluccio, Fein, and Davis 1994). Information on child and family functioning, subsequent maltreatment, and foster care reentry is needed to gauge how well children fare after they return home.
This review provides many lessons for further research on family preservation and particularly for the National Evaluation of Family Preservation Services. To begin, it is evident that evaluations must use the most rigorous methods whenever possible, that is, randomized experimental designs. The story of family preservation research shows that early uncontrolled studies were quite misleading, when viewed in light of later more rigorous studies. It is also evident that if the objective of placement prevention is to be seriously addressed, both programmatically and evaluatively, the problem of targeting must be solved. Whether targeting of these programs can in fact be substantially improved remains an open question, but further efforts should be made before giving up on this issue.
In the view of many, the most promising approach to family preservation is the Homebuilders model. Unfortunately, this approach has not been subjected to large scale, well controlled evaluation, so this should have high priority in planning future evaluations. Beyond attention to this one approach, we need further exploration of the differential effects of various models and the question "do models matter?" needs to be addressed. We also need to get into the details of work with families, to explore the effects of differences in dose, types of services, and other activities with clients.
We know little about the differential impact of services such as these on various subgroups of families. A crucial step in pursuing this issue is the specification of groups to examine. We have no clearly delineated diagnostic system to rely upon here. Do we define groups in terms of presenting problems, family structure, history of involvement with public systems, ages of children and parents, or some other characteristic? Beyond this, there is the overriding question of the interaction between family characteristics and services: what works best for whom?
It seems likely that evaluations will continue to use placement of children as a principal outcome measure, but clearly they should also measure other outcomes, in order to detect potential benefits of these programs. These other outcomes should include maltreatment subsequent to referral, as well as various measures of family and child functioning. It is also clear that families should be followed for some time after the completion of service, to attempt to determine the persistence of effects. How long the follow up period should be is a matter of some debate, but it seems reasonable to follow families for at least a year and possibly two. In this connection, the effects of after care services have rarely been considered in evaluations and should be examined in future work.
Finally, we need to pay more attention to the effects of contextual factors, including community characteristics and availability of community services. It is likely that future evaluations will take place in the context of major changes in the public welfare and child welfare systems in this country, and evaluations must find ways to account for the effects of these changes.
50 The detection of targeting problems in reunification programs is a somewhat different matter than in placement prevention programs. Presumably, all cases referred to reunification programs have a child in placement at the time of referral, so the objective of reunification may be considered to be relevant (if not appropriate) for all cases. Targeting problems could arise in two ways: the referral of cases in which reunification would take place in the absence of the program and referral of cases in which reunification is not possible within the time limits of the program. The first of these problems would be revealed by high reunification rates in randomized control groups while the second problem would be seen in low reunification rates in the experimental group. Of course, low rates of reunification in the group receiving reunification services may be seen as either a targeting problem or as an indication of ineffectiveness of the service.