This evaluation of family preservation programs was designed to assess the extent to which key goals of the programs are being met: the goals of reducing foster care placement, maintaining the safety of children, and improving family functioning.
The assessment of effects on placement and safety of children was based on administrative data which were available on families for at least one year after the beginning of service. Family functioning was assessed through interviews with caretakers at the beginning of service, one month later (at the end of service for the family preservation group), and a year after the beginning of service. Interviews with caseworkers were also conducted at the beginning and one month points.
No significant differences were found between the experimental and control groups on family level rates of placement, case closings, or subsequent maltreatment. There were a few child and family functioning items in which the experimental group displayed better outcomes than the control group in at least one of the states. However, these results did not occur in more than one state. It was found that family preservation programs in two states resulted in higher assessments by clients of the extent to which goals have been accomplished and of overall improvement in their families' lives.
Reducing foster care placement. In none of the three states were there statistically significant differences between the experimental and control groups on family level rates of placement or case closings (see Table 5). In Kentucky, placement rates at the end of one year were 23 and 24 percent for the experimental and control groups, respectively. In New Jersey and Tennessee, the percents were about 28 and 22 percent.
As to be expected with any program, some of the families assigned to family preservation programs did not receive the services or received a minimal dosage of the services. Also, a small number of the families in the control group were actually provided family preservation services. To address these issues, analyses were conducted in which these cases were dropped (secondary analysis). Results of the secondary analyses were quite similar to the primary analyses, also showing no significant differences between the groups in rates of placement.5
The ideal family preservation case is one in which there has been a recent significant crisis in the family, resulting in the maltreatment that triggers the possibility of removal of the child from the home. Subsamples of cases that approached this ideal were examined. Again in these analyses there were no statistically significant differences between the experimental and control groups in placement rates over time.
In addition to placement rates at various points in time, placement was examined in terms of proportion of time in substitute care subsequent to random assignment. No significant differences were found in care days for the families in any of the three states. In Kentucky both the experimental and control group children spent an average of 6 percent of the days subsequent to random assignment in care. In New Jersey, experimental group children spent an average of 6 percent of that time in placement compared to 4 percent for the control group children. In Tennessee, experimental group children spent an average of 10 percent of that time in placement, compared to 5 percent for the control group children.
NOTE: Primary analyses included all cases randomly assigned, except for cases that were determined to be inappropriate referrals. Secondary analyses dropped two categories of cases: families in the control group that were actually provided family preservation services ("violations") and families in the experimental group that received no or little service ("minimal service cases"). "Refined" analyses were limited to subgroups that were thought to represent better targeted cases. Most of the analyses above are of records of placements in administrative data. In Kentucky and Tennessee data were also available from case records on placements with relatives that were not recorded in the administrative data. Those data are included in the rows labeled "including relative placement."
Targeting. Since these programs were intended to prevent the placement of children, the target group for the services was families in which at least one child was "in imminent risk of placement." As in previous studies, it was found that most of the families served were not in that target group. This is shown by the placement rate within a short period of time in the control group, indicating the placement experience in the absence of family preservation services. In all three states, the placement rate in the control group within one month was quite low. It would, therefore, have been virtually impossible for the programs to be effective in preventing imminent placement, since very few families would have experienced placement within a month without family preservation services.
A number of subgroups that were thought to represent better targeting were examined. These included cases coming directly from the investigation of an allegation of abuse or neglect, cases with recent substantiated allegations of abuse or neglect, and, in Kentucky, a subgroup of cases in which workers had submitted petitions to the court for placement or some other court-ordered intervention. In none of these subgroups did placement rates in the control group within one month exceed 12 percent. Hence, even in these more refined (from the standpoint of targeting) subgroups, the intended target group was not in evidence.
It should be noted that the results found here occurred despite efforts in this project to improve targeting. In Kentucky and New Jersey a special screening form, developed by the evaluation team, was employed to rate the risk to children with the intent that cases with intermediate risk would be referred to the program. In Kentucky efforts were made to divert to family preservation cases that had been referred to the court. In Tennessee, special training efforts were instituted to address concerns about targeting.
Child Safety. Maltreatment subsequent to the beginning of service was generally not related to experimental group membership, except for one subgroup in Tennessee. Subsequent maltreatment was measured by the occurrence or nonoccurrence of a substantiated allegation of abuse or neglect following an investigation of such an allegation. The rate of subsequent maltreatment was relatively low, about 18 percent of the families in Kentucky had a substantiated allegation within one year of random assignment; in New Jersey the rate was 12 percent and in Tennessee, 25 percent. In Tennessee, in those families with an allegation within 30 days prior to random assignment, the experimental group children experienced fewer substantiated allegations than children in the control group did.
The findings of little difference between the experimental and control groups in subsequent maltreatment can be read in two ways. It indicates that families served by family preservation were no more likely than families not receiving the services to be subjects of allegations of harm. In this sense, children were, by and large, kept safely at home while receiving family preservation services. However, children in both groups were primarily in their homes, and family preservation did not result in lower incidence of maltreatment compared with children in the control group.
Subgroups. In an effort to identify groups of cases for which family preservation is effective, subgroups of Kentucky and New Jersey cases were examined.6 Subgroups were defined in terms of problems of the family (e.g., substance abuse, financial difficulties, and depression) and family structure. Within these subgroups, experimental and control groups were compared on placement and substantiated allegations subsequent to random assignment. Only one significant difference was found. Among single mothers in New Jersey, those in the experimental group were less likely to have a subsequent substantiated allegation than those in the control group. No subgroups were found in which there were effects on placement in either state.
Family functioning. In a few areas of family functioning, in one or the other of the states, families in the experimental group appeared to be doing better at the end of services (see Table 6). There were very few differences at the year follow-up and in changes over time. Those differences that did appear (primarily at the end of services) were not consistent across states and were not maintained. Family functioning was assessed through caretaker and caregiver interviews at three points in time, shortly after the beginning of services, four to six week later (at the end of services for the Homebuilders group), and again a year after services began. Areas assessed included life events, economic functioning, household condition, child care practices, caretaker depression, child behavior, and caretaker functioning. It can be said that family preservation services may have small, apparently short-term, effects on some areas of functioning. There was one item with some consistency across sites, the overall assessment of improvement by caretakers. At post treatment, a significantly larger proportion of experimental group caretakers in Kentucky and New Jersey generally thought there was "great improvement" in their lives. In Tennessee, although not significant, results tended in the same direction.
5. It should be noted that the most rigorous approach to analysis requires that cases be maintained in the groups to which they were randomly assigned. Random assignment is used to assure that the groups are as similar as possible at the outset of service. Removing cases from the groups or switching cases from one group to another threatens group equality and allows for the possibility that post-treatment differences could be explained by factors other than service. In particular, it is likely that violations and minimal service cases differ in systematic ways from other cases. Hence, the secondary analyses should be viewed with caution.
6. The number of cases in Tennessee was too small to allow subgroup analysis.