Several studies examined effects of services on measures of family functioning.33 Some studies have simply examined change in these measures over time, but change can often be explained by maturation, statistical regression, and a host of external influences. Hence, we are interested in comparing the status of the treatment group following services and the status that group would have attained at the same point in time in the absence of the intervention. Alternatively, we might compare amounts of change over the same period of time. Randomized experiments provide the best estimates of such effects.
In the New Jersey study, both the treatment and control groups made gains on the Moos Family Environment Scale, Interpersonal Support Evaluation List, and Child Well-Being Scales, but there were few statistically significant differences between groups in the amount of change (Feldman 1991).34
In Meezan and McCroskey's (1993) study, six scales were used to measure family functioning: parent-child interactions, living conditions of the family, interactions between caregivers, supports available to parents, financial conditions of the family, and developmental stimulation of children. Families in both in-home services and regular services generally reported that they did not have significant problems in family functioning at case opening and did not see significant change in these areas at case closing. However, families in the in-home services group reported more improvements in living conditions and financial conditions at one year after termination, compared to families who received regular child protective services. Parents in the program group also reported more improvements in their children's behavior between referral and case closing, although there were no differences between groups one year after services had ended. In contrast to parents' views of family functioning, workers who provided home-based services reported that the families had significant problems in all areas of family functioning at case opening and made significant improvements in four of six domains at case closing. (The four areas in which improvements were noted were: parent-child interactions, living conditions, supports available to families, and developmental stimulation given to children.) The validity of workers' ratings of change in cases in which they are invested is open to question. Further, since caseworkers' reports were not available for the control group, we cannot be sure that changes reported by workers were due to the services provided.
Measures of child and family functioning in eight domains were obtained through interviews with a subsample of parents in three of the six sites in the Illinois experiment (Schuerman, Rzepnicki, and Littell 1994). The domains were: housing conditions, economic conditions, physical child care, discipline and emotional care of children, children's academic adjustment, children's conduct, children's symptomatic behavior, and parental coping skills. A few statistically significant differences between groups were found (most were in favor of the program group), but these gains were modest and did not last over time.35
33 See Wells and Whittington 1993; Berry 1992; Fraser, Pecora, and Haapala 1991; Yuan et al. 1990; Mitchell, Tovar, and Knitzer 1989; and Jones 1985. Other studies have compared measures of functioning for clients who received home-based services and those with children in foster care (e.g., Wald, Carlsmith, and Leiderman 1988).
34 Measurements on social support and the Family Environment Scale were taken at three points in time, at the beginning of service, at termination (six weeks after referral for the control group), and three months post termination. Child well being was measured only at Time 1 and Time 2. On only one of four social support measures (availability of people to do things with) was there a difference between experimentals and controls in time 2 to time 3 change, favoring the experimental group. On child well being there was no difference in change scores overall, one of three subscales (parental role performance) did show a significant difference in change, favoring the experimental group. On the Family Environment Scale there were no differences in change between time 1 and time 2 on any of 10 subscales. There were time 1 to time 3 differences in change on two scales, family cohesion and personal growth, again favoring the experimental group.
35 In each of the three waves of interviews in Chicago, differences were found favoring the family preservation group in one of the eight domains of functioning, but these improvements were not stable over time. Family preservation clients reported fewer problems in housing at the first interview, fewer problems in physical child care at the second interview, and fewer problems in children's academic adjustment in the third interview. An analysis of change over time indicated that the proportion of problems reported in children's academic adjustment in Chicago tended to decrease over time in Family First cases, while it increased among regular services cases. In the other two areas, differences over time favoring the Family First group were found in one or two of the domains, but these effects were modest and short-lived. Overall, the program had no significant impact on parents' feelings of self-efficacy or on the availability of informal social support. Improvements in the receipt of informal support were quite limited and disappeared over time. The program had no lasting effects on the use of formal services.