[Main Page of Report | Contents of Report]
This chapter describes the outcomes of the programs in Kentucky, New Jersey, and Tennessee, the three Homebuilders states in the evaluation. Outcomes for Philadelphia are presented in Chapter 4 of this volume.
The outcomes we examined were the placement of children in substitute care following random assignment to the experimental or control group, subsequent reports of maltreatment and a number of measures of child and family functioning. The focus is on comparisons between the experimental and control groups. Analyses we have designated as "primary" were conducted on all randomly assigned cases except those that were determined to be inappropriate referrals. This includes cases in which the assignment was violated (cases assigned to the control group that were given family preservation services) and cases assigned to family preservation that received no or little such service. Insofar as family preservation services have effects, inclusion of these cases in the analysis will tend to reduce the observed differences between the groups. However, the most rigorous approach to analysis requires that we retain these cases in the group to which they were assigned in order to maintain the statistical equivalence of the groups at the outset of the experiment, which is the reason for random assignment in the first place.
It is likely that violations and minimal service cases differ in systematic ways from other cases (perhaps not detected in the measurements of the study), hence, switching them to the other group would result in groups that were not equivalent at the beginning. It can be argued that inclusion of minimal service cases in the experimental group is quite proper on other grounds: the implementation of any program will involve some cases that do not receive the service, and estimates of impact ought to take that into account. We did conduct analyses ("secondary" analyses) in which the violations and minimal service cases were dropped, so as to examine differences between cases that actually received the intended treatment (family preservation or regular services). This analysis must be viewed as only suggestive, since it does not preserve the initial statistical equivalence of the groups created by random assignment. In fact, the results of the secondary analyses were usually similar to those of the primary analysis. We note in footnotes when the secondary analysis differed substantially from the primary analysis. The secondary analysis tables are presented in Appendix I, Volume 3.
Some analyses were also conducted on a more "refined" sample in which we attempted to focus on cases that approached a conception of "ideal" family preservation cases. Family preservation services are designed for families in crisis, presumably the crisis surrounding a recent allegation of maltreatment, the investigation of that allegation, and the threat of removal of a child. Theoretically, this state of crisis makes families more willing to seek and respond to help. As indicated in Chapter 7 in Volume One of the report, many of the families did not appear to conform to this specification of the target group. In Kentucky and New Jersey we looked at two subgroups of cases, those with a recent substantiated allegation recorded in the administrative data (within three months prior to referral to family preservation services) and those in which an investigative worker was involved. In Tennessee, nearly all of the cases came from investigating workers, so we looked at those cases with an allegation within 30 days prior to referral.
[Go To Contents]
A principal goal of family preservation services is the prevention of placement into substitute care, so that must be the first (though not the last) outcome examined. Placement included foster care, institutions and residential treatment programs, group homes, and adoptive placements. (23) We are initially concerned with the character and timing of the first placement of a child following random assignment. We collected data on placement prior to November 30, 1999 in Kentucky, September 30, 1999 in New Jersey, and August 31, 1999 in Tennessee. (24) Although data were provided at the individual level, most of the analyses are presented at the family level. (25) In Kentucky, the administrative files contained data on 1130 children in 345 families, 172 in the experimental group and 171 families in the control group. One hundred thirty-nine children in 61 families (36%) in the experimental group experienced placement subsequent to random assignment compared to 96 children in 55 families (32%) in the control group. In New Jersey, administrative data were available on 1290 children in 442 families, 275 in the experimental group and 167 in the control group. One hundred sixty-six children in 109 families (40%) in the experimental group were placed compared to 55 children in 48 families (29%) in the control group.
In Tennessee, multiple sources of data were used to calculate the rate of subsequent placement. A statewide management information system (CORS) provided information on formal paid placements. Additionally, case record reviews provided information on unpaid relative placements. In Tennessee, placement data were available on 468 children in 140 families, 93 in the experimental group and 47 in the control group. In the analysis of CORS data, forty-six children in 23 families (25%) in the experimental group experienced placement subsequent to random assignment compared to 25 children in 10 families (21%) in the control group. Including unpaid relative placements, 60 children in 29 families (31%) in the experimental group experienced placement subsequent to random assignment compared to 31 children in 13 families (28%) in the control group. These differences were not statistically significant at the family level in Kentucky, New Jersey or Tennessee (see Table 3-1 for types of placements after random assignment).
A comparison of these percents is, however, misleading, because of varying periods of risk of placement. The proper approach to the analysis of such data is survival analysis, in which the proportions of cases placed at each point in time following random assignment in each group are compared, accounting for the numbers of cases that "survive" to that point. We examined survival curves for each group and determined whether these curves were statistically different. Family level analyses were based on the first date of placement of any child in the family if a placement occurred.
| Kentucky | ||
|---|---|---|
| Type | N | % |
| Foster care | 144 | 64.0 |
| Private institution | 69 | 30.7 |
| Foster care, medically fragile | 6 | 2.7 |
| Child psychiatric hospital | 4 | 1.8 |
| Not specified | 2 | 0.8 |
| Total | 225 | 100 |
| New Jersey | ||
|---|---|---|
| Type | N | % |
| Foster care | 102 | 46.1 |
| Juvenile family crisis | 47 | 21.2 |
| Residential treatment | 35 | 15.9 |
| Group home | 17 | 7.7 |
| Public institution | 8 | 3.6 |
| Shelter care | 5 | 2.3 |
| Adoptive | 4 | 1.8 |
| Relative | 3 | 1.4 |
| Total | 221 | 100 |
| Tennessee | ||
|---|---|---|
| Type | N | % |
| Foster care | 31 | 44.3 |
| Relative home | 9 | 12.9 |
| Trial home | 6 | 8.5 |
| Residential | 6 | 8.5 |
| Continuum contract | 4 | 5.7 |
| Non-relative home | 4 | 5.7 |
| Adoptive home | 3 | 4.3 |
| Runaway | 2 | 2.8 |
| Shelter | 2 | 2.8 |
| Independent living | 2 | 2.8 |
| Detention | 1 | 1.4 |
| Total | 71 | 100 |
| Note: Includes only placements recorded in administrative data. There were additional unpaid relative placements (see text). | ||
Kentucky. The family level analysis of subsequent placement is displayed in Figure 3-1. (26) These survival curves show the proportion of families remaining intact (without placement of a child) at each point in time following random assignment. The curves begin at 1, indicating that at the time of random assignment, all children were at home. The curves then decline as children enter care. The higher curve at any point represents the group with fewer placed children at that point. The curves are adjusted for cases that are "right censored." For example, cases that were not observed for a full year following random assignment are dropped in the calculation of the percentage remaining intact ("surviving") at one year. The Wilcoxon statistic indicates that the survival rates for the experimental and control groups are not statistically different. At the one-year interval, 25 percent of experimental group families and 24 percent of control group families experienced substitute care placement. At the end of two years, 32 percent of the experimental group and 27 percent of the control group families experienced substitute care placement.
"Refined" groups analyses were also conducted, limiting the sample to cases referred by investigative workers and to those families with substantiated allegations within the three months prior to random assignment. Significant differences did emerge for families with a substantiated allegation within three months prior to random assignment. In the primary analysis of those families coming from an investigative worker, 26 percent of the experimental group and 15 percent of the control group experienced subsequent placement within one year after the random assignment date. For those with recent substantiated allegations, 29 percent of the experimental group and 13 percent of the control group experienced subsequent placement within one year (significant at .05 level).
An additional "refined" group was available for analysis in Kentucky. Prior to random assignment, workers submitted petitions to the court for placement or some other court ordered intervention on 66 families. Administrative data were available for all 66 families (32 in the experimental group, 34 in the control group). Survival analyses were conducted to explore the relationship between family preservation services and subsequent placement. At one year after random assignment, 22 percent of the experimental group and 29 percent of the control group experienced placement; a nonsignificant difference.
Figure 3-1
First Placement after Random Assignment (Families)
In addition to the administrative data on placement, in Kentucky the Westat site coordinator attempted to document all placements subsequent to random assignment, based on her contacts with caseworkers. The administrative data file contained placements not recorded by the site coordinator, and vice versa. The only systematic difference between these data sources was the documentation of relative placement. Relatives are generally not paid for placements in Kentucky, so these data were not recorded in the administrative files. Survival analyses were conducted with a combination of caseworker and administrative placement records. If either data source recorded a placement event, that family was coded as experiencing subsequent placement. The first documented date of placement, taken from either source, was selected for analysis. The patterns of placement in these analyses are similar to those reported above. At one year, 27 percent of the experimental group and 32 percent of the control group families experienced placement, a nonsignificant difference.
In addition to survival analyses, placement can be examined in terms of the proportion of time in substitute care subsequent to random assignment. If family preservation services are effective in preventing placements, we would expect them to result in lower numbers of days in foster care. Family preservation might also result in shorter stays in care, once children are placed. Comparison of days in care provides a beginning look at the question of whether family preservation results in lower costs of foster care (of course, a complete cost-effectiveness analysis must also factor in the differential costs of family preservation and regular services).
The proportion of time in care is calculated by dividing the number of days in care by the number of days of possible care (number of days between random assignment and the date of administrative data collection). As the proportions are calculated at the family level, the number of days in care represents the total number of care days summed across all children within a particular family. Similarly, the number of possible care days represents the total number of possible care days summed across all children within a particular family. The number of possible care days is adjusted for a child's eighteenth birthday and for births since random assignment. For both primary and secondary analyses, in both the experimental and control groups children spent an average of 6 percent of the days subsequent to random assignment in care.
New Jersey. The family level analysis of placements is shown in Figure 3-1. (27) More families in the experimental group experienced placement of a child than in the control group (at one year, 29% of the experimental group vs. 22 percent of the control group; at two years, 39 percent of the experimental group vs. 28 percent of the control group) although the differences are not significant. It might be noted that in the analyses the survival curves for the two groups tend to begin to diverge at about 6-7 months, that is, at about that time more children in the experimental group are being placed. We do not have a ready explanation for this divergence.
Refined groups analyses in New Jersey revealed statistically significant differences. In the primary analysis of those families coming from an investigative worker, 25 percent of the experimental group and 15 percent of the control group experienced subsequent placement within one year of the random assignment date. For those with recent substantiated allegations, 25 percent of the experimental group and 14 percent of the control group experienced subsequent placement.
As to the proportion of time that children spent in care in New Jersey, experimental group children spent an average of 6 percent of that time in placement, compared to 5 percent for the control group children (not a significant difference).
Tennessee. Survival rates at the family level were first calculated using only the CORS and then including relative placement (the "any evidence" analysis) data. The family level analyses of subsequent placement is displayed in Figure 3-1. (28) The Wilcoxon statistic indicates that the survival rates for the experimental and control groups are not statistically different. In the analysis of CORS data, 23 percent of experimental group families and 19 percent of control group families experienced substitute care placement within one year subsequent to random assignment. In the "any evidence" analysis, 28 percent of the experimental group families and 23 percent of control group families experienced placement within one year subsequent to random assignment.
As in Kentucky and New Jersey, a "refined" group was available for analysis in Tennessee. Ninety-three families had an allegation within 30 days prior to random assignment. The Wilcoxon statistic for the survival analysis of placement in these families indicates that the survival rates of the two groups are not statistically different. In the analysis of CORS data, 17 percent of the experimental group and 15 percent of the control group experienced subsequent placement within one year of random assignment. In the "any evidence" analysis, 22 percent of the experimental group and 21 percent of the control group experienced subsequent placement within one year of random assignment.
As to the proportion of time that children spent in care in Tennessee, experimental group children spent an average of 10 percent of that time in placement, compared to 5 percent for the control group children. This difference is nonsignificant.
In a number of analyses of subsequent placement in these states, more experimental group families experienced placement than did control group families. In a few analyses, fewer experimental group families experienced placement. However, none of these analyses were statistically significant; in none of these states can the data be taken as firm evidence that family preservation resulted in more placements. Nor is there evidence that it resulted in fewer.
Imminent Risk of Placement. The family preservation programs in these states are designed to prevent the unnecessary removal of children by serving families with children who are at imminent risk of out-of-home placement. (29) One way to explore the accuracy of the "imminent risk" designation is to examine the proportion of control group families that experienced placement within a short time after random assignment. Since the control and experimental groups were randomly assigned and are expected to be statistically equivalent before services are begun, the proportion of families experiencing placement in the control group indicates the proportion of referred families that would have experienced placement in the absence of receiving family preservation services. We looked at control group placement rates 30 days after random assignment, believing that time period provided a liberal interpretation of "imminent risk." If a significant proportion of the control group experienced placement within 30 days of random assignment, one could argue that the program was appropriately targeted. At the time of random assignment, referring workers were asked to designate those children who were considered "at risk."
In Kentucky, in the first 30 days following random assignment, in the primary analysis 4 percent of at risk children in the experimental group were placed compared to 3 percent of control group at risk children. At the family level, 6 percent of the experimental group families and 5 percent of the control group families experienced placement within the first 30 days subsequent to random assignment. The percentages were similar in the investigative group (8% of the experimental compared with 5% of the control group), and among those with recent substantiated allegations (6% of the experimental group compared with 3% of the control group).
In New Jersey, of those children judged to be at risk, 4 percent of the control group and 3 percent of the experimental group were placed in 30 days. At the family level, 5 percent of the families in the experimental group experienced placement of at least one child within one month of random assignment, compared to 6 percent of the control group. Rates of imminent placement were similar in the "refined" group analyses. Of those families coming from an investigative worker, 3 percent of the experimental group and 5 percent of the control group experienced placement within 30 days of random assignment. For those families with a substantiated allegation within three months prior to random assignment, 8 percent of the experimental group and 5 percent of the control group experienced placement within 30 days.
In Tennessee, rates of placement within one month were somewhat higher than in Kentucky and New Jersey. Of those children judged to be at risk, 13 percent of the control group and 11 percent of the experimental group were placed in 30 days. There were no relative placements within the first 30 days subsequent to random assignment. Thus, there are no differences between the CORS and "any evidence" analysis. At the family level in Tennessee, the CORS administrative data indicates that 11 percent of both the experimental and control groups experience placement within 30 days subsequent to random assignment. Rates of imminent placement were similar in the "refined" group analyses. Of those families with a recent allegation (within 30 days prior to random assignment), 7 percent of the experimental group and 12 percent of the control group experienced a CORS placement within 30 days of random assignment.
Although the percentages of placement within one month were somewhat higher in Tennessee, in all three states, these percentages were quite low. The numbers of interest here are those for the control group, indicating the targeting efficiency of the program in these three sites is very low.
[Go To Contents]
Hazard analyses permit the examination of the effects of multiple independent variables (in addition to experimental group membership) on rates of placement. They also provide somewhat more precise estimates of the effect of experimental group membership, since they control for the effects of the other variables in examining experimental-control group differences. In addition, they allow for the examination of "interactions" between other variables and experimental-control group membership, to see if the effects of experimental group membership differ for subgroups of the sample. We conducted Cox regression analyses of placement hazards using as predictor variables case characteristics available in the administrative data. Case characteristics in the administrative data are quite limited. Unlike the survival analyses which were conducted at the family level, hazard analyses were done at the child level because we wanted to include in them characteristics of children.
Kentucky. We examined the effects of the child's age, race, prior placement, and prior substantiated allegation of maltreatment, together with experimental group membership, on rates of placement. Regarding main effects, there were no significant predictors of placement. However, there was a significant interaction between experimental group membership and prior placement. Examining the differences in placement rates between the experimental and control groups by whether or not the child had had a prior placement indicates that among those with a prior placement, there is little difference in placement rates (32% for the family preservation group and 34% for the control group) while there is a significant difference for those without prior placement (22% for the experimental group and 14% for the control group). The interactions between experimental group membership and age and prior substantiated allegations were not significant.
New Jersey. New Jersey hazard analyses indicate that older age and prior placement increase the hazard rate significantly (p < .05; prior placement by 88% and each year of age by 3%). Experimental group membership was also significant (p < .08; experimental group membership increases the hazard rate by 97%). The interactions between experimental group membership and race, age, prior placement, and prior substantiated allegations were not significant.
Hazard analyses were also performed to examine the effect of county on placement. These analyses were conducted at the family level. Burlington county was chosen as the reference category, as it had the highest rate of placement. Thus, rates of placement in the other New Jersey counties are compared to the placement rates of Burlington. In addition to the county variables, experimental group and interactions of county with experimental group were entered into the regression equation. The hazard of placement for families was decreased by 67 percent for Ocean county, 73 percent for Monmouth county, 47 percent for Essex county, 57 percent for Bergen county, and 74 percent for Passaic county. The coefficient associated with Camden county was non-significant. There were no significant effects of experimental group or of county-experimental group interactions. This indicates that even after removing county variation, there are no significant differences between the experimental and control groups, nor does the effect of experimental group vary by county.
Tennessee. We examined the effects of the child's age, race, prior placement, prior allegation within 30 days of random assignment, prior substantiated allegation within 30 days of random assignment, and experimental group membership on rates of placement. Similar to Kentucky and New Jersey, we also explored interactions between experimental group membership and child characteristics. No significant interactions emerged. Only prior substantiated allegation had a significant effect on the likelihood of placement subsequent to random assignment. In the analysis of the CORS administrative data, a substantiated allegation within the last 30 days prior to random assignment increased the hazard rate by 209 percent. When unpaid relative placements were included ("any evidence") prior substantiation increased the hazard by 173 percent.
[Go To Contents]
Subsequent maltreatment of children is a second important outcome to be examined. Family preservation programs are intended to lower the risk of harm to children while keeping them at home, and subsequent maltreatment is an indicator of such risk. Furthermore, the justification for family preservation programs rests on the belief that the safety of children is not compromised when their families are referred to these programs, so examination of subsequent maltreatment rates is important to determine whether children, in fact, are safe in these programs.
As with placement, data on subsequent maltreatment come from the administrative data files of the states. As is almost always the case in studies like this, our data do not record actual maltreatment, but only investigated reports of maltreatment. Some abuse and neglect goes unreported, and, because not every report is investigated, there are cases of harm that are reported but not investigated.
As with the analyses of subsequent placement, survival graphs were developed to compare the timing of subsequent, substantiated allegations of maltreatment. (30) Again, survival analyses were conducted at the family level for both the primary and secondary analysis groups as well as for the "refinement" groups.
Kentucky. Two hundred twenty-three children in 99 families (58%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared with 206 children in 100 families (58%) in the control group. The distribution of the various types of allegations is as follows: 2 percent dependency, 7 percent emotional, 63 percent neglect, 25 percent physical abuse, and 8 percent sexual maltreatment. As families can be the subjects of multiple allegations on any given day, these percentages do not sum to 100. Not all investigations result in substantiated allegations. One hundred forty-three children in 63 families (37%) in the experimental group were the subjects of substantiated allegations of maltreatment compared with 118 children in 57 families (33%) in the control group. The difference was not statistically significant at the family level. The distribution of substantiated allegations is as follows: 1 percent dependency, 4 percent emotional, 72 percent neglect, 20 percent physical abuse, and 3 percent sexual maltreatment.
Figure 3-2 displays the survival curves for substantiated allegations in the primary analysis. At one year subsequent to random assignment, 24 percent of the experimental group and 21 percent of the control group families experienced substantiated reports of maltreatment. Although a higher percentage of families in the "refined" analyses experienced substantiated allegations of maltreatment, similar patterns emerged. For the investigative group, 27 percent of the experimental group and 24 percent of the control group experienced a substantiated allegation of maltreatment in the one-year interval. For those families with a substantiated allegation within the three months prior to random assignment, 25 percent of the experimental and 21 percent of the control group experienced substantiated allegations of maltreatment within a year subsequent to random assignment. For the group on which petitions had been submitted to court for placement or other orders, 22 percent of the experimental group and 33 percent of the control group experienced a substantiated allegation within one year subsequent to random assignment, a nonsignificant difference.
Figure 3-2
First Substantial Allegation after Random Assignment (Families)
The survival analyses indicate that experimental and control group families had a very similar likelihood of substantiated reports of maltreatment subsequent to random assignment.
New Jersey. One hundred seventy-eight children in 94 families (34%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared to 101 children in 60 families (36%) in the control group. Fifty-eight children in 34 families (12%) in the experimental group experienced a substantiated allegation of maltreatment following random assignment, compared to 42 children in 29 families (17%) in the control group. In none of the survival analyses conducted were there significant differences between the experimental and control groups. Figure 3-2 shows substantiated allegations at the family level. About 11 percent of families in both groups have substantiated allegations within one year.
Patterns of substantiated allegations were similar for the "refined" group analyses, none of which showed significant differences between groups. Of those families coming from an investigative worker, 7 percent of the experimental group and 10 percent of the control group had a substantiated allegation within one year subsequent to random assignment. For those families with a substantiated allegation within three months prior to random assignment, 10 percent of the experimental group and 16 percent of the control group had a substantiated allegation within one year subsequent to random assignment.
Tennessee. Allegation data were available for 482 children in 144 families. Sixty-four children in 36 families (38%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared with 61 children in 26 families (54%) in the control group. The differences were not statistically significant at the family level. The distribution of the various types of allegations is: 66 percent physical abuse, 20 percent supervision/neglect, 2 percent sexual abuse/medical, and 12 percent other (includes allegations such as failure to thrive, truancy, and unruly child). Forty-four children in 25 families (26%) in the experimental group were the subjects of substantiated allegations of maltreatment compared with 42 children in 18 families (38%) in the control group. These differences were not statistically significant at the family level. The distribution of the various types of substantiated allegations is: 66 percent physical abuse, 20 percent supervision/neglect, 1 percent sexual abuse/medical, and 13 percent other.
Figure 3-2 displays the survival curves for substantiated allegations in the primary analysis. At one year subsequent to random assignment, 24 percent of the experimental group and 25 percent of the control group families experienced substantiated reports of maltreatment. Survival rates were also calculated for those families with an allegation within 30 days prior to random assignment. Significant differences emerged for subsequent allegations and near significant differences emerged for subsequent substantiated allegations. Of those families with a recent allegation, 28 percent of the experimental group and 52 percent of the control group experienced an allegation within one year subsequent to random assignment. Similarly, 18 percent of the experiment group and 30 percent of the control group experienced a substantiated allegation within one year subsequent to random assignment. These differences suggest that in Tennessee family preservation reduced the likelihood of subsequent maltreatment for those families with recent allegations.
[Go To Contents]
In Kentucky and New Jersey, we examined a number of subgroups of cases to determine whether we could detect differences between experimental and control groups on placement and substantiated allegations subsequent to random assignment within each subgroup. The number of cases in Tennessee was not sufficient to support subgroup analysis. The results are shown in Table 3-2. Most of the subgroups were defined in terms of problems existing at the time of the initial interview. For both placement and substantiated allegations the table shows the number of cases in each subgroup and the percentage of cases in the subgroup experiencing the event within 12 months. The analysis involved determining the significance of the difference between the experimental and control groups in the occurrence of the event within twelve months, and in the survival curves as a whole. (31) The first row of the table shows the results for the Kentucky and New Jersey samples as a whole. Except for substance abuse, the definitions of the subgroups were taken from the initial caretaker interview. Very few caretakers acknowledged substance use in the first interview, so that subgroup was determined from information in both the caretaker and caseworker initial interviews.
None of the 36 experimental-control group comparisons were significant at the .05 level. In the analysis so far, efforts to find subgroups for which family preservation service was related to reduced placement have been unsuccessful.
Family preservation services are sometimes thought to lead to quicker case closings in the public agency and less frequent subsequent involvement with the child welfare agency. Administrative data on case closings and subsequent case openings were examined to determine the effects of these services on case closings and subsequent reopenings.
Kentucky. Of the 255 cases that were open in the public agency at the time of the referral to family preservation services, 180 (71%) were closed some time after the referral and 75 (30%) remained open as of November 30, 1999 (the last date of observation for these analyses). Survival analyses were performed to examine the lengths of time between the referral to family preservation services and the first closing of the case. As shown in Figure 3-3, significant differences were found between the experimental and control groups.
Of the 180 cases that were open at the time of the referral to family preservation services and closed some time after that referral, 10 cases were re-opened again before November 30, 1999. Five of these 10 cases were in the experimental group, and five were in the control group.
| Subgroup | Kentucky | New Jersey | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Placement | Substantiated allegations | Placement | Substantiated allegations | |||||||||||
| In 12 mos | In 12 mos | In 12 mos | In 12 mos | |||||||||||
| N | Overall % |
pa | Survival p |
Overall % |
pa | Survival p |
N | Overall % | pa | Survival p |
Overall % |
pa | Survival p |
|
| Overall | 345 | 25 | 23 | 442 | 27 | .09b | 11 | |||||||
| Substance abuse | 37 | 29 | 29 | 53 | 26 | 20 | ||||||||
| No substance abuse | 244 | 23 | 19 | 326 | 28 | 12 | ||||||||
| Problems with bills | 157 | 26 | 25 | 195 | 27 | 13 | ||||||||
| Problems with daycare | 99 | 22 | 25 | 111 | 26 | 14 | ||||||||
| Depressionc | 152 | 26 | 24 | 168 | 33 | 11 | ||||||||
| Problems with punishment | 205 | 26 | 23 | 262 | 28 | 11 | ||||||||
| Problems with school | 145 | 23 | 22 | 202 | 31 | 11 | ||||||||
| Problems with employment | 193 | 24 | 22 | 192 | 26 | 14 | ||||||||
| Single mother | 129 | 21 | 19 | 116 | 26 | 17 | ||||||||
| a Fisher exact,
two tail b Experimental group more likely to experience placement c Caretakers with depression scores above median for the state |
||||||||||||||
Figure 3-3
First Case Closing after Random Assignment (Families)
New Jersey. Of the 441 cases with administrative case closing data, 427 were open at the time of random assignment. Of these 427 cases, 263 (62%) were closed subsequent to the referral to family preservation services. Figure 3-3 shows the results of a survival analysis in which we examined the time to case closing for the 421 cases that were open at the time of the referral to family preservation services. There was no significant difference between the experimental and control groups in the rate or timing of case closings after the referral date. Of the 263 cases that were closed after the referral to family preservation services, 81 cases (31%) were reopened. There was no significant difference between the experimental and control groups in the proportion of cases that were reopened (32% in the experimental group, 29% in the control group).
Tennessee. Of the 147 families with case opening and closing data, 111 were open in the public agency at the time of the referral to family preservation services. Of these 111 families, 96 (87%) were closed some time after the referral and 15 (14%) remained open as of August 31, 1999 (the last date of observation for these analyses). Survival analyses were performed to examine the lengths of time between the referral to family preservation services and the first closing of the case. As indicated by the survival curves in Figure 3-3, no significant differences were found between the experimental and control groups. Of the 96 cases that were open at the time of the referral to family preservation services and closed some time after that referral, 17 cases were re-opened again before August 31, 1999. There was a significant difference in the rate of reopening. Eight (12%) of the 66 experimental group cases that were closed subsequently reopened, compared with 9 (30%) of the 30 control group cases (p < .05).
[Go To Contents]
Family preservation services are intended to result in improved functioning of children and families. This goal is sought both for its own intrinsic value as well as an intermediate objective in the prevention of subsequent maltreatment and placement; parents who are functioning better and better parent-child relationships should result in lower risk of abuse or neglect.
In our interviews with caretakers and caseworkers we asked a number of questions tapping various aspects of functioning. We asked most of these questions in all three interviews with caretakers (at the beginning of service, four to six weeks after service began, and one year after the beginning of service) and in the two interviews with caseworkers (at the beginning of service and four to six weeks later). In the initial interview, we usually asked respondents to answer in terms of circumstances in the last three months. In the post-treatment and followup interviews, we asked in terms of "since we last talked to you [at the time of the initial interview or the post-treatment interview]." To indicate the effects of family preservation services, we can compare the experimental and control groups on the responses to these questions in the second and third interviews and on change between interviews. We report on the responses to a number of individual items in our interviews. In addition, we combined the responses to many questions into summated scales. (32) We examined differences between experimental and control groups in each state in the average levels of these scales at post-treatment and at followup and we examined changes over time in these averages using multivariate repeated measures analysis. (33) The results of the analyses of the scales are shown in Tables 3-3, 3-4, and 3-5 and in Figure 3-4. (34)
| Post-Treatment | followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Means | Multivariate ps | Univariate ps-time | Univariate ps-Grp-time interaction | |||||||||||||||
| N | Ma | pb | N | M | p | N | Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf |
Post v. Followg |
Initial v. later |
Post v. Follow |
||
| Positive life events | C | 146 | .12 | 119 | .22 | 108 | .16 | .12 | .21 | .42 | .001 | .41 | .001 | |||||
| E | 148 | .14 | 130 | .21 | 117 | .18 | .14 | .20 | ||||||||||
| Negative life events | C | 146 | .04 | 119 | .07 | 108 | .08 | .03 | .07 | .40 | .001 | .27 | .001 | .001 | ||||
| E | 148 | .03 | 130 | .09 | 117 | .10 | .03 | .09 | ||||||||||
| Life events depression | C | 145 | .35 | 119 | .37 | 107 | .45 | .36 | .38 | .40 | .001 | .38 | .001 | |||||
| E | 147 | .36 | 130 | .40 | 117 | .50 | .37 | .42 | ||||||||||
| Economic functioning | C | 142 | .17 | 118 | .17 | 105 | .22 | .15 | .18 | .08 | .001 | .27 | .001 | |||||
| E | 144 | .22 | 127 | .20 | 111 | .32 | .23 | .20 | ||||||||||
| Punishment | C | 143 | .16 | 113 | .15 | 101 | .22 | .17 | .15 | .49 | .001 | .36 | .001 | .09 | ||||
| E | 147 | .17 | 121 | .15 | 109 | .25 | .18 | .15 | ||||||||||
| Child aggression | C | 146 | 1.36 | 119 | 1.24 | 108 | 1.56 | 1.44 | 1.29 | .84 | .001 | .96 | .001 | .05 | ||||
| E | 148 | 1.33 | 130 | 1.29 | 117 | 1.57 | 1.44 | 1.32 | ||||||||||
| School problems | C | 112 | .22 | 96 | .25 | 78 | .29 | .24 | .27 | .14 | .03 | .41 | .01 | |||||
| E | 101 | .20 | 97 | .19 | 77 | .26 | .21 | .18 | ||||||||||
| Child withdrawn | C | 146 | .89 | 119 | .99 | 108 | 1.08 | .89 | 1.00 | .99 | .001 | .29 | .001 | |||||
| E | 148 | .93 | 130 | .93 | 117 | 1.08 | .97 | .93 | ||||||||||
| Stolen things or arrested | C | 146 | .34 | 119 | .39 | 108 | .31 | .34 | .39 | .76 | .75 | .07 | .03 | |||||
| E | 148 | .32 | 130 | .31 | 117 | .44 | .35 | .32 | ||||||||||
| Child substance abuse | C | 146 | .07 | 119 | .03 | 108 | .05 | .07 | .04 | .40 | .35 | .67 | ||||||
| E | 148 | .04 | 130 | .02 | 117 | .03 | .03 | .03 | ||||||||||
| Child problems | C | 146 | 2.20 | 119 | 1.89 | 108 | 2.44 | 2.23 | 2.18 | .87 | .001 | .47 | .001 | |||||
| E | 148 | 2.05 | 130 | 2.04 | 117 | 2.56 | 2.14 | 2.07 | ||||||||||
| Negative child behaviors | C | 140 | .34 | 107 | .33 | 98 | .37 | .34 | .34 | .95 | .001 | .47 | .001 | |||||
| E | 139 | .34 | 120 | .34 | 106 | .39 | .34 | .33 | ||||||||||
| Positive child behaviors | C | 142 | .71 | 109 | .67 | 99 | .70 | .69 | .67 | .78 | .31 | .84 | ||||||
| E | 142 | .71 | 121 | .68 | 109 | .69 | .69 | .67 | ||||||||||
| Household condition | C | 142 | .02 | 119 | .02 | 102 | .05 | .02 | .02 | .29 | .002 | .45 | .001 | |||||
| E | 147 | .02 | 129 | .01 | 111 | .03 | .01 | .01 | ||||||||||
| Depression (SCL-90) | C | 145 | .79 | 119 | .67 | 107 | .95 | .77 | .70 | .67 | .001 | .31 | .001 | |||||
| E | 146 | .74 | 130 | .79 | 115 | .96 | .74 | .83 | ||||||||||
| Positive child care practices | C | 140 | .85 | 107 | .82 | 94 | .87 | .85 | .83 | .55 | .09 | .97 | .06 | |||||
| E | 143 | .82 | 116 | .81 | 103 | .85 | .84 | .81 | ||||||||||
| Negative child care practices | C | 141 | .14 | 109 | .12 | 97 | .17 | .15 | .12 | .57 | .001 | .22 | .001 | .05 | ||||
| E | 144 | .13 | 117 | .13 | 104 | .20 | .14 | .13 | ||||||||||
| a Means of
control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three |
||||||||||||||||||
| Post-Treatment | followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf |
Post v. Followg |
Initial v. later |
Post v. Follow |
|||||||||
| Positive life events | C | 133 | .15 | 107 | .23 | 83 | .19 | .16 | .25 | .05 | .001 | .99 | .001 | |||||
| E | 210 | .13 | 166 | .21 | 129 | .17 | .13 | .22 | ||||||||||
| Negative life events | C | 133 | .04 | 107 | .09 | 83 | .13 | .04 | .10 | .46 | .001 | .49 | .001 | .001 | ||||
| E | 210 | .05 | 166 | .11 | 129 | .11 | .04 | .09 | ||||||||||
| Life events depression | C | 133 | .42 | 106 | .47 | 83 | .48 | .38 | .44 | .93 | .001 | .33 | .001 | |||||
| E | 210 | .39 | 165 | .42 | 128 | .52 | .39 | .41 | ||||||||||
| Economic functioning | C | 132 | .34 | .02 | 107 | .36 | 83 | .39 | .33 | .33 | .07 | .06 | .31 | .06 | ||||
| E | 209 | .25 | 167 | .31 | 129 | .29 | .24 | .30 | ||||||||||
| Punishment | C | 131 | .25 | .04 | 105 | .21 | 80 | .31 | .25 | .22 | .03 | .001 | .93 | .001 | .03 | |||
| E | 209 | .20 | 167 | .18 | 129 | .27 | .20 | .17 | ||||||||||
| Child aggression | C | 134 | 1.68 | .09 | 107 | 1.38 | 84 | 1.89 | 1.77 | 1.37 | .05 | .001 | .26 | .001 | .001 | |||
| E | 210 | 1.48 | 167 | 1.32 | 130 | 1.62 | 1.44 | 1.28 | ||||||||||
| School problems | C | 118 | .22 | 96 | .34 | 69 | .36 | .26 | .36 | .009 | .001 | .84 | .01 | .003 | ||||
| E | 197 | .20 | 157 | .28 | 121 | .29 | .19 | .26 | ||||||||||
| Child withdrawn | C | 134 | .40 | 107 | .41 | 84 | .62 | .42 | .37 | .38 | .001 | .51 | .001 | |||||
| E | 210 | .40 | 167 | .38 | 130 | .50 | .41 | .33 | ||||||||||
| Stolen things or arrested | C | 134 | .37 | 107 | .42 | 84 | .58 | .35 | .44 | .60 | .001 | .99 | .001 | .05 | ||||
| E | 210 | .29 | 167 | .48 | 130 | .55 | .31 | .41 | ||||||||||
| Child substance abuse | C | 134 | .16 | 107 | .20 | 84 | .26 | .13 | .18 | .94 | .01 | .71 | .02 | .10 | ||||
| E | 210 | .17 | 167 | .27 | 130 | .23 | .14 | .22 | ||||||||||
| Child problems | C | 134 | 2.64 | 107 | 2.88 | 84 | 3.00 | 2.70 | 2.99 | .57 | .001 | .45 | .003 | .09 | ||||
| E | 210 | 2.64 | 167 | 2.83 | 130 | 3.03 | 2.62 | 2.73 | ||||||||||
| Negative child behaviors | C | 130 | .33 | .04 | 105 | .34 | 81 | .41 | .34 | .35 | .005 | .001 | .84 | .001 | .001 | |||
| E | 207 | .28 | 163 | .32 | 126 | .35 | .28 | .30 | ||||||||||
| Positive child behaviors | C | 132 | .74 | 106 | .75 | 83 | .73 | .75 | .77 | .63 | .01 | .69 | .02 | .02 | ||||
| E | 208 | .73 | 163 | .76 | 128 | .72 | .73 | .77 | ||||||||||
| Household condition | C | 134 | .06 | 107 | .04 | 84 | .06 | .06 | .03 | .78 | .05 | .38 | .04 | |||||
| E | 210 | .05 | 167 | .05 | 129 | .06 | .05 | .04 | ||||||||||
| Depression (SCL-90) | C | 134 | 1.00 | .08 | 105 | .85 | 83 | 1.01 | .84 | .82 | .92 | .001 | .71 | .001 | ||||
| E | 209 | .83 | 166 | .82 | 127 | 1.04 | .89 | .77 | ||||||||||
| Positive child care practices | C | 128 | .76 | 103 | .76 | 77 | .79 | .77 | .77 | .43 | .11 | .67 | .06 | |||||
| E | 206 | .77 | 163 | .80 | 124 | .82 | .78 | .79 | ||||||||||
| Negative child care practices | C | 129 | .18 | .02 | 101 | .14 | 76 | .21 | .18 | .14 | .07 | .001 | .34 | .001 | .06 | |||
| E | 207 | .14 | 162 | .13 | 124 | .19 | .13 | .13 | ||||||||||
a Means of control and experimental groups |
||||||||||||||||||
| Post-Treatment | followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf |
Post v. Followg |
Initial v. later |
Post v. Follow |
|||||||||
| Positive life events | C | 37 | .14 | 36 | .31 | 28 | .20 | .15 | .32 | .53 | .001 | .76 | .001 | |||||
| E | 80 | .18 | 74 | .31 | 61 | .24 | .19 | .31 | ||||||||||
| Negative life events | C | 37 | .05 | 36 | .14 | 28 | .14 | .06 | .15 | .13 | .001 | .77 | .10 | .001 | ||||
| E | 80 | .05 | 74 | .11 | 61 | .10 | .04 | .12 | ||||||||||
| Life events depression | C | 31 | .36 | 36 | .35 | 28 | .49 | .40 | .34 | .85 | .001 | .66 | .001 | |||||
| E | 80 | .34 | 74 | .34 | 61 | .50 | .35 | .34 | ||||||||||
| Economic functioning | C | 37 | .25 | 36 | .33 | 28 | .43 | .25 | .31 | .06 | .001 | .86 | .001 | |||||
| E | 80 | .18 | 73 | .22 | 59 | .31 | .16 | .19 | ||||||||||
| Punishment | C | 37 | .13 | 36 | .10 | 28 | .21 | .12 | .11 | .45 | .001 | .11 | .001 | .09 | .10 | |||
| E | 76 | .13 | 71 | .07 | 54 | .28 | .15 | .08 | ||||||||||
| Child aggression | C | 37 | .81 | 36 | .86 | 28 | .89 | .86 | .86 | .97 | .11 | .18 | .09 | |||||
| E | 80 | .86 | 74 | .59 | 61 | 1.05 | .93 | .61 | ||||||||||
| School problems | C | 32 | .17 | 35 | .20 | 22 | .35 | .25 | .25 | .11 | .003 | .65 | .002 | |||||
| E | 65 | .15 | 63 | .18 | 46 | .23 | .15 | .20 | ||||||||||
| Child withdrawn | C | 37 | .27 | 36 | .28 | 28 | .68 | .25 | .32 | .94 | .001 | .10 | .01 | |||||
| E | 80 | .38 | 74 | .23 | 61 | .59 | .41 | .23 | ||||||||||
| Stolen things or arrested | C | 37 | .19 | 36 | .47 | 28 | .50 | .25 | .50 | .27 | .004 | .66 | .04 | .007 | ||||
| E | 80 | .19 | 74 | .34 | 61 | .43 | .18 | .31 | ||||||||||
| Child substance abuse | C | 37 | .03 | 36 | .03 | 28 | .00 | .04 | .04 | .44 | .97 | .31 | ||||||
| E | 80 | .03 | 74 | .04 | 61 | .08 | .03 | .05 | ||||||||||
| Child problems | C | 37 | 2.08 | 36 | 2.03 | 28 | 2.39 | 2.14 | 2.04 | .49 | .02 | .70 | .005 | |||||
| E | 80 | 1.72 | 74 | 1.70 | 61 | 2.33 | 1.84 | 1.80 | ||||||||||
| Negative child behaviors | C | 34 | .21 | 35 | .22 | 26 | .33 | .23 | .23 | .52 | .001 | .81 | .001 | |||||
| E | 72 | .21 | 71 | .19 | 55 | .30 | .22 | .20 | ||||||||||
| Positive child behaviors | C | 34 | .81 | 36 | .90 | .07 | 26 | .83 | .80 | .90 | .69 | .03 | .26 | .01 | ||||
| E | 72 | .83 | 70 | .86 | 54 | .81 | .83 | .85 | ||||||||||
| Household condition | C | 36 | .07 | 34 | .10 | 25 | .13 | .08 | .14 | .09 | .28 | .38 | ||||||
| E | 78 | .06 | 71 | .06 | 58 | .09 | .07 | .06 | ||||||||||
| Depression (SCL-90) | C | 37 | .76 | 36 | .83 | 28 | 1.00 | .81 | .72 | .89 | .008 | .64 | .002 | |||||
| Positive child care practices | E | 80 | .70 | 74 | .73 | 61 | 1.03 | .71 | .73 | |||||||||
| C | 36 | .82 | 34 | .96 | 24 | .90 | .89 | .94 | .70 | .05 | .71 | .01 | ||||||
| E | 71 | .88 | 67 | .93 | 48 | .90 | .86 | .95 | ||||||||||
| Negative child care practices | C | 35 | .09 | 33 | .07 | 25 | .11 | .09 | .08 | .38 | .001 | .09 | .001 | .03 | ||||
| E | 72 | .09 | 66 | .06 | 50 | .18 | .09 | .07 | ||||||||||
| a Means of
control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that the average initial score is equal to the average of post-treatment and final scores g Test of hypothesis that average post-treatment score is equal to the average final score |
||||||||||||||||||
We report on the differences between the experimental and control groups as they were initially formed (the "primary analysis"). Results of the secondary analyses (dropping violations and minimal service cases) were usually similar. They are reported in footnotes when they were materially different.
We looked at a large number of differences between groups on functioning variables and as will be seen, some of these comparisons revealed statistically significant differences, usually favoring the experimental group. However, as will also be seen, the differences are not consistent across states or across time.
In the caretaker interviews, there was a 15 item "life events" inventory asking about the occurrence of both positive and negative events in the last 3 months in the initial interview or since the previous interview in the post-treatment and followup interviews (Appendix K, Volume 3, Initial Caretaker Interview, p. 7). We formed three scales from this inventory, positive life events, negative life events, and a scale of those life events that might reflect depression in the caretaker (we had a more formal depression measure as well, described below). In none of the three states were there statistically significant differences between the experimental and control groups in the scores on these measures at the post-treatment or followup interviews nor were there differences between the groups in changes over time (see Tables 3-3, 3-4, and 3-5). It will be noted that generally the levels of these scales dropped in the second interview compared to the first and then increased again in the third interview. This is likely due to the differences in the time periods referenced in the questions asked at the three points in time (the last three months in the initial interview, last 4 to 6 weeks in the post-treatment interview, and last 10 to 11 months in the followup interview).
In the second and third interviews, caretakers were again asked questions about problems in the family. These questions paralleled those asked in the first interview, except this time caretakers were asked to respond to questions with regard to the time "since we last spoke to you" (see Tables 3-6 and 3-7). We analyze each of these problem items separately.
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | Control | Experimental | Control | Experimental | ||||||||||
| N | % | N | % | p | N | % | N | % | p | N | % | N | % | p | |
| Problems | |||||||||||||||
| Felt blue or depressed | 145 | 41 | 147 | 37 | 133 | 44 | 210 | 48 | 37 | 46 | 80 | 40 | |||
| Felt nervous or tense | 145 | 46 | 147 | 46 | 131 | 49 | 210 | 44 | 37 | 41 | 80 | 40 | |||
| Just wanted to give up | 145 | 17 | 147 | 15 | 133 | 26 | 210 | 20 | 35 | 29 | 79 | 23 | |||
| Overwhelmed with work or family responsibility | 145 | 37 | 148 | 47 | 0.08 | 133 | 50 | 210 | 44 | 37 | 32 | 80 | 33 | ||
| Felt you had few or no friends | 145 | 14 | 147 | 18 | 133 | 20 | 209 | 18 | 37 | 38 | 79 | 19 | 0.03 | ||
| Not enough money for food, rent, or clothing | 145 | 39 | 148 | 44 | 133 | 52 | 210 | 40 | 0.03 | 37 | 46 | 80 | 43 | ||
| Gotten in trouble with the law | 144 | 3 | 147 | 3 | 133 | 2 | 210 | 1 | 37 | 3 | 80 | 0 | |||
| Had too much to drink in a week | 145 | 2 | 148 | 1 | 133 | 2 | 209 | 2 | 37 | 5 | 80 | 3 | |||
| Used drugs several times a week | 145 | 1 | 148 | 0 | 133 | 2 | 209 | 1 | 37 | 5 | 80 | 3 | |||
| Economic Items | |||||||||||||||
| Had difficulty paying rent | 141 | 13 | 144 | 20 | 133 | 29 | 209 | 18 | 0.02 | 37 | 24 | 80 | 15 | ||
| Had difficulty paying electric/heat | 142 | 20 | 144 | 28 | 132 | 30 | 209 | 26 | 37 | 27 | 80 | 21 | |||
| Had difficulty buying enough food | 145 | 15 | 146 | 17 | 132 | 28 | 209 | 22 | 37 | 22 | 80 | 11 | |||
| Had difficulty buying clothes | 145 | 17 | 146 | 21 | 132 | 47 | 208 | 33 | 0.01 | 37 | 27 | 80 | 24 | ||
| Positive Items | |||||||||||||||
| Have you felt happy | 145 | 82 | 147 | 89 | 0.09 | 131 | 82 | 210 | 81 | 37 | 84 | 80 | 93 | ||
| Gotten together with anyone to have fun/relax | 145 | 64 | 148 | 64 | 133 | 65 | 210 | 59 | 37 | 38 | 80 | 75 | 0.001 | ||
| Doing a pretty good job raising kids | 144 | 94 | 147 | 91 | 130 | 88 | 209 | 91 | 36 | 92 | 79 | 96 | |||
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | p | Control | Experimental | p | Control | Experimental | p | |||||||
| N | % | N | % | N | % | N | % | N | % | N | % | ||||
| Problems | |||||||||||||||
| Felt blue or depressed | 119 | 44 | 130 | 42 | 107 | 55 | 166 | 48 | 36 | 44 | 74 | 47 | |||
| Felt nervous or tense | 119 | 47 | 130 | 53 | 105 | 53 | 166 | 47 | 36 | 31 | 73 | 38 | |||
| Just wanted to give up | 119 | 18 | 130 | 23 | 105 | 27 | 164 | 25 | 36 | 19 | 73 | 14 | |||
| Overwhelmed with work or family responsibility | 119 | 39 | 130 | 42 | 105 | 51 | 165 | 50 | 36 | 44 | 74 | 36 | |||
| Felt you had few or no friends | 119 | 24 | 130 | 25 | 107 | 15 | 166 | 24 | .07 | 36 | 8 | 74 | 19 | ||
| Not enough money for food, rent, or clothing | 118 | 44 | 130 | 46 | 106 | 46 | 166 | 45 | 36 | 42 | 74 | 49 | |||
| Gotten in trouble with the law | 119 | 6 | 130 | 4 | 107 | 1 | 166 | 4 | 36 | 3 | 74 | 3 | |||
| Had too much to drink in a week | 119 | 3 | 130 | 3 | 106 | 0 | 166 | 2 | 36 | 3 | 74 | 7 | |||
| Used drugs several times a week | 119 | 3 | 130 | 0 | 106 | 0 | 166 | 1 | 36 | 3 | 74 | 4 | |||
| Economic Items | |||||||||||||||
| Had difficulty paying rent | 118 | 20 | 127 | 20 | 107 | 34 | 167 | 27 | 36 | 39 | 74 | 20 | .04 | ||
| Had difficulty paying electric/heat | 118 | 19 | 126 | 25 | 107 | 36 | 167 | 37 | 36 | 42 | 74 | 32 | |||
| Had difficulty buying enough food | 119 | 14 | 129 | 15 | 107 | 35 | 167 | 26 | 36 | 19 | 73 | 14 | |||
| Had difficulty buying clothes | 119 | 15 | 128 | 19 | 107 | 42 | 167 | 35 | 36 | 31 | 73 | 21 | |||
| Positive Items | |||||||||||||||
| Have you felt happy | 119 | 89 | 130 | 89 | 106 | 83 | 166 | 87 | 36 | 92 | 73 | 89 | |||
| Gotten together with anyone to have fun/relax | 119 | 73 | 130 | 69 | 107 | 57 | 166 | 64 | 36 | 61 | 74 | 69 | |||
| Doing a pretty good job raising kids | 112 | 91 | 123 | 93 | 104 | 90 | 166 | 92 | 36 | 92 | 74 | 96 | |||
| NOTE: "FE" indicates significance determined by Fisher's exact test | |||||||||||||||
Kentucky. At the post-treatment interview, on 8 of the 9 problem questions, there were no differences between the experimental and control groups in responses. On the question of whether the caretaker felt overwhelmed with work or family responsibilities, a greater proportion of caretakers in the experimental group responded affirmatively at post-treatment than did caretakers in the control group (47% vs. 37%, p = .08). (35) In addition to the items about problems, caretakers were asked three questions about positive aspects of their lives: "gotten together with anyone to have fun or relax," "felt happy," and "felt that considering everything you're doing a pretty good job raising your kids." The primary analysis revealed that 89 percent of caretakers in the experimental group responded that they "felt happy" since the last interview as compared to 82 percent of caretakers in the control group (p = .09). For both the experimental and control groups, 64 percent responded affirmatively to the question of "getting together with anyone to have fun or relax," and over 90 percent responded affirmatively that they were "doing a pretty good job raising [their] kids." There were no significant differences between experimental and control groups on these last two items in either the primary or secondary analyses.
At the followup interview, there were no significant differences between experimental and control groups on any of these items.
New Jersey. On 8 of the 9 problem questions, there were no significant differences between the experimental and control groups in responses at post-treatment. On the overall question about the economic condition of family, "have you felt you just didn't have enough money for food, rent, or clothing?" 52 percent of the control group said yes, compared to 40 percent of the experimental group significantly different at p = .03, although in the secondary analysis the difference was not significant. (36) On none of the positive questions were there significant differences between groups in either the primary or secondary analyses.
At followup differences between groups approached significance for only one item, "felt you had few or no friends." More experimental group respondents replied affirmatively to this item (24% vs. 15% p = .07).
Tennessee. There was a significant difference between groups on only one of the nine problem questions at post-treatment. Fewer experimental group respondents reported they felt they had few or no friends (19% vs. 38%, p = .03). As to positive items, experimental group respondents far more often reported they "had gotten together with someone to have fun or relax" (75% vs. 38%, p < .001). At followup there were no differences between groups on any of these items.
In addition to the general item in the problem inventory on not having enough money for food, rent, or clothing, we asked four specific questions about difficulties in paying for the essentials of living (rent, electric service and heating, food, and clothes) (see Tables 3-6 and 3-7). These items were examined individually and were combined into an overall economic functioning scale. (See Tables 3-3, 3-4, and 3-5 for analyses of the scale).
Kentucky. On the scale, primary and secondary analyses revealed no significant differences in the average proportion of affirmative responses to the four items either at post-treatment or at followup. (37), (38) There also were no differences between groups in changes over time.
New Jersey. On the economic functioning scale, the experimental group had a lower average proportion of affirmative responses to these items at post-treatment (.25 vs. .34, p = .02) although the difference was not significant in the secondary analysis. The difference was not significant at followup nor were there significant differences in change over time. There were, however, significant differences on two of the specific items at post-treatment. Control group respondents more often reported difficulties paying rent (29% vs. 18%, p = .02) and also more often reported difficulties in buying clothes (47% vs. 33%, p = .01). These differences were not significant in the secondary analysis. At followup, there continued to be a difference in regard to buying food, although it was not significant (26% of the experimental group vs. 35% of the control group p = .12). At followup, the groups were similar on the other three items.
Tennessee. Control group respondents more often reported problems on the economic functioning scale at both post-treatment and followup, although the differences were not significant. There were also no significant differences between the groups in change over time. On individual items, there were no significant differences between groups at the post-treatment interview. At followup, significantly fewer experimental group respondents reported difficulties paying rent (20% vs. 39%, p = .04). There were no significant differences on the other three items at followup.
Caretakers were asked 10 questions about problematic conditions in the home (e.g., nonfunctioning heating, plumbing, or electrical systems; peeling paint; broken windows or doors).
Kentucky. In Kentucky, the experimental and control groups did not differ on the average proportions of the presence of such conditions at post-treatment or followup, nor did they differ on change in these proportions over time (see Table 3-3). On only one of the specific items were there any differences at post-treatment. Five percent of caretakers in the control group reported having broken windows or doors that were not fixed as compared to 1 percent in the experimental group (Fisher's exact p-value = .034). (39) There were no significant differences on any of the individual items at followup.
New Jersey. In New Jersey the experimental and control groups did not differ on the average proportions of the presence of such conditions at post-treatment or followup, nor did they differ on change in these proportions between interviews (see Table 3-4). There were no significant differences between the groups on any of the ten individual items at post-treatment. At followup, 8 percent of the experimental group respondents reported "a lot of peeling paint" compared to 2 percent of the control group respondents (p = .04). Also, 4 percent of the experimental group and none of the control group respondents reported that cooking appliances did not work (Fisher's exact p-value = .09).
Tennessee. There were no significant differences between experimental and control groups on the overall scales of household condition at either post-treatment or followup, nor were there differences in change over time (see Table 3-5). At post-treatment, more control group respondents reported bare electrical wires (8% vs. 0%, p = .08; Fisher's exact p-value = .03) while more experimental group respondents reported living in an "unsafe building because of illegal acts (5% vs. 0%, p = .05). There were no significant differences on any individual items at followup.
Caretakers were asked a series of yes-no questions about child care practices at the end of treatment and in the last three months at followup (both positive and negative). The results from these questions are shown in Table 3-8 and 3-9. In addition, three scales were formed using these items: positive child care practices (5 items), negative child care practices (10 items), and punishment (5 items, all of which are also included in the negative child care practices scale). Results from the scales are shown in Tables 3-3, 3-4 and 3-5.
Kentucky. In the post-treatment interview, "punishment for not finishing food" was the only item for which there were significant differences between the experimental and control groups, with a greater proportion of the control group responding affirmatively. There were no items on which there were significant differences at followup.
There were no significant differences between the experimental and control groups with regard to the positive and negative child care practice scales at the time of the second or third interviews. Both groups responded affirmatively to over 80 percent of the positive items and less than 15 percent of the negative items at both points in time. There were also no significant differences in change over time, in both groups there was a decline in the number of negative child care practices over time. With regard to the 5 items that pertain to punishment, caretakers in the experimental group responded affirmatively to a greater average proportion of punishment items than did the caretakers from the control group (.25 vs .20, p = .067) at the initial interview. At post-treatment, the average proportion of punishment items answered affirmatively were nearly the same (.17 for experimental group vs. .16 for the control group). Thus, from the first interview to the second, the reduction in the average proportion of punishment items endorsed was greater for the experimental group than for the control group (.09 fewer vs. .04 fewer, p = .054). (40) However, across the three points in time, there were no significant differences in change.
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | p | Control | Experimental | p | Control | Experimental | p | |||||||
| N | % | N | % | N | % | N | % | N | % | N | % | ||||
| Lost temper when child got on nerves | 145 | 43 | 147 | 46 | 132 | 63 | 209 | 59 | 37 | 27 | 77 | 35 | |||
| Found that hitting child was good | 143 | 6 | 147 | 7 | 131 | 12 | 208 | 8 | 37 | 19 | 76 | 8 | 0.09 | ||
| Hitting child harder that meant to | 143 | 8 | 147 | 6 | 131 | 10 | 208 | 5 | 37 | 5 | 75 | 8 | |||
| Out of control when punishing child | 144 | 24 | 146 | 24 | 131 | 40 | 209 | 30 | 0.05 | 36 | 11 | 76 | 12 | ||
| Have you praised your children | 144 | 94 | 146 | 92 | 132 | 92 | 209 | 92 | 37 | 92 | 76 | 91 | |||
| Listened to music together w/child | 144 | 86 | 146 | 86 | 132 | 86 | 209 | 82 | 37 | 92 | 77 | 90 | |||
| Tied child with cord- string-belt | 142 | 0 | 146 | 0 | 132 | 1 | 209 | 0 | 37 | 0 | 76 | 1 | |||
| Gone to amusement park, pool, picnic | 143 | 71 | 145 | 65 | 130 | 46 | 208 | 37 | 0.08 | 37 | 43 | 76 | 68 | 0.01 | |
| Uncomfortable hugging child | 126 | 5 | 134 | 6 | 95 | 5 | 144 | 6 | 33 | 9 | 72 | 4 | |||
| Encouraged child to read book | 137 | 92 | 140 | 90 | 126 | 82 | 202 | 91 | 0.02 | 34 | 94 | 72 | 96 | ||
| Have children handled household chores | 127 | 79 | 125 | 80 | 126 | 75 | 200 | 83 | 0.09 | 34 | 91 | 72 | 93 | ||
| Not let children into the house | 138 | 2 | 137 | 2 | 127 | 8 | 202 | 4 | 33 | 3 | 72 | 1 | |||
| Punished for not finishing food | 139 | 7 | 139 | 1 | 0.02 | 126 | 6 | 204 | 5 | 33 | 0 | 71 | 0 | ||
| Blamed child w/ things not their fault | 142 | 39 | 142 | 33 | 128 | 28 | 204 | 20 | 0.07 | 35 | 31 | 75 | 44 | ||
| Let child to play where not allowed | 138 | 2 | 138 | 4 | 123 | 1 | 200 | 4 | 33 | 0 | 71 | 1 | |||
| Unable to find someone to watch children | 144 | 9 | 146 | 12 | 131 | 21 | 206 | 12 | 0.04 | 35 | 20 | 73 | 27 | ||
| NOTE: "FE" indicates significance determined by Fisher's exact test | |||||||||||||||
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | Control | Experimental | Control | Experimental | ||||||||||
| N | % | N | % | p | N | % | N | % | p | N | % | N | % | p | |
| Lost temper when child got on nerves | 113 | 44 | 121 | 41 | 106 | 60 | 167 | 57 | 36 | 22 | 72 | 18 | |||
| Found that hitting child was good | 113 | 6 | 121 | 4 | 105 | 5 | 167 | 4 | 36 | 8 | 71 | 6 | |||
| Hitting child harder that meant to | 113 | 3 | 121 | 5 | 105 | 6 | 167 | 1 | .06 | 36 | 6 | 71 | 3 | ||
| Out of control when punishing child | 112 | 21 | 121 | 23 | 105 | 32 | 167 | 25 | 36 | 14 | 71 | 7 | |||
| Have you praised your children | 113 | 89 | 121 | 91 | 106 | 92 | 167 | 94 | 36 | 97 | 72 | 93 | |||
| Listened to music together w/child | 113 | 82 | 121 | 84 | 106 | 81 | 167 | 86 | 36 | 94 | 69 | 93 | |||
| Tied child with cord- string-belt | 113 | 0 | 121 | 2 | 106 | 1 | 167 | 1 | 36 | 0 | 70 | 0 | |||
| Gone to amusement park, pool, picnic | 113 | 74 | 121 | 70 | 106 | 48 | 167 | 44 | 35 | 94 | 70 | 90 | |||
| Uncomfortable hugging child | 102 | 4 | 110 | 5 | 74 | 5 | 116 | 6 | 35 | 6 | 66 | 3 | |||
| Encouraged child to read book | 107 | 83 | 115 | 81 | 103 | 87 | 160 | 89 | 34 | 100 | 67 | 96 | |||
| Have children handled household chores | 101 | 75 | 104 | 75 | 103 | 70 | 160 | 83 | .02 | 34 | 94 | 63 | 89 | ||
| Not let children into the house | 106 | 0 | 115 | 0 | 103 | 3 | 162 | 3 | 33 | 3 | 64 | 3 | |||
| Punished for not finishing food | 109 | 3 | 116 | 2 | 103 | 2 | 162 | 2 | 33 | 3 | 65 | 3 | |||
| Blamed child w/ things not their fault | 109 | 34 | 116 | 42 | 103 | 21 | 162 | 23 | 33 | 15 | 66 | 14 | |||
| Let child to play where not allowed | 109 | 1 | 114 | 0 | 101 | 2 | 160 | 1 | 33 | 0 | 65 | 3 | |||
| Unable to find someone to watch children | 111 | 3 | 122 | 0 | 103 | 17 | 165 | 18 | 33 | 21 | 67 | 19 | |||
New Jersey. On two of the items there were significant differences between the experimental and control groups at post-treatment: "have things sometimes gotten out of control when you punished your children?" happened more often in the control group and "have you encouraged your child to read a book?" which was done more often by experimental group respondents. At followup, there were two other items with differences: more control group respondents said they hit their child harder than they meant to (6% vs. 1%, p = .06) while more experimental group respondents said they had the children handle household chores (83% vs. 70%, p = .02).
On the scales, there were no significant differences between the experimental and control groups with respect to the positive child care practice items at either post-treatment or followup. At both points in time, both groups responded affirmatively to over 75 percent of the items. There were significant differences between the experimental and control groups with respect to the negative child care practice items at post-treatment. Caretakers in the experimental group responded affirmatively to 14 percent of the items whereas caretakers in the control group responded affirmatively to 18 percent of the items (p = .02). The difference disappeared at followup. At post-treatment, the experimental group significantly less often used punishment (.20 vs. .25; p = .04), but the difference was not statistically significant in the secondary analysis (p = .08) or in the primary analysis at followup (p = .15). There were no significant differences between groups in the change in changes over time in proportion of negative child care practice or punishment items.
Tennessee. At post-treatment, on one item there was a significant difference between experimental and control groups; more experimental group respondents indicated they had gone to an amusement park, pool, or picnic (68% vs. 43%, p = .01). There were no items with significant differences at followup. There were no significant differences in any of the three scales at either post-treatment or followup, nor in changes over time. The experimental group had a higher negative child care practices score at the first interview, and declined more than the control group, resulting in a nearly significant multivariate time-group interaction (p = .09) and a significant univariate comparison of the first interview score with the average of the scores from the later two interviews (p = .03).
In both the initial and post-treatment interviews we administered the SCL-90 (41) depression scale to measure the level of depression of the caretaker. (42) In none of the three states were there significant differences between the groups in scores on this scale at the post-treatment or followup interviews or in changes over time (see Tables 3-3, 3-4, and 3-5). Scores at post-treatment were, on average, less than those in the initial interview for both groups and the reduction was greater for the experimental group (Kentucky: .23 less for the experimental group and .14 less for the control group; New Jersey: .19 less for the experimental group, .09 less for the control group; Tennessee: .32 less for the experimental group, .19 less for the control group), though the differences were not statistically significant.
In all three interviews, we asked 35 questions about specific child behaviors, both positive and negative. Questions were phrased in terms of "any of the children" and some questions were age specific. Responses to these questions were used to form various scales: aggression (3 items), school problems (5 items), positive child behaviors (10 items), and negative child behaviors (21 items, including the aggression and school problems items). Analyses of these scales are shown in Tables 3-3, 3-4, and 3-5.
Kentucky. Neither the primary nor the secondary analyses revealed any significant differences between the groups in scores on any of these scales at post-treatment or at followup or in the change over time. Specific items on whether the child was withdrawn, had stolen things or been arrested, or had engaged in substance abuse did not reveal significant differences between groups at post-treatment or followup. For the scale of having stolen things or been arrested, the experimental group scored higher at the initial interview and declined between the first and second interview, while the control group increased between the first and second interviews and again at the followup interview. As a result, the multivariate interaction between time and group was nearly significant (p = .07) and the univariate difference between the groups in the difference between the first interview and the average of the later two interviews was significant at p = .03.
New Jersey. In the primary analysis there were no significant differences between groups on these scales at post-treatment or at followup, except for the overall negative child behaviors scale at post-treatment, on which the experimental group was lower (an average of 28% of the items vs. 33%; p = .04). (43) For none of these scales was there a significant difference between groups on change over time.
Specific items on whether the child was withdrawn, had stolen things or been arrested, or had engaged in substance abuse did not reveal significant differences between groups at post-treatment or followup or in change over time.
Tennessee. There were no significant differences between the experimental and control groups in the average scores on these scales at post-treatment or at followup, nor were there significant differences in change over time. The difference for positive child behaviors at followup was nearly significant, with the control group scoring higher (p = .07). The specific items on whether the child was withdrawn, had stolen things or been arrested, or had engaged in substance abuse did not reveal significant differences between groups at post-treatment or followup or in change over time.
[Go To Contents]
In the post-treatment interview, caretakers were asked about general changes in their families' lives since entering the study. Results are shown in Table 3-10 and 3-11. At post-treatment, in Kentucky and New Jersey, relative to control group caretakers, a significantly larger proportion of experimental group caretakers generally thought there was "great improvement" in their lives. This difference was significant in both the primary and secondary analyses. In the Tennessee secondary analysis, results
| Kentucky | New Jersey | Tennessee | ||||
|---|---|---|---|---|---|---|
| Control % |
Experimental % |
Control % |
Experimental % |
Control % |
Experimental % |
|
| Primary analysis: | p = .02 | p = .001 | p = n.s. | |||
| Great improvement | 16 | 22 | 9 | 16 | 32 | 32 |
| Some improvement | 31 | 42 | 41 | 52 | 32 | 42 |
| Same | 42 | 29 | 34 | 20 | 22 | 14 |
| Somewhat or a great deal worse | 12 | 6 | 16 | 12 | 14 | 13 |
| Kentucky | New Jersey | Tennessee | ||||
|---|---|---|---|---|---|---|
| Control % |
Experimental % |
Control % |
Experimental % |
Control % |
Experimental % |
|
| Primary analysis: | p = n.s. | p = n.s. | p = n.s. | |||
| Great improvement | 34 | 36 | 30 | 28 | 53 | 36 |
| Some improvement | 37 | 38 | 36 | 42 | 31 | 41 |
| Same | 18 | 16 | 17 | 16 | 8 | 15 |
| Somewhat or a great deal worse | 11 | 9 | 16 | 13 | 3 | 7 |
tended in the same direction, though not significantly (p = .09). At followup,
differences
between the groups in Kentucky and New Jersey had nearly disappeared. In
Tennessee, control group respondents more often thought there was "great
improvement," although it was not a significant difference.
[Go To Contents]
The caseworker interviews also contained questions regarding child and family functioning, in an effort to provide another perspective on these issues. In interpreting caseworker reports, it should be noted that experimental group caseworkers were Homebuilders workers, while control group respondents were the public agency workers responsible for the cases at the time of the interview. It is likely that there are differences between these groups of caseworkers in the knowledge they have of the cases, since Homebuilders workers had much more intensive involvement and that involvement began before the first research interview. In addition, it may be that there are systematic differences in these groups of workers in the approaches they take to the assessment of family problems. Hence, interpretations of comparisons between responses of workers serving each of the groups must be made with caution.
Caseworkers were asked nine questions tapping various aspects of caretaker functioning on a five-point scale from 0 for not adequate to 4 for very adequate. Table 3-12 provides a list of these nine questions and a summary of the results from the initial and post-treatment interviews.
Kentucky. At the initial interview, significant or nearly significant differences were found on three items, with the experimental group scoring more adequate on average: ability to provide food (p = .02), responding patiently to child's questions (p = .06), and attending to children's health needs (p = .08). On a scale averaging the nine ratings for each case, the difference between means of the experimental and control groups approached significance, with the experimental group having a higher mean (p = .06). At post-treatment there were no significant differences in the primary analysis. However, in the secondary analysis, caretakers from the experimental group were rated higher (more adequate) than those from the control group with respect to whether they attended to the children's health needs (p = .04). As for the scale averaging the nine ratings, no differences were found between the experimental and
| Kentucky | Control | Experimental | p | ||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Initial: | |||||
| Caretaker ability to provide food | 130 | 2.68 | 114 | 2.96 | 0.02 |
| Caretaker ability giving affection | 132 | 2.63 | 125 | 2.82 | |
| Caretaker respect for child's opinions | 119 | 2.38 | 106 | 2.58 | |
| Respond patiently to child's questions | 122 | 2.16 | 110 | 2.44 | 0.06 |
| Respond to child's emotional needs | 137 | 2.15 | 122 | 2.35 | |
| Provide learning opportunities | 127 | 2.17 | 110 | 2.35 | |
| Setting firm/consistent limits/rules | 130 | 1.68 | 116 | 1.88 | |
| Adequate supervisor/responsible childcare | 140 | 2.14 | 123 | 2.39 | 0.10 |
| Attending to children's health needs | 135 | 2.76 | 114 | 3.00 | 0.08 |
| Caretaker functioning, 9 items, average of nonmissing items, higher = better | 118 | 2.25 | 102 | 2.48 | 0.06 |
| Post-treatment: | |||||
| Caretaker ability to provide food | 145 | 2.88 | 154 | 2.97 | |
| Caretaker ability giving affection | 147 | 2.82 | 157 | 2.81 | |
| Caretaker respect for child's opinions | 135 | 2.58 | 144 | 2.45 | |
| Respond patiently to child's questions | 138 | 2.43 | 148 | 2.34 | |
| Respond to child's emotional needs | 145 | 2.28 | 156 | 2.28 | |
| Provide learning opportunities | 144 | 2.38 | 154 | 2.42 | |
| Setting firm/consistent limits/rules | 145 | 2.09 | 150 | 1.99 | |
| Adequate supervisor/responsible childcare | 152 | 2.50 | 158 | 2.59 | |
| Attending to children's health needs | 150 | 2.93 | 157 | 3.08 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 142 | 2.56 | 151 | 2.55 | |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
| New Jersey | Control | Experimental | p | ||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Initial: | |||||
| Caretaker ability to provide food | 119 | 3.24 | 224 | 3.20 | |
| Caretaker ability giving affection | 120 | 2.88 | 229 | 2.62 | 0.03 |
| Caretaker respect for child's opinions | 118 | 2.42 | 219 | 2.32 | |
| Respond patiently to child's questions | 117 | 2.44 | 220 | 2.27 | |
| Respond to child's emotional needs | 118 | 2.37 | 228 | 2.23 | |
| Provide learning opportunities | 114 | 2.83 | 220 | 2.50 | 0.005 |
| Setting firm/consistent limits/rules | 126 | 2.11 | 228 | 1.93 | |
| Adequate supervisor/responsible childcare | 130 | 2.80 | 238 | 2.71 | |
| Attending to children's health needs | 125 | 3.34 | 214 | 3.17 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher = better | 107 | 2.65 | 211 | 2.44 | 0.02 |
| Post-treatment: | |||||
| Caretaker ability to provide food | 137 | 3.36 | 246 | 3.34 | |
| Caretaker ability giving affection | 141 | 2.93 | 256 | 2.70 | 0.04 |
| Caretaker respect for child's opinions | 130 | 2.55 | 247 | 2.42 | |
| Respond patiently to child's questions | 140 | 2.51 | 248 | 2.37 | |
| Respond to child's emotional needs | 149 | 2.43 | 258 | 2.37 | |
| Provide learning opportunities | 137 | 2.89 | 247 | 2.60 | 0.01 |
| Setting firm/consistent limits/rules | 147 | 2.37 | 252 | 2.14 | 0.06 |
| Adequate supervisor/responsible childcare | 149 | 2.95 | 258 | 2.79 | |
| Attending to children's health needs | 148 | 3.35 | 252 | 3.25 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 140 | 2.79 | 249 | 2.66 | 0.10 |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
| Tennessee | Control | Experimental | p | ||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Initial: | |||||
| Caretaker ability to provide food | 38 | 2.79 | 53 | 3.11 | |
| Caretaker ability giving affection | 42 | 2.76 | 60 | 2.92 | |
| Caretaker respect for child's opinions | 34 | 2.23 | 52 | 2.77 | 0.01 |
| Respond patiently to child's questions | 32 | 2.22 | 53 | 2.57 | |
| Respond to child's emotional needs | 40 | 2.05 | 59 | 2.47 | 0.04 |
| Provide learning opportunities | 39 | 2.64 | 56 | 2.55 | |
| Setting firm/consistent limits/rules | 36 | 2.33 | 57 | 2.01 | |
| Adequate supervisor/responsible childcare | 44 | 2.32 | 61 | 2.95 | 0.005 |
| Attending to children's health needs | 43 | 2.65 | 59 | 3.18 | 0.03 |
| Caretaker functioning, 9 items, average of nonmissing items, higher = better | 30 | 2.53 | 51 | 2.60 | |
| Post-treatment: | |||||
| Caretaker ability to provide food | 41 | 2.98 | 74 | 3.32 | 0.06 |
| Caretaker ability giving affection | 45 | 2.73 | 80 | 2.95 | |
| Caretaker respect for child's opinions | 40 | 2.35 | 74 | 2.84 | 0.01 |
| Respond patiently to child's questions | 38 | 2.26 | 76 | 2.67 | 0.04 |
| Respond to child's emotional needs | 42 | 2.26 | 81 | 2.59 | 0.06 |
| Provide learning opportunities | 44 | 2.64 | 78 | 2.64 | |
| Setting firm/consistent limits/rules | 43 | 2.04 | 79 | 2.38 | |
| Adequate supervisor/responsible childcare | 46 | 2.52 | 82 | 2.93 | 0.04 |
| Attending to children's health needs | 45 | 2.96 | 78 | 3.13 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 42 | 2.51 | 77 | 2.82 | 0.04 |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
control groups at post-treatment. Looking at change over time, on one item, respecting child's opinions, the ratings for the control group increased over time (.19 change), whereas the ratings for the experimental group decreased slightly over time (-.06 change), a difference that is significant (p = .05). The differences between groups in change on the overall scale averaging the nine ratings was not significant.
New Jersey. At the initial interview, on two items there were significant differences between the experimental and control groups, the control group scoring more adequate on average: caretaker's ability in giving affection (p = .03) and the caretaker's ability to provide learning opportunities (p = .005). On the scale averaging the nine ratings for each case, there was a significant difference between means of the experimental and control groups, the control group having a higher mean (p = .02). At post-treatment, the control group scored higher (more adequate functioning) on the same two items as before. On the scale of nine items the control group scored slightly higher, although the difference was nonsignificant. As to change over time, on one item ("respecting child's opinions"), the control group had, on average, more positive change than the experimental group. The difference in degree of change was significant at .05 (this result also held in the secondary analysis, p = .05). Differences between groups in change on the overall scale were not significant.
Tennessee. At the initial interview there were four items on which the groups were significantly different, the experimental group scoring higher on all four: caretaker respect for child's opinions (p = .01), response to child's emotional needs (p = .04), adequate supervision (p = .005), and attending to the child's health needs (p = .03). At post-treatment, five items had differences between groups significant at .06 or lower, all favoring the experimental group: caretaker ability to provide food, respect for child's opinions, response to child's emotional needs, adequate supervision, and respond patiently to child's questions. The average of all nine items was also significantly different for the groups. On one item, setting firm and consistent limits, there was a significant difference in the amount of change over time, the experimental group increased by an average of .31, while the control group declined by an average of .29 (p = .01). On the scale of nine items there was no significant difference between the groups in change over time.
As in the caretaker interview, we asked caseworkers about conditions in the home. Caseworkers were asked 13 yes-no questions, some positive and some negative. These items were combined in a scale which indicated that in Kentucky and New Jersey at post-treatment, control group families had, on average, a significantly better household condition than did experimental group families (Kentucky: p = .014; New Jersey: p = .02). In both states, for both groups the analysis of change over time indicated a slight improvement in the condition of the household. The difference between the experimental and control groups in change over time was not significant in either state. In Tennessee, there was no difference between groups at post-treatment or in change over time (both groups declined by .01).
Caseworkers were asked a number of questions about problems experienced by children, caretakers, or other adult household members (question 19 on the initial caseworker interview, question 17 on the post-treatment caseworker interview). Twenty-one of these problems concerned the caretakers.
Kentucky. At post-treatment, in the primary analysis, caseworkers reported that the experimental group caretakers had, on average, 31 percent of the problems compared to 25 percent for the control group, a difference significant at p = .0005. (44) There were no significant differences in change in caretaker problems between the interviews in either the primary or secondary analyses.
New Jersey. At the post-treatment interview, on average, in the primary analysis caseworkers reported that experimental group caretakers had 23 percent of the problems compared to 21 percent of the control group, a nonsignificant difference. (45) There were no significant differences in change in caretaker problems between the interviews in either the primary or secondary analyses.
Tennessee. At post-treatment, caseworkers reported that experimental group caretakers had 18 percent of the problems compared to 21 percent of the control group, a nonsignificant difference. There was a significant difference between the groups in change over time, the experimental group improving more than the control group (-.08 vs. -.03, p = .05).
Twelve of the items on the caseworker problem inventory concerned the children. In Kentucky at post-treatment, the percentage of child problems for the experimental group was, on average, 27 percent compared to an average of 25 percent for the control group, a nonsignificant difference. (46) There were no significant differences in change in child problems between interviews in either the primary or secondary analyses.
In New Jersey at post-treatment, the average of the percentages of child problems was 25 percent for the experimental group and 27 percent for the control group, a nonsignificant difference. (47) There were no significant differences in change in child problems between interviews in either the primary or secondary analyses.
In Tennessee, the average percentages of child problems in the two groups at post-treatment were very close (18% for the control group, 19% for the experimental group). The difference between the groups in change over time was not significant.
[Go To Contents]
We performed regression analyses on a number of family functioning outcomes measured at the post-treatment interview and at followup. The analyses were intended to control for the effects of a number of variables, thereby providing more sensitive tests of the effects of family preservation, and to examine the effects of the variables on the outcomes. The dependent variables in these analyses were some of the scales of functioning discussed above: caretaker depression, child aggression, punishment, child school problems, difficulty paying bills, positive life events, negative life events, positive child behaviors, negative child behaviors, household condition, positive child care practices, and negative child care practices. Independent variables in these analyses were assignment group (experimental or control), caretaker's age, caretaker's race, family composition, caretaker's educational attainment, caretaker's employment status, residential stability, use of income support programs, caretaker's history of abuse and/or neglect, regular access to an automobile, and time to interview (days between random assignment and post-treatment/followup interview). The analyses also included the initial scores for the dependent variable, thereby controlling the level at post-treatment or followup for the initial value. Interactions between control variables and experimental group were also examined, only a few were found to be significant. (48)
Caretaker's age, caretaker's race, family composition, caretaker's educational attainment, caretaker's employment status, use of income support programs, caretaker's history of abuse and/or neglect have all been examined in previous studies of outcomes in child welfare and have often been found to be predictive. Residential stability and regular access to a car have been less often examined. Since transportation and housing assistance are commonly provided in family preservation service models, the inclusion of such variables seems justifiable. Moreover, prior research does support a relationship between residential stability and major depression (49) and child adjustment. (50) Similarly, transportation (or lack there of) has been found to be related to participation in social programs (51) and family functioning. (52) We included time to interview because of the fact that that varied considerably and might have affected the degree of change that we were observing.
Regression analyses were conducted at the family level for both the post-treatment and followup measures. The coefficients are displayed in Table 3-13 and 3-14. All of the coefficients are shown for the initial measure of the outcome variable and for experimental group. Coefficients for other variables are shown if they were significant at p = .1 or lower. Most of the analyses are ordinary least squares regressions, logistic regressions were used for dichotomous or highly skewed variables. Generally, the initial measure was the strongest predictor. Although the size of these coefficients decreased between the post-treatment and followup interview, the majority of such coefficients remained significant. The positive direction of the coefficients indicates that caretakers with higher initial values also had higher post-treatment and followup values.
In regard to the post-treatment analyses, experimental group families generally had better outcomes, but the differences were significant in only three analyses. In New Jersey, the experimental group had lower depression scores and lower negative child care practices than the control group when controlled for the other independent variables. In the analysis without controlling for the other variables, the result for depression was in the same direction, but not significant (p = .08). The result for negative child care practices without the control variables was also in the same direction and significant (p = .02). In Tennessee the experimental group had fewer negative life events in the regression analysis. The difference between groups in the uncontrolled analysis was not significant. (53) Three differences significantly in favor of the experimental group in the uncontrolled analyses were no longer significant in the regression analysis, all in New Jersey: caretaker use of punishment, negative life events, and positive child behaviors.
At the followup interview, the regression analysis indicates that family preservation clients had lower levels of child aggression in Tennessee, fewer school problems in Kentucky, and fewer problematic conditions in the home in Tennessee. There were no significant differences between groups in the uncontrolled comparisons.
Regarding the remaining independent variables, there was little consistency in whether or not a variable had an effect and even in the direction of the effect. The following discussion focuses on those variables significant at p = .05 or lower. At post-treatment, the variables that most often showed effects were education and the caretaker having a history of being maltreated.
| Dependent Measures | Initial Measure | Experim. Group Assignment |
Care-taker Age |
Single Mother |
Ethnic Minority |
Abuse Neglect History |
Education | Employ- ment |
Income Support |
Access to Car |
Housing Stability |
Time to Interview |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Caretaker depression | ||||||||||||
| Kentucky2 | .596** | -.053 | .083 | .144** | .078 | |||||||
| New Jersey2 | .690** | -.202** | -.048 | .102* | -.170** | |||||||
| Tennessee2 | .606** | -.064 | ||||||||||
| Child aggression | ||||||||||||
| Kentucky | .522** | -.051 | ||||||||||
| New Jersey | .589** | -.044 | .101* | -.079 | ||||||||
| Tennessee | .533** | -.004 | ||||||||||
| Punishment | ||||||||||||
| Kentucky1 | 9.81** | 1.16 | 2.02* | |||||||||
| New Jersey | .529** | -.079 | ||||||||||
| Tennessee | .281** | -.012 | ||||||||||
| Child school problems | ||||||||||||
| Kentucky | .539** | -.020 | .111 | |||||||||
| New Jersey | .381** | -.041 | .107 | |||||||||
| Tennessee | .654** | -.073 | ||||||||||
| Difficulty paying bills | ||||||||||||
| Kentucky | .608** | .023 | ||||||||||
| New Jersey | .632** | -.061 | -.080 | -.088 | .107* | |||||||
| Tennessee | .513** | -.020 | .155 | .142 | ||||||||
| Positive life events | ||||||||||||
| Kentucky | .218** | .055 | .160** | .124* | ||||||||
| New Jersey | .330** | -.074 | -.108* | .104 | -.139* | |||||||
| Tennessee | .330** | .020 | ||||||||||
| Negative life events | ||||||||||||
| Kentucky1 | 1.85* | .833 | 2.69** | |||||||||
| New Jersey | .278** | .008 | .132* | .097 | ||||||||
| Tennessee | .100 | -.768** | -.290** | -.707** | ||||||||
| Positive child behaviors | ||||||||||||
| Kentucky | .489** | -.002 | -.089 | .120* | ||||||||
| New Jersey | .579** | -.032 | ||||||||||
| Tennessee | .525** | .098 | -.187* | |||||||||
| Negative child behaviors | ||||||||||||
| Kentucky | .592** | -.012 | ||||||||||
| New Jersey | .581** | -.078 | ||||||||||
| Tennessee | .647** | -.016 | ||||||||||
| Household condition | ||||||||||||
| Kentucky1 | 9.01** | .961 | .414* | .948 | ||||||||
| New Jersey1 | 5.66** | .744 | 2.01* | |||||||||
| Tennessee | .429** | -.004 | ||||||||||
| Positive child care | ||||||||||||
| Kentucky | .401** | -.041 | .113* | .128* | ||||||||
| New Jersey | .566** | .007 | -.081 | |||||||||
| Tennessee | .575** | .069 | .268** | |||||||||
| Negative child care | ||||||||||||
| Kentucky | .569** | -.078 | -.115* | |||||||||
| New Jersey | .571** | -.119* | .083 | .116* | ||||||||
| Tennessee | .371** | -.136 | -.177 | |||||||||
1 Logistic regression, Exp (B) displayed |
||||||||||||
| Dependent Measures | Initial Measure | Experim. Group Assignment |
Care- taker Age |
Single Mother | Ethnic Minority | Abuse Neglect History | Education | Employ-ment | Income Support | Access to Car | Housing Stability | Time to Interview |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Caretaker depression | ||||||||||||
| Kentucky2 | .552** | .068 | .116* | |||||||||
New Jersey2 |
.518** | -.051 | -.124* | |||||||||
| Tennessee2 | .443** | -.011 | ||||||||||
| Child aggression | ||||||||||||
| Kentucky | .363** | .022 | ||||||||||
| New Jersey | .417** | .031 | -.120* | |||||||||
| Tennessee | .347** | -.226* | .173 | |||||||||
| Punishment | ||||||||||||
| Kentucky1 | 6.67** | .750 | .977* | |||||||||
| New Jersey | .288** | -.085 | -.126 | -.108 | ||||||||
| Tennessee | .271* | -.146 | .293** | |||||||||
| Child school problems | ||||||||||||
| Kentucky | .254** | -.147* | .253** | .180** | ||||||||
| New Jersey | .272** | -.098 | ||||||||||
| Tennessee | .451** | -.002 | .290** | |||||||||
| Difficulty paying bills | ||||||||||||
| Kentucky | .396** | -.016 | .127* | |||||||||
| New Jersey | .537** | -.001 | .108* | -.108* | ||||||||
| Tennessee | .290** | -.132 | ||||||||||
| Positive life events | ||||||||||||
| Kentucky | .260** | -.044 | .117 | .227** | ||||||||
| New Jersey | .081 | -.049 | -.165* | |||||||||
| Tennessee | .261* | -.081 | -.198 | .304* | ||||||||
| Negative life events | ||||||||||||
| Kentucky1 | 1.71* | 1.18 | ||||||||||
| New Jersey | .356** | .023 | -.176** | |||||||||
| Tennessee | .116 | -.127 | .272* | .236* | ||||||||
| Positive child behaviors | ||||||||||||
| Kentucky | .250** | .028 | .182** | |||||||||
| New Jersey | .294** | -.002 | .165* | |||||||||
| Tennessee | .192 | -.064 | -.231* | .301** | ||||||||
| Negative child behaviors | ||||||||||||
| Kentucky | .385** | -.058 | .137* | |||||||||
| New Jersey | .404** | -.016 | -.121* | |||||||||
| Tennessee | .344** | -.097 | .200 | |||||||||
| Household condition | ||||||||||||
| Kentucky1 | 3.86* | 1.38 | .271 | |||||||||
| New Jersey1 | 3.59** | 1.40 | 2.51* | |||||||||
| Tennessee | .045 | -1.24** | -.317 | .207 | ||||||||
| Positive child care | ||||||||||||
| Kentucky | .370** | .014 | -.193** | .133* | ||||||||
| New Jersey | .164* | .044 | -.200** | |||||||||
| Tennessee | .110 | -.056 | ||||||||||
| Negative child care | ||||||||||||
| Kentucky | .340** | .020 | ||||||||||
| New Jersey | .311** | -.050 | -.135* | -.143* | ||||||||
| Tennessee | .195 | -.085 | .296* | |||||||||
| 1 Logistic
Regression, Exp (B) displayed 2 Depression scores transformed using log transformation * p < .05, ** p < .01. All coefficients for experimental group assignment and initial measure are shown, regardless of significance. All other entries without stars are significant at .1. |
||||||||||||
Caretaker education was related to three post-treatment outcomes in Kentucky. More education was associated with more punishment, more positive life events, and worse household condition. In Tennessee more education was related to fewer negative life events. In Kentucky, having a history of maltreatment was related to higher depression and more negative life events. In New Jersey, history of maltreatment was related to higher depression and children being more aggressive. Income support, ethnic minority, and caretaker employment all were predictors in 3 of the 36 post-treatment regressions. Time to interview was significant in only one of the regressions.
At followup, time to interview emerged as a predictor in 4 of the 36 regression equations, in all cases related to an increase (worsening). Other variables often related to outcome were caretaker age and education. In New Jersey, older caretakers had fewer positive life events and had children who were less aggressive. In Tennessee, older caretakers had children with fewer positive behaviors. In Kentucky, older caretakers had higher depression scores, less often engaged in positive child care practices, and had children with more school problems.
In New Jersey at followup, caretaker education was related to 3 outcomes. More education was related to lower depression, less difficulty paying bills, and more positive child behaviors. In Kentucky, more education was related to more difficulty paying bills, more positive life events, and more positive child behaviors. More education in Tennessee is related to more child school problems. In Kentucky, caretaker's employment is related to more negative child behaviors, more child school problems, and more positive child care practices. Caretaker employment in New Jersey is associated with fewer negative child care practices. Ethnic minority caretakers in New Jersey had fewer negative life events, engaged in fewer negative child care practices, and had children with fewer negative behaviors.
The 72 regression equations for post-treatment and followup contain a fair number of significant coefficients, but there is little consistency across states or across outcomes.
In summary, regression models were constructed to explore the relationship between caretaker demographic characteristics and experimental group and family functioning. Other than the initial value of the measures, relatively few significant relationships emerged. Moreover, these relationships were not consistent across the states. As to the effects of family preservation services, these data do not support a strong relationship between these services and better family functioning.
[Go To Contents]
The preceding analysis focuses on differences between cases assigned to family preservation and the control group. But some cases in the experimental group received relatively low levels of service while cases in the control group may have received more services than usual. One can examine the relationship between the amount of service provided, regardless of whether the case is in the experimental or control group, and outcomes. In this analysis we measured level of service in two ways, both drawn from the post-treatment interview with the caretaker: the caretaker's report of the number of contacts with the worker and the caretaker's
| KY, NJ, TN | |||
|---|---|---|---|
| N | Exp(B) | p | |
| Placement within 18 months and caseworker contact | 749 | ||
| Constant | .39 | ||
| Number of times met with worker since last interview | 1.000 | .89 | |
| Number of placements within 18 months after random assignment date | 208 | ||
| Placement within 18 months and caseworker activities | 749 | ||
| Constant | .40 | ||
| Number of caseworker activities (as reported by caregiver) | .99 | .68 | |
| Number of placements w/in 18 months after random assignment date | 208 | ||
| Substantiated allegation within 18 months and caseworker contact | 753 | ||
| Constant | .25 | ||
| Number of times met with worker since last interview | 1.003 | .12 | |
| Number of substantiated allegations 18 months after random assignment date | 161 | ||
| Substantiated allegation within 18 months and caseworker activities | 753 | ||
| Constant | .26 | ||
| Number of CW activities (as reported by caregiver) | 1.01 | .68 | |
| Number of substantiated allegations 18 months after random assignment date | 161 | ||
report of the number of caseworker activities. The relationship between these variables and placement and subsequent substantiated allegations of harm was examined through logistic regression, reported in Table 3-15. As can be seen in the table, there were no significant relationships between the level of service and these outcomes, all of the odds ratios for the predictive variables were quite close to one.
The analysis to this point has examined the effects of services in an undifferentiated way, by looking at the relationship between the amount of services and outcomes, by either comparing the outcomes of the experimental and control groups or, in the last section, examining the relationship between amount of services as determined by the number of contacts and outcomes. But it is possible that the services provided were not responsive to the particular problems of families. To the extent that this is the case, it would explain the relatively small effects of services on outcomes. Furthermore, an examination of specific problems and specific services might reveal effects that are obscured in the global analyses presented thus far.
We explored this possibility in a limited way by looking at three prominent problems experienced by families in the study, financial difficulties, problems with discipline of children, and depression. After identifying families with these problems, we determined the extent to which services provided might have addressed these issues, the extent of "match" between problems and services provided. Finally, we determined whether there was a relationship between the extent of match of services and problems with outcomes. We conducted the analyses only on cases in the family preservation group in the three Homebuilders states for which we had interviews at the beginning and end of service and contact forms giving us information on services provided. We limited the sample to the Homebuilders group so as to look at the implementation of a particular, well defined model, thus limiting extraneous variance due to variation in approach. It was thought that within that group we would most likely find a match between problems and services. We also believed that the contact form data, on which this analysis depended, was more complete and of a higher quality for the experimental group. Furthermore, in general, control group cases did not receive a high enough volume of services to reveal a match.
We combined the samples from the three states, a total of 292 families with 886 children. Women were the caretakers in 89 percent of these families and the caretakers were, on average, 36 years old. Forty-eight percent were African American, 47 percent were white.
Economic problems were determined from responses on the caretaker interviews to questions about difficulties paying rent, paying electric and heating bills, buying food, and buying clothes for the children. Any family responding positively to any one of these four questions at the initial interview was classified as having economic problems (n = 157, 54%). The presence of disciplinary problems was determined from seven questions on the initial caretaker interview. (54) Again, if any one of these questions was answered affirmatively, the case was classified as having disciplinary problems (n = 221, 76%). Caretaker depression was determined from the SCL-90 depression scale. Individuals with scores higher than the median (determined separately for men and women) were categorized as having some problems with depression.
Services specific to particular problems were determined from the contact forms completed by family preservation caseworkers. Services responding to economic problems were providing emergency cash or paying bills; buying food; helping to find housing; providing clothing, furniture, or supplies; and discussions of money management. Services directed at disciplinary practices were discussions of discipline of children, child's anger management, dealing with violence in the family, caretaker interaction with the child, supervision of children, and child development. Services directed at caretaker depression were discussions of depression, other caretaker emotional problems, social skills, and adult companionship. The extent of service response to particular problems was calculated as the simple count of the number of times caseworkers checked a particular item on the contact forms submitted for the family.
There are a number of limitations in this procedure. First, we looked at only three problem areas, areas that we thought we could identify relatively easily. Further, the measurement of service response is clearly not ideal, service data were not constructed in a way that would make them straightforwardly parallel to problems, so the development of service measures in this analysis is quite post-hoc. In addition, obviously caseworkers may have chosen, for good reason, not to respond to a particular problem, perhaps because another problem was more pressing or more tractable, so that the failure to respond to a particular problem should not be viewed as an indication of the failure of casework in the case. There was also considerable overlap of these problems (83 families reported all three problems at the initial interview, 82 reported two of them, while 24 reported none of them). Despite these limitations, tendencies toward the matching of services and problems should show up in the data, though perhaps not as prominently as would have been the case had data collection been explicitly directed at exploring these issues.
To determine the match between problems and services, we calculated the average number of times a service was provided to cases with the problem (the number of contact forms recording the service) and compared that to the average number of times the service was provided to cases that did not have the problem. The results are shown in Table 3-16.
Families with economic problems significantly more often than families without such problems received three of the five services identified as responding to these problems. We conclude that there is moderate evidence of a match between problems and services in the economic area. Families with discipline problems received two of five services significantly more often than those without these problems, some indication of a match. As to depression, caseworkers significantly more often discussed depression with caretakers with high levels of depression than with those with lower levels. For the other three services identified as possibly responding to caretaker depression there were no significant differences. Discussion of "other caretaker emotional problems" occurred more often with those with higher levels of depression, although the difference is not significant. We conclude, therefore, that there is some match between problems and casework response, a match that might have shown itself more strongly had measurement of services been designed to reveal it.
| Cases with the problem | Cases not having the problem | |
|---|---|---|
| Economic problems, n = | 157 | 134 |
| Services: | ||
| Emergency cash or paying bills | 0.57** | 0.23 |
| Buying food | 1.26** | 0.63 |
| Helping to find housing | 0.37 | 0.36 |
| Clothing, furniture, supplies | 0.92 | 0.66 |
| Money management discussion | 2.05** | 0.93 |
| Discipline problems, n = | 221 | 68 |
| Services, discussion of: | ||
| Discipline of child | 7.35** | 5.08 |
| Child anger management | 4.98* | 3.33 |
| Family violence | 2.53 | 1.73 |
| Caretaker interaction with child | 6.32 | 5.82 |
| Supervision of children | 3.45 | 3.76 |
| Depression, n = | 152 | 140 |
| Services, discussion of: | ||
| Caretaker depression | 2.83** | 1.43 |
| Other caretaker emotional problems | 2.37 | 1.80 |
| Adult companionship | 1.32 | 0.99 |
| Social skills | 2.42 | 2.85 |
| * p < .05, ** p < .01. | ||
Effect of Match of Services to Problems on Outcomes.
We next attempted to determine whether, for cases experiencing each of the three problem categories, service responses specific to the problem made a difference in outcome. The outcomes we examined were improvements in the specific problem at the time of the post-treatment caretaker interview and subsequent placement and maltreatment. Because families in the experimental group were not randomly assigned to varying levels of specific kinds of services, it can be assumed that there are selection biases operating in determining levels of service. Hence, for the examination of effects on the level of the problem at the post-treatment interview, two-stage least squares techniques were used in which the provision of specific services was modeled in the first stage and the effects of services on outcomes were determined in the second stage. Demographic characteristics were included in both stages and the level of the problem at the initial interview was included in the second stage. A variable reflecting the caretaker's response in the post-treatment interview to a question on whether additional services were needed was also included in both stages. First stage instruments were chosen based on their prediction of levels of service and lack of association with the outcome variables. In all of the analyses the instruments were jointly significant in the prediction of levels of service.
We report here on the results of the second stage of these analyses, the determination of the effect of specific services on problem level at the post-treatment interview. For all three problems, as would be expected, the initial level of the problem was significantly and positively related to the post-treatment level. For the group experiencing economic problems, of the five services thought to respond to the problem, only one was significantly related to post-treatment problem levels, the provision of cash assistance was positively related to level of economic problems post-treatment. That is, the more cash assistance provided, the higher the levels of economic problems. It is likely that this seemingly contrary finding simply reflects the fact that families with considerable economic difficulties are more likely to get cash assistance but are also more likely to continue to experience those problems. The variable reflecting need for additional concrete services was also positively related to post-treatment levels of economic problems.
For the group with discipline problems, of the five services in the second stage equation, two were significantly related to the level of the problem at post-treatment. The higher the level of discussion of discipline of children, the lower the level of the problem while the greater the discussion of child anger management, the higher the level of the problem. For cases with above median levels of depression, two of the four services thought to respond to the problem were significantly related to level of the problem at post treatment. More discussion of depression in caseworker contacts was related to higher levels of depression post-treatment while more discussion of adult companionship was related to decreased post-treatment depression.
The sparseness of positive findings in this analysis leads us to conclude that there is little evidence here of positive effects of concrete or clinical services on these three problem areas.
To examine the effects of specific services on placement and maltreatment following entry into the study, hierarchical linear modeling was used in order to account for the fact that we have multiple children in some families and their outcomes are not independent. (55) To deal with selection issues, predicted values of services, determined from the first stage of the 2SLS analyses, were entered at level two (the family level) and these terms were used to determine the effects of services on outcomes. Again, separate analyses were done for each problem group.
For the economic difficulties subgroup, two of the five problem specific services were significantly and negatively related to the likelihood of subsequent maltreatment: provision of clothing/furniture/supplies and housing assistance. Cash assistance and provision of clothing/furniture/supplies were significantly and negatively related to the likelihood of subsequent placement. For families with problems in discipline of children, none of the six services were related to subsequent maltreatment, while discussions of violence in the family were related to a decrease in likelihood of placement and discussions of child anger management were related to an increase. In families in which the caretaker was measured as having higher than median levels of depression at the beginning of service, none of the problem specific services were related to subsequent maltreatment. However, discussion of depression was related to an increase in likelihood of placement while discussions of social skills and of companionship/friendship were related to a decrease in placement.
No consistent patterns emerge in the analysis of the effects of specific services on subsequent maltreatment and placement. There is some indication that within the economic problems group, services directed at these problems have some beneficial effects on these outcomes. Within the discipline problems and depression groups, results are mixed, some services are related to increases in subsequent maltreatment and placement while others are associated with decreases. No clear conclusions can be drawn.
[Go To Contents]
Information from the caretaker interviews, the caseworker interviews, and the administrative data were analyzed for indications of differences between the experimental and control groups subsequent to the referral to the family preservation program. Tables 3-15 and 3-16 contain a summary of those outcomes on which we found significant differences between the experimental and control groups in any state for the primary analyses (p < .05). Items in bold are those on which the experimental group had better outcomes, those in italics are those on which the control group had better outcomes.
In none of the three states were there significant differences between the experimental and control groups on family level rates of placement or case closings. Subsequent maltreatment was generally not related to experimental group membership, except for one subgroup in Tennessee. In Tennessee, in those families with an allegation within 30 days prior to random assignment, the experimental group children experienced fewer substantiated allegations than children in the control group.
In Tables 3-17 and 3-18 there are a number of child and family functioning items in which the experimental group displayed better outcomes than the control group in one of the states. It should be noted that the results have not been adjusted for the multiplicity of significance tests performed. That is, these significant items surfaced out of a large number of items and scales examined. In such a situation it is to be expected that some items will show significant differences simply by chance, so the appearance of a few significant differences should not be taken as an indication of superiority of one group over another, particularly when the results are not confirmed in more than one state. On only two items were differences found in two states: caretakers' assessment of whether goals had been accomplished and their assessment of overall change. We are inclined to believe that family preservation programs as represented in these states do result in higher assessments by clients of the extent to which goals have been accomplished and of overall change, since differences on those items were found in both states. Beyond that, we are unable to claim consistent evidence of positive effects of family preservation services. (56)
| Caretaker Interview: Proportion of affirmative answers to yes/no questions | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Kentucky | New Jersey | Tennessee | |||||||
| Control % |
Exp % |
p | Control % |
Exp % |
p | Control % |
Exp % |
p | |
| Is apartment/house rented (vs. owned) | 75 | 89 | 0.005 | 70 | 68 | 69 | 75 | ||
| Got together with anyone to have fun | 64 | 64 | 65 | 59 | 38 | 75 | 0.001 | ||
| Felt had few or no friends | 14 | 18 | 20 | 18 | 38 | 19 | 0.03 | ||
| Had difficulty buying clothes | 17 | 21 | 47 | 33 | 0.008 | 27 | 24 | ||
| Out of control when punishing child | 24 | 24 | 40 | 30 | 0.05 | 11 | 12 | ||
| Punished for not finishing food | 7 | 1 | 0.02 | 6 | 5 | 0 | 0 | ||
| Unable to find someone to watch child | 9 | 12 | 21 | 12 | 0.04 | 20 | 27 | ||
| Encouraged child to read a book | 92 | 90 | 82 | 91 | 0.02 | 94 | 96 | ||
| Have goals been accomplished | 63 | 77 | 0.02 | 52 | 71 | 0.001 | 81 | 84 | |
| Assessment of overall change: | 0.02 | 0.001 | |||||||
| Great improvement | 16 | 22 | 9 | 16 | 32 | 32 | |||
| Some improvement | 31 | 42 | 41 | 52 | 32 | 42 | |||
| Same | 42 | 29 | 34 | 20 | 22 | 14 | |||
| Somewhat or a great deal worse | 12 | 6 | 16 | 12 | 14 | 13 | |||
| Caretaker Scales: | |||||||||
| Difficulty paying bills (proportion of 4 items) | 0.17 | 0.22 | 0.34 | 0.25 | 0.02 | 0.25 | 0.18 | ||
| Negative child care practices (proportion of 10 items) | 0.14 | .0.13 | 0.18 | 0.14 | 0.02 | 0.09 | 0.09 | ||
| Punishment (proportion of 5 items) | 0.16 | 0.17 | 0.25 | 0.20 | 0.04 | 0.13 | 0.13 | ||
| Negative child behaviors (proportion of 21 items) | 0.34 | 0.34 | 0.33 | 0.28 | 0.04 | 0.21 | 0.21 | ||
| Change in proportion of punishment items from initial to post-treatment interviews | -0.04 | -0.09 | 0.05 | -0.05 | -0.07 | -0.07 | -0.13 | ||
| Change in proportion of negative child care practices from Initial to post-treatment interviews | -0.02 | -0.06 | 0.04 | -0.04 | -0.05 | -0.01 | -0.08 | 0.02 | |
| Ability giving affection (higher = more adequate) | 2.83 | 2.83 | 2.93 | 2.70 | 0.04 | 2.73 | 2.95 | ||
| Providing learning opportunities for child (higher = more adequate) |
2.38 | 2.42 | 2.89 | 2.60 | 0.008 | 2.64 | 2.64 | ||
| Respecting child's opinions (higher = more adequate) | 2.58 | 2.45 | 2.55 | 2.42 | 2.35 | 2.84 | 0.01 | ||
| Responding patiently to child's questions (higher = more adequate) | 2.43 | 2.34 | 2.44 | 2.27 | 2.26 | 2.67 | 0.04 | ||
| Adequate supervision / Responsible child care (higher = more adequate) | 2.50 | 2.59 | 2.80 | 2.71 | 2.52 | 2.93 | 0.04 | ||
| Household condition (proportion of 13 items, higher = worse condition) | 0.10 | 0.13 | 0.01 | 0.09 | 0.11 | 0.02 | 0.12 | 0.12 | |
| Caretaker problems (proportion of 21 items, higher = more problems) | 0.25 | 0.31 | 0.0005 | 0.21 | 0.23 | 0.21 | 0.18 | ||
| Caretaker functioning (higher = better) | 2.56 | 2.55 | 2.79 | 2.66 | 0.10 | 2.51 | 2.82 | 0.04 | |
| Respecting child's opinions (change in average ratings from Time 1 to Time 2)** | 0.19 | -0.06 | 0.05 | 0.27 | 0.04 | 0.05 | 0.06 | 0.14 | |
| Setting firm/consistent limits/rules (change in average ratings from Time 1 to Time 2) ** | 0.35 | 0.22 | 0.33 | 0.25 | -0.29 | 0.29 | 0.01 | ||
| Caretaker Problems (Change in proportion of 21 items; lower = less at Time 2) | -0.06 | -0.04 | -0.05 | -0.04 | -0.03 | -0.08 | 0.05 | ||
| NOTE: This table only includes items
with a primary analysis p-value less than .05 in at least one of the states;
p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. ** Scale for change in ratings: -4 = ability decreased greatly over time, 0 = no change in ability over time, +4 = ability increased greatly over time |
|||||||||
| Proportion of affirmative answers to yes/no questions | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Control % |
Exp % |
p | Control % |
Exp % |
p | Control % |
Exp % |
p | |
| Has spouse held full time job | 81 | 78 | 86 | 68 | .05 | 100 | 85 | ||
| Had difficulty paying rent | 20 | 20 | 34 | 27 | 39 | 20 | .04 | ||
| Have children handled household chores | 75 | 75 | 70 | 83 | .02 | 94 | 89 | ||
| NOTE: This table only includes items
with either a primary p-value less than .05 in at least one of the states;
p-values greater than .10 are not reported Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. |
|||||||||
There are a few items on which the control group had better outcomes, nearly all of them on measures provided by caseworkers. We are not inclined to read too much into these results, since experimental group caseworkers generally knew the families better and there may well have been significant differences in the ways that workers serving the two groups saw families and judged their functioning.
[Go To Contents]
23. The full list of New Jersey service codes that were included is: public institution, teaching family placement, para-foster care income maintenance, juvenile-family crisis shelter placement, relative placement, foster care placement, residential treatment placement, finalized adoption placement, selected adoption placement - pending, maternity home care, group home placement, independent living, and shelter care placement. Four of these categories did not actually occur in the data: teaching family placement, para-foster care income maintenance, finalized adoption placement, and selected adoption placement - pending. In Kentucky placement (as reflected in the variable FACTYPE), included: adoption, foster care, private institution/boarding schools, family treatment home, unmarried parent, other, children's psychiatric hospital, and foster care medically fragile. The data did not include adoption, family treatment home, and unmarried parent. In Tennessee, placements included: foster care, relative home, trial home, residential care, continuum contract, non-relative home, adoptive home, runaway, shelter, independent living, and detention.
24. Cases entered the study at varying points in time. In Kentucky, cases entered between May 7, 1996 and February 13, 1998; in New Jersey, cases entered between November 6, 1996 and February 26, 1998; and in Tennessee, between November 19, 1996 and May 26, 1998.
25. There are two reasons for focusing on family-level analyses. First, we are not confident that the administrative data allow for accurate identification of children to be included in the risk pool (what would be the denominator in a rate of placement calculation). Children are identified as belonging to a family through a case number. The analysis requires that we identify children who are in the home at the time of random assignment (or who are born or return to the home subsequently). In these states, children apparently often retain a family case number even when they are not in the home, and the administrative data do not allow us to verify the location of the child at the time of random assignment (or even sometimes at the time of an event such as placement). This problem is alleviated in analyses at the family level, since we know that the family is at risk of having a child placed (as long as there are any children in the family).
As to the accuracy of the "numerator" in our analyses, we focus on the first event (e.g., placement) in the family, subsequent to random assignment. It is possible that the first event occurs with regard to a child identified with a family but not living in that family at the time of the event. We judge the likelihood of that occurring to be small (the effects of this source of error would be similar in a family and child level analysis). In addition, subsequent events involving other children identified with the family but not in the family at the time of the event would not affect the family level analysis, while they would create inaccuracies in a child level analysis.
The second reason for focusing on the family level has to do with a "clustering" effect in the child level analysis. Clustering refers to the lack of independence between children within the same family of observations of such things as placement. If one child is removed from the home, the remaining children are more likely to experience placement. The "clustering effect" leads to an underestimate of the significance levels when analyses are conducted at the child level. Conducting the analyses at the family level is one approach to resolving this dilemma.
We did conduct a few analyses at the child level, when we wanted to take into account child characteristics, but it should be remembered that significance levels in those analyses are downwardly biased.
26. In Kentucky, the ratio of assignment to experimental and control groups was 50-50.
27. In New Jersey, approximately 60 percent of the cases were assigned to the experimental group.
28. In Tennessee, approximately two-thirds of the cases were assigned to the experimental group.
29. Kentucky policy specifies that imminent risk includes children who are at risk of commitment as dependent, abused, or neglected; who are identified through the Regional Interagency Council, an interdepartmental unit, as severely emotionally disturbed; or whose families are in conflict such that they are unable to exercise reasonable control of the child. Both the referring worker and family members shall believe that without immediate intensive intervention, out-of-home placement is imminent. At the time of this study, New Jersey targeted family preservation services for families at imminent risk of having at least one child enter placement. The referring worker must have based the assessment of imminent risk on a face-to-face interview with the family no more than 5 days prior to the referral. The family must need services immediately and the worker must determine that other, less intensive, services have been used, are not appropriate, or are not available. In Tennessee, CPS intake workers complete a risk assessment form to identify high, intermediate, low, or no risk. High risk cases are identified as cases where "the child or children in the home are at imminent risk of serious harm if there is no intervention in the situation." A typical high risk case might involve such factors as: 1) a vulnerable child; 2) a history of previous maltreatment; 3) an active perpetrator who has continued access to the child; and 4) no available support or family strengths to offset the stated risks.
30. Analyses were also done on all allegations, whether substantiated or not. The results were very similar, although, of course, rates for all allegations were higher.
31. The six months analyses and survival analyses are obviously not independent.
32. Often we used average responses or proportions of positive responses rather than sums of responses to items. This was done in order to have scores for individuals when there were a few missing items on the scales. If an individual had too many missing items (usually 1/3rd or more) the score was declared missing. Rules for the calculation of all scales are given in Appendix J.
33. In multivariate repeated measures analysis, three main hypotheses are tested, first, that the scores for the experimental group, averaged over the three points in time are equal to those of the control group, (the "group" hypothesis); second, that the averages of the groups at each point in time are the same (the "time" hypothesis); and third, that there is no interaction between time and group. It is the third hypothesis that is central, indicating whether the groups change in different ways.
34. Variables in Tables 3-3, 3-4 and Figure 3-4 are described in Vol. 3, Appendix J.
35. This difference was slightly greater and statistically significant in the secondary analysis (48% vs. 35%, p= .04).
36. In the secondary analysis, fewer experimental group respondents reported health problems (12% vs. 21% for the control group, p = .04).
37. The control group had a slightly lower average proportion of affirmative responses to these items at post-treatment (.17 vs. .22, p = .16).
38. In the primary analysis, at post-treatment, a greater proportion of the experimental group reported difficulties paying rent (20% vs 13%, p = .13) and electric or heat bills (28% vs. 20%, p = .11). In the secondary analysis, differences were smaller and p-values for both items were above .20.
39. This difference was maintained but not significant in the secondary analysis (5% vs. 1%, Fisher's exact p-value = .078).
40. In the secondary analysis, there was again a .09 reduction in the average proportion of punishment items endorsed by the experimental group and a .04 reduction for the control group (p = .03).
41. Derogatis, L. R., Lipman, R. S., & Covi, L. (1973) SCL-90: An outpatient psychiatric rating scale -- preliminary report. Psychopharmacology Bulletin, 9 (1), 13 - 28.
42. Reliability analysis yielded a Cronbach's alpha of .92 at initial, .93 at post-treatment, and .92 at follow-up in Kentucky; .95 at initial, .94 at post-treatment, and .95 at follow-up in New Jersey; and .91 at both initial and post-treatment, and .90 at follow-up in Tennessee.
43. This difference was also significant for the secondary analysis (28% vs. 33%, p = .006).
44. In the secondary analysis, the difference was maintained and remained significant (31% vs. 24%, p = .0004).
45. In the secondary analysis, the average percents were 24 percent for the experimental group and 21% for the control group (p = .06).
46. In the secondary analysis, however, the difference increased and approached significance with 29 percent for the experimental group and 24 percent for the control group, p = .06.
47. The difference for the secondary analysis was also not significant (25% vs. 28%, p = .12).
48. The significant interactions with experimental group were as follows. For depression at post-treatment in New Jersey, there was an interaction of experimental group with single motherhood; for single mothers, there was no relationship between experimental group and depression, for other caretakers, the control group had higher depression scores. Also for depression at post-treatment in New Jersey, there was an interaction with employment; for those employed at the initial interview, there was no difference between the experimental and control groups, for those unemployed, the control group had higher depression scores. For negative life events at post-treatment in Tennessee, there was an interaction with income support; for those not receiving income support the control group had more negative life events, for those receiving income support, there was no difference between the experimental and control groups in negative life events. For household condition at follow-up in Tennessee, there was an interaction between age of caretaker and experimental group; in the control group there was no relationship between age and household condition while in the experimental group, older caretakers had worse household conditions.
49. Brown, D., Ahmed, F., Gary, L., & Milburn, N. (1995) Major depression in a community sample of African Americans. American Journal of Psychiatry 152(3), March 373-378.
50. Humke, C. & Schaefer, C. (1995) Relocation: A review of the effects of residential mobility on children and adolescents. Psychology; a quarterly journal of human behavior, 32(1), 16-24.
51. Honig, A. & Pfannestiel, A. (1991) Difficulties in reaching low-income new fathers: Issues and cases. Early Child Development & Care 77, 115-125.
52. Baxter, A., & Kahn, J. (1999) Social support, needs and stress in urban families with children enrolled in an early intervention program. Infant-Toddler Intervention 9(3), September 239-257.
53. The differing results for the uncontrolled analysis and the regression analysis may be due to the significant interaction in the regression equation of experimental group and income support.
54. The questions were: have you lost your temper when your children got on your nerves, have you found that hitting your child was a good way to get him/her to listen, have you sometimes found yourself hitting your child harder than you meant to, have things sometimes gotten out of control when you punished your child, have you punished your child by tying him/her up with a rope, cord, string, or belt, have you sometimes punished your child by not letting him/her into the house, have you punished your child for not finishing the food on his/her plate.
55. Because the dependent variable was dichotomous, the logit link function was used, transforming the outcome into log-odds. Hence, the analysis actually used a hierarchical non-linear model.
56. The reader is reminded of the findings reported in Chapter 7 indicating that experimental group caretakers generally had more positive views of service and of their relationships with workers than control group caretakers.
Main Page of Report | Contents of Report
Home Pages:
Human Services Policy
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)