Evaluation of the New York City Home Rebuilders Demonstration

IMPLEMENTATION OF THE HOMEREBUILDERS DEMONSTRATION

Chapter 3:
Section 3.1  Administrative Data

Contents

1. Case Characteristics
2. Outcomes from the Administrative Data
3. Refinements on the Above Analysis
4. Goals
5. Summary

Data from New York State on the foster care placement histories of 2987 children in the experiment were analyzed. This figure excludes 4 children who died during the experiment, 17 children who were born after the experiment began and who were subsequently placed, 5 children who were first placed after the experiment began, 14 children who were in placement both before and after the experiment began on July 1, 1993 but were not in placement on that date, and 9 children who were on trial discharge or otherwise absent from their placement homes on that day. The numbers of children by agency and experimental group are shown in Table 3-1. In the three "true experimental agencies" (HD, LF, and MM) there were 640 control cases and 696 experimental cases.

Table 3-1.  Number of cases by agency and experimental group

3.1.1  Case Characteristics

Sex. Overall, 51 percent of the children were female. There were some differences in sex distributions among agency groups as indicated in Table 3-2.

Table 3-2.  Gender by agency and experimental group (percents)

Ethnicity.  In 17 percent of the cases, the administrative data did not record the race of the child. Of the remainder, 74 percent were African American, 7 percent Puerto Rican, 14 percent other Hispanic, 2 percent white, and 2 percent other (Table 3-3). Again, there were small differences among agencies and experimental groups. As can be seen, the OTT comparison group and the NYF experimental group had somewhat more Puerto Rican and other Hispanic children (a combined total of 34% and 32% respectively) and somewhat fewer African American children (65% and 63%) than the other groups (it should be remembered that the groups in these agencies were not randomly assigned). In MM, there was also a bit of a difference, 11 percent of the experimental group and 21 percent of the control group were Puerto Rican or other Hispanic.

Table 3-3.  Ethnicity by agency and experimental group (percents)

Note:  Due to rounding, percents may not all equal 100%

Age.  The median age of children at the time the experiment began on July 1, 1993 was 5. Seven percent were less than 1 year old, 19 percent were 1 or 2 years old, 28 percent were at least 3 but less than 6, 31 percent were between 6 and 11, and 14 percent were 11 or more (Table 3-4). Again, there were some differences across agencies. Children in OTT were generally younger than those in other agencies.

Table 3-4.  Age of children by agency and experimental group (percents)

Note:  Due to rounding, percentages may not always equal 100%.

Placement Histories.  The vast majority of children (88%) were in their first placements at the time the experiment began. There were no important differences among the agencies and experimental groups on this characteristic.

Length of Time in Placement.  At the time the experiment began, the length of time children had been in placement varied widely from 1 day to 11.6 years. The median length was 2.6 years, the mean 2.9 years. The interquartile range was from 1.2 to 4.3 years. There was some variation among agencies and groups (Table 3-5). Children in LF, JC, and OTT had been in care a shorter period of time than those in the other agencies. The differences between experimental and comparison groups are significant in LF, NYF, and OTT at the .03 level. In LF, MM, and OTT, the experimental group had been in care a shorter period of time than the comparison group, on average, while in NYF the comparison group had been in care a shorter period of time. The averages in HD were very close. The differences between experimental and control groups in LF and MM appeared despite random assignment of cases in those agencies.

As will be seen, length of time in care at the beginning of the experiment was related to outcomes of the experiment.

Table 3-5.  Years in placement at the beginning of the experiment by agency and experimental group

Type of Placement. On July 1, 1993, 72 percent of the children were in nonrelative foster boarding homes, 24 percent were in "approved relative care," and the remainder in various forms of group homes. There were substantial differences among agency groups in these percentages (Table 3-6). LF and OTT had far fewer relative placements than other agencies while JC had far more.

Table 3-6.  Type of placement at the beginning of the experiment by agency and experimental group (percents)

Note:  Due to rounding, percentages may not always equal 100%.

Permanency Goal.  The permanency goal on July 1, 1993 was return home for 65 percent of the children and adoption for 32 percent. The remaining children had goals of independent living, "other primary resource," or "placement prevention." Again, there were differences among the agencies (Table 3-7). LF and OTT had far fewer cases with the goal of adoption than other agencies.

Table 3-7.  Permanency goal at the beginning of the experiment by agency and experimental group (percents)

Note:  Due to rounding, percentages may not always equal 100%

Summary.  Children in the HomeRebuilders experiment were almost entirely African American or Hispanic. Over one-half were less than 6 years of age. Most were in their first placements in nonrelative foster homes and had been in placement a fairly long period of time, on average a little less than 3 years. The original intent of the experiment was to include only children for whom the goal was return to birth parents, but that requirement was relaxed and about one-third of the children had goals of adoption at the time they entered the project. On many of these characteristics, there was variation among agencies and between experimental and comparison groups within agencies. The differences among agencies in child characteristics suggest that the agencies were dealing with somewhat different populations of children. The differences between experimental and control groups in the true experimental agencies occurred in spite of random assignment.

3.1.2  Outcomes from the Administrative Data

The rapid achievement of permanency and reduction in days in care were central objectives of the HomeRebuilders program. Two measures were examined: the percentage of children whose cases were closed and the number of days in care subsequent to July 1, 1993. Obviously, these are highly associated measures. The first of these measures concerns the first time cases were closed following July 1, 1993. The primary reasons for case closing were return home and adoption, although a few cases were closed for other reasons, for example, when the child aged out of the system. The number of days in care takes into account all placements subsequent to July 1, 1993, but subtracts days in which the child was on trial discharge or otherwise absent from the foster care placement. The intent of this measure is to capture the number of days of paid foster care. It can be used as a rough indication of the costs of foster care for a child.

The status of cases may be examined at various points in time. In the following, we examine status at various points following the beginning of the experiment up to April 1997, the last date for which we have data. Outcomes were examined only within agencies. In most of the analyses, there is considerable variation among agencies so that combining across agencies would not be meaningful, since it would obscure the between agency variations. In OTT and NYF, cases were not randomly assigned, so comparisons between the experimental and comparison groups and the significance levels of those comparisons are less meaningful. In JC there was no comparison group.

Percentages of Cases Closed.  Table 3-8 shows the status of children at two points in time, December 31, 1995, when the experiment was ended and April 1997. As can be seen, there is considerable variation among the agencies and comparison groups. By December 1995, in most of the groups fewer than 50 percent of the cases were closed. By April 1997, this figure had increased but varied considerably among the groups, from 44 percent in the HD experimental group to 78 percent in JC. As to comparisons between experimental and control groups, in LF the experimental group had closed significantly more cases than the control group by December 1995 (p < .001). By April 1997 the difference had narrowed somewhat (p = .06). In MM, the control group had closed more cases than the experimental group by December 1995 (p = .03); proportions of closed cases by April 1997 were very close. In HD, the difference between the experimental and control groups in December 1995 was not significant, but by April 1997 the control group had closed more cases (p = .02). As to the nonexperimental agencies, in OTT differences were not significant at either point in time while in NYF, the experimental group had closed significantly more cases at both points in time (both differences,

p < .001). In JC the proportion of closed cases was relatively high, but it is difficult to know what to make of this, given the lack of a comparison group in that agency.

Table 3-8.  Status of Children (percents)
at 12/31/95

Note:  Significant differences between experimental and control groups in percent closed are indicated by symbols attached to the compared percentages:  # = p < .1; * = p < .05; *** = p < .001.

at 4/30/97

Note:  Significant differences between experimental and control groups in percent closed are indicated by symbols attached to the compared percentages: # = p < .1; * = p < .05; *** = p < .001.

Of interest is the relatively low proportion of children in the experimental group who were returned home. The group with the highest proportion of children returned home was the LF experimental group, 39 percent by December 1995 and 44 percent by April 1997. Agencies varied considerably in the extent to which they used return home, and conversely, adoption, as routes to case closing. Particularly interesting is a comparison of LF and NYF, two agencies in which the experimental group closed significantly more cases than the comparison group. LF returned more children home and used adoption much less than NYF. Reasons for closing as percentages of all cases closed can also be examined. (see Appendix Table B-6). Except in OTT, as a proportion of closed cases, the experimental groups had fewer adoptions and more returns home than the comparison groups.

Figure 3-1 provides more detail on percentages of cases closed over time. The graphs indicate the widening and then narrowing of the differences between the experimental and control groups in LF. In NYF the increasing difference between the groups is evident, with the experimental group closing ever more cases over time. Of interest is what happens to these graphs following the end of the program. In the LF experimental group the graph flattens after that point, so that the difference between experimental and control groups narrows. In HD, the experimental group graph also flattens while the control graph continues to rise, leading to greater differences between the groups. The flattening of the graphs for the experimental groups might reflect the withdrawal of the fiscal incentives of the program, but these dynamics are not seen in the other experimental groups.

Days in Care.  Table 3-9 shows the average days in care for children in the experiment at the end of 1995 and by April 1997. The results are similar to those in the analysis of closed cases in that the experimental groups in LF and NYF have significantly lower levels of care days than the comparison groups at both points in time. At the end of the project in December 1995, the LF experimental group had 13 percent fewer days in care than the control group and the NYF experimental group had 7 percent fewer days in care. By April 1997, the LF experimental group had 15 percent fewer days in care than the control group and the NYF experimental group had 14 percent fewer days in care. In this analysis, in no agency does the comparison group have significantly lower days in care than the experimental group at either point. Also consistent with the case closing analysis, JC had relatively low levels of average care days.

Table 3-9.  Mean days in care by agency and experimental group

Figure 3-2 displays number of care days at several points in time. In HD, MM, and OTT, the curves for the experimental and comparison groups are remarkably similar, while the lower days in care in the experimental groups in LF and NYF is evident.

It is of interest to examine lengths of time in care for differing outcomes, in particular, for adoption versus other outcomes. Considering only those cases that closed by April 1997, as would be expected, time in care was considerably higher for children that were adopted.

For all cases that were closed, the total length of time in care was 60 percent higher for adopted children. The differences in LF between experimental and control groups in lengths of time in care may be due in part to the much lower number of adoptions in the LF experimental group. However, it must be noted that NYF, with a significant difference between experimental and comparison groups in days in care, had a large number of adoptions overall and little difference between experimental and comparison groups in proportions of closed cases that were adoptions (67% vs. 70%).

3.1.3  Refinements on the Above Analysis

The original conception of the HomeRebuilders experiment called for limiting the sample to cases that were not in relative care and that had a goal of return home. However, these restrictions were subsequently relaxed and a number of children were included in the experiment who were in relative care and a number who had adoption as a goal at the beginning. Analyses only of children in nonrelative foster care and in nonrelative foster care with a goal of return home are shown in Appendix Tables B-8 and B-9. The results are very similar to those for all children shown above.

Regression Analyses. A regression analysis was performed to "control for" characteristics of children at the time the experiment began. The dependent variable in the analysis was days in care through April 1997 while the independent variables were child characteristics available in the administrative data: ethnicity, sex, age, days in care at 7/1/93, goal at the beginning of the experiment, and experimental versus control group. Separate analyses were performed for each of the agencies. The results (shown in Appendix Table B-10) indicate that, controlling for the other factors in the equations, the HomeRebuilders groups in LF and NYF continued to have significantly fewer days in care than the comparison groups in those agencies. The effects in reduction in days in care were slightly larger than the uncontrolled effects shown above. The effects on days in care for the other agencies were not significant.

The results also indicate that in all agencies, Hispanic children tended to spend less time in care than African Americans, although the effects of ethnicity were significant only in HD. Sex was not significant, except in NYF, where males had longer times in care. More previous placements tended to result in more days in care while longer length of time in care at the beginning of the experiment tended to result in a lower number of subsequent days in care. A goal of return home had mixed effects in the various agencies. In LF, it led to a lower average number of days in care (p < .1) while in MM it led to more days in care (p < .05).

3.1.4  Goals

Goals for the case often changed in the course of work with the family. Table 3-10 shows goals at the time the experiment began by goals at the time the experiment ended in December 1995.

The most frequent goal change was from return home to adoption (39% of cases with an initial goal of return home were changed to adoption). When adoption was the initial goal, it was infrequently changed (90% of initial adoption goals remained adoption at December 31, 1995).

Table 3-10.  Permanency goal at December 31, 1995
by goal at beginning of the experiment
(numbers, row percents in parentheses)

Note:  "Other" goals include independent living, "other primary resource," and "placement prevention."

Changes in goals by agency are shown in Appendix Table B-11. For cases with an initial goal of return home, the variation among agencies in percent of cases with changed goals was not large (from 48% in the LF control group to 66% in the HD control group and NYF experimental group). However, the proportion of changes from return home to adoption did vary considerably, from 19 percent in the LF experimental group to 53 percent in the HD experimental group. Most cases with an initial goal of adoption continued to have that goal in the agency subgroups. The exception is the LF experimental group, where changes occurred in 80 percent of the small number of cases (15) with an initial goal of adoption.

It has been suggested that "managed care" programs like HomeRebuilders should result in quicker decisions on courses of action to take. It was thought that in HomeRebuilders, changes of goals might occur more quickly than in the comparison group. Thus, dates of the last goal change were examined. Details are shown in Appendix Table B-12. Although there were few significant differences, the date of the last goal change was, on average, earlier in the experimental groups, consistent with expectations. The exception was decisions to move to adoption in LF, where such decisions took longer in the experimental group.

3.1.5 Summary

Of the three true experimental agencies, the LF experimental group had more case closings and fewer days in care than the control group. In the other two experimental agencies, there is little evidence of a positive effect of HomeRebuilders on case closings and days in care. Among the nonexperimental agencies, the NYF experimental group had significantly more case closings and fewer days in care than the comparison group. The results for LF and NYF were confirmed in a regression analysis controlling for certain characteristics of the children.

About two-thirds of the children began the experiment with a permanency goal of return home; nearly all of the remainder had a goal of adoption. Many of the original goals of return home were subsequently changed to adoption, so that by the end of the experiment, in December 1995, the proportions had substantially reversed, with 55 percent of the children having been adopted or having a goal of adoption.