Evaluation of the New York City Home Rebuilders Demonstration

IMPLEMENTATION OF THE HOMEREBUILDERS DEMONSTRATION

Chapter 3:
Section 3.2  Interviews with Caretakers

3.2.1  Caretaker Interview Sample Description

In a sample of the cases included in the administrative data analysis, caretakers and their caseworkers were interviewed. The interviews were designed to provide data on services delivered to the families and children and on child and family functioning following the experiment.

Services were examined in order to describe the activities undertaken on behalf of these families and to determine differences between experimental and comparison groups in those activities. At the outset of the project, it was intended that the experimental group would receive more intensive services directed at reunifying the children or bringing about other permanent arrangements. The analysis aimed at determining whether, in fact, the services provided to the two groups were different.

The effects of the experimental intervention on child and family functioning was also examined. It is not entirely clear how one should think about child and family functioning as an outcome of this effort. On the one hand, it would seem that any new program should strive to enhance the functioning of the clients it serves and should be judged at least in part on the extent to which improvement occurs. On the other hand, the primary intervention in HomeRebuilders was a fiscal one, directed at lowering the number of days in foster care. Hence, one might view the program as a success if care days were lowered while functioning was not negatively affected.

The sample of cases for caretaker and caseworker interviews was restricted to cases that had a goal at the beginning of the experiment of return home and at least one child under 13 years of age. Thus, the interview sample is not parallel to the entire group of cases analyzed in the previous section. Furthermore, strictly speaking, the experimental and control group interview samples cannot be considered to have been randomly assigned, in part because response rates were not perfect (see Appendix A) and may have led to some differences between experimental and control groups in the families actually interviewed. In addition, the process of determining the caretaker to be interviewed may have resulted in differences between groups in the characteristics of the interviewees. Interviews were held with caretakers in both the experimental and comparison groups in HD, LF, MM, and OTT. Caretakers in the comparison groups in NYF and JC were not interviewed (there was no comparison group in JC). Because of the complexity of some cases, the determination of the caretaker to be interviewed required detailed rules, which are given in Appendix A. A total of 433 caretakers were interviewed. Thirty percent were birth parents, 26 percent relatives, and 43 percent nonrelatives. There were 52 percent adoptive or preadoptive respondents and 11 percent foster parents. Relative adoptive or preadoptive caretakers accounted for 14 percent, nonrelative adoptive or preadoptive for 38 percent. Relatively few respondents were foster parents. The breakdown of respondent category by agency and experimental group is shown in Table 3-11.

Table 3-11.  Caretaker respondent type by agency and experimental group (percents)

Within agencies, there were relatively few differences between experimental and comparison groups. In OTT, no comparison group cases were adoptive or preadoptive relatives, while 13 percent of the experimental group was in that category. In MM, 19 percent of the control group were foster parents or nonadoptive relatives, while 8 percent of the experimental group was in that category. In LF, there are two important differences between the experimental and control group -- more experimental group respondents were birth parents (with or without all of the children having been returned; 34% of the experimental group vs. 20% for the control group) and fewer were adoptive or preadoptive nonrelatives (31% vs. 43%).

There were some differences across agencies. HD had fewer birth parents and more adoptive or preadoptive nonrelatives than other agencies. NYF had fewer adoptive and preadoptive nonrelatives, while LF had fewer adoptive and preadoptive relatives.

Respondent Characteristics. The respondents were 95 percent women, 72 percent African American, 24 percent Hispanic, and 4 percent white (2 respondents were an "other" race-ethnicity). As shown in Table 3-12, race/ethnicity varied by agency.

JC and NYF had fewer African American and more Hispanic respondents than the other agencies. In HD, LF, and MM, the race/ethnicity distributions in the control and experimental groups were very similar. In OTT, there were more African American respondents in the experimental group (84%) than the comparison group (65%) and fewer Hispanic respondents (11% in experimental group, 32% in comparison group).

Table 3-12.  Respondent ethnicity by agency (percents)

Note:  Due to rounding percentages may not always equal 100%.

The average age of respondents was 46 (s.d.: 11.8). The average size of the respondents' households was 5.3, with an average of 3.5 children. Forty-nine percent of the children in the homes were female. The median age of the youngest child was 4 (mean of 4.9), the oldest, 12 (mean 11.5). Thirty-two percent of the respondents were married, 21 percent single, 23 percent never married, and divorced and widowed accounted for 12 percent each. Birth parents were more likely to be never married (45%) and less likely to be married (20%). In MM, 52 percent of the control group were married and 17 percent never married compared to 32 percent and 15 percent in the experimental group. Sixty percent of the respondents had less than a high school education, 15 percent had graduated from high school, and 24 percent had at least some college. There were no strong differences in education by type of respondent, although adoptive relatives and foster parents/other relatives more often had an 8th grade or less education (19% and 14% respectively, compared to 7% of birth parents and other adoptive parents). There were some differences in caretaker education among agencies and between experimental and control groups within agencies as shown in Table 3-13. Within the true experimental agencies, the control group in MM had a larger proportion of cases with eighth grade or less than the experimental group, although the numbers are quite small (7 versus 2). In LF, the experimental group had lower levels of education than the control group while in HD the opposite was true. These differences may have affected the comparison of the experimental and control groups on outcomes that are reported below.

Table 3-13.  Respondent educational level by agency and experimental group (percents)

A majority (52%) of respondents reported they were unemployed and not looking for work (45% of birth parents, 60% of adoptive relatives, 53% of other adoptive parents, and 56% of foster parents/other relatives), 34 percent were employed, and 14 percent were looking for work. Birth parents were a bit more evenly split among these three categories (45%, 27%, and 28%). There were some differences among agencies (Table 3-14). There were no strong differences between experimental and ccomparison groups within agencies, except in LF where 38 percent of the control group were employed and 40 percent not looking for work compared to 20 percent and 57 percent in the experimental group: (again, this difference may have affected comparisons in LF). Most (75%) of the respondents rented their residences, including 93 percent of birth parents, 76 percent of relatives, and 62 percent of nonrelatives.

Table 3-14.  Respondent employment status by agency (percents)

Summary. Consistent with the characteristics of the child sample, most respondents were African American or Hispanic. Nearly all were women and their average age was 46. Their households tended to be large, an average of over 5 members including 3.5 children. More than one-half of the respondents had less than a high school education. Only about one-third were employed and another one-seventh were looking for work.

3.2.2  Experience of Respondents with Maltreatment

It is often thought that child maltreatment is an intergenerational phenomenon, that parents who were abused as children are more likely to abuse their own children. Respondents were asked whether they were abused or neglected as children (Table 3-15). Fourteen percent said they had been abused, including 29 percent of birth parents, 10 percent of relatives, and 6 percent of nonrelatives. Eleven percent had been neglected, including 23 percent of birth parents, 10 percent of relatives, and 4 percent of nonrelatives. Six percent (26) had been in foster homes or institutions, including 18 birth parents, 3 relatives, and 5 nonrelatives. Evidently, birth parent respondents were more likely to have been maltreated.

Table 3-15.  Experience of caretakers with maltreatment and foster care (Percents)

3.2.3  Outcomes

The interviews included a number of questions that measure outcomes. Some of these items had very little variation in response. The relationship between membership in experimental and comparison groups was examined for those items that had at least minimal variation. In addition, summations of various individual items formed scales. Differences between the experimental and comparison groups were analyzed within the four agencies in which we interviewed caretakers in both groups (HD, LF, MM, and OTT). Three of these agencies (HD, LF, and MM) were considered "true experimental" agencies. We did not analyze combinations of all experimental groups against all comparison groups, since such combinations would have little meaning, given the lack of control groups in JC and NYF and the nonrandom formation of groups in OTT.

Three categories of significant findings were identified, those at or below p = .1, .05, and .01. Probabilities reported are two-tail, based on Chi-square, Fisher exact, or t-tests, as appropriate. T-tests of differences in group means were used for scales and for a few items involving ordered categorical responses. A large number of statistical tests were performed and this invariably leads to the identification of spurious results, so caution must be used in interpreting isolated significant findings.

Parent and Family Functioning. Parent and family functioning was examined only for respondents who were birth parents. Although the functioning of other caretakers and their families is not unimportant, HomeRebuilders was originally conceived as a program that would enable birth parents to regain custody of their children. The focus of the intervention was to be on the functioning of those parents. Furthermore, when other goals were pursued (most notably, adoption) involving other caretakers, the extent of involvement of the agencies with those other caretakers varied considerably, making comparisons problematic. Unfortunately, the numbers of birth parents within experimental and comparison groups within agencies was very small, ranging in these analyses from 6 to 19, making it difficult to detect significant differences. Table 3-16 summarizes the results.

Table 3-16.  Birth parent and family functioning

There were two series of yes-no questions. The first (Personal Life Events) asked whether any of 14 "things have happened to you in the last three months," and included both positive ("gotten together to have fun or relax," "felt happy") and negative ("felt blue or depressed," "felt nervous or tense," "felt you had few friends," "not enough money for essentials," "overwhelmed by work or family responsibilities") experiences (caretaker interview, question 41). Seven of these items had too little variation to analyze further. On none of the remaining seven items were there any significant (p < .1) differences between comparison groups in any agency. Sometimes the experimental group had better outcomes, other times the comparison group. Four of the items could be thought of as reflecting depression and were combined into a "life events depression" scale. None of the comparisons of the birth parents in the experimental and comparison groups were significant on this scale.

The second series of yes-no questions was 15 items tapping caretaking behaviors ("In the last three months, have you . . .") (caretaker interview, question 55). Seven of these items had too little variation to compare experimental and comparison groups. Of the remaining eight items, five were positive ("praised your child for doing well," "listened to music together," "gone to an amusement park, pool, etc.," "encouraged your child to read a book," and "had your child handle household chores on a regular basis") and three were negative ("lost your temper when your child got on your nerves," "things got out of control when punishing your child," and "blamed your child when things were not his or her fault"). Five of the fifteen items formed a "punishment" scale.

On the eight items with adequate variation, none of the 32 within agency comparisons of birth parent respondents proved significant. For the punishment scale derived from these items, there was one significant difference: the experimental group in LF had a higher mean score (higher level of punishment) than the control group (p < .05).

The interview included a family life events inventory of 15 items ("has anything like this happened to you or someone in your household in the past three months") (caretaker interview, question 40) from which were derived two scales, positive life events (6 items) and negative life events (8 items). None of the within agency experimental-comparison group comparisons for birth parents were significant on these scales. Four yes-no questions were asked about difficulties in paying for things, from which a scale was derived consisting of a count of yes responses (caretaker interview, question 52). Only in OTT was there a significant difference on this scale: the experimental group had fewer problems in paying for things than the comparison group (p < .1).

Respondents were asked for their overall assessment of changes in family life since the agency began work with them (caretaker interview, question 69). A five point scale, ranging from great improvement (1) to a great deal worse (5) was provided. Examining means on this item, there were no significant differences between experimental and comparison groups within agencies, for either all respondents or for birth parents only. Means were lower (better) for the experimental group in HD and LF and the comparison group in OTT. In MM the means were the same for all respondents and lower in the control group for birth parents. Responses to this question, collapsed into three categories, were also examined in crosstabulations, (the categories "same," "somewhat worse," and "a great deal worse," were collapsed, the latter two having very few responses). Analyzing all respondents in the true experimental agencies combined, experimental group respondents more often said "great improvement" (41% vs. 28% for the control group) and less often rated matters as same or worse (28% vs. 40%). The significance of this difference was .11. The result held in HD and LF (nonsignificant for HD, p = .095 for LF) while it was reversed for OTT (comparison group respondents more often rated things as greatly improved). In MM, the percentages in the three categories for the experimental and control groups were almost the same. Separate analysis of birth parent responses yields similar nonsignificant patterns.

Child Outcomes. A series of 35 yes-no child behavior questions were used to derive four scales (caretaker interview, question 57):

• Proportion of 25 negative child behaviors,
• Proportion of 10 positive child behaviors,
• Number of four child aggression items yes (these items were among the negative behaviors), and
• Proportion of five school problem items yes (also among the negative behaviors).

These scales were analyzed both for all respondents and for birth parents separately. In the all respondents analysis, no significant differences were found between experimental and comparison groups in any of the agencies. For birth parents, there was one significant difference: in OTT, average scores for child aggression were lower in the experimental group than in the comparison group (p < .05).

Even more than in the case of parental functioning, issues arise in the interpretation of child behavior as an outcome of these services. Most of the services of programs like HomeRebuilders are directed to parents. Therefore, the effects on child behavior are necessarily indirect and might be expected to be relatively small.

Summary. Given the large number of comparisons examined, little can be made of the few significant differences noted above. Nonsignificant differences often favored the comparison group. There is no conclusive evidence in the interviews of caretakers that the HomeRebuilders group had better child and family functioning than the comparison group.

3.2.4  Services

Respondents were asked a number of questions about social program involvement, services received, and caseworker activities. A few indexes were constructed from these items and again the focus was on differences between experimental and comparison groups within agencies. These items were examined both for all respondents and for birth parents separately. The results are shown in Table 3-17.

Respondents were asked about involvement in the last 3 months in 11 programs, including income support programs (e.g., food stamps, WIC, and AFDC) and various treatment programs (alcoholism or drug addiction programs, marriage counseling, and community mental health programs) (caretaker interview, question 53). There was enough variation in involvement in six of these programs to make comparisons between experimental and comparison groups. In the analysis of all respondents, 3 of the 24 within agency comparisons were significant. A scale was constructed of the number of income support programs received. There were no significant differences in means of this variable within agencies. Also determined was whether respondents had been in any treatment program. For all respondents, in only one group did this percentage exceed 15 percent (in the JC treatment group it was 19%) and it was often less than 10 percent. For birth parents only, the percentages were higher (reaching 60% of 15 respondents in JC, but 33% or lower in the other groups). None of the comparisons between experimental and comparison groups were significant on this variable.

Table 3-17.  Caretaker respondents' reports of program involvement and services received

In the analysis of birth parent respondents' involvement in programs, there were two (out of 24) significant differences within agencies, both in MM. In MM, experimental group birth parent respondents less often received food stamps and AFDC (both comparisons, p < .05).

Respondents were also asked about the receipt during the past 2 years of 13 services, including day care, help in finding a place to live, medical or dental care, and transportation (caretaker interview, question 54). Nine of these services had enough variation to proceed with between group analyses. In the analysis of all respondents, 7 of the 36 within agency comparisons were significant. In all but one of these comparisons, experimental group families more often received the service. In all three true experimental agencies, experimental group clients more often received help finding a place to live, although the differences were nonsignificant.

A scale consisting of a count of these 13 services received was constructed. In the all-respondents analysis, there were no significant differences within agencies, although in all of the four comparison group agencies the experimental group had a higher average number of services received. For birth parents alone, experimental group respondents in MM had a significantly higher level of service use (p < .05) and in LF, experimental group families had a nonsignificantly lower level of use. Otherwise, the results paralleled those for the all-respondents analysis.

Respondents were asked whether their caseworkers had engaged in any of 17 activities, including help with money, transportation, advice about job training programs and school, and help with handling feelings (caretaker interview, question 64). Twelve of these items had enough variation for analysis (see Table 3-18). In the all-respondents analysis, 4 of the 48 comparisons in the control group agencies were significant. In the birth parent analysis, there were five (of 48) significant differences, all in the "wrong" direction, that is, the control group more often experienced the activity. Four of these differences were in MM and one was in LF. On an overall index of caseworker activities, caseworkers for control group birth parents in MM engaged in more kinds of activities (p < .1); there were no significant differences in the other agencies.

Table 3-18.  Caretaker respondents' reports of caseworker activities

Nonadoptive and preadoptive respondents were asked whether their workers helped them see their good qualities and their problems (caretaker interview, question 68). None of the within agency comparisons were significant on these two items. For the true experimental group agencies, experimental group respondents more often reported the worker helped them see their problems, but the differences were not significant. For birth parents only, there were no significant differences, but the directions of associations were similar to those for all respondents.

Birth parents whose children had returned home were asked whether there was discussion with the caseworker, foster parent, and child about the child's return home. None of the comparisons within agencies were significant, although the numbers in the agency subgroups were quite small. These respondents were also asked whether the number and length of visits with children had increased in the month before the children returned home. There were no significant differences between the experimental and comparison groups. They were also asked whether they had received any help from the agency since the return of the children. Again, there were no significant differences between experimental and comparison groups. The percentage varied widely in subgroups (from 0% to 56%) and in LF, MM, and OTT such after return involvement was higher for the control group, although again, the numbers were very small.

Respondents were asked about the frequency of contact with the worker in the last 3 months. In nearly all of the comparisons of experimental and comparison groups, for both all respondents and birth parents alone, members of the comparison group had more contact with the worker. This may be due to the fact that experimental group cases were closed earlier than comparison group cases, so that more of the experimental group cases had no opportunity to meet with the worker during the three months prior to the interview.

Finally, respondents were asked whether the worker listened to their concerns and understood their situations. For the birth parents in HD and LF, experimental group respondents more often answered "most of the time" on these two questions, though the differences were not significant. There was little difference between groups in the all respondents analysis of these two questions.

Relationships with Foster Parents. Birth parents were asked about their relationships with the foster parents responsible for their children. The results are shown in Table 3-19. The small number of cases in the analysis hampers interpretation. In LF and MM the experimental group had more contact with foster parents than in the control group. The reverse was true in HD. As to the five activities that were asked about, there were no strong trends in any agency; within agencies, the foster parents in the experimental group engaged in some activities more than the comparison group and in other activities, less than the comparison group. In HD and LF, birth parents in the experimental group rated their overall relationships with foster parents higher than did control group parents; there was little difference between the groups in MM and OTT.

Table 3-19.  Involvement of parents with foster parents (percents, Ns in parentheses)

Summary.  One would expect to find that the experimental group received more services than the comparison group and that these services would be more intensive. A liberal reading of the above findings might conclude that the experimental group received slightly more and more intensive services. But the evidence in that direction is not strong, and there are a number of instances in which the comparison group received more service.