Evaluation of Family Preservation and Reunification Programs: Interim Report. 4.3.4 Evaluation of Family Preservation and Reunification Services


Executive staff in New Jersey expressed early interest in participation in this evaluation to obtain a thorough assessment of their family preservation services. FPS services in New Jersey had been operational for almost ten years. They were recently expanded to all counties. The emphasis, while originally focused on adolescents with family problems, was undergoing a shift to maltreatment cases involving young children.

In addition to such things as maturity of the program and the use of the Homebuilders model, New Jersey also met the study's criterion that there was not saturation of FPS services. To avoid the ethical concern of denying services to families, sites were considered where service demand exceeded the number of slots available. Ten counties were identified as possible sites. They were Warren, Ocean, Bergen, Cape May, Monmouth, Salem, Cumberland, Essex, Hudson and Middlesex. DYFS administrators decided on the final sites to be included. They wanted a balance of northern and southern counties as well as urban and suburban ones. In addition, the DYFS random assignment study had been conducted in four counties. DYFS wanted to limit the research burden on these counties. DYFS selected seven counties that agreed to participate: Bergen, Burlington, Camden, Essex, Monmouth, Ocean and Passaic Counties. As seen in Figure 4-1, this resulted in a cluster of three counties in the Northern part of the state and four in central New Jersey. A target of 500 cases was set for New Jersey to allow for adequate subsample analysis.

DYFS requested a 60-40 split of cases in experimental and control groups. Having a better than 50% chance of obtaining family preservation services was thought to encourage caseworkers to make referrals.

Figure 4.1. New Jersey counties participating as evaluation sites (shown in color)

Figure 4.1. New Jersey counties participating as evaluation sites (shown in color)

The procedures for targeting and screening were determined with Central Office DYFS staff, then individualized with counties to fit their service delivery procedures. DYFS administrators, while interested in participating in the evaluation, had concern about the random assignment. They wanted to work out all ethical and procedural concerns before allowing the evaluators to talk with county staff. As a result, state FPS administrators did not include county DYFS staff or FPS administrators in discussions with evaluators until workplan, procedures, and protocols were completed. This delayed and possibly lost some of the "buy-in" by the local administrators and workers.

The screening protocol, developed for the evaluation and discussed in Chapter 2, was offered to counties to assist with targeting cases for family preservation. It was asked that the tool be completed for all cases considered for FPS, as well as children being referred for placement into foster care.9 For counties that had a formal pre-placement conference, the screening protocol would be completed during that meeting. For other cases, a referring worker would complete a screening tool with her or his supervisor prior to submitting a referral to the screener. All counties, except Passaic County, agreed to use the screening protocols for cases referred to FPS and randomly assigned to the evaluation. For placement cases, DYFS staff from all counties felt they could not commit to using the protocol, because it would be considered a paperwork burden to staff.

We received screening protocols on 56 percent of the 442 net study cases.10 In addition, workers completed protocols on 15 cases that were not referred for random assignment. Our intent had been for workers to use the screening tool for all cases considered for family preservation services. However, this did not occur. Of the screening protocols for cases that were randomly assigned, 60 percent (147) were experimental and 40 percent (99) were control. Table 4-6 presents a breakdown of item responses for each of the screening questions.

The screening protocol asked nine questions to establish a risk score. The worker and his or her supervisor were to complete the form at the time the case was reviewed for referral to FPS. The purpose of the form was to have workers reassess certain conditions of the case to make sure it was appropriate for family preservation services. The form was not intended to replace worker judgement, but to give them an opportunity to review their decisions about the appropriateness of the case for FPS.

Table 4-6.
Item Response For Screening Protocol

Screening Protocol Question

Cases Randomly Assigned (%) (N=245) Cases Not Randomly Assigned (%) (N=15)
1. Number of previous substantiated abuse and neglect reports:
None 29 15
One 43 54
Two 24 23
Unknown 4 8
Total % 100% 100%
2. Substantiated report of abuse and neglect within the last six months:
No 59 59
Yes 32 33
Unknown 8 8
Total % 100% 100%
3. Has a child been previously removed and placed in substitute care because of maltreatment?
No 65 50
Yes 27 42
Unknown 9 8
Total % 100% 100%
4. Has a perpetrator currently living in the family made threats of physical harm to the family in the last two weeks?
No 71 69
Yes 18 23
Unknown 11 8
Total % 100% 100%
5. Perpetrator in family ever convicted of a crime against a person:
No 68 92
Yes 4 --
Unknown 26 8
Total 100% 100%
6. Perpetrator in family abuses drugs:
No 60 69
Yes 19 8
Unknown 21 23
Total 100% 100%
7. At least one of the victims 3 years old or less:
No 76 77
Yes 23 23
Unknown 1 --
Total % 100% 100%
8. Single-female-headed household:
No 49 47
Yes 45 53
Unknown 6 --
Total % 100% 100%
9. Any income from employment:
No 59 47
Yes 33 53
Unknown 8 --
Total% 100% 100%
10. Total Score    
0 8 7
1 12 13
2 20 13
3 22 27
4 24 20
5 9 13
6 3 --
7 1 7
8 1 --
Total % 100% 100%
Average 2.9 3.1

Guidelines provided to the workers said that cases receiving a score greater than 2 and less than 5 fell within reasonable risk, and should be referred. Cases with a score of less than 2 might not be considered at risk and cases with a score greater than 5 might have too high risk. Although workers could refer cases outside the 2-5 range, they were asked to provide the reason they believed the case should receive family preservation services. Examples of reasons that were offered for scores below 2 are acting–out teenagers and teenagers with suicidal tendencies. The majority of cases received a score between 2 and 5 (75 percent). Only 5 percent of the cases had a score greater than 5 and 20 percent had a score less than 2. In New Jersey, caseworkers indicated they did not believe the risk scale sufficiently addressed the problems of teenagers, and therefore there were cases that did not score as high as they should have.

The workers completing the screening protocols depicted the majority of the children having previous abuse and neglect allegations (67 percent), but less than one-third of them within the last six months. Workers reported that 23 percent of cases had a child age three or less. The reader is reminded that these findings are based on screening protocol data completed by workers at the time of referral to family preservation. Overall scores on experimental and control cases were similar and are not presented here.

The second column of the table provides a breakdown of the responses to the screening protocol for the cases not submitted for random assignment. As there are so few of these cases, comparisons with randomly assigned cases are problematic. The average score for the two groups is similar, 3.1 for the non-study cases, and 2.9 for the study cases.

Workers mainly completed screening forms only for families that they actually referred.

10 Workers in Passaic County did not complete screening protocols for the study. Excluding Passaic County from the total, we received protocols for 246 or 62% of the 399 net study cases.