Preparation and training for the experiment were conducted in the winter and early spring of 1996. Training sessions were held with both DCBS screeners and FPP program coordinators. During the one-day training sessions study procedures were reviewed including use of study forms, the screening protocol, random assignment procedures, and the role of the study site coordinator.
In addition, initial group meetings were conducted with caseworkers and supervisors from each unit. Workers were very resistant to random assignment and concerned that it would deny services to families, cause extra tasks, and delays in referrals for their caseloads.
A site coordinator was hired locally and provided by the study to assist DCBS and FPP staff with case information needed for random assignment and the conduct of interviews.
Referral to FPP. Prior to implementing study procedures, workers identified families they felt were appropriate to receive family preservation services, got supervisor approval for the referral, and then made the referral to the public agency screener through a referral form. The screener was then responsible for determining whether or not the referral was appropriate and contacting the family preservation agency to see if there were any openings. If there were time periods when referrals were low, the screener was also responsible for working with caseworkers to identify appropriate new referrals. The plans for implementing the evaluation built upon these procedures. As almost equal numbers of referrals came from the intake and ongoing units, it was decided to maintain this practice. Procedures for the two types of units only varied in the definition of an eligible case. Procedures established to refer cases were:
- 1. When a worker decided to refer a case for family preservation services, he or she determined the family's willingness to participate in the program;
- 2. The worker discussed the referral with his or her supervisor;
- 3. The worker called the screener to see if there was an opening;
- 4. If there was an opening, the worker referred the case and the DCBS screener determined eligibility for family preservation;
- 5. Eligibility was determined using Kentucky's current review procedures. The screener was also asked to complete a screening protocol that contained a scoring procedure for determining risk. If the screener's decision differed from the recommendation indicated by the score, the circumstances were described of the case that supported the decision that was made (see section on targeting);
- 6. Once the case was determined eligible, the screener notified Westat that an eligible case was ready for assignment. A computerized program was used to randomly assign a case as experimental or control;
- 7. The screener notified the worker that the family had been accepted into the experimental group or assigned to the regular service group;
- 8. If the case was assigned to the experimental group, it was referred for family preservation services.
It was expected that cases referred from intake were cases with recent abuse/neglect reports. Cases referred from ongoing were defined as: the family unit was maintained at home without family preservation service and a new situation emerged which indicated that without family preservation services, the children would be placed in out-of-home care.
Targeting. As discussed earlier, DCBS administrators were concerned that FPP was not always targeted at families in which placement for a child was imminent. The study implemented two new procedures to help improve targeting and identify those cases that were at "imminent risk of placement." The first procedure was to identify potentially "high risk" eligible family preservation cases that were not being referred for services. In Louisville, there was a Court Liaison who reviewed every case in which a petition to the court was being filed because there was concern for the child's safety or the case was being referred for foster care placement. Efforts were made to include these cases in the study as more "severe" cases that might not otherwise be referred for family preservation services. For each case in which a petition was filed, we asked that a Worker Safety Checklist be completed by the worker (see Appendix E). For all recently investigated cases in which a petition was being filed, the workers were asked to complete a checklist which covered such issues as: whether or not they were considering foster care, whether the location of the primary caretaker was known, whether the caretaker refused to care for the children, whether the caretaker was chemically dependent without a plan for treatment, whether there was potential for recurring risk of sexual abuse, whether the caretaker was willing to work with an agency, and whether the caretaker was the perpetrator of harm to the child. These questions were developed in conjunction with family preservation and public agency staff to identify potential foster care cases, yet screen out those cases that would not be eligible for family preservation.
The study Site Coordinator reviewed the checklists using established criteria to ascertain whether or not the case should be sent to the screener for family preservation eligibility determination. These criteria excluded cases where workers indicated they were not considering foster care placement, the caretaker could not be located, the caretaker refused to care for the children, the caretaker was chemically dependent without a current treatment plan, or there was potential for recurring risk of sexual abuse. To help ensure that previously referred cases were not re-referred for family preservation, the procedures also included a question about previous referrals.
Throughout the 22 months of data collection, the Site Coordinator reviewed 2103 petitions. Of those petitions, 177 (8 percent) were identified as potentially eligible for family preservation. Of those referred to the screener, 42 percent (74) were randomly assigned. For 51 percent (53) of the cases not enrolled, the reason was that there was no space available in family preservation. Insufficient information accounted for another 23 cases (Table 3-6).
|Number of petitions reviewed||2103|
|Total petitions sent to FP screener||177|
|- Cases enrolled in experiment||74|
|- Cases not enrolled||103|
|Reasons cases not reviewed|
|- No space||53|
|- Insufficient information||23|
|- Worker plans to enroll||11|
The second procedure to help tighten targeting had the screener use a protocol to review cases referred to her. Screening protocols were developed by the study team to aid the screener. The protocol provided the screener the opportunity to review his or her decision by using a risk index based on factual items such as previous substantiated complaints, more than one maltreated child in the family, previous foster care placements, and the presence of substance abuse. The instrument yielded a score, the midrange values of which were thought to suggest referral to family preservation. Guidelines provided to the screener stated that cases receiving a score greater than 2 and less than 5 fell within reasonable risk, and should be referred. Cases with a score of less than 2 might not be considered at risk. Cases with a score greater than 5 might have too high risk. Although cases outside the 2-5 range could be referred, the screener was asked to provide the reason she believed the case should receive family preservation services. The screener was asked to explain why she was still referring cases that fell below or above the midrange. The screener was told that the tool was not to be used to determine referral, but as a review of her decision. As indicated earlier, copy of the protocol is presented in Appendix B.
Although there were many cases referred to the screener that were not sent to family preservation, screening tools were only completed on those cases referred to family preservation. Table 3-7 presents a breakdown of item responses for each of the screening questions. A total of 327 protocols were completed, approximately 91 percent of the 358 cases randomly assigned. The majority (77%) of the screener's scores for the cases referred to family preservation fell in the midrange between 3 and 5, with an average score of 4.2. The screener's comments about why she referred cases with scores below 3 and above 5 focused on the above 5 scores. She did not provide comments when a case with a score below 3 was referred. However, for the cases with scores above 5, the comments indicated that family preservation was necessary to prevent placement and procedures were in place to ensure safety.
The screening protocol depicted the majority of children (85%) having previous abuse and neglect allegations, with 85 percent of the allegations within the last six months. About one-third of the families had a child previously placed in substitute care. The reader is reminded that these findings are based on the screener's knowledge of the case at the time of referral to FPP.
Throughout the 22 months of data collection, approximately 683 cases were referred to the Louisville screener for family preservation. Of these cases, 323 were randomly assigned to FPP or the control group. A monthly breakdown of the number of cases referred to the screener and then referred for randomization for Louisville is provided in Table 3-8. Only DCBS referrals to family preservation were considered for random assignment. Excluded from the study were cases referred by non-DCBS sources, and family reunification cases.
The number of cases referred for random assignment stayed constant for the first year of the study. A slight decrease in referrals was experienced in the second year. There was not an immediate rise in referrals as the study ended. Referrals in March and April 1998 maintained at 28 and 29 per month respectively. Conversations with the screener indicated that it became more difficult to continue to get workers to refer cases for family preservation, as they experienced having more of their cases go control.