Evaluation of Family Preservation and Reunification Programs: Final Report - Volume One. 2.3 Random Assignment and Case Enrollment Status


Random Assignment. Individual referral and random assignment procedures were developed for each site. These procedures built upon existing agency referral procedures to family preservation. In both Kentucky and New Jersey, the screener made random assignment referrals. Random assignment began in May 1996 in Kentucky and in November 1996 in New Jersey and Tennessee. In Kentucky and New Jersey random assignment ended in February 1998, and in Tennessee, random assignment ended in May 1998. In Philadelphia, cases were entered into the study from March 1997 to June 1999.

Cases were referred to the screener, who, depending upon the site, either determined if the case was appropriate for family preservation or merely made sure that space was available. The screener then called Westat for assignment of the case. The Westat assignment clerk asked for some basic information about the case. In most instances random assignment was done while the screener stayed on the telephone. The screener then mailed or faxed the family preservation referral form to provide more details about the case. This form was used to fill in the study's random assignment form. (see Appendix C).

Westat personnel used a computer program to randomly assign the case to either the experimental or control group. For those cases randomly assigned to the experimental group, the

Table 2-1
Assignment of Cases by County
  Jefferson Fayette Total KY
Randomly assigned 165 158 13 22 178 180
Inappropriate referrals 3 3 -- 3 3 6
Net study cases 162 155 13 19 175 174
New Jersey
  Camden Burlington Ocean Monmouth Essex Bergen Passaic Total NJ
Randomly assigned 20 40 23 51 29 42 24 27 49 66 24 29 13 33 182 288
Inappropriate referrals 1 1 3 4 - 1 1 2 4 4 4 -- 2 1 15 13
Net Study cases 19 39 20 47 29 41 23 25 45 62 20 29 11 32 167 275
Tennessee--Shelby County Philadelphia
  C E   C E
Randomly assigned 52 101 Randomly assigned 149 213
Inappropriate referrals 3 3 Inappropriate referrals 5 4
Net study cases 49 98 Net study cases 144 209

family received family preservation services. For those cases assigned to the control group, the family received other services provided by the agency.

Case Enrollment and Status. Table 2-1 shows the number of cases enrolled by county. A 50/50 experimental/control assignment was planned in Kentucky, and a 60/40 experimental/control assignment in New Jersey and Philadelphia. Tennessee began with a 60/40 experimental/control assignment which changed to 70/30 about six months into the study. The actual proportions assigned to each group fell within the expected range.

Some eligible cases were not referred for random assignment and did not get into the study but did receive family preservation services. Exceptions were granted only with the approval of state officials who reviewed the case and determined whether to bypass the study. The state was asked to report exceptions, but sometimes these cases were only detected during review of agency logs and screener telephone calls. Over the course of the study, there were 5 exceptions in Kentucky, 33 exceptions in New Jersey, and none in Tennessee or Philadelphia.

In Kentucky a total of 358 cases were randomly assigned by the Department of Social Services (DSS), (12) 323 in Jefferson County (Louisville) and 35 in Fayette County (Lexington). (13) Of these, 9 were determined to be inappropriate referrals and were excluded from the analyses (6 in the experimental group and 3 in the control group). The 9 inappropriate referrals included 3 reunification cases, 4 cases in which the children identified as at risk were out of the home, and 1 case where the custodial parent was incarcerated (in one case the reason for inappropriate referral was not identified). After removing the 9 inappropriate referrals, there were 174 net study cases in the experimental group and 175 net study cases in the control group.

The New Jersey evaluation involved programs in seven counties. A total of 470 cases were randomly assigned from the Department of Youth and Family Services, 288 in the experimental group and 182 in the control group. Of the 470 cases that were randomly assigned, 28 cases were determined to be inappropriate referrals (13 in the experimental group and 15 in the control group). Seventeen of these inappropriate referral cases were reunification cases. The remaining inappropriate referrals included foster care cases, cases with no child at risk in the home, or cases that had previously received family preservation services and were being re-referred for a "booster" session. After removing the 28 inappropriate referrals, there were 275 net study cases in the experimental group and 167 net study cases in the control group. The numbers of cases in each county in New Jersey are too small to allow for separate analyses of data by county, so we combine them in all analyses of this report.

The Tennessee evaluation in Shelby County included 153 cases randomly assigned by the Division of Children's Services (DCS). Of these, six were determined to be inappropriate referrals and were excluded from the analyses (3 in each of the groups). The inappropriate referrals were due to no children under the age of 13 in the home (one case), three reunification cases, and one case with children in foster care. The sixth inappropriate referral was screened out by DCS. After removing the six inappropriate referrals, there were 49 net study cases in the control group and 98 net study cases in the experimental group.

In Philadelphia, 362 cases were randomly assigned, nine of which were inappropriate referrals, five from the control group and four from the experimental group. The nine inappropriate referrals include reunification cases, cases in which the children identified as at risk were out of the home, one case that was already receiving services, and cases from units that were not participating in the study. After removing the inappropriate referrals, there were 144 net cases in the control group and 209 net cases in the experimental group for a total of 353 net study cases.

The basic analysis of differences between experimental and control groups concerned those cases labeled "Net Study Cases." However, in a few cases the group assignment was violated, that is, the group to which a family was assigned was switched. Although cases that were deemed to require family preservation should have been designated as exceptions, we allowed each state 6 "approved violations," that is, the state central office could switch the groups following random assignment, upon application from the local office. Despite the allowance of 6 violations, 9 Kentucky cases were switched from the control to the experimental group, 8 of these switches were approved and 1 additional violation was unapproved. New Jersey had 24 violations, 19 approved and 5 unapproved, 14 percent of the net study cases assigned to the control group. In Tennessee, three cases were switched from the control to the experimental group and in Philadelphia there were five switches. There were no recorded switches from the experimental group to the control group in any of the states.

Some cases in the experimental group were provided minimal services because of refusal by the family to participate, failure of the family to comply with initial expectations of the program, or because the provider agency turned the case back. Turnbacks occurred when family preservation services workers were unable to contact the family or the family did not meet the criteria for service (in a few such cases, children were not considered to be at risk). There were 53 minimal service cases in Kentucky, 5 noncompliance, 18 refusals, and 31 turnbacks. (14) In New Jersey, 44 cases assigned to the experimental group received minimal services because of refusal (14 cases), noncompliance by the caretaker (7 cases), or because the case was turned back by the family preservation agency (23 cases). Tennessee had 11 minimal service cases because of refusal (4 cases), the DCS worker never followed through (1 case), the family preservation agency turned back the case due to safety issues (3 cases), and children placed in foster care (3 cases). Seventeen of the 52 minimal service cases in Kentucky had at least one caseworker contact. One case had more than 5 contacts. In New Jersey, of the 44 minimal service cases, on 31 (70%) we had at least one contact. Seven of the 31 families had more than 5

contacts. In Philadelphia, there were 67 minimal service cases in the experimental group and 4 in the control group. The distribution of violations and minimal service cases is shown in Table 2-2.

View full report


"report1.pdf" (pdf, 770.39Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®