Evaluation of Family Preservation and Reunification Programs: Interim Report. 2.3 Random Assignment and Case Enrollment Status

01/08/2001

Random assignment. Individual referral and random assignment procedures were developed for each site. These procedures built upon existing agency referral procedures to family preservation. In both Kentucky and New Jersey, the screener made random assignment referrals. Random assignment began in May 1996 in Kentucky and in November 1996 in New Jersey and Tennessee. In Kentucky and New Jersey random assignment ended in February 1998, and in Tennessee, random assignment ended in May 1998.

Cases were referred to the screener, who, depending upon the site, either determined if the case was appropriate for family preservation or merely made sure that space was available. The screener then called Westat for assignment of the case. The Westat assignment clerk asked for some basic information about the case. In most instances random assignment was done while the screener stayed on the telephone. The screener then mailed or faxed the family preservation referral form to provide more details about the case. This form was used to fill in the study’s random assignment form. (see Appendix C).

Westat personnel used a computer program to randomly assign the case to either the experimental or control group. For those cases randomly assigned to the experimental group, the family received family preservation services. For those cases assigned to the control group, the family received other services provided by the agency.

Case enrollment and status. Table 2-1 shows the number of cases enrolled in New Jersey, Kentucky, and Tennessee by county. A 50/50 experimental/control assignment was planned in Kentucky, and a 60-40 experimental/control assignment in New Jersey, Tennessee began with a 60/40 experimental/control assignment which changed to 70/30 about six months into the study. The actual proportions assigned to each group fell within the expected range.

Some eligible cases were not referred for random assignment and did not get into the study but did receive family preservation services. Exceptions were granted only with the approval of state officials who reviewed the case and determined whether to bypass the study. The state was asked to report exceptions, but sometimes these cases were only detected during review of agency logs and screener telephone calls. Over the course of the study, there were 5 exceptions in Kentucky, 33 exceptions in New Jersey, and none in Tenessee.

In Kentucky a total of 358 cases were randomly assigned by the Department of Social Services (DSS),3 323 in Jefferson County (Louisville) and 35 in Fayette County (Lexington)4. Of these, 9 were determined to be inappropriate referrals and were excluded from the analyses (6 in the experimental group and 3 in the control group). The 9 inappropriate referrals included 3 reunification cases, 4 cases in which the children identified as at risk were out of the home, and 1 case where the custodial parent was incarcerated (in one case the reason for inappropriate referral was not identified). After removing the 9 inappropriate referrals, there were 174 net study cases in the experimental group and 175 net study cases in the control group.

Table 2-1.
Assignment of Cases by County
Kentucky
  Jefferson Fayette Total KY
C E C E C E
Randomly Assigned 165 158 13 22 178 180
Inappropriate Referrals 3 3 - 3 3 6
Net Study Cases 162 155 13 19 175 174

 

New Jersey

  Camden Burlington Ocean Monmouth Essex Bergen Passaic Total NJ
C E C E C E C E C E C E C E C E
Randomly Assigned 20 40 23 51 29 42 24 27 49 66 24 29 13 33 182 288
Inappropriate Referrals 1 1 3 4 - 1 1 2 4 4 4 -- 2 1 15 13
Net Study Cases 19 39 20 47 29 41 23 25 45 62 20 29 11 32 167 275

 

Tennessee

  Shelby
C E
Randomly Assigned 52 101
Inappropriate Referrals 3 3
Net Study Cases 49 98

Note: C = Control Group, E = Experimental Group

The New Jersey evaluation involved programs in seven counties. A total of 470 cases were randomly assigned from the Department of Youth and Family Services (DYFS), 288 in the experimental group and 182 in the control group. Of the 470 cases that were randomly assigned, 28 cases were determined to be inappropriate referrals (13 in the experimental group and 15 in the control group). Seventeen of these inappropriate referral cases were reunification cases. The remaining inappropriate referrals included foster care cases, cases with no child at risk in the home, or cases that had previously received family preservation services and were being re-referred for a “booster” session. After removing the 28 inappropriate referrals, there were 275 net study cases in the experimental group and 167 net study cases in the control group. The numbers of cases in each county in New Jersey are too small to allow for separate analyses of data by county, so we combine them in all analyses of this report.

The Tennessee evaluation in Shelby County included 153 cases randomly assigned by the Division of Children’s Services. Of these, six were determined to be inappropriate referrals and were excluded from the analyses (3 in each of the groups). The inappropriate referrals were due to no children under the age of 13 in the home (1 case), 3 reunification cases, and one case with children in foster care. The sixth inappropriate referral was screened out by DCS. After removing the six inappropriate referrals, there were 49 net study cases in the control group and 98 net study cases in the experimental group.

The basic analysis of differences between experimental and control groups concerned those cases labeled “Net Study Cases.” However, in a few cases the group assignment was violated, that is, the group to which a family was assigned was switched. Although cases that were deemed to absolutely require family preservation should have been designated as exceptions, we allowed each state 6 “approved violations,” that is, the state central office could switch the groups following random assignment, upon application from the local office. Despite the allowance of six violations, 9 Kentucky cases were switched from the control to the experimental group, 8 of these switches were approved and 1 additional violation was unapproved. New Jersey had 24 violations, 19 approved and 5 unapproved, 14 percent of the net study cases assigned to the control group. In Tennessee 3 cases were switched from the control to the experimental group. There were no recorded switches from the experimental group to the control group in any of the three states.

Some cases in the experimental group were provided minimal services because of refusal by the family to participate, failure of the family to comply with initial expectations of the program, or because the provider agency turned the case back. Turnbacks occurred when family preservation services workers were unable to contact the family or the family did not meet the criteria for service (in a few such cases, children were not considered to be at risk). There were 52 minimal service cases in Kentucky, 4 noncompliance, 18 refusals, and 30 turnbacks. In New Jersey, 44 cases assigned to the experimental group received minimal services because of refusal (14 cases), noncompliance by the caretaker (7 cases), or because the case was turned back by the family preservation agency (23 cases). Tennessee had eleven minimal service cases because of refusal (4 cases), the DCS worker never followed through (1 case), the family preservation agency turned back the case due to safety issues (3 cases), and children placed in foster care (3 cases). Seventeen of the 52 minimal service cases in Kentucky had at least one caseworker contact. One case had more than 5 contacts. In New Jersey, of the 44 minimal service cases, on 31 (70%) we had at least one contact. Seven of the 31 families had more than 5 contacts.

The distribution of violations and minimal service cases is shown in Table 2-2. In the analyses of Chapters 6, 7, and 8 we conducted analyses in which we dropped the violations and minimal service cases from the sample (the “secondary” analyses).

Table 2-2. 
Violations and Minimal Service Cases by County
Kentucky
  Jefferson Fayette Total KY
C E C E C E
Net Study Cases 162 155 13 19 175 174
Violations 9 - - - 9 --
Minimal Service - 47 - 5 - 52

 

New Jersey

  Camden Burlington Ocean Monmouth Essex Bergen Passaic Total NJ
C E C E C E C E C E C E C E C E
Net Study Cases 19 39 20 47 29 41 23 25 45 62 20 29 11 32 167 275
Violations 1 - - - 6 - 3 - 6 - 6 - 2 -- 24 --
Minimal Service - 6 - 8 - 5 - 1 - 13 - 3 -- 8 -- 44

 

Tennessee

  Shelby
C E
Net Study Cases 49 98
Violations 3 --
Minimal Service 1 10

Note: C = Control Group, E = Experimental Group


Kentucky state social services have since been reorganized. DSS merged with the Department for Social Insurance to become the Department for Community Based Services.

4 In both Kentucky and New Jersey, two families were randomly assigned twice. The second of these assignments was considered an inappropriate referral and was dropped from this count.