Shortly after random assignment began in November 1996, study staff observed that some county screeners misinterpreted criteria for the evaluation and made inappropriate referrals. This was a particular problem for counties with multiple screeners (Essex and Passaic), as well as during periods when screeners were on vacation and substitute screeners were used. Sometimes, screeners would make a referral and subsequently additional information became available indicating the case was an inappropriate referral. Often these cases were identified by evaluation staff upon review of the DYFS referral materials or in conversation with screeners. These could occur either in the treatment or control group.
Many of the inappropriate referrals were reunification cases, not eligible for the study. Since our criteria excluded cases after placement of seven days, often screeners were not informed by caseworkers that the children were already out of home for an extended period prior to the referral. DYFS issued a memorandum to DYFS screeners clarifying that reunification cases where the children at risk were in care less than seven days should be included in the study. This appeared to help reduce inappropriate referrals in some counties. There was some concern that reunification as a reason for an inappropriate referral was being used to game the system by removing a case from the study that went "control." However, overall, 28 cases were identified as inappropriate referrals, 15 in the control group, and 13 in the experimental group.