It was through the site coordinator activities that many aspects of the study integrity were controlled. This was accomplished in a variety of ways. The site coordinators served as the points of contact between the home office and agency liaisons. They monitored performance by the participating agencies, alerted the home office to problems, and became actively involved in resolving problems as they arose.
The site coordinator (SC) was responsible for tracking down needed information to complete interviews (e.g., addresses, caseworker names). Additionally, the SC monitored the status of individual cases to report changes in service end dates, or to identify and seek explanations for cases in which the assignment to regular or experimental services appeared to have been violated. These included cases that should have been but were not referred to random assignment, cases that were randomly assigned but did not get referred to the appropriate service provider, and cases that were not eligible for the study, but were receiving family preservation services. This was accomplished by comparing results of random assignment to agency logs on a monthly basis. State and local personnel were provided monthly reports delineating the cases assigned, their status, and problem areas.
The site coordinator also had a weekly meeting with the public agency screeners and private agency liaisons to review concerns and problems. By keeping in touch with caseworkers and persons in critical positions to the project, the SC was able to gather information about changes in policies, procedures, and staff so that necessary changes could be made. In Kentucky, Tennessee, and Philadelphia there was one site coordinator for one site, while in New Jersey, two site coordinators traveled across seven counties.
|Net study cases||162||155||13||19||175||174|
|Cases with administrative data||160||155||13||17||173||172|
|Note: Administrative data on one KY case in the experimental group contained only opening and closing data on an adult family member. No data on placements or reports of maltreatment were available for this case.|
|Net study cases||19||39||20||47||29||41||23||25||45||62||20||29||11||32||167||275|
|Cases with administrative data||19||39||20||47||29||41||23||25||44||62||20||29||11||32||167||275|
|Net study cases||49||98||Net study cases||144||209|
|Cases with administrative data||48||96||Cases with administrative data||144||205|
10. The Family Preservation and Family Support Implementation Study was selecting sites at the same time. It was decided that conducting both studies in the same site would be too burdensome for states: therefore, Alabama, Arizona, Texas, and Los Angeles, California were eliminated as candidates for the second round of site visits.
11. The study was also conducted in Lexington for a limited period of time.
12. Kentucky state social services have since been reorganized. DSS merged with the Department for Social Insurance to become the Department for Community Based Services.
13. In both Kentucky and New Jersey, two families were randomly assigned twice. The second of these assignments was considered an inappropriate referral and was dropped from this count.
14. One control group case in Kentucky was classified as "minimal service" because the family moved to another state shortly after the referral.
15. Staff indicated there were no contacts for 25 cases in Kentucky (18E and 7C); 14 cases in New Jersey (5E and 9C), 7 cases in Tennessee (6C and 1E.), and 143 cases in Philadelphia (58C and 85E).
16. Transferring cases was not a problem for experimental cases as they went directly to a family preservation worker.
17. One case in the experimental group was a turnback where the second interview was conducted with the public agency worker and the first interview was conducted with the FPS worker 20 days after the second interview had already been conducted. For this case and four others where there were less than 10 days between the two caseworker interviews, computed scores measuring the change between initial and post-treatment interviews were dropped from the caseworker data.
18. The difference in times to interview for the caretaker post-treatment interviews was nearly significant: experimental group, 51.2 days vs. 53.7 days for the control group, p = 0.056.
19. Two cases in the experimental group were closed by the time the worker was contacted for the initial interview, so both caseworker interviews were conducted on the same day. In all, 9 sets of initial and post-treatment interviews (3 caretaker and 6 caseworker) were conducted with less than 10 days between completion dates. For these cases, computed scores measuring the change between initial and post-treatment were dropped from the caseworker data.
20. For all cases in Kentucky, we calculated the length of time between the last activity recorded in the administrative data before referral to family preservation services and the date of referral to family preservation services. For each of these 20 cases, there was no recorded activity within 3 years prior to the referral date. It appears that for these cases, recent administrative data were not obtained from the DSS system.
21. In the course of the evaluation, Kentucky changed administrative data systems, which resulted in some difficulties in the retrieval of administrative data.