Evaluation of Family Preservation and Reunification Programs: Final Report: Volume 2: Philadelphia

4. Philadelphia

[Main Page of Report | Contents of Report]

Contents

• 4.1 Introduction
• 4.2 The Philadelphia Families
  • 4.2.1 Family Problems
  • 4.2.2 Child Problems
  • 4.2.3 Caretaker Abuse or Neglect as a Child
  • 4.2.4 Previous Allegations and Placement
  • 4.2.5 Social Program Participation
• 4.3 Summary of Sample Description
• 4.4 Services
  • 4.4.1 Services During the Treatment Period
  • 4.4.2 Summary of Services During the Treatment Period
  • 4.4.3 Services During the Followup Period
  • 4.4.4 Summary of Followup Services.
• 4.5 Outcomes
  • 4.5.1 Substitute Care Placement Following Random Assignment
  • 4.5.2 Allegations of Maltreatment Following Random Assignment
  • 4.5.3 Family and Child Functioning - Caretaker Interviews
  • 4.5.4 Information from Caseworkers on Functioning
  • 4.5.5 Summary of Outcome Data

Endnotes

4.1 Introduction

Philadelphia was selected for the evaluation because it employed an alternative, somewhat less intensive, longer term approach, and therefore provides some contrast with the Homebuilders sites. In addition, the programs in Philadelphia that were examined were thought to focus on families with substance abuse problems which was not the case in the other sites.

As in the other states, the design for the evaluation in Philadelphia was an experiment in which families were randomly assigned to either the family preservation program (the experimental group) or to other, "regular," services of the child welfare system (the control group). In Philadelphia, both the experimental group and control group received services from private agencies under contract with the public child welfare agency. The public agency has a specialized family preservation unit that develops selection criteria, approves families to receive family preservation services and works closely with the private providers. All other in-home services, known as SCOH (Services to Children in their Own Homes), are delivered by private providers and monitored by the Department's caseworkers. During the evaluation period, experimental cases received family preservation services and control cases received SCOH services. Family preservation is a three-month program that requires workers to spend at least 5 to 10 hours per week with the family.

A description of data collection and sample size in Philadelphia is found in Volume 1, Chapter 6 of this report.

[Go To Contents]

4.2 The Philadelphia Families

Descriptive information about the Philadelphia families was gathered from the initial interviews with caretakers (n = 263) and is summarized in Table 4-1. Because families were randomly assigned, we would expect the families in the experimental and control groups to be similar at the time of random assignment, and for that reason, the sample is described as a whole. Differences between the two groups were examined and there were no characteristics on which the groups differed to a statistically significant degree.

The respondents were primarily women (95%). Most (91%) of the respondents were birth mothers, 5 percent were biological fathers, just under 3 percent were grandparents, and

Table 4-1
Description of the Philadelphia Families at Time of Initial Interviews

	N	%
Gender of caretaker/respondent	263
Male		5.3
Female		94.7
Race of caretaker/respondent	263
African American (not Hispanic)		80.0
Caucasian (not Hispanic)		15.0
Hispanic		2.3
Other		1.9
Respondent's education level	263
Elementary school or less		3.8
Some high school		61.0
High school graduate or obtained GED		18.7
College		11.0
Special education or vocational schooling		4.2
Respondent's marital status	263
Married		9.5
Divorced		6.8
Separated		11.0
Widowed		3.0
Never married		69.0
Respondent's relationship to youngest child	263
Birth mother		90.5
Biological father		4.6
Grandparent		2.7
Other relative		2.3
Household composition	263
Birth mother, no other adults		49.8
Birth mother & 1 male adult		19.8
Birth mother & extended family*		18.6
Biological father*		4.6
Other relative caretaker*		4.6
Other**		2.7
	N	Mean
Age of respondent	260	31.78
Age of youngest child	263	3.36
Age of oldest child	263	9.83
Number of kids	263	3.40
Number of adults	263	1.60
* These categories may also include other non-related adults in the home **Includes: nonrelative caretaker, adoptive or step-parent, birth mother & non-related females, or birth mother, and more than one non-related male

just over 2 percent were other relatives. The racial composition of the respondents was mostly African American (not Hispanic), at 80 percent, with 15 percent Caucasian (not Hispanic), and the remainder Hispanic and other. The average age of the respondents was 32 (s.d. = 9.11). Just under 4 percent of the respondents had less than a high school level education, 61 percent had some high school, 19 percent had graduated from high school or obtained a GED, 11 percent had at least some college education, and 4 percent had special education or vocational schooling. Approximately 10 percent of the respondents indicated they were married, 7 percent divorced, 11 percent separated, 3 percent widowed, and 69 percent never married. Twenty-two percent reported that they were living with a spouse or partner. At the time of the first interview, 17 percent of the respondents indicated they were employed, 43 percent were unemployed and looking for work, and 40 percent were unemployed and not looking for work. Overall, 65 percent of the respondents rented their homes.

On average, these families were comprised of 1.6 adults and 3.4 children for an overall average family size of 5.0 persons. The average age of the youngest child in the family was 3.45 years (s.d. = 3.75), and the average age of the oldest child in the family was 9.8 years (s.d. = 4.47). Respondents were also asked to provide information regarding the relationship of other adults in the home relative to the youngest child in the home. This information was then used to determine household composition for these families. Approximately one half of the households were headed by a single birth mother, 20 percent had a birth mother residing with one male adult, 19 percent had a birth mother and extended family, 5 percent were headed by a biological father, and 5 percent were headed by another relative caretaker.

4.2.1 Family Problems

Problems and strengths identified by Philadelphia caretakers are summarized in Table 4-2. Most (96%) respondents felt they were "doing a pretty good job raising [their] kids." Still, data from the time one interviews provides us with some sense of the difficulties these families faced as caretakers were asked whether or not they had experienced certain problems in the last month. With regard to emotional problems, 62 percent of the respondents reported feeling "blue or depressed," 53 percent reported feeling nervous or tense, 52 percent were overwhelmed by work or family responsibilities, 33 percent said they had just wanted to give up at some point in the last month, and 35 percent felt they had few or no friends.

With regard to financial difficulties, 56 percent responded that in the past month they did not feel they had enough money for food, rent, or clothing. In response to more specific

Table 4-2
Philadelphia Caretaker Problems and Strengths, Caretaker Initial Interview(% responding yes)

Problems	Control	Experimental	p
Felt blue or depressed	58	65
Felt nervous or tense	50	54
Just wanted to give up	35	32
Overwhelmed with work or family responsibility	48	55
Felt you had few or no friends	35	35
Not enough money for food, rent, or clothing	60	54
Gotten in trouble with the law	0	3
Had too much to drink in a week	7	3
Used drugs several times a week	6	10
Economic Items
Had difficulty paying rent	25	25
Had difficulty paying electric/heat	38	35
Had difficulty buying enough food	18	21
Had difficulty buying clothes	40	40
Positive Items
Have you felt happy	69	77
Gotten together with anyone to have fun/relax	0	3
Doing a pretty good job raising kids	95	97

questions about difficulties paying bills in the past 3 months, 25 percent reported difficulty paying rent, 36 percent reported difficulty paying electric or heating bills, 20 percent difficulty buying food for the family, and 40 percent difficulty buying clothes for their children.

Although the Philadelphia program was intended to be focused on substance abuse, only five percent of respondents acknowledged having too much to drink several times a week, and 8 percent reported using drugs several times a week. When caretakers were asked whether a child or children they care for went through alcohol or drug withdrawal when born, 8 percent responded affirmatively. ⁽⁵⁷⁾ Less than 2 percent of respondents indicated they had gotten in trouble with the law in the past month.

4.2.2 Child Problems

Table 4-3 shows caretakers' responses to questions about problems of children in the families. About three-fifths of respondents reported that their child "gets upset easily," and almost three-fourths reported that at least one child throws tantrums. Approximately one-fifth of the caretakers reported school problems for a child in the family; 19 percent had been absent a lot, 25 percent had failed a class, and 22 percent had been temporarily suspended from school. Approximately one-third of the caretakers reported aggressive behavior by the child; 31 percent indicated a child in their family "fights a lot with other kids" and 33 percent reported that the child is aggressive toward the caretaker.

4.2.3 Caretaker Abuse or Neglect as a Child

When asked two separate questions about whether they had been abused or neglected as a child, 32 percent of the respondents reported having been abused and 23 percent neglected. Eighteen percent responded affirmatively to both questions, and overall, 37 percent of the caretakers reported having either been abused, neglected, or both as a child. Thirteen percent of caretakers had been in a foster home or institution. Experimental and control groups did not differ significantly with respect to these previous experiences. ⁽⁵⁸⁾

4.2.4 Previous Allegations and Placement

Historical reports of maltreatment and historical records of placement in substitute care were contained within the administrative data files. Three hundred and thirty-one (95%) of the families had been investigated for maltreatment prior to random assignment. Two hundred and eighty-three (81%) of the families had experienced at least one substantiated allegation prior to random assignment. ⁽⁵⁹⁾ The administrative files reported four types of allegations; physical abuse, neglect, sexual maltreatment, and other. The allegation just prior to random assignment was of primary interest. This particular allegation provides some indication of reason for referral to

Table 4-3
Concerns and Problems Regarding Children, Philadelphia Caretaker
Initial Interview (% responding yes regarding any child that the respondent cares for)

	N	%
Asked about all children…
Child went through alcohol withdrawal at birth	262	3
Child went through drug withdrawal when born	262	8
Child doesn't show much interest in what is going on	259	17
Child is smaller/lighter than other children	262	21
Child get(s) upset easily	259	59
Asked for children over 3 months old…
Is/Are funny and makes you laugh	251	99
Like(s) to share things with others	248	90
Throw(s) tantrums	251	70
Is/are shy and withdrawn	251	36
Is/are outgoing and friendly	252	97
Is/are good looking	252	100
Fight(s) a lot with other kids	247	31
Has/have language problems	246	18
Asked for children over 4 years old…
Is/are very aggressive toward you	217	33
Has/have a special talent in music	217	51
Like(s) animals	217	95
Is/are good at sports	216	79
Usually does the right thing	217	86
Hangs with friends you don't like	215	25
In the past 3 months, has any child you care for…
Gone to church regularly	216	42
Been absent from school a lot	207	19
Run away from home overnight	199	5
Been temporarily suspended from school	205	22
Been expelled from school	205	4
Taken care of younger children	204	36
Took something that didn't belong	216	24
Absent from school/no good reason	205	9
Received special education at school	206	25
Failed any classes	205	25
Received counseling	205	28
Asked for any child over age 7…
In the last 3 months, has any child been arrested	174	7
Asked only for children over age 10…
Has child age 11 or older had alcohol problems	115	0
Has child age 11 or older had a drug problem	114	4
Has any girl age 12 to 18 been pregnant	57	4
Has any boy age 14 to 18 fathered a child	19	0

family preservation. The distribution of last substantiated allegation prior to random assignment is as follows: 29 percent physical abuse, 66 percent neglect, 2 percent sexual maltreatment, and 3 percent other.

Regarding substitute care placement, 131 children in 63 (18%) families had experienced placement prior to random assignment. For these cases, on average, 42.5 months elapsed between the last day of care and random assignment. In the placement spell just prior to random assignment the average length of time in substitute care was 6.7 months. ⁽⁶⁰⁾

4.2.5 Social Program Participation

In the initial interview, respondents were asked whether they or anyone else in the household had participated in various social programs within the past 3 months. The overall rates of participation are provided in Table 4-4. Approximately four-fifths indicated that they received food stamps, just over two-thirds received AFDC, slightly less than half received WIC, about one quarter received social security disability, and less than one-tenth received a housing voucher. Overall, respondents indicated that they participated in an average of 2.3 of the 5 income support programs listed (s.d. = 1.2) and 90 percent of the sample participated in at least one of the five programs. There were no significant differences between experimental and control groups in the rate of participation in income support programs. Reports of participation in alcoholism, drug treatment, marriage counseling, and job training programs were less than 10 percent for each. Slightly less than a third of the sample reported participation in Head Start or another pre-school program.

[Go To Contents]

4.3 Summary of Sample Description

Most of the respondents to the first interview were women and birth mothers of the youngest child in the home. Eighty percent of the respondents were African-American (not Hispanic) and 15 percent were Caucasian. Approximately two-thirds of the respondents had not graduated from high school, slightly more than two-thirds were never married, and over four-fifths were unemployed. About half of the households were headed by a single birth mother, and

Table 4-4
Philadelphia Participation in Social Programs Prior to Initial Interview

Program	Percent
Food stamps	80.2
Job training	23.3
WIC	46.2
AFDC	70.1
Housing vouchers	7.3
Social security disability	25.2
Alcoholism program	5.7
Drug treatment program	14.5
Marriage counseling	0.8
Community mental health program	11.5
Head Start/pre-school	36.9

the average age of the respondents was just under 32 years. On average there were 1.6 adults and 3.4 children in the home, with the average age of the youngest child 3.4 years and the average age of the oldest child 9.8 years.

Over half of the respondents answered affirmatively to each of three questions about emotional difficulties: "feeling blue or depressed," "feeling nervous or tense," and "feeling overwhelmed with work or family responsibility." More than half of the respondents also indicated that they did not have enough money for food, rent, or clothing. Eleven percent said they had problems with alcohol or drugs, and just over one-third reported that they had been abused, neglected, or both as a child.

Ninety percent of the respondents indicated that they participated in at least one of 5 income support programs: AFDC, food stamps, WIC, Social Security disability, and housing vouchers. The rate of participation was less than 10 percent for alcoholism treatment, drug treatment, marriage counseling, and job training. About one-third of the respondents indicated participation in Head Start or another pre-school program.

Ninety-five percent of the families in the study had an investigation prior to their referral for family preservation services and 80 percent had at least one substantiated allegation. Seventeen percent of the families experienced placement of at least one child prior to the referral for family preservation services. With respect to sample characteristics, there were no significant differences between the experimental and control groups.

[Go To Contents]

4.4 Services

4.4.1 Services During the Treatment Period

In the second interview with caretakers and caseworkers, we asked questions about services offered and received during the period since random assignment. Experimental and control group responses to these questions were compared. This analysis documents the services received by both groups (thereby beginning to get into the "black box" of services) and determines whether the experimental group in fact did receive more services and more intense services than the control group.

Caseworker Activities. Caretakers were asked to indicate whether the caseworker provided help with a number of specific problems. Table 4-5 shows the number of affirmative responses in each group. According to caretakers, the most common activities in which workers engaged were discussing discipline, providing transportation, and telling caretakers about other agencies that offer services. Of the 19 items on which caretakers were questioned, the control group workers reportedly engaged in one activity, advising on substance abuse, more than the experimental group workers (26% vs. 18%), however, the difference was not statistically significant (p = .16). For 8 of the 19 items, experimental group workers reportedly engaged in the activity significantly more often than control group workers (all at p = .05 or less). ⁽⁶¹⁾ The activities engaged in more often by experimental group workers include: help with money for other things, providing transportation, talking with caretaker about discipline, advising how to get medical care, discussing how to get a better place, advising on job training programs, talking about how to get a paying job, and advising on how to continue school.

Table 4-5
Philadelphia Caretaker Reports of Caseworker Activities, Post-treatment Interview

	Control %	Experimental %	p
Caseworker helped with money for rent/elect./phone	3	4
Caseworker helped with money for other things	5	22	.001
Caseworker provided transportation	35	50	.03
Caseworker discussed proper feeding of child	22	28
Caseworker talked with you about discipline	32	53	.002
Caseworker talked with you on relations with spouse	13	20
Caseworker helped you clean house	6	7
Caseworker helped with painting/house repairs	01	4
Caseworker discussed how to get childcare	21	32	.08
Caseworker helped with welfare/food Stamps	4	9
Caseworker advised how to get medical care	10	23	.02
Caseworker talked with you how to handle anger	31	37
Caseworker advised you on substance abuse	26	18
Caseworker discussed with you how to get a better place	25	38	.05
Caseworker advised on job training programs	23	36	.04
Caseworker talked about how to get a paying job	19	33	.02
Caseworker advised on how to continue school	21	34	.03
Caseworker arranged for some childcare	5	8
Caseworker told you about other agencies	39	47

A total count of the number of these 19 caseworker activities reported by caretakers also shows significant differences between the experimental and control groups. Caretakers in the experimental group reported an average of 4.6 caseworker activities (n = 148, s.d. = 3.8) while caretakers in the control group reported an average of 2.9 caseworker activities (n = 113, s.d. = 3.1) (p = .001). ⁽⁶²⁾ When asked which of the caseworker activities were especially helpful, experimental group caretakers judged significantly more activities to be helpful than did control group caretakers (2.2 vs. 1.5, p = .02). ⁽⁶³⁾

Social Program Participation. In the second interview, caretakers were again asked about their participation in the set of social programs listed in Table 4-6, except this time they were asked to report their participation since the time of the first interview. The proportions of involvement were remarkably similar to those in the first interview with a change of 2 percent or

Table 4-6
Philadelphia Participation in Social Programs, Post-treatment Interview

Program	C %	E %	p
Food stamps	78	80
Job training	20	26
WIC	40	44
AFDC	65	70
Housing vouchers	10	5
Social security disability	29	23
Alcoholism program	8	7
Drug treatment program	13	16
Marriage counseling	2	2
Community mental health program	13	13
Head Start/pre-school	29	32
Note: C = Control Group, E = Experimental Group

less for most programs. Exceptions to this were a 4.1 percent decrease in participation in WIC services, and a 6.1 percent decrease for participation in Head Start or Pre-school programs, perhaps due to the aging of youngest children. There were no significant differences between experimental and control groups for participation in any of the social programs at the time of the second interview. No significant differences were found with respect to the total number of income support programs or treatment programs in which respondents participated since the time of the first interview.

Caretakers' Reports of Services. In the second interviews, caretakers were asked if they had received any of a set of specific services in the time since the first interview. As shown in Table 4-7, there were no significant differences between the experimental and control groups in receipt of any of these services in the primary analysis. ⁽⁶⁴⁾ In a separate question, caretakers were asked whether the agency provided homemaker services or the assistance of a parent aide.

Table 4-7
Philadelphia Caretaker Report of Services, Post-treatment Interview

	Control %	Experimental %	p
Daycare	11	15
Help in finding a place to live	9	9
Staying at an emergency shelter	4	3
Medical or dental care	33	39
Transportation	25	39
Education services/GED	9	11
Parent education/training classes	16	37
Legal services	7	10
Counseling	21	26
Respite care	3	3
Homemaker services	1	1
A parent aide to help you	1	3

Fewer than 3 percent of all caretakers reported having a homemaker or receiving assistance from a parent aide, with no significant differences reported between the experimental and control groups (again, see Table 4-7 for details). When caretakers were asked whether they did not receive any services they felt were needed, 21 percent of the control group responded affirmatively and 18 percent of the experimental group responded affirmatively, a difference that was not statistically significant. ⁽⁶⁵⁾

Relationship with Caseworker. Table 4-8 shows results from a number of questions in which caretakers were asked about their relationships with caseworkers. Reports of the quality of the relationship were positive overall, with more than half of the respondents from both the experimental and control group indicating that the caseworker listened to their concerns, understood their situation, and agreed on the goals most of the time. A greater proportion of

Table 4-8
Philadelphia Caretakers' Reports on Relationship with Caseworker, Post-treatment Intervie

	Control %	Experimental %	p
Worker listened to your concerns most of the time	81	80
Worker understood your situation very well	82	82
You and worker agreed on goals most of the time	67	71	.10
Did worker sometimes talk with you about issues that were not easy to talk about?	27	36
Caseworker helped you to see your good qualities	68	82	.01
Caseworker helped you to see your problems	74	76
Did you see your caseworker
More often than you wanted	21	25
As often as you wanted	59	57
Not often enough	20	18

experimental group caretakers felt their workers helped them see their good qualities (82% vs. 68%, p = .01). With respect to the frequency of contact with the workers, approximately 20 percent of caretakers from both the experimental and control groups indicated they did not see their caseworkers often enough. A slightly greater proportion of caretakers in the experimental group indicated they saw their workers "more often than [they] wanted" (25% vs. 21%) and a slightly greater proportion of caretakers in the control group indicated they saw their workers "as often as [they] wanted" (59% vs. 57%). ⁽⁶⁶⁾

Caseworkers' Reports of Services. In the second interview, caseworkers were asked whether they had helped any member of the family with any of 25 services, such as child care, homemaker services, income programs, treatment programs of various sorts, and health care. Table 4-9 provides a list of these 25 items and the proportion of caseworkers who indicated this service was provided. Caseworkers from the experimental group reported helping their clients with an average of 4.9 of these services (s.d. = 3.5), while caseworkers from the control group reported helping their clients with an average of 3.8 of these services (s.d. = 3.0; p = .0004). ⁽⁶⁷⁾ In the primary analyses, an examination of individual services reveals 6 services that were provided significantly more often to the experimental group than to the control group (significance levels were all at p = .05 or less). These services include: childcare or babysitting, parent training, other housing assistance, emergency financial assistance, recreational services, and household management. No services were provided significantly more often to the control group. ⁽⁶⁸⁾

Table 4-9
Philadelphia Caseworkers' Report of Services Provided to Family, Post-treatment Interview

	Control %	Experimental %	p
Childcare or baby sitting	6	16	.02
AFDC or other public income (except SSI)	6	12
SSI for adult or child	3	0	.06 (FE)
Food stamps	6	9
Drug treatment	17	16
Alcoholism treatment	8	8
Legal aid	4	9
Help with education	19	24
Respite care	6	6
Parent training	34	62	.001
Health care	28	35
Inpatient mental health	2	2
Outpatient mental health/counseling	20	24
Health assessment	23	31
Housing financial assistance	3	8	.08
Other housing services	10	21	.02
W.I.C.	8	6
Emergency financial assistance other than housing	23	54	.001
Job training	7	11
Emergency shelter	1	2
Recreational services	26	38	.05
Family planning	23	21
Self help groups	9	16
Household management	21	38	.004
Homemaker services	6	9
Other	16	13
N	99	151
Note: "FE" indicates significance determined by Fisher's exact test

Contact Data. One page contact reports were to be completed by all caseworkers for each face-to-face contact with a family member during the time period designated for family preservation services. On these forms, workers provided information about the date, time, persons involved, and the services delivered during each contact. Some data on contact forms are presented in Table 4-10. At least one contact form was submitted for 85 (59%) of the control group cases and 124 (59%) of the experimental group cases. The following analyses were limited

Table 4-10
Philadelphia Contact Forms

	C	E	p
Number of cases with at least one form submitted	85	124
Average number of forms per case	9.9	17.6	.01
Average number of home visits	7.8	13.9	.01
Average number of visits with caretakers	8.2	15.4	.01
Average number of visits with the other parent	1.2	1.5
Average number of visits with children	7.3	13.3	.01
Average number of contacts with service
Concrete Services	C	E	P
Transportation	2.5	5.2	.01
Buying food	1.3	1.0
Child care	2.1	1.2	.06
Clothing, furnishings, and supplies	0.9	1.5	.07
Topics of Discussion
Discipline of children	3.5	4.4
Goals	5.7	7.2	.10
Caretaker's interaction with children	4.4	4.2
Child's anger management	1.8	2.0
Supervision of children	4.1	4.3
Money management	3.4	5.1	.05
Employment	2.7	4.6	.01
Conflict among adults	1.9	2.7	.10
Note: C = Control Group, E = Experimental Group The contact forms contained additional concrete services and topics of discussion (see Appendix K, Volume 3). Only those that were most often reported are shown here. Entries are average numbers of times per family that an item was reported, for those families with at least one form submitted.

to those families with contact reports. On average, more contact forms were submitted for the experimental group than for the control group (17.6 vs. 9.9; p < .001). In addition to the overall number of reports submitted, the experimental group received significantly more home visits (13.9 vs. 7.8; p < .001), visits with caretakers (15.4 vs. 8.3; p < .001), and visits with children (13.3 vs. 7.3; p < .001). As experimental group families received significantly more contacts than the control group families, they also received significantly more individual activities. The most common concrete service was the provision of transportation (an average of 5.2 times for the experimental group vs. 2.5 times for the control group; p < .001). Child care was the second most common concrete service (an average of 1.2 times for the experimental group vs. 2.1 times for the control group; p < .1). Additional concrete services included clothing, furnishings, and supplies (1.5 times for the experimental group vs. 0.95 for the control group; p < .1).

Contact forms also captured general information about the topic of discussion, counseling, or instruction. The most common topics of discussion were the goals of working together (7.2 times in the experimental group vs. 5.7 times in the control group; p < .1), money management (5.1 vs. 3.4; p < .05), employment (4.6 vs. 2.7; p < .01), and conflict among adults (2.7 vs. 1.9; p < .1).

Additional data on experimental group contacts are shown in Table 4-11. These data confirm the fact that services to the experimental group often did not begin until sometime after random assignment. Only 8 percent of the cases had an in-home contact within 7 days and a relatively small proportion of contacts occurred in the first month.

Table 4-11
Philadelphia Experimental Group Contacts

	N	%
Number of families with contact data	124	60
Total number of contact forms submitted	2182
Contacts in week 1	18	1
Contacts in month 1	479	16
Contacts in month 2	912	30
Contacts in month 3	825	27
In-home contact within 72 hours	3	2
In-home contact with 7 days	10	8
Concrete service within 7 days	8	7
Hours of contact	N	Mean
Average hours of contact overall	122	34.1
Average hours contact in month 1	89	8.8

4.4.2 Summary of Services During the Treatment Period

The caretaker interview, the caseworker interview, and the contacts data all confirmed the expectation that the experimental group would receive more and more intensive services than the control group. At time two, caretakers in the experimental group reported an average of 4.6 caseworker activities as compared to 2.9 for the control group. In addition to caseworker activities, caretakers were asked about specific services received. The following services were among those most often reported by caretakers: counseling, transportation, parent education or training, and medical or dental services. Differences between the experimental and control groups for the caretaker interview data include a significantly greater proportion of experimental group caretakers responding affirmatively to nine questions regarding caseworker activities. These nine activities and the response rates are provided in Table 4-12.

Consistent with the information provided by caretakers, caseworkers reported providing more services to families in the experimental group (an average of 4.9 services) than those in the control group (an average of 3.8 services, p = .001). For 6 of the specific services listed, a greater proportion of caseworkers in the experimental group reported providing services as compared to caseworkers in the control group (see Table 4-12).

The contact form data were consistent with both the caretaker and caseworker interview data in supporting the conclusion that the experimental group families received more services than did the control group families. An average of 17.6 contact forms were received for the experimental group as compared to an average of 9.9 contact forms for the control group. The contact forms also indicate that the experimental group received home visits, visits with the caretaker, and visits with the children significantly more often than did the control group. In response to questions pertaining to the nature of the relationship with the caseworker, caretakers from the experimental group were significantly more likely to indicate their workers helped them to see their good qualities.

4.4.3 Services During the Followup Period

When caretakers were interviewed a year after random assignment, they were asked some of the same questions about services received, this time since the last interview (since the end of family preservation services for the experimental group and during a comparable period for the control group). Tables 4-13, 4-14, and 4-15 show analyses of these questions.

Table 4-12
Summary of Services for Philadelphia, Post-treatment Interview

Caseworker Activities:
Proportion of affirmative answers by caretakers to yes/no questions	Control %	Experimental %	p
Caseworker helped with money for other things	5	22	.001
Caseworker provided transportation	35	50	.03
Caseworker talked with you about discipline	32	53	.002
Caseworker advised how to get medical care	10	23	.02
Caseworker discussed with you how to get a better place	25	38	.05
Caseworker advised on job training programs	23	36	.04
Caseworker talked about how to get paying job	19	33	.02
Caseworker advised on how to continue school	21	34	.03
Caseworker helped you see good qualities	68	82	.01
Caseworker talked about how to get paying job	19	33	.02

	Control Mean	Experimental Mean	p
Caretaker report of number of caseworker activities	2.9	4.6	.0001
Caretaker report of number of "helpful" caseworker activities	1.5	2.2	.02

Services Provided:
Proportion of affirmative answers by caretakers to yes/no questions	Control %	Experimental %	p
Childcare or baby sitting	6	16	.02
Parent training	34	62	.001
Other housing services	10	21	.02
Emergency financial assistance	23	54	.001
Recreational services	26	38	.05
Household management	21	38	.004

	Control Mean	Experimental Mean	p
Caseworker report of number of services provided	3.4	4.9	.0004
Note: This table only includes items with a primary p-value less than or equal to .05 Items in bold indicate significant findings in favor of the experimental group.

Table 4-13
Philadelphia Caretaker Reports of Caseworker Activities, Followup Interview

	Control %	Experimental %	p
Caseworker helped with money for rent/electricity/phone	4	4
Caseworker helped with money for other things	18	16
Caseworker provided transportation	29	32
Caseworker discussed proper feeding of child	16	13
Caseworker talked with you about discipline	32	24
Caseworker talked with you on relations with spouse	16	8	.08
Caseworker helped you clean house	3	6
Caseworker helped with painting/house repairs	2	1
Caseworker discussed how to get child care	14	17
Caseworker helped with welfare/food Stamps	9	10
Caseworker advised how to get medical care	10	14
Caseworker talked with you how to handle anger	28	19	.10
Caseworker advised you on substance abuse	18	19
Caseworker discussed with you how to get a better place	24	18
Caseworker advised on job training programs	22	21
Caseworker talked about how to get a paying job	23	16
Caseworker advised on how to continue school	29	18	.05
Caseworker arranged for some child care	2	6
Caseworker told you about other agencies	33	29

Table 4-14
Philadelphia Participation in Social Programs, Followup Interview

Program	C %	E %	p
Food stamps	79	75
Job training	21	31	.09
WIC	38	40
AFDC	64	68
Housing vouchers	12	10
Social security disability	33	22	.07
Alcoholism program	10	04
Drug treatment program	17	13
Marriage counseling	1	2
Community mental health program	9	14
Head Start/pre-school	57	52
Note: C = Control Group, E = Experimental Group

Table 4-15
Philadelphia Caretaker Report of Services, Followup Interview

	Control %	Experimental %	p
Day care	17	25
Help in finding a place to live	11	10
Staying at an emergency shelter	4	6
Medical or dental care	36	38
Transportation	23	28
Education services/GED	9	16
Parent education/ training classes	17	29	.03
Legal services	7	10
Counseling	23	29
Respite care	1	1
Homemaker services	0	1
A parent aide to help you	2	3

Caseworker Activities. Caretaker reports of caseworker activities since the post-treatment interview are shown in Table 4-13. Only one item showed significant differences between experimental and control groups in the primary analysis. Compared to caretakers in the experimental group, a significantly greater proportion of caretakers in the control group reported that their caseworkers advised them on how to continue school (29% vs. 18%; p = .05). ⁽⁶⁹⁾

Participation in Social Programs. As indicated in Table 4-14, there were no significant differences between the experimental and control groups with respect to involvement in social programs during the post-treatment period. ⁽⁷⁰⁾

Caretaker Report of Services. Table 4-15 indicates that there was only one service in which there is a significant difference between experimental and control groups in reported receipt of services during the post-treatment period. A greater proportion of experimental group respondents reported receiving parent education/training classes (29% vs. 17%; p = .03). ⁽⁷¹⁾

4.4.4 Summary of Followup Services.

There were few significant differences between experimental and control groups on report of service as shown in Table 4-16. A significantly greater proportion of caretakers in the control group reported that, in the period of time since the post-treatment interview, their caseworker advised them on how to continue school. On the other hand, a significantly greater proportion of caretakers from the experimental group reported receiving parent education/training classes since the time of the post-treatment interview.

Table 4-16
Summary of Services in Philadelphia, Followup Interview

Caseworker Activities: (Proportion of affirmative answers to yes/no questions)	Control %	Experimental %	p
Caseworker advised on how to continue school	29	18	.05

Services Provided: (Proportion of affirmative answers to yes/no questions)	Control %	Experimental %	p
Parent education/training classes	17	29	.03
Note: Table only includes items with a primary p-value of .05 or less. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group.

[Go To Contents]

4.5 Outcomes

The following outcome analyses compare the experimental and control groups. As previously discussed, there were a few cases in which the original random assignments were violated, that is control group families were given FPS services or minimal services were provided. There were also cases which are thought to have not received services before the post-treatment interview due to a delay in the assignment of a worker to that case. Therefore, analyses were conducted for the primary analysis group (the original random assignment group), the secondary analysis group (dropping the violations and minimal services cases from the original random assignment group), and, for interview data, the tertiary analysis group (dropping additional cases which appear to have not had a worker assigned in time for the post-treatment interview to determine effects of service). ⁽⁷²⁾ Secondary and tertiary analyses are reported in footnotes.

4.5.1 Substitute Care Placement Following Random Assignment

Family preservation is believed to prevent unnecessary placement in substitute care. Prevention of placement is not as central an objective of family preservation in Philadelphia as in other locations, but it is, nonetheless, an outcome of interest. Table 4-17 provides data at the individual level for type of first placement after random assignment. ⁽⁷³⁾ The administrative files contained subsequent placement data on 349 families, 205 in the experimental group and 144 families in the control group. Although the data were at the child level, the analyses are presented at the family level. ⁽⁷⁴⁾

Table 4-17
Type of First Placement After Random Assignment, Child Level

Philadelphia
Type	N	Percentage
Foster care, institution	98	42.1
Foster care, home	92	39.5
Emergency shelter	29	12.4
Foster care, group	14	6.0
Total	233	100

In the experimental group, 148 children in 65 families (32%) experienced placement subsequent to random assignment. In the control group, 85 children in 37 families (26%) experienced placement subsequent to random assignment. The differences were not statistically significant at the family level.

As in the other states, a simple comparison of overall percentages is not the most appropriate way to analyze these placement data. As families were randomly assigned at various points in time (between March 12, 1997 and June 23, 1999), the risk periods (amount of time eligible to experience placement) varied between families. The administrative data were collected on October 31, 2000 and therefore the minimum risk period was approximately sixteen months, and the maximum was more than 44 months. Hence, survival analyses were conducted to account for the varying risk periods.

Child level data were aggregated to the family level for the following survival analyses. The family level survival analyses were developed based on all 1,212 children in the administrative data. Families survived if no child experienced subsequent placement. For those families with subsequent placement, the first placement date of any child in that particular family was used to calculate the time interval between random assignment and first subsequent placement. If multiple children were removed from a single home, the date of first placement was selected.

The family level analysis of subsequent placement is displayed in Figure 4-1. The survival curves and Wilcoxon statistic indicate that the survival rates are not statistically different between the experimental and control groups. At the one year interval, 18 percent of experimental group families and 15 percent of control group families experienced substitute care placement in the primary analysis. ⁽⁷⁵⁾ The survival analyses suggest that there were no differences between the rates of placement in the experimental and control groups. A summary of placement rates at various points in time following random assignment is shown in Table 4-18.

In addition to survival analyses, placement can be examined in terms of the proportion of time in substitute care subsequent to random assignment. The proportion is calculated by dividing the number of days in care by the number of days of possible care (number of days between random assignment and the date of administrative data collection). As the proportions are calculated at the family level, the number of days in care represents the total number of care days summed across all children within a particular family. Similarly, the number of possible care days represents the total number of possible care days summed across all children within a

Figure 4-1
First Placement after Random Assignment (Families)

Table 4-18
Summary of Philadelphia Placement Data, Survival Analyses Families Experiencing Placement of at Least One Child Within Specified Periods of Time

	1 month		6 months		12 months		18 months
	C %	E %	C %	E %	C %	E %	C %	E %
Primary analysis	1	1	12	10	15	18	20	24
Secondary analysis	1	1	13	9	16	15	19	21
Note: C = Control Group, E = Experimental Group

particular family. The number of possible care days is adjusted for a child's eighteenth birthday. In the experimental group, children spent an average of 6 percent of the days subsequent to random assignment in care. In the control group, children spent an average of 4 percent of the days subsequent to random assignment in care. This difference is not statistically significant.

4.5.2 Allegations of Maltreatment Following Random Assignment

Two hundred sixty-eight children in 110 families (54%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared with 161 children in 69 families (50%) in the control group. The difference was not statistically significant at the family level. The distribution of the various types of allegations is as follows: 43 percent physical abuse, 53 percent neglect, and 4 percent sexual maltreatment. One hundred twenty-three children in 60 families (29%) in the experimental group were the subjects of substantiated allegations of maltreatment compared with 67 children in 32 families (22%) in the control group. This difference was not statistically significant at the family level. The distribution of substantiated allegations is as follows: 53 percent physical abuse, 43 percent neglect, and 4 percent sexual maltreatment.

As with the analyses of subsequent placement, survival graphs were developed to compare the timing of subsequent substantiated allegations of maltreatment. ⁽⁷⁶⁾ Again, survival analyses were conducted for both the primary and secondary analysis groups. Child level data were aggregated at the family level. There were no significant differences between the experimental and control groups. Figure 4-2 displays the survival curves for the primary analysis group. At one year, 20 percent of the experimental group and 13 percent of the control group families experienced substantiated reports of maltreatment subsequent to random assignment. At two years, 25 percent of the experimental group and 18 percent of the control group families experienced substantiated reports of maltreatment subsequent to random assignment. ⁽⁷⁷⁾ The survival analyses indicate that experimental families did not experience fewer substantiated reports of maltreatment subsequent to random assignment.

Sub-group Analysis. We examined a number of subgroups of cases to determine whether we could detect differences between experimental and control groups on placement and substantiated allegations subsequent to random assignment within each subgroup. The results are shown in Table 4-19. Most of the subgroups were defined in terms of problems existing at the time of the initial interview. For both placement and substantiated allegations the table shows the number of cases in each subgroup, the percentage of cases in the subgroup experiencing the event within 6 months, the significance of the difference between the experimental and control groups in the occurrence of the event within six months, and the p value for the analysis of differences in survival curves. The first row of the table shows the results for the Philadelphia sample as a whole. Except for substance abuse, the definitions of the subgroups were determined from the initial caretaker interview. Very few caretakers acknowledged substance use in the first interview, so that subgroup was determined from information in both the caretaker and caseworker initial interviews.

As can be seen, nearly all of the experimental-control group comparisons shown are not significant. Of the 18 comparisons in the table, only one is significant at .05, that for

Figure 4-2
First Substantiated Allegation after Random Assignment (Families)

Table 4-19
Philadelphia Subgroup AnalysesSignificance Levels of Differences Between
Experimental and Control Groups

		Placement			Substantiated allegations
		In 6 mos			In 6 mos
Subgroup	N	Overall %	pa	Survival p	Overall %	pa	Survival p
Overall	349	9.5			10.0		.098
Substance abuse	72	11.1			11.1
No substance abuse	186	9.1	.076		8.6
Problems with bills	151	11.9			10.6
Problems with daycare	143	9.8			10.5
Depressionc	139	12.9			8.6		.089
Problems with punishment	163	11.0			9.8
Problems with school	98	13.3			6.1		.038b
Problems with employment	56	10.7			7.1
Single mother	129	11.6			13.2
a Fisher exact, two tail b Experimental group more likely to experience subsequent allegation c Caretakers with depression scores above median for the site

problems with school. Among those who identified a child having problems with school, those in the experimental group were more likely to have a substantiated allegation than those in the control group. In the analysis so far, efforts to find subgroups for which family preservation service was related to reduced placement have been unsuccessful.

4.5.3 Family and Child Functioning - Caretaker Interviews

Life Events. In both the initial and second interviews, caretakers were asked to respond to a 15 item "life events" inventory asking about the occurrence of both positive and negative

events (see Appendix K, Volume 3, Initial Caretaker Interview, p. 7, and Interim Caretaker Interview, p. 8). Three scales were formed from this inventory: positive life events, negative life events, and a scale of those life events that might reflect depression in the caretaker (we had a more formal depression measure as well, described below). In the post-treatment interview, the proportion of positive life events reported by caretakers in the experimental group was significantly higher than the proportion reported by caretakers in the control group (.19 vs. .15; p = .05). ⁽⁷⁸⁾ The proportion of positive life events reported by caretakers in the experimental group remained higher in the followup interview (.23 vs. .20), however, the difference was not statistically significant. On the measures of negative life events and life events reflecting depression there were no statistically significant differences between the experimental and control groups at the time of the post-treatment or followup interviews. ⁽⁷⁹⁾

Problems. In the post-treatment and followup interviews, caretakers were again asked questions about problems in the family. These questions paralleled those asked in the first interview (see Section 4.2.1 Family Problems above, under Section 4.2 The Philadelphia Families), except this time caretakers were asked to respond to questions with regard to the time "since we last spoke to you." Tables 4-21 and 4-22 display these items and the proportion of affirmative responses at the time of the post-treatment and followup interviews. At the time of the post-treatment interview, there were no significant differences between the experimental and control groups responses to any of the nine items about problems in the family. ⁽⁸⁰⁾ At the time of the followup interview, no significant differences were found on eight of the nine items in the primary analysis. However, on the question about the overall economic condition of the family,

Table 4-20
Philadelphia Family and Child Functioning Scales

		Post-Treatment			Followup			Multivariate Repeated Measures
		N	Ma	pb	N	M	p	N	Means			Multivariate ps			Univariate ps-Time		Univariate ps-Grp-time interaction
									Initial	Post	Follow	Grpc	Timed	Time -Grpe	Initial v. laterf	Post v. Followg	Initial v. later	Post v. Follow
Positive life events	C	113	.15	.05	90	.20		70	.13	.15	.21		.001		.003	.001
	E	148	.19		135	.23		102	.17	.18	.23
Negative life events	C	113	.06		90	.08		70	.13	.11	.14		.02			.03
	E	148	.05		135	.08		102	.10	.09	.12
Life events depression	C	113	.38		89	.35		69	.43	.34	.33	.09	.001		.001
	E	148	.41		135	.41		102	.53	.40	.42
Paying bills	C	113	.27		90	.21	.08	70	.33	.23	.20		.02		.005
	E	148	.31		134	.28		100	.32	.31	.30
Income support	C	113	2.20		90	2.25		70	2.40	2.38	2.43
	E	148	2.22		135	2.14		102	2.24	2.23	2.15
Treatment programs	C	113	.36		90	.37		70	.39	.41	.40
	E	148	.37		135	.32		102	.28	.39	.32
Punishment	C	112	.16		89	.16		69	.20	.15	.15	.06	.002		.001
	E	148	.19		132	.17		98	.25	.20	.17
Child aggression	C	113	1.32		90	1.13		70	1.34	1.30	1.14
	E	148	1.25		135	1.21		102	1.16	1.24	1.24
School problems	C	93	.14		80	.16		56	.15	.14	.17					.07
	E	123	.13		116	.17		79	.18	.14	.19
Child withdrawn	C	113	.52		90	.42		70	.51	.51	.43
	E	148	.61		135	.54		102	.48	.63	.56
Stolen things or arrested	C	113	.20		90	.13	.02	70	.27	.19	.10		.03	.04	.01			.02
	E	148	.16		135	.26		102	.26	.18	.27
Child substance abuse	C	113	.03		90	.01		70	.03	.01	.01
	E	148	.01		135	.02		102	.00	.00	.03
Child problems	C	113	1.59		90	1.68		70	1.61	1.56	1.67
	E	148	1.76		135	1.67		102	1.61	1.84	1.66
Negative child behaviors	C	108	.25		88	.22		65	.26	.25	.23
	E	143	.25		131	.25		96	.25	.25	.25
Positive child behaviors	C	109	.80		89	.79		66	.78	.79	.79
	E	145	.81		134	.80		97	.81	.81	.79
Household condition	C	112	.09		90	.06	.05	69	.12	.08	.06
	E	147	.09		135	.10		101	.10	.10	.10
Depression (SCL-90)	C	113	.96		89	.79		69	1.05	.89	.78		.006		.003
	E	148	1.00		135	.83		102	.98	.95	.88
Positive child care practices	C	108	.90		87	.91		66	.90	.90	.91
	E	142	.88		129	.88		93	.90	.88	.88
Negative child care practices	C	108	.13		86	.13		66	.15	.13	.13			.008			.002
	E	147	.15		130	.15		94	.19	.15	.14
a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three

Table 4-21
Philadelphia Caretaker Problems and Strengths, Caretaker Post-treatment Interview (% responding yes)

	Control		Experimental		p
	N	%	N	%
Problems
Felt blue or depressed	113	45	148	43
Felt nervous or tense	113	43	147	46
Just wanted to give up	113	22	148	29
Overwhelmed with work or family responsibility	113	41	146	47
Felt you had few or no friends	112	22	148	25
Not enough money for food, rent, or clothing	113	48	148	59	.08
Gotten in trouble with the law	113	2	148	3
Had too much to drink in a week	112	4	148	2
Used drugs several times a week	113	8	148	6
Economic Items
Had difficulty paying rent	113	19	148	20
Had difficulty paying electric/heat	113	28	148	33
Had difficulty buying enough food	113	26	148	31
Had difficulty buying clothes	113	34	148	42
Positive Items
Have you felt happy	112	83	148	86
Gotten together with anyone to have fun/relax	113	51	148	53
Doing a pretty good job raising kids	112	93	148	95

Table 4-22
Philadelphia Caretaker Problems & Strengths, Caretaker Followup Interview (% responding yes)

	Control		Experimental		p
	N	%	N	%
Problems
Felt blue or depressed	90	46	135	49
Felt nervous or tense	90	38	135	44
Just wanted to give up	89	18	135	25
Overwhelmed with work or family responsibility	89	38	135	46
Felt you had few or no friends	90	31	135	27
Not enough money for food, rent, or clothing	90	33	135	49	.02
Gotten in trouble with the law	90	0	135	1
Had too much to drink in a week	90	3	135	2
Used drugs several times a week	90	0	135	2
Economic Items
Had difficulty paying rent	90	18	134	20
Had difficulty paying electric/heat	90	29	134	29
Had difficulty buying enough food	90	16	134	27	.05
Had difficulty buying clothes	90	21	134	37	.01
Positive Items
Have you felt happy	89	89	135	90
Gotten together with anyone to have fun/relax	90	57	135	57
Doing a pretty good job raising kids	88	97	134	96

"have you felt you did not have enough money for food, rent, or clothing?" 49 percent of the experimental group and 33 percent of the control group responded affirmatively (p = .02). ⁽⁸¹⁾

In addition to the items about problems, caretakers were asked three questions about positive aspects of their lives: "getting together with anyone to have fun or relax," "felt happy and "felt that considering everything you're doing a pretty good job raising your kids." For the experimental and control groups combined, at post-treatment, 85 percent responded affirmatively to the question of whether they "felt happy," 53 percent responded affirmatively to the question of "getting together with anyone to have fun or relax," and 94 percent responded affirmatively that they were "doing a pretty good job raising [their] kids." At followup, 90 percent of respondents (experimental and control groups combined) reported that they "felt happy," 57 percent responded affirmatively to the question of "getting together with anyone to have fun or relax," and 96 percent reported that they were "doing a pretty good job raising [their] kids." ⁽⁸²⁾

Economic Functioning. In addition to the general item on not having enough money for food or rent, caretakers were asked four specific questions about difficulties in paying for the essentials of living (rent, electricity and heating, food, and clothing). When these items were combined into a scale, all analyses (primary, secondary, and tertiary) revealed no significant differences in the average proportion of affirmative responses to the four items at the time of the post-treatment interview. At the time of the followup interview, the average proportion of affirmative responses to the four items was greater for the experimental group than the control group (.28 vs. .21) but the difference was not statistically significant in the primary analysis (p = .08). ⁽⁸³⁾ Using repeated measures to look at changes in the scale responses over time, results indicate a decline in the average proportion of affirmative responses to this scale of economic functioning for both groups (p = .02). These changes over time did not differ significantly for the experimental and control groups.

Looking at the 4 individual items that comprised the scale of economic functioning, there were no significant differences between experimental and control groups at the time of the post-treatment interview. At the time of the followup interview, primary analyses revealed significant differences between experimental and control group respondents on 2 of the 4 items. A greater proportion of the experimental group respondents reported having difficulty buying enough food (27% vs. 16%; p = .05), and difficulty buying clothes (37% vs. 21%; p = .01). ⁽⁸⁴⁾

Household Condition. Caretakers were asked 10 questions about problematic conditions in the home (e.g., nonfunctioning heating, plumbing, or electrical systems; peeling paint; broken windows or doors). The experimental and control groups did not differ on the average proportions of the presence of such conditions at the time of the post-treatment interview. At the time of the followup interview, the average proportion of problematic conditions present was greater for the experimental group than for the control group (.21 vs. .06; p = .05). Repeated measures analysis revealed no significant changes over time and no significant differences between the two groups averaged over time.

On only one of the specific items regarding problematic conditions in the home were there any differences in the primary analysis of the post-treatment interview. Twenty-three percent of caretakers in the experimental group and 13 percent of the caretakers in the control group reported that "there were not enough basic necessities such as chairs, tables, beds, cribs, mattresses, or not enough basic necessities such as blankets, sheets, pots or dishes" (p = .03). ⁽⁸⁵⁾At the time of the followup interview, primary analysis revealed significant differences between the experimental and control group on one of the 10 specific household condition items. Four percent of the experimental group and none of the control group caretakers reported that the electricity did not work for more than a day at a time since the post-treatment interview (Fisher's exact p-value = .05). ⁽⁸⁶⁾

Child Care Practices. In both the post-treatment and followup interviews, caretakers were asked a series of yes-no questions about child care practices in the last three months (both positive and negative). The results from these questions are shown in Tables 4-23 and 4-24.

Table 4-23
Philadelphia Caretaker Reports of Child Care Practices, Post-treatment Interview

	Control		Experimental		p
	N	%	N	%
Lost temper when child got on nerves	112	53	148	52
Found that hitting child was good	112	4	148	6
Hitting child harder that meant to	112	5	148	11
Out of control when punishing child	111	18	147	24
Have you praised your children	112	95	147	98
Listened to music together w/child	112	95	148	92
Tied child with cord- string-belt	112	0	147	1
Gone to amusement park, pool, picnic	111	85	147	77
Uncomfortable hugging child	105	10	138	13
Encouraged child to read book	108	98	142	99
Have children handled household chores	105	76	140	71
Not let children into the house	105	1	140	1
Punished for not finishing food	107	6	140	3
Blamed child w/ things not their fault	107	21	139	22
Let child to play where not allowed	107	15	139	12
Unable to find someone to watch children	111	42	144	46

Table 4-24
Philadelphia Caretaker Reports of Child Care Practices, Followup Interview

	Control		Experimental		p
	N	%	N	%
Lost temper when child got on nerves	89	52	132	44
Found that hitting child was good	89	6	132	9
Hitting child harder that meant to	89	4	132	7
Out of control when punishing child	89	17	132	26
Have you praised your children	89	99	132	96
Listened to music together w/child	89	93	132	92
Tied child with cord- string-belt	89	0	130	0
Gone to amusement park, pool, picnic	90	79	133	73
Uncomfortable hugging child	87	10	131	11
Encouraged child to read book	87	99	129	98
Have children handled household chores	85	84	128	80
Not let children into the house	84	1	126	2
Punished for not finishing food	86	1	130	7	.05 (FE)
Blamed child w/ things not their fault	86	21	130	24
Let child play where not allowed	86	19	129	16
Unable to find someone to watch children	88	47	133	33	.04
NOTE: "FE" indicates significance determined by Fisher's exact test

Three scales were formed using items that appear in Tables 4-23 and 4-24: positive child care practices (5 items), negative child care practices (10 items), and punishment (5 items, all of which were also in the negative child care practices scale).

At the time of the post-treatment interview, primary analyses revealed no significant differences between experimental and control groups on any of the items. ⁽⁸⁷⁾ At followup, a significantly greater proportion of experimental group respondents responded affirmatively that they "punished [child] for not finishing food" (7% vs. 1%; Fisher's exact p-value = .05). ⁽⁸⁸⁾

There were no significant differences between the experimental and control groups with regard to the positive and negative child care practice scales at the time of either the post-treatment or the followup interview. At each point in time, caretakers from both groups responded affirmatively to over 80 percent of the positive items and less than 15 percent of the negative items. Repeated measures analyses revealed no significant changes over time in the positive child care practices scale. There was a small but significant decrease in the proportion of negative child care practices and in the proportion of affirmative answers to the punishment items for both groups averaged over time (see Table 4-20 and Figure 4-3). For all scales, there were no significant interactions between group and time variables, indicating that the pattern of means over time was similar for both the experimental and control groups.

Caretaker Depression. In all three interviews, we administered the SCL-90 depression scale to measure the level of depression of the caretaker. ⁽⁸⁹⁾ There were no differences between the groups in scores on this scale at the time of the post-treatment or followup interview. Results of the repeated measures analysis indicate significant decreases over time in the depression scores for both groups averaged together (see Table 4-20 and Figure 4-3). The pattern of declining depression scores was the same for both the experimental and control group. ⁽⁹⁰⁾

Child Behavior. We asked 35 questions about specific child behaviors, both positive and negative. Questions were phrased in terms of "any of the children" and some questions were age specific. Responses to these questions were used to form various scales: aggression (3 items), school problems (5 items), positive child behaviors (10 items), and negative child behaviors (21 items, including the aggression and school problems items). Neither the primary nor the secondary analyses revealed any significant differences between the groups in scores on any of these scales at the time of the post-treatment or followup interviews (see Table 4-20 and Figure 4-3). Furthermore, none of the hypotheses tested in the repeated measures analysis resulted in significant effects for any of the levels of analyses (primary, secondary, or tertiary).

Figure 4-3
Child and Family Functioning Over Time (Families)

Specific items on whether the child was withdrawn, or had engaged in substance abuse also did not reveal significant differences between groups at either point in time (post-treatment or followup) or in any of the analysis (primary, secondary, or tertiary). A scale measuring two items asking whether any of the children had stolen things or been arrested did result in significant differences in the primary analysis of the followup interview and of the effects over time. This scale was calculated by summing the "yes" responses to the two items, resulting in scale scores ranging from 0 to 2. At the time of the followup interview, the scale score was significantly higher for the experimental group than for the control group (.26 vs. .13; p = .02). Repeated measures analysis indicated that the pattern of scores over time was significantly different for the experimental and control groups, particularly in the time period between the post-treatment interview and the followup interview. In the control group, the average score for caretakers responding that their child had stolen things or been arrested consistently declined over time, but in the experimental group, the average score declined between the initial and post-treatment interviews and returned to the original level in the followup interview (see Table 4-20). For further interpretation of these results, scale scores of 1 and 2 were collapsed and Chi-square analyses were used to examine the proportion of caretakers from each group responding affirmatively to either item at each point in time.

At the time of the initial interview, 22 percent of caretakers from both the control and experimental groups reported that their child had stolen things and/or been arrested in the last three months. At the time of the post-treatment interview, 17 percent of caretakers from both the control and the experimental groups reported that their child had stolen things and/or been arrested since the time of the initial interview. At the time of the followup interview, a significantly greater proportion of caretakers from the experimental group reported that their child had stolen things and/or been arrested since the time of the post-treatment interview (24% vs. 13%; p = .04).

Overall Assessment of Improvement by Caretakers. In both the post-treatment and followup interviews, caretakers were asked about general changes in their family lives since entering the study (see Tables 4-25 and 4-26). At the time of the post-treatment interview, 27 percent of experimental group caretakers generally thought there was "great improvement" in

Table 4-25
Philadelphia Caretakers' Assessments of Overall Change
Since First Interview, Post-treatment Interview

	Control %	Experimental %
	p = .07
Great improvement	17	27
Some improvement	46	51
Same	27	17
Somewhat or a great deal worse	6	5
Not ascertained	4	1

Table 4-26
Philadelphia Caretakers' Assessments of Overall Change
Since Post-treatment Interview, Followup Interview

	Control %	Experimental %
	p = n.s.
Great improvement	38	36
Some improvement	38	40
Same	17	21
Somewhat or a great deal worse	7	4
Not ascertained	1	0

their lives, compared to 17 percent of control group caretakers ( p = .07). ⁽⁹¹⁾ When response categories were collapsed to reflect "some or great improvement," things are "just the same," or "somewhat or a great deal worse," a significantly greater proportion of experimental group caretakers reported "some or great improvement" (77% vs. 63%; p = .05). ⁽⁹²⁾

At the time of the followup interview, slightly more than a third of respondents reported "great improvement" and three quarters of respondents reported "some or great improvement," with no significant differences between the experimental and control groups in any of the levels of analysis (primary, secondary, or tertiary).

4.5.4 Information from Caseworkers on Functioning

In interpreting caseworker reports, it should be noted that while both SCOH and family preservation services were provided by private agency workers, it is likely that there are differences between these groups of caseworkers in the knowledge they have of the cases. It is expected that family preservation workers had much more intensive involvement with the families they worked with. Hence, interpretations of comparisons between responses of workers serving each of the groups must be made with caution.

Given the low response rate on caseworker initial interviews (35 percent for the control group and 54 percent for the experimental group), analysis of initial caseworker interview items was not conducted. This lack of data from the caseworker initial interview makes it difficult to interpret differences between the experimental and control groups at the time of the post-treatment interview. It is unknown whether those differences existed at the time of random assignment or whether they reflect differential changes during the treatment period.

Caretaker Functioning. Caseworkers were asked nine questions tapping various aspects of caretaker functioning on a five-point scale from 0 for not adequate to 4 for very adequate. Table 4-27 provides a list of these nine questions. At the time of the post-treatment interview, there were no significant differences between experimental and control group caseworkers on any of these nine items or on the scale averaging the nine items. ⁽⁹³⁾

Table 4-27
Philadelphia Caseworkers' Assessments of Caretakers' Parental Functioning,
Post-treatment Interview

	Control		Experimental		p
	N	Mean	N	Mean
Caretaker ability to provide food	89	2.90	145	3.05
Caretaker ability giving affection	92	2.95	145	3.05
Caretaker respect for child's opinions	83	2.59	131	2.80
Respond patiently to child's questions	86	2.50	138	2.64
Respond to child's emotional needs	89	2.56	144	2.69
Provide learning opportunities	90	2.30	143	2.57	.07
Setting firm/consistent limits/rules	89	2.19	140	2.45	.10
Adequate supervisor/responsible childcare	92	2.54	148	2.80	.10
Attending to children's health needs	89	2.90	144	3.03
Caretaker functioning, 9 items, average of nonmissing items, higher=better	88	2.63	139	2.79
Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate

Household Condition. As in the caretaker interview, we asked caseworkers about conditions in the home. Caseworkers were asked 13 yes-no questions, some positive and some negative. These items were combined in a scale which indicated the proportion of household condition problems. At the time of the post-treatment interview, experimental group families had, on average, significantly fewer problems with household condition than did control group families (.13 vs. .16; p = .05). ⁽⁹⁴⁾

Caretaker Problems. Caseworkers were asked a number of questions about problems experienced by children, caretakers, or other adult household members (question 19 on the first caseworker interview, question 17 on the second caseworker interview). Twenty-one of these problems concerned the caretakers. At the time of the post-treatment interview, caseworkers reported that the caretakers had, on average, 26 percent of the 21 problems. There were no significant differences between the experimental and control groups. ⁽⁹⁵⁾

Child Problems. Twelve of the items on the caseworker problem inventory concerned the children. At the time of the post-treatment interview, the percentage of child problems for the experimental group was, on average, 19 percent compared to an average of 15 percent for the control group, a nonsignificant difference. ⁽⁹⁶⁾

4.5.5 Summary of Outcome Data

Information from the caretaker interviews, the caseworker interviews, and the administrative data were analyzed for indications of differences between the experimental and control groups subsequent to the referral to the family preservation program. Tables 4-28 and 4-29 contain a summary of those outcomes on which we found significant differences between the experimental and control groups in the primary analysis (p < .05). Items in bold are those on which the experimental group had better outcomes, those in italics are those on which the control group had better outcomes.

There were no significant differences between experimental and control groups on family level rates of placement. Subsequent maltreatment was generally not related to experimental group membership either.

Table 4-28
Summary of Outcomes in Philadelphia, Post-treatment Interview

Caretaker Scales:	Control Mean	Experimental Mean	p
Positive life events	.15	.19	.05
Caseworker Scales:
Household Condition (higher = worse condition)	.16	.13	.05
NOTE: This table only includes items with a primary p-value less than or equal to .05. Items in bold indicate significant findings in favor of the experimental group.

Table 4-29
Summary of Outcomes in Philadelphia, Caretaker Followup Interview

Proportion of affirmative answers to yes/no questions
	Control %	Experimental %	p
Not enough money for food, rent, or clothing	33	49	.02
Had difficulty buying enough food	16	27	.05
Had difficulty buying clothes	21	37	.01
Punished children for not finishing food	1	7	.05 (FE)
NOTE: This table only includes items with a primary p-value less than or equal to .05. Italicized items indicate significant findings in favor of the control group. "FE" indicates significance determined by Fisher's exact test.

Caretaker Scales:	Control Mean	Experimental Mean	p
Stolen things or arrested	.13	.26	.02
Number of 10 household condition problems	.06	.10	.05
NOTE: This table only includes items with a primary p-value less than or equal to .05. Italicized items indicate significant findings in favor of the control group.

As shown in Tables 4-28 and 4-29, there were few significant differences between experimental and control groups in analyses of child and family functioning items. It should also be noted that the results have not been adjusted for the multiplicity of significance tests performed. That is, these significant items surfaced out of a large number of items and scales examined. In such a situation it is to be expected that some items will show significant differences simply by chance, so the appearance of a few significant differences should not be taken as an indication of superiority of one group over another. Overall, we are unable to claim consistent evidence of positive effects of the family preservation services in Philadelphia that were examined in this study.

[Go To Contents]

Endnotes

57. There was a small, non-significant difference between the experimental group, of whom 5.8 percent said yes, and the control group, of whom 11.2 percent responded yes (p = .11).

58. Although not significantly different, compared to the control group, the experimental group did have a somewhat higher proportion of respondents who reported having been neglected as a child (27% vs. 19%; p = .15).

59. The state of Philadelphia reports 8 possible outcomes for reports of maltreatment; (1) indicated, perpetrator admitted, (2) indicated, medical evidence, (3) not substantiated, (4) pending determination, (5) substantiated, (6) unfounded, (7) indicated - investigating, and (8) unable to complete.

60. Placement spells are defined as any consecutive period of time in substitute care and may consist of several distinct placements (i.e., several different foster homes).

61. The results of "secondary" analyses, dropping violations of random assignment and cases receiving minimal service show slightly greater differences between the experimental and control groups (see Appendix). Here, the same 8 significant primary analysis items show significant differences in favor of the experimental group at p = .01 or lower, and one additional item showed significant differences in the same direction at p < .05. All nine items that showed significant differences in the secondary analysis remained significantly different in the tertiary analysis (see Appendix).

62. These differences remained significant and in the same direction for both secondary and tertiary analyses (for magnitude and significance levels, see Appendix).

63. Differences remained significant and in the same direction for both secondary and tertiary analyses (for magnitude and significance levels, see Appendix).

64. In the secondary analysis (dropping violations and minimal service cases) and the tertiary analysis (dropping additional cases that may not have had a worker assigned), a significantly greater proportion of experimental group caretakers reported receiving transportation and parent education/training classes (see Appendix for magnitude and significant levels).

65. The difference was, however, significant in the secondary analysis (21% control vs. 14% experimental, p = .001) and the tertiary analysis (22% control vs. 13% experimental, p = .003).

66. In addition to slight changes in the magnitude of the difference in whether workers helped caretakers see their good qualities, secondary analysis revealed that a significantly greater proportion of experimental group caretakers reported that they and their worker agreed on goals most of the time (75% vs. 70%, p = .03). Tertiary analysis revealed no additional items with significant differences (see Appendix).

67. When violations and minimal services cases were excluded, the difference between the groups was even larger (5.3 vs. 3.1, p = .0001).

68. SSI for adult or child was reportedly provided more often to the control group than the experimental group, and the difference was marginally significant by Fisher's Exact test (3% vs. 0%; p = .06). In the secondary analyses, excluding violations and minimal service cases, 10 services were provided significantly more often to the experimental group than to the control group (again, significance levels were all at p = .05 or less). In addition to the 6 primary analysis items showing differences in favor of the experimental group, secondary analyses indicate that the following services were also provided significantly more often by the experimental group: health assessment, housing financial assistance, self help groups, and homemaker services (see Appendix for magnitude of difference and significance levels). In the secondary analyses no services were provided significantly more often to the control group than the experimental group.

69. None of the items showed significant differences between experimental and control group caretakers in the secondary analysis or the tertiary analysis.

70. Secondary and tertiary analyses did not result in any significant differences either.

71. This difference remained significant in the secondary analysis (31% vs. 16%; p = .02) and was marginally significant in the tertiary analysis (31% vs. 18%; p = .06).

72. Tertiary analyses were not performed on caseworker interview data due to the fact that all of the 29 additional cases dropped for this level of analysis were missing both caseworker interviews and results would therefore be the same as for the secondary analysis.

73. In determining placements, we depended on the variable "factype" in the administrative data. The specific categories for this variable included: adoption, foster care, private institution/boarding schools, family treatment home, unmarried parent, other, children's psychiatric hospital, and foster care medically fragile.

74. Due to the "clustering effect," analyses at the child level are misleading. Clustering refers to the lack of independence between children within the same family of observations of such things as placement. One could argue that if one child is removed from the home, the remaining children are more likely to experience placement. The "clustering effect" leads to an underestimate of the significance levels when analyses are conducted at the child level. Conducting the analyses at the family level is one approach to resolving this dilemma.

75. Fifteen percent of the experimental group, and 16 percent of the control group experienced substitute care placement within a year in the secondary analysis.

76. Analyses were also done on all allegations, whether substantiated or not. The results were very similar, although, of course, rates for all allegations were higher.

77. Figure 4-2 also displays the survival curves for the secondary analysis group. At one year, 22 percent of the experimental group and 13 percent of the control group experienced substantiated reports of maltreatment.

78. This difference remained significant in both the secondary and tertiary analyses (see Appendix for magnitude of difference and significant levels).

79. These results held for the secondary and tertiary analyses.

80. These results were maintained in the secondary and tertiary analyses.

81. This difference was slightly larger and more significant in the secondary analysis (53% experimental and 33% control; p = .006) and the tertiary analysis (53% experimental and 31% control; p = .005). Tertiary analysis also revealed significant differences in the proportion of respondents indicating they "just wanted to give up," with a greater proportion of experimental group respondents answering affirmatively (29% vs. 15%; p = .04).

82. There were no significant differences between experimental and control groups on these items in the secondary or tertiary analyses.

83. The difference was greater and marginally significant in the secondary analysis (.30 vs. .21; p = .06). In the tertiary analysis, the difference was greater still and it was statistically significant (.31 for the experimental group and .19 for the control group; p = .02).

84. Differences on both items remained significant in the secondary and tertiary analyses (see Appendix for magnitude of differences and significance levels).

85. This difference increased and remained significant in the secondary analysis (25% vs. 11%; p = .009) and the tertiary analysis (26% vs. 11%; p = .01). In the tertiary analysis, one additional item resulted in significant differences at the time of the post-treatment interview. Six percent of the experimental group caretakers and none of the control group caretakers reported that "there were bare electric wires" (Fisher's exact p-value = .02).

86. This difference was not significant in either the secondary or the tertiary analysis. However, tertiary analyses revealed that two different items resulted in significant differences between experimental and control group caretakers at the time of the follow-up interview. Nine percent of the experimental group caretakers and none of the control group caretakers reported that the "plumbing did not work" (Fisher's exact p-value = .01). Seventeen percent of the experimental group caretakers and six percent of the control group caretakers reported that "a lot of paint was peeling" (p = .04).

87. In the secondary analysis, "hitting child harder than meant to" was the only item for which there were significant differences between the experimental and control groups, with a greater proportion of the experimental group responding affirmatively (13% vs. 5%, p = .03). The difference was not significant in the tertiary analysis.

88. This difference was not significant in the secondary or tertiary analysis.

89. Reliability analysis yielded a Cronbach's alpha of .92 at initial interview, .90 at post-treatment, and .94 at follow-up.

90. These results held for the secondary and tertiary analyses.

91. This difference was greater and statistically significant in the secondary analysis (31% vs. 18%; p = .02) and the tertiary analysis (31% vs. 16%; p = .02).

92. These results were also maintained in the secondary analysis (80% vs. 63%; p = .02) and tertiary analysis (79% vs. 62%; p = .03).

93. Secondary analyses resulted in significant differences on three of the individual items. Relative to caseworkers in the control group, caseworkers in the experimental group rated caretakers higher (more adequate) on "respecting children's opinions" (2.84 vs. 2.56; p = .05), "setting firm limits for children" (2.50 vs. 2.18; p = .05), and "providing adequate personal supervision" (2.86 vs. 2.51; p = .04).

94. These results were maintained in the secondary analysis (.13 vs. .17; p = .03).

95. This was also true in the secondary analysis.

96. This result held in the secondary analysis.

Where to?

Top of Page | Contents

Main Page of Report | Contents of Report

Home Pages:
Human Services Policy
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)