HUD/HHS/VA Collaborative Initiative to Help End Chronic Homelessness: An Evaluation of an Initiative to Improve Coordination and Service Delivery of Homeless Services Networks. Results

02/13/2007

Sample Characteristics

Analyses were based on two types of measures, as described above, one that used each participating agency at each of the 11 CICH sites as the unit of analysis, and the other that used dyadic relationships among the agencies as the unit of analysis. An average of 6.7 agencies (standard deviation=1.66) were surveyed at each site in each wave. Data were available for 80 agencies surveyed in wave one; 72 in wave two; and 70 in wave three.(4)

Data were available for a substantially greater number of dyadic relationships — 528 in wave one; 474 in wave 2; and 462 in wave 3. Data were thus available for an average of 44.4 dyadic relationships (standard deviation=19.8) per site in each survey year. Table 4 provides descriptive information on each system level measure and Table 5 presents a summary of bivariate correlations among the 12 measures.

Table 4:
Descriptive Characteristics: System-Level Measures Across All Time Points
Variable Groups Variables N Mean Scores SD Range

System Connectedness And Integration

Integrative Practices

215

2.08

.52

0-3

Coalition Involvement

187

2.25

.72

0-3

Working Together

210

2.66

.51

0-3

Involvement of Local Organizations: Number

211

7.65

2.42

0-12

Dyadic Joint Service Planning and Coordination

1,366

1.82

.87

0-3

Dyadic Trust and Respect

1,369

2.64

.60

0-3

Emphasis On Providing Housing Services And On The Associated Goal Of Ending Homelessness

Emphasis on Transitional, Emergency, and Affordable Housing as Contrasted with Permanent Supported Housing

206

2.23

.72

0-3

Change in Emphasis from Emergency Shelter and Transitional Housing to Permanent Supported and Non-supported Housing

208

2.12

.52

1-3

Development Of Homeless Services Management Information System

Client and Services Information Available

208

.84

.29

0-1

CICH Management Information System Exists

199

1.51

1.01

0-4

Use of Evidence-Based mental health Practices

Use of  Evidence Based Practices

208

2.30

.44

.11-3

Funds Transfer and Influence

Fiscal Relationship Exists

1,355

.31

.46

0-1

 

Table 5:
Correlations of All Measures at Agency Level

Variables 1 2 3 4 5 6 7 8 9 10 11 12

1=Integrative Practices (N=215)

1.00

 

 

 

 

 

 

 

 

 

 

 

2=Coalition Involvement (N=187)

.39 (.0001)

1.00

 

 

 

 

 

 

 

 

 

 

3=Working Together (N=210)

.53 (.0001)

.25 (.001)

1.00

 

 

 

 

 

 

 

 

 

4=Involvement of Local Organizations: Number (N=211)

.35 (.0001)

.28 (.0001)

.33

(.0001)

1.00

 

 

 

 

 

 

 

 

5= Dyadic Joint Service Planning and Coordination (By Rating Agency) (N=217)

.47 (.0001)

.07

 (.35)

.23 (.001)

.23 (.0009)

1.00

 

 

 

 

 

 

 

6=Dyadic Trust and Respect (By Rating Agency) (N=221)

.45 (.0001)

.11

 (.13)

.26 (.0001)

.30 (.0001)

.52 (.0001)

1.00

 

 

 

 

 

 

7= Emphasis on Transitional, Emergency, and Affordable Housing as Contrasted with Permanent Supported Housing (N=206)

.27 (.0001)

.008 (.91)

.23 (.0012)

.29 (.0001)

.20 (.0035)

.29 (.0001)

 

 

 

 

 

 

8= Change in Emphasis from Emergency Shelter and Transitional Housing to Permanent Supported Housing (N=208)

-.084 (.23)

-.14 (.059)

-.216 (.0019)

-.036 (.61)

-.014 (.84)

0.12 (.083)

 

 

 

 

 

 

9=Client and Services Information Available (N=208)

.32 (.0001)

.21 (.004)

.26 (.0002)

.22 (.0014)

.27 (.0001)

.05

 (.47)

.105 (.14)

-.053 (.45)

1.00

 

 

 

10=CICH Management Information System Exists (N=199)

.25 (.0005)

.13 (.085)

.20 (.0052)

.20 (.0053)

.24 (.0009)

.15 (.038)

-.004 (.96)

.13 (.071)

.28

(.0001)

1.00

 

 

11= Average Degree Evidence Based Practices Used (N=208)

.66 (.0001)

.23 (.002)

.56 (.0001)

.33 (.0001)

.38 (.0001)

.46 (.0001)

.24

(.001)

-.031 (.66)

.22 (.0002)

.20 (.004)

1.00

 

12=Funds Transfer and Influence (N=215)

.20

(.0029)

-.008

(.91)

.11

(.12)

.16

(.023)

.49

(.0001)

.28

(.0001)

.13 (.07)

.14 (.043)

.12 (.09)

.20

(.004)

.16 (.02)

1.00

System Integration

Results for both the implementation of system integration practices, and the integration outcome measures showed positive trends (Table 6). Over the study period there was a significant increase of 15% in the measure of implementation of practices designed to encourage system integration. While there was no significant changes from wave one to wave three in the level of involvement of the local homeless coalition, or in how well the agencies at each site globally rated the way they worked together, this mostly likely reflects a ceiling effect since participating sites had achieved high baseline scores (2.4-2.6 out of a possible 3.0) prior to program implementation, i.e., during and even prior to the development of their CICH proposals.

Table 6:
System Integration Over Time
Measures of System Connectedness and Integration N Mean Scores Time Comparison

Wave 1

Wave 2

Wave 3

Percent Change (Wave 1 to Wave 3)

P*

Pairwise Comparison**

Integrative Practices

204

1.94

2.06

2.23

14.9%

.003

W3>W1

Coalition Involvement

187

2.36

2.09

2.29

-3.0%

.049

W1>W2

Working Together

210

2.59

2.70

2.69

3.9%

.30

None

Involvement of Local Organizations: Number

203

7.34

7.73

8.21

11.9%

.12

W3>W1

Dyadic Joint Service Planning and Coordination

1,366

1.63

1.85

2.04

25.2%

.0001

W3>W2>W1

Dyadic Trust and Respect

1,369

2.58

2.68

2.67

3.5%

.005

W3>W1, W2>W1

 

*Proc mixed used in which autocorrelation controlled for with respect to site for those measures based on key informant observations (site and ID for those based on relationship characteristics)

 **Only those comparisons with significance in which p<.05 reported

 

There were significant increases in two other key measures of system integration (Table 6). Strong results were observed on the measure of joint planning and coordination, which increased by 25% over the study period and showed highly significant change (p<.0001) (see Figure 1). The dyadic measure of trust and respect also increased by a statistically significant 3.5%. The smaller magnitude of change on this measure similarly reflects a high baseline level (2.6 out of a possible 3.0) that left little room for improvement on the underlying metric.

Figure 1.
System Integration over Time

Figure 1. System Integration over Time. See text for explanation.

Analyses of correlations between the implementation measure and the level of system integration actually achieved showed that greater implementation of integration practices was highly and significantly associated with greater system integration and better system performance on virtually all measures (see first column of Table 5).

Other System Changes

Table 7 presents data that address the second study question; i.e., was the implementation of the CICH initiative associated with changes in the type of housing provided at CICH sites, in the implementation of homeless management information systems (HMIS), or in the use of evidence-based mental health practices?

Table 7:
Comparisons Over Time of Other System Characteristics
Variable Groups Variables N Mean Scores Time Comparison
Wave 1 Wave 2 Wave 3 Percent Change (Wave 1 to Wave 3) P* Pairwise Comparison**

Emphasis On Providing Housing Services And On The Associated Goal Of Ending Homelessness

Emphasis on Transitional, Emergency, and Affordable Housing as Contrasted with Permanent Supported Housing (Factor Score)

206

2.33

2.21

2.15

-7.7%

.32

None

Change in Emphasis from Emergency Shelter and Transitional Housing to Permanent Supported Housing (Factor Score)

208

2.12

2.07

2.18

2.8%

.47

None

Development Of Homeless Services Management Information System

Client and Services Information Available

208

.75

.86

.90

20.0%

.0009

W3>W1, W2>W1

CICH Management Information System Exists  

199

1.23

1.47

1.90

54.5%

.0006

W3>W1, W3>W2

Use of Evidence-Based mental health Practices

Use of  Evidence Based Practices (Mean Rating)

204

2.12

2.35

2.41

13.7%

.0002

W3>W1, W2>W1

Funds Transfer and Influence

Fiscal Relationship Exists

1,355

.29

.32

.33

13.8%

.52

None

 

*Proc mixed used in which autocorrelation controlled for with respect to site for those measures based on key informant observations (site and ID for those based on relationship characteristics)

 **Only those comparisons with significance in which p<.05 reported

 

There was no significant change in the degree to which agencies at CICH sites emphasized the provision of emergency, transitional, and affordable housing without support, as contrasted with permanent supported housing, nor was there significant change in the assessment of change in types of housing emphasized at CICH sites. The lack of change in housing emphasis may reflect ceiling effects since many sites selected for CICH were already committed to developing permanent supported housing.

There was, however, a significant increase of 20% in the reported ability of CICH agencies to obtain information about clients served and services delivered to them by the CICH network. There was also a significant 54.5% increase in the measure of implementation of a homeless management information system.

The 13.7% increase in the measure of the use of evidence-based mental health practices progressed monotonically from wave one to wave three and was also highly statistically significant (p=.0002).

Change by Site and Agency Type

There were few differences among sites, or among types of agencies, in the magnitude of changes in system-wide performance measures.(5)  Significant variation in change by site was observed on the dyadic measure of joint planning and coordination (see Table 8 columns 3-5).

Table 8:
Comparison of Measures by Site and Agency Type
Variable Groups Variables N Site Comparison
(All Time Points)
Agency Type Comparison
(All Time Points)
Interaction Effect — Site and Time Sites that Showed Significant Change Over Time Interaction Effect — Agency Type and Time Types of Agencies that Showed Significant Change Over Time

System Connectedness And Integration

Integrative Practices

215

(p=.49)

NA

(p=.85)

NA

Coalition Involvement

187

(p=.91)

NA

(p=.57)

NA

Working Together

210

(p=.70)

NA

(p=.83)

NA

Involvement of Local Organizations: Number

211

(p=.96)

NA

(p=.91)

NA

Dyadic Joint Service Planning and Coordination

1,366

(p=.02)

Los Angles (p=.0001) Columbus (p=.0001)

 Fort Lauderdale (p=.0007) San Francisco (p=.0015) Portland (p=.0195)

(p=.06)

Lead (p=.0045)

 Housing (p=.0087)

 Mental Health (p=.0017)

 VA (p=.0001)

 Other (p=.0005)

Dyadic Trust and Respect

1,369

(p=.28)

NA

(p=.11)

Lead (p=.0046)

Emphasis On Providing Housing Services And On The Associated Goal Of Ending Homelessness

Emphasis on Transitional, Emergency, and Affordable Housing as Contrasted with Permanent Supported Housing

206

(p=.72)

NA

(p=.86)

NA

Change in Emphasis from Emergency Shelter and Transitional Housing to Permanent Supported Housing

208

(.97)

NA

(p.=62)

NA

Development Of Homeless Services Management Information System

Client and Services Information Available

208

(p=.73)

NA

(p=.81)

NA

CICH Management Information System Exists

199

(p=.60)

NA

(p=.17)

NA

Use of Evidence-Based mental health Practices

Use of  Evidence Based Practices

208

(p=.49)

NA

(p=.74)

NA

Funds Transfer and Influence

Fiscal Relationship Exists

1,355

(p=.74)

NA

(p=.25)

NA

 

Analysis of the site changes over time showed five of the eleven sites with significant increases in the level of joint planning and coordination, in contrast to the other six sites that did not show significant change (Table 8). Closer examination of the data indicated that although on average these six sites started at higher level of joint planning and coordination than the other five sites, they all scored less than two (with three being the highest possible score).

Examination of differences in change across types of agencies revealed significant differences in changes on both the measure of joint planning and coordination and on the measure of trust and respect. Five of the seven agency types showed significant change on the joint planning and coordination measure (Table 8 second panel). The lead agency alone showed a significant increase in trust and respect in comparison to the other agencies.

Fiscal Relationship Comparisons

There was no significant increase or decrease in the prevalence of fiscal relationships(6) among agencies participating in CICH (see last row of Table 9) suggesting that participating in CICH was not associated with the formation of these types of relationships. Table 7 presents comparisons between pairs of agencies (i.e., dyads) with and without a fiscal relationship. Cross sectionally, agencies in dyads characterized by fiscal relationships had substantially higher ratings on the measure of joint planning and coordination (55.5% higher) and on the measure of trust and respect (8.6% higher) than those dyads without a fiscal relationship. However, there were no significant differences in the magnitude of change over the study period between pairs of agencies with and without fiscal relationships. Increases in system integration observed over time cannot therefore be attributed to the development of stronger fiscal relationships.

Table 9:
Funds Transfer Relationship and Measures of Integration
Variables N Mean Scores Presence of Fiscal Relationship Interaction Effect — Fiscal Relationship and Time
No Relationship Relationship Exists Percentage Difference

Dyadic Joint Service Planning and Coordination

1,326

1.55

2.41

55.5%

(p<.0001)

(p=.10)

Dyadic Trust and Respect

1,322

2.57

2.79

8.6%

(p<.0001)

(p=.95)

View full report

Preview
Download

"report.pdf" (pdf, 296.36Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apa.pdf" (pdf, 145.6Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®