This report highlights several positive trends in the characteristics and activities of CICH networks over the course of this initiative. The most notable trend was the significant increase over the study period in the implementation of practices that encourage system integration, as well as in levels of system integration themselves, particularly on the measure of joint service planning and coordination. We also found that the implementation of practices intended to encourage system integration was significantly and positively correlated with measured levels of integration. Future reports will evaluate the effects of system integration on service delivery and client outcomes.
It is further encouraging that there was a significant increase in the availability of information on client and service delivery and in the implementation of homeless management information systems as well as in the use of evidence-based mental health practices. Each of these findings reflects movement towards meeting CICH program goals.
There were no significant changes in ratings of the type of housing provided or in the prevalence of dyadic fiscal relationships. The CICH sites were selected as the best candidates out of a field of over 100 applicants and as a result of this selection process these sites appear to have been focused on the provision of permanent supported housing even before the initiative began.
Significant variation among sites or agency types in the amount of change they experienced over the study period was limited to two dyadic measurements of joint planning and coordination and trust and respect. These modest cross-site differences partly reflect lack of statistical power due to the small number of cases as well as to ceiling effects on these measures since these sites were selected for their demonstrated capacity to function as a coordinated network of agencies.
While a major emphasis in CICH was placed on encouraging system integration it was also of interest that agencies with ongoing fiscal relationships had significantly higher levels of joint planning and coordination as well as trust and respect, although these relationships do not explain the changes in integration measures observed over the course of the initiative.
As with previous evaluations of initiatives intended to increase system integration (i.e., the Program on Chronic Mental and the Access to Community Care and Effective Services and Supports program) we found that the CICH initiative was associated with increasing levels of coordination and communication as well as trust and respect among service agencies. Given appropriate motivation, technical support, and resources, participating agencies achieved increased levels of system integration and furthered the implementation of both homeless management information systems and evidence-based practices.
This preliminary evaluation report does not address the issue of whether the increases in system integration, or in the implementation of either information systems or evidence-based practices, were associated with improved client outcomes. Although substantial data will be available on client outcomes, the small number of sites and the high level of integration at the beginning of the project may limit our eventual ability to demonstrate such relationships.
Several limitations of this evaluation need mention. Most importantly, all data reported here were based on interviews with a small number of key informants. We did not have access to objective measures of site performance, a weakness that is broadly inherent in research on service systems due to their complexity, the large number of individuals and organizations that comprise them, and the many exogenous environmental factors that influence system operation (Rosenheck et al. 2001).
Secondly, this study did not use an experimental evaluation design through which sites would have been randomly assigned to a treatment or a control group exposed to different interventions, nor did we use a quasi-experimental study design, such as matched site comparisons. Although the pre- and post-implementation data for this evaluation are suggestive, other factors including reporting biases may have been responsible for the measured system changes.
While these limitations prevent us from concluding definitively that the CICH initiative caused the observed system changes at these 11 evaluation sites, the data presented are clearly consistent with such an inference, and suggest that site level initiatives in the CICH program successfully accomplished the program objectives.
"report.pdf" (pdf, 296.36Kb)