In response to the problem of chronic homelessness, the U.S. Departments of Housing and Urban Development (HUD), Health and Human Services (HHS), and the U.S. Department of Veterans Affairs (VA) launched a new 3-year federal Initiative in October 2003 through the U.S. Interagency Council on Homelessness entitled The Collaborative Initiative to Help End Chronic Homelessness (CICH). Through this Initiative, persons experiencing chronic homelessness receive permanent supported housing funded by HUD, and supportive primary healthcare and mental health services provided by the Health Resources Services Administration (HRSA), the Substance Abuse and Mental Health Services Administrations (SAMHSA) of HHS, and by the Veterans Health Administration (VHA) of VA.
The three federal agencies sponsoring the Initiative (HUD, HHS & VA) enlisted the VA Northeast Program Evaluation Center (NEPEC) to conduct a national evaluation of CICH network collaboration and client outcomes to monitor the implementation and effectiveness of the $35-million Initiative by using a common evaluation methodology across all 11 CICH sites.
Previous reports have shown improved client outcomes in housing and health status and higher levels of collaboration and trust and increased use of evidence-based practices at these 11 sites.
It has been hypothesized that increased collaboration and trust between organizations and use of evidence-based practices will increase service delivery and improve outcomes. In this report, network data reflecting collaboration, trust, and use of evidence-based practice at the time clients enrolled in CICH were merged with 12-month client outcome data to examine the association of inter-agency relationships at the start of the program and client outcomes during the first year of program participation.
Key informants from the local agencies providing CICH housing and supportive services at each site were identified in fall 2003 through an initial "network definition" telephone interview, and then interviewed annually over three years over the telephone by NEPEC evaluation staff beginning in 2004. Key informants were asked to report on levels of inter-agency service delivery and collaboration (through the annual "network participation" telephone interview) along four key dimensions: use of best practices, collaborative planning, trust and respect, and exchanging of resources.
Upon entering the program and the national evaluation, basic socio-demographic and clinical status data on clients were documented. Clients were also asked about their use of a wide range of services and reported on a wide range of housing, health status, and other outcome domains thought to be possibly influenced by the multi-faceted CICH intervention. Local VA research staff at each site administered these baseline assessments, and quarterly follow-up assessments thereafter, primarily through in-person interviews with evaluation participants. Client baseline and quarterly follow-up data during the first year of treatment were merged with data from the network survey administered prior to each client's entry into the evaluation. Thus, network data from one of three annual network surveys administered near the beginning of each calendar year were merged with client data on the basis of site codes and date of program entry, thereby constituting the "baseline" network survey for each client.
Mixed linear regression models were used to examine the relationship between network measures and client measures (service use and outcomes), adjusting for potentially confounding client characteristics — i.e., baseline client characteristics found to be bivariately associated with each network measure.
Significant associations at p<.05 were found in 10 of 64 associations of network measures and measures of client service use relationships, most notably in association with the best practices network measure (which accounted for five of these ten significant relationships). Paradoxically, clients treated by service networks implementing a greater number of best practices received fewer preventive healthcare procedures over time, were less likely to discuss health behaviors with their doctor, were less likely to be visited in the community by a case manager, had fewer total service providers, although they were more likely to have a money manager, than CICH clients at sites reporting less use of best practices.
Significant associations at p<.05 were found in only 2 of 72 network-client outcome relationships examined, again both involving the best practices network measure. The use of best practices was associated with a greater client trust towards their doctor, as well as with decreased total service costs, presumably due to the significantly lower levels of service use as described above. If a Bonferroni adjustment was used to adjust the test of significance for multiple comparisons, the level of significance should be lowered from p<.05 to p<.001 (initial p<.05 level divided by 34 outcome measures). Using the more conservative tests, the only statistically significant relationship found between the network measures and measures of client service use or outcomes was the increased likelihood of receiving money management services at sites using more evidence-based practices.
Possible explanations for these overall non-significant findings include high initial levels of collaboration between agencies at the start of the project (ceiling effect), limited variability among network measures across sites (homogeneity effect), and either non-existent or a weak influence of network characteristics on 12-month client service use and outcomes (small size effect).
These preliminary findings suggest that neither use of evidence-based practices nor measures of collaboration and trust among CICH network agencies were significantly associated with either client service use or client outcomes during clients' first year of entering the program. Questions regarding the association of changes in network collaboration and client service use and outcomes over the entire 3-year program follow-up period will be addressed in a subsequent report after CICH client data collection is completed in fall 2007.