HUD/HHS/VA Collaborative Initiative to Help End Chronic Homelessness: An Evaluation of an Initiative to Improve Coordination and Service Delivery of Homeless Services Networks. Methods


Surveys and Data Collection

The CICH program began between March and August 2004 at 11 sites: Los Angeles, Martinez (Contra Costa), and San Francisco, California; Philadelphia, Pennsylvania; New York, New York; Denver, Colorado; Chicago, Illinois; Columbus, Ohio; Chattanooga, Tennessee; Fort Lauderdale, Florida; and Portland, Oregon (See Table 1 for a brief description of each site) and are scheduled to run for a three year period after start up. Sites varied in the number of chronically homeless that they would serve, from 50 to 100 people.

Table 1:
Site Overviews



Broward County, Florida ~ HHOPE Program The HHOPE program, a collaboration of the Broward County Human Services Department, Homeless Initiative Partnership Administration, provides scattered site housing and supportive services to severely and persistently mentally ill and chronically homeless individuals through Shelter Plus Care. The project is implementing a Housing First approach and using a modified ACT team.  Their goal is to serve 80 individuals over the life of the project.
Chattanooga, Tennessee ~ The Collaborative Initiative Chattanooga’s Collaborative Initiative, coordinated by the Fortwood Center, serves chronically homeless individuals in scattered site housing.  The Initiative is implementing the Housing First approach and an ACT team to provide wrap-around services for clients in housing.  The goal of the Initiative is to serve 50 individuals over three years, in housing subsidized through Shelter Plus Care.
Chicago, Illinois ~ ACT Resources for Chronically Homeless (ARCH) Led by the Chicago Department of Human Services, ARCH targets chronically homeless individuals with mental health, substance abuse, and/or co-occurring disorders.  They are using Shelter Plus Care vouchers to secure 59 tenant-based permanent housing units. The units are both scattered site and clustered.  ARCH uses a Housing First approach and an ACT team. Their goal is to bring about significant expansion of permanent supportive housing, coordination and maximization of mainstream resources, and expansion of evidence-based service strategies to meet the complex needs of chronically homeless people.
Columbus, Ohio ~ Rebuilding Lives PACT Team Initiative (RLPTI) RLPTI is led by Southeast, which contracts project management to the Community Shelter Board.  The project serves chronically homeless individuals with severe mental disabilities or co-occurring substance abuse and mental illness.  For this initiative, they have five clustered site housing units through a Supportive Housing Program grant and use a Housing First approach. They use a PACT model and have incorporated several evidence-based practices.  One of RLPTI’s main goals is to increase the behavioral healthcare system in Franklin County, particularly by increasing Southeast’s capacity and treatment slots. In addition, they plan to increase income supports and entitlements for the chronically homeless. The goal of the RLPTI is to house and serve 108 individuals.
Contra Costa, California ~ Project Coming Home (PCH) Led by the Contra Costa Office of Homeless Programs, PCH serves chronically homeless individuals using a Health, Housing, and Integrated Services Network (HHISN).  Through Shelter Plus Care, PCH uses a housing first, scattered site model facilitated through partnerships with the housing authority and Shelter Inc.  The goals of PCH include: increasing the effectiveness of integrated systems of care by providing comprehensive services and treatment, linked to housing; increasing the use of mainstream resources that pay for services and treatment; and supporting the development of infrastructures that sustain housing, services treatment, and inter-organizational partnerships beyond the federal initiative.  Over a five-year period, they expect to contact 5,250 chronically homeless individuals and house 155 individuals.
Denver, Colorado ~ Denver Housing First Collaborative (DHFC) DHFC, a Shelter Plus Care grant is a collaboration of agencies led by Colorado Coalition for the Homeless.  It seeks to provide coordinated housing and treatment to chronically homeless individuals with disabilities, substance abuse, severe and persistent mental illness, co-occurring disorders, and/or chronic physical illness.  DHFC uses a Housing First approach and an ACT team.  Housing is both scattered site and clustered.  DHFC aims to serve 100 clients in year one.
Los Angeles, California ~ Skid Row Collaborative The Collaborative, which has a Shelter Plus Care Grant, is coordinated by the Skid Row Housing Trust, and seeks to serve chronically homeless and disabled persons.  It uses the Health, Housing, and Integrated Services Network (HHISN) model.  To reach their goal of assisting clients into permanent housing, the Collaborative is expanding mental health and co-occurring treatment services by adding a team of case manager specialists in mental health and substance abuse and peer advocates to provide outreach, engagement, support and recovery services/ treatment, and case management.  They have a goal of housing and serving 62 individuals. The project has already had contact with 140 homeless individuals in its first year.
New York, New York ~ In Homes Now (IN)/Project Renewal IN, a Supportive Housing grant, is coordinated by Project Renewal serves chronically homeless individuals who are active substance abusers in New York City.  IN uses a Housing First approach and an Intensive Integrated Service Team to supplement existing programs (Continuum of Care and Pathways to Housing) for which active substance users are not eligible.  The project’s goal is to house and provide comprehensive services for 40 individuals from the target population in scattered-site SRO apartments located in Manhattan and the Bronx.
Philadelphia, Pennsylvania ~ Home First With the City of Philadelphia as the lead agency, Home First serves homeless individuals who have serious mental illness and/or co-occurring disorders and who have among the highest number of documented days in the city’s emergency shelter and residential behavioral health system. They use a Housing First approach and an ACT team.  The project intends to serve approximately 85 chronically homeless individuals over the life of the Supportive Housing Program project.
Portland, Oregon ~ The Community Engagement Program (CEP III)/ Central City Concern CEP III, which has a Shelter Plus Care grant, is coordinated by Central City Concern.  The project focuses on the “hardest to serve” of Portland’s chronically homeless population – those with a significant disability (i.e., physical health, mental health, and/or substance abuse issues) and/or co-occurring disorders.  Based on ACT and the Housing First approach, clients are housed in scattered site, clustered, or Shelter Plus Care units.  The project’s main goal is to demonstrate an effective model in reducing chronic homelessness for people with co-occurring disorders. CEP III seeks to serve 100 clients in the first year and 150 over the life of the project.
San Francisco, California ~ Direct Access to Housing (DAH) The San Francisco Department of Public Health is the lead agency for the Direct Access to Housing initiative, which has a Supportive Housing Program grant.  They are creating 70 units of permanent supportive housing through an expansion of their DAH program at the Empress Hotel.  DAH serves chronically homeless individuals with disabilities, using a supportive housing model.

Each site identified core agencies to participate in an annual key informant survey involving representatives of the lead agency and partnering agencies that provided housing assistance, mental health care, substance abuse services, primary care, and the local Veterans Health Administration facility. A "network definition survey" was administered at the start of the initiative (in November/December 2003) to identify participating agencies and key informants at each site. Key informants were those identified by program leaders as the most knowledgeable about the activities of each agency at each site.

A second more extensive "network participation survey" was then administered in three waves, between November 2003 to March 2006, to the key informants identified at each of the participating agencies. The first survey wave occurred before CICH began (from November 2003 to March 2004). The second and third survey waves occurred at the end of the first and second years of operation (from November 2004 to February 2005 and from January to March 2006, respectively). Surveys were sent to all key informants prior to the interviews, which were conducted over the telephone by national evaluation staff at the Northeast Program Evaluation Center. In some sites one agency provided more than one key service (e.g., the mental health agency also provided substance abuse service) and only one survey was conducted for that agency. At other sites, in contrast, more than one agency provided a particular service and key informants at both agencies were interviewed. Thus, the number of agencies at each site ranged from five to nine. Furthermore, at some agencies more than one key informant was identified and interviews were jointly held.


Five dimensions of CICH service systems were assessed: 1) system connectedness and integration (Coccozza et al., 2000; Interagency Council on Homelessness 2003; Morrissey et al., 2002); 2) emphasis on providing permanent supported housing services (Interagency Council on Homelessness 2003); 3) development of homeless services management information systems (Magnobosco-Bower JL 2001); 4) use of evidence-based mental health practices (DHHS 1999; New Freedom Commission on Mental Health 2003; SAMHSA, 2003); and 5) the existence of interagency fiscal relationships. Because each measurement domain could be addressed in several ways, multiple measures were used in each domain.

Most of the measures were based on items in which each key informant characterized the overall service network at their site from the perspective of their agency. However, elaborating on methods developed by Morrissey et al. (2002), 16 questions asked key informants about the relationship of their agency to each of the other participating agencies in their local network (see items 32-45 in Appendix A). While respondents answered questions about the performance of their network as a whole only once, they gave four to nine responses (depending on the number of agencies in the core CICH network) to questions concerning their dyadic relationship with each other agency. Creation of scales using these 16 items involved multiple steps. First, we conducted a separate exploratory factor analysis of the 16 items. Factor analysis indicated that data could be summarized in four factors that explained 74% of the variation in the 16 measures (see Table 2 for factor loadings for individual items). One of the four factors represented joint planning and cooperation with other agencies, and a second addressed trust and respect between agencies. For these two factors the relevant items were averaged to create scales characterizing each dyadic relationship. The other two factors addressed interagency funding flows: and were used to create a measure indicating whether a fiscal relationship existed between each pair of agencies.

Table 2:
Four-Factor Solution to a Factor Analysis of 15 Dyadic Integration/Coordination
Items from the Network Participation Survey
Burden Item Factor Loading
Factor 1: Joint Planning and Cooperation  

   Send clients to or receive homeless clients from this agency


   Co-locate staff and/or services in the same location with this agency 


   Cooperate in serving clients 


   Communicate clearly with this agency in serving clients 


   Agree on goals with this agency for  serving clients


   Cooperate with this agency in planning


   Communicate clearly with this agency in planning 


   Send information to or receive information from this agency for coordination, control, planning, or evaluation 


   Have written documents (e.g., MOUs) or flow charts specifying working relationship with this agency 


Factor 2: Trust and Respect


   Trust this agency 


   Respect this agency 


Factor 3: Send Funds to or Influence Flow of Funds to Other Agency


   Send funds to this agency 


   Influence flow of funds to agency


Factor 4: Receive Funds From or Influences Flow of Funds to Your Agency


   Receive funds from this agency 


   To what extent does this agency influence the flow of funds to you


In the sections below we describe specific measures in each of the five domains.

1) System connectedness and integration: Six measures were used to assess connectedness and integration, four of which were based on key informant ratings of the overall CICH network and two that were based on dyadic (i.e. interagency) relationship scales, as described above.

The first system-level measure assessed the extent to which each network implemented specific interventions that were designed to result in greater system integration (Morrissey 2002). This measure is the average implementation rating given to 20 potential interventions, 12 of which were developed to evaluate the implementation of integration strategies in the ACCESS program (Cocozza et al. 2000; Morrissey et al. 2002) and the remainder of which were developed specifically for the CICH initiative. Each item represented the degree to which each of the 20 strategies was implemented, on a score from zero to three with zero representing "none" or no effort and three representing "a lot". Examples of these strategies are presence of: a) a system integration coordinator position, b) an interagency coordinator body, c) a formal strategic plan, d) co-location of services, e) cross-training, or f) client tracking systems (see Appendix A for a full list of these strategies — items 46-65) (Cronbach's coefficient alpha=.88; Range .2 - 3).

The second system-level integration measure assessed the extent to which the local homelessness coalition provided material resources, or political or institutional support for the implementation of the CICH initiative. This "coalition participation" measure is the average of five items which addressed how important the coalition had been to the CICH network in providing: 1) support for the development of the initial application; 2) material resources, 3) political and institutional support; 4) help in implementing the initiative; and 5) guidance in the shaping of the goals of the initiative (Cronbach's coefficient alpha=.84 for all five items). Items were scored from zero to three with zero representing "not at all" and three representing "very" or "a lot" (see appendix A for details on the specific items in this scale — 86-90).

The third measure of system connectedness and integration was a global assessment of how well the agencies at a site worked together to solve problems. This scale is the average of four items that focus on how well agencies jointly identified, understood and fixed service delivery problems, as well as how effectively they worked together to address client service needs (Cronbach's coefficient alpha=.83). Here too, each item was scored on a zero to three scale where zero represented "not at all" and three represented "very well" (see Appendix A items 18a-18d).

The fourth system integration scale assessed the total number of different types of agencies that were involved with the CICH network. Key informants at each surveyed agency were asked about agencies that were on the initial grant application for the CICH network and agencies as well as types of local organizations that were not included among the core agencies (e.g., soup kitchens, law enforcement agencies, private businesses, etc.) (Cronbach's coefficient alpha=.75; Range 3 - 12). Items in this scale were dichotomized so that answers of two or three ("somewhat involved" or "very involved") were scored as a one, and other responses were scored as zero (see Appendix A items 11a-11l).

The last two scales were based on items derived from the dyadic measures of the relationships between pairs of agencies and addressed a) joint planning and cooperation and b) trust and respect within each dyadic relationship. First, as suggested by the factor analysis described above, nine items were averaged to constitute a scale representing inter-agency joint planning and coordination. Specific items in this scale addressed: cooperation in serving clients, goal congruence, client referral, cooperative planning, co-location of staff and services, information sharing and communication (see items 32-36 and 40-43 in appendix A for details).

The second dyadic interagency scale, a measure of interagency trust and respect, was constructed as the average of two items addressing the extent to which this relationship was characterized by: 1) trust and 2) respect.

Items in all dyadic scales were scored from zero to three with zero representing "none" and three representing "a lot".

2) Emphasis on providing diverse housing services. Two measures were derived from an exploratory factor analysis of eight items and addressed: a) degree of emphasis in the network, as a whole, on providing various types of housing; and b) changes in the types of housing emphasized by each CICH network (for further details see items 13a-13d and 15a-15d in Appendix A). Factor analysis indicated that these items could be summarized in two scales, one reflecting an emphasis on emergency shelter as well as transitional and unsupported housing and the other reflecting a change in these emphases (see Table 3 for factor loadings for individual items).

Table 3:
Two-Factor Solution to a Factor Analysis of 8 Housing Emphasis Items
from the Network Participation Survey
Burden Item Factor Loading

Factor 1: Current Housing Emphasis by CICH network


    Emergency Shelter


    Transitional Housing


    Affordable Housing




Factor 2:  Change in Housing Emphasis by CICH Network


    Emergency Shelter


    Transitional Housing


    Permanent Supported Housing



3) Development of homeless services management information systems (HMIS). Two measures were created to evaluate the degree to which systems existed at the network level for management of client, service delivery and outcome information. The first measure was the average of three items that documented whether a system was available to generate data on: 1) client characteristics, 2) housing and service delivery, or 3) client outcomes. These three items were scored from zero or one, with zero representing "no" and one representing "yes".

The second measure was a single item which asked if an HMIS specific to the CICH initiative had been implemented. There were five possible answers to this question: 1) neither planned for, nor currently available for use, 2) being planned or developed, 3) in use by some network members, 4) in use by most network members, or 5) in use by all network members. The first answer was scored as zero while the last answer was scored as four (see items 19, 21, 23, and 25 in Appendix A for more details).

4) Evidence-based practices. A single scale was used to measure the degree to which each of the 11 CICH networks was viewed by its core agencies as having implemented evidence-based practices. This scale was based on the 18 practices identified in 2003 by the Substance Abuse and Mental Health Services Administration (SAMHSA) to be solidly evidence-based in a report entitled "An Overview of Mental Health and Substance Abuse Services and Systems Coordination Strategies" (SAMHSA 2003). Examples of these practices are: a) multidisciplinary treatment teams or intensive case management (e.g. Assertive Community Treatment [ACT]); b) self-help programs; c) the housing-first model of supportive housing; d) opiate substitution; and e) family psycho-education (for further details see items 66 to 82 in Appendix A). Items for each of the 18 evidence-based practices measured the extent to which these practices were implemented at each site and were scored from zero to three, with zero representing "none" or no effort and three representing "a lot". The values for the 18 items, one for each evidence-based practice, were averaged for each survey respondent to create this scale (Cronbach's coefficient alpha= .86; Range .11 - 3).

5) Resource flows and influence over resource flows. Finally, a scale was constructed to measure whether an active fiscal relationship existed between each pair of agencies. This measure was based on four items based on the factor analysis described above. First, two intermediate scales were created that characterized the transfer of funds or influence over the transfer of funds between agencies. The first of these dyadic scales (sending or influencing funds sent), was based on responses to two questions concerning the extent to which each agency: a) directly sent funds to each other agency or b) influenced the flow of funds to other agencies from third parties. The second of these scales (receiving funds or influencing receipt of funds) was based on answers to a related pair of questions concerning the extent to which each agency: a) received funds directly from each other agency or b) influenced the receipt of funds from each other agency. The answers to these individual items were scored from zero to three with zero representing "no funds exchanged" and three representing "a lot of funds exchanged". These two intermediate scales were constructed by averaging each pair of items. In the second step, a final, dichotomous measure was created that had a value of one if either of the two intermediate scales had a value of one or greater (i.e., a rating of at least "a little" on either of the funds transfer measures). Otherwise the scale had a value of zero (no funds transfer). For every pair of agencies this scale indicates whether there existed any resource transfer, or influence over resource transfer from a third party between the two agencies in the dyad.


Separate analyses were conducted to address each of the central study questions.

System Integration: The first question concerned the degree to which the CICH project was associated with the implementation of interventions designed to increase system integration, and/or with resultant increases in system coordination, or in trust and respect among agencies at each site. To address this question. a series of analyses were conducted in which the dependent variables were the six system connectedness and integration measurements discussed previously, and the independent variable was a categorical variable representing the year of the project (time). This variable had a value of one to three, with one indicating measurement conducted before the project started and values of two or three indicating measurements that occurred in subsequent years.

In these analyses random effects were modeled using an unstructured covariance matrix, thereby adjusting standard errors for the correlated nature of the data in these models (i.e., for the potential correlation of observations from the same agency for different years). This technique is often referred to as hierarchical linear modeling (HLM)(3) (38).

The second part of the first evaluation question was whether the implementation of practices designed to facilitate system integration was associated with observed increases in system coordination as well as in trust and respect among providers at each site. To investigate this issue we examined the correlation of measures reflecting the implementation of integration strategies, goals and activities with the measures of system coordination and the measure of trust and respect. Correlations at the agency level were done using the PROC CORR procedure of the SAS ® software system.

Other System Changes: Analysis of data to answer the second study question also focused on system change, in this case, in the types of housing provided, in the degree to which information systems were implemented, and in the extent to which evidence-based mental health practices were deployed. The analyses of these data were similar to those described for measures of integration. Separate HLM models were examined in which the dependent variables were the measures of system performance and the independent variable represented the year of the study.

Site and Agency Change: The third system-level question concerned variation across sites and agency types in integration efforts and outcomes, and in other system-level performance measures.

In the analysis of the degree to which the 12 system performance measures changed to different degrees across sites, analysis of covariance was used to test the relationship between a categorical variable with 11 levels representing the site of the responding agency, a variable representing time, and the variable of primary interest, a term that represented the interaction of site and time. These analyses thus examined whether there were significantly different degrees of change over time between sites, on each dependent measure. For those dependent measures in which the interaction term was significantly related to the dependent measure (with an alpha level set to less than 0.15 because of the small number of observations), we further examined each measure over time to identify specific sites at which there was statistically significant change over time. The PROC GLM procedure of SAS ® was used to conduct these analyses.

The same analytic approach was used to examine variation across agency types over time on these measures. In these analyses a measure that represented the type of respondent agency was substituted for the measure of site. This measure is a categorical variable that represents each of the seven types of agencies, i.e., the lead agency; the local Veterans Health Administration facility; or an agency that provided housing assistance, mental health care, substance abuse services, primary care, or another service.

Fiscal Integration. The final system level question addressed the association between the presence of a funding relationship between each pair of agencies and the degree of joint planning and cooperation, as well as trust and respect. In this analysis, we first used HLM to examine two models in which the independent measure was the dyadic indicator of the existence of a fiscal relationship. Random effects were modeled for site, agency, and time. An additional set of models addressed the interaction of fiscal relationship and time. These models explored whether there were differences in the amount of change in the dependent measures between dyads with and without a fiscal relationship.

View full report


"report.pdf" (pdf, 296.36Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report


"apa.pdf" (pdf, 145.6Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®