A brief set of individual characteristics were measured through a structured screening form administered at first contact, prior to enrollment into the program. This form also recorded the date of enrollment in the program.
Further data were obtained during the baseline interview among those who agreed to participate in the evaluation. These data were collected to allow comparisons between people screened but not enrolled into the program and those who were enrolled, both program-wide and at each participating site, as well as a comparison of enrolled clients who did and who did not give consent to participate in the evaluation.
Due to the intensive and comprehensive nature of the Collaborative Initiative and the heterogeneity of the target population, an extensive array of service use and client outcome measures were chosen to document the diverse processes and outcomes of the program.
Receipt of four types of services typically needed by chronically homeless people were documented: primary health care services, mental health & substance abuse treatment, case management, and the overall integration of service delivery. The following seven client outcome domains were examined: housing status, community adjustment, mental health status, substance use, physical health status, income, and health care costs.
A brief explanation of each of these measures follows.
CICH intake and outreach staff completed a structured form on each person screened, which documented: 1) basic socio-demographic information (i.e., age, gender, and race/ethnicity); 2) eligibility characteristics (i.e., single individual vs. parent, and type of chronic homelessness experience a current episode of homelessness lasting a year or longer vs. 4 or more episodes of homelessness during the past 3 years); 3) each of three disabling condition(s) (i.e., mental health, substance abuse or medical) identified at screening and confirmed by evaluation staff by clarifying those conditions noted by clinicians at screening with program intake staff and by asking clients during the baseline assessment interview; 4) outreach location and the agency initiating outreach; 5) outreach clinician observations of clinical problems; 6) response to early interactions with intake/outreach staff; and, 7) date enrolled into the program (if applicable). Measures of time (days) from screening to enrollment (among those enrolled into the program), and time from enrollment to the baseline assessment interview (among CICH clients) were also documented.
Supplemental socio-demographic and clinical information (e.g., veteran status, disabling condition(s) were collected following written informed consent, along with service use and client outcome data.
Primary health care services
First, CICH clients were asked whether they had a "usual health care provider", and whether they had health insurance?
Then the number of routine, preventive healthcare procedures received during the past year was assessed from a list of 12 gender-neutral procedures, plus either four male-specific procedures or two female-specific procedures (Gelberg, 2003). Examples of gender-neutral procedures include measurement of height, weight, blood pressure, cholesterol, hearing and vision, testing for diabetes, tuberculosis, hepatitis, and routine dental care. Male-specific procedures included prostate exam, rectal exam, colonoscopy, and PSA testing. Female-specific procedures included PAP testing and breast exam.
Clients reporting unhealthy behaviors were also asked the number of such behaviors that they had discussed with a healthcare professional during the previous year. These behaviors included drinking alcohol among drinkers, smoking among smokers, and diet/nutrition among those who were obese at baseline.
The total number of outpatient medical visits made during the past 3 months was also included as a primary health treatment measure.
Finally, the trust in physician scale (Anderson & Dedrick, 1990), an 11-item measure, was used to assess the level of trust felt by a patient with his/her primary doctor/physical healthcare provider. Item responses range from 1=totally disagree to 5=totally agree. The scale score is computed as the mean response to these 11 items, with a higher score reflecting greater trust felt by CICH clients and their primary provider (alpha=.91). Sample items include: "My health care provider is usually considerate of my needs and puts them first", "I trust my health care provider so much that I always try to follow his/her advice," and "If my health care provider tells me something is so, then it must be true".
Mental health services and substance abuse treatment
Clients were further asked whether they could identify a primary mental health or substance abuse treatment provider, as well as the total number of outpatient mental health visits and the total number of outpatient substance abuse treatment visits received during the previous 3 months. A fourth measure addressed participation in self-help groups (i.e., Alcoholics Anonymous or Narcotics Anonymous)
A 7-item therapeutic alliance scale was used to measure the strength of the therapeutic relationship experienced by CICH clients with their primary mental health or substance abuse provider (Neale & Rosenheck, 1995). Scale scores range from 0 to 5 reflecting greater strength of relationship, and were calculated as the average response to the 7 items (alpha=.94; mean=4.4; range 0-6). For example, clients were asked, "How often does your provider perceive accurately what your goals are?", and "How often are the goals of your work with your provider important to you?".
Finally, clients' experience of personal choice in selecting mental health or substance abuse services was measured using a 5-item "consumer choice" scale (Monahan et. al., 2005), which was computed as the average response (alpha=.89; mean=4.0; range=1-5), ranging from 1=strongly disagree to 5=strongly agree, with higher scores reflecting greater client choice. The scale included the following items: 1) I felt free to do what I wanted about going for treatment; 2) I chose to go for treatment; 3) It was my idea to obtain treatment; 4) I had a lot of control over whether I went for treatment; and, 5) I had more influence than any one else on whether I went for treatment.
Clients were also asked whether they could identify a primary case manager, and whether they were visited by a case manager in a community setting (i.e., either at home or at some other place in the community other than a service agency or healthcare facility setting) during the previous 3 months.
Clients were further asked whether they had a money manager ("a person or organization which helps you manage your money") and whether they had had any contact with their landlord, either in-person or by telephone, during the past 3 months? Money management has been identified as an important ingredient in the approach to helping homeless people developed by Tsemberis and colleagues (2000) and landlord-tenant relationships have been found to be associated with housing outcomes among persons with mental health and/or substance abuse problems (Kloos et. al., 2002).
Both objective and subjective measures were used to evaluate the integration or coordination of diverse CICH services. Services integration is defined here to be the extent to which the key components characterizing the CICH intervention (listed below) were provided to clients. An objective measure of overall services integration was based on calculation of the proportion of total component services received by clients using a series of six dichotomous service component measures including: 1) independent housing 2) case management, 3) general medical care, 4) substance abuse treatment, 5) mental health services, and 6) VA services. If the client did not have need of a particular service or was not eligible to receive VA services, the score for that particular item was coded as missing. Scores on non-missing values were averaged and the total ranged from 0 to 100%, with a higher score representing more fully integrated service delivery (overall mean at baseline=.64; sd=.23).
A second objective measure of overall service delivery represented the total number of outpatient health visits of all kinds received during the previous 3 months. This measure was calculated as the sum of medical visits, mental health visits, and substance abuse treatment visits. Clients were further asked to estimate the total number of different individual service providers assisting them during the past 3 months.
To supplement these objective measures, a 5-item subjective scale was developed to measure the extent to which the delivery of these services was perceived to be well coordinated or fragmented. The five component items included one which asked about the coordination of services ("How often do providers work together to coordinate your care?", and four which addressed fragmentation (i.e., "How often is one provider unaware of information about your care that another provider has?", and "How often do these providers seem unaware of what the others are doing for you?", and "How often do you have to tell the same information to several providers?"). The response set to these items was 0=rarely, 1=sometimes, and 2=often, and the scale score is the average response across the 5 items, after recoding fragmentation items so that higher scores reflect a higher degree of coordination of services and less fragmentation (alpha=.80; mean=1.3; range=0-2 over all time points).
The primary goal of the Collaborative Initiative was the rapid and sustained placement of chronically homeless people into long-term housing. Clients were asked at each interview the number of days during the past 3 months that they were housed in each of nine settings, as well as where they were residing at the time of each assessment. The number of days housed was defined as living in their own place, someone else's place, or in an SRO hotel or boarding home. SRO hotels were considered residences because some sites used such housing as the primary housing resource for CICH clients (e.g., Los Angeles and San Francisco). Nights spent in shelters, outdoors, or in vehicles or abandoned buildings were classified as representing "homeless" housing status.
Clients who were living in their own places were also asked to report their level of satisfaction with their housing using a 20-item housing satisfaction scale developed by Tsemberis and colleagues (2003), as part of the SAMHSA Supported Housing Initiative (CMHS, 2001). Responses to these items ranged from 1=very dissatisfied to 5=very satisfied, so the higher the score, the greater the satisfaction with housing. The scale score was calculated as the mean of the responses to these 20 items (alpha=.89). Sample items include, "The amount of choice you had over the place you live", "How close you live to family and friends", "The safety of your neighborhood", and "The amount of privacy you have".
To evaluate how well CICH clients were integrated into and engaged in community life, they were asked whether they had participated in each of 16 common activities (e.g., visiting with close friends/relatives/neighbors, going to a grocery store, restaurant, retail store, bank, movie, library, park, or reading a newspaper) during the previous 2 weeks (Katz, 1963). A "community involvement" scale was then calculated as the total number of these activities, and ranged from 0 to 16, with a higher score indicating greater participation in community activities.
Social support networks were assessed by questions asking the number of types of persons who would be available to help them about three different types of assistance: a short-term loan of $100, a ride to an appointment, or someone to talk with if they felt suicidal (Vaux et. al., 1987).
Additional single-item measures addressed a) whether clients knew any of their neighbors well, b) the number of days spent in jail during the past 3 months, and c) satisfaction with life overall (subjective quality of life), scored on a 7-point terrible=1 to delighted=7 scale, with a higher score indicating greater satisfaction with life. (Lehman, 1988).
CICH client income was expected to rise as a result of participating in the program, both through increased access to public support benefit payments and through employment. Clients were asked whether they had received any of several types of public support income during the past month, and if so, the amount of such income. Information on days of employment and employment income were also obtained, along with informal types of income. Responses to these items were summed to create a measure of total income.
Mental health and physical health status
The Medical Outcomes Study Short Form (SF)-12 mental health subscale (Ware et. al., 1998), three subscales from the Brief Symptom Inventory (BSI) (Derogatis & Spencer, 1982), and an observed psychotic behavior rating scale (Dohrenwend, 1982) were used to evaluate mental health status. The SF-12 mental and physical health subscales assess the overall level of functioning in their respective domains, with scores ranging from 0 to 100, and a score of 50 representing normal level of functioning in the general population. Each 10 point-interval represents one standard deviation (above or below) the general population norm.
Three BSI subscales were used reflecting major domains of subjective distress: psychoticism (e.g. hallucinations, delusional beliefs, disorganized thinking), depression, and anxiety. The BSI score presented in this report is the mean value for these three sub-scales, ranging from 0=never experience symptom to 4=very often experience symptom.
Finally, a measure of observed psychotic behaviors included 10 types of psychotic behavior observable by evaluation staff during the course of baseline and follow-up interviews (e.g., auditory or visual hallucinations, delusions, agitation/aggression, inappropriate behavior or speech, incoherent speech). Each of these behaviors was coded 0=not at all to 3=a lot, based on staff observations, and the total scale score was computed as the average score across these 10 items (alpha=.76; mean=0.2; range=0-2.6 over all time points).
Items from the Addiction Severity Index (ASI) documented the number of days each client drank to intoxication and whether they had used any illicit drugs during the previous month. Alcohol and drug sub-scales (McLellan et. al., 1980) measured alcohol and drug use problems using multiple items combined in a standard comparable score ranging from 0 to 1, and which included items such as, "Do you feel like you have a problem with alcohol now?", "How many days in the past 30 have you experienced drug problems?", and "How troubled or bothered have you been in the past 30 days by your own alcohol/drug problems?". A higher score on an these ASI sub-scales reflects a greater, more serious substance use problem.
Service costs were estimated for four aggregated types of care: medical/dental treatment, mental health services, substance abuse services, and the total for all three types of services. These estimates were computed by multiplying the number of visits/days of care reported by standard estimates of the unit cost of each type of care. Unit costs were estimated on the basis of data average unit cost data compiled for a recent NIMH funded cost effectiveness study of treatment of schizophrenia (Rosenheck et. al., 2006) as follows:
- Inpatient medical = $1,866; emergency room medical = $96; outpatient medical = $70; dental = $50
- Inpatient mental health = $1,059; emergency room mental health = $86; outpatient mental health = $82; day hospital mental health = $329; drop-in mental health = $82; consumer support mental health = $16
- Inpatient substance abuse = $435; emergency room substance abuse = $86; outpatient substance abuse = $25; residential treatment for substance abuse = $40; AA/NA = $16
For example, if during the previous 3 months a CICH client reported spending three days in the hospital, two days in the emergency room, and had six outpatient visits for a physical health problem, and one dental visit, then his/her medical/dental quarterly service cost for that quarter would have been $6,260 ((3*$1,866) + (2*$96) + (6*$70) + (1*$50)).
Total health costs were also sub-grouped across types of service into inpatient and outpatient costs. Inpatient costs were calculated as the sum of the following five service costs: inpatient medical, mental health, substance abuse, day hospital, and residential treatment. Outpatient costs were similarly calculated as the sum of each of the various types of outpatient costs multiplied by their unit costs listed above.