HUD/HHS/VA Collaborative Initiative to Help End Chronic Homelessness: Preliminary Client Outcomes Report. Statistical Analysis


Characteristics of those screened, enrolled in the program, and consenting to the evaluation

Basic demographic and clinical characteristics of all individuals screened for possible enrollment into the program are presented, for both the entire sample and for each site. Analysis of variance and chi square tests are used to identify the significance of differences in individual characteristics across sites at screening.

Next, demographic and clinical characteristics of individuals screened but not enrolled into the project are compared with those of individuals who enrolled in the project and with those who gave consent to participate in the evaluation, again using data from the screening form with the significance of differences evaluated by chi-square and independent samples t-tests.

We next present baseline characteristics of all CICH clients participating in the evaluation with a comparison of differences across sites, using data obtained from the baseline interview. The significance of differences across sites were evaluated using one-way analysis of variance (ANOVA) and chi-square tests.

Service use and outcome: Change over time

Repeated measures mixed regression models then were used to identify changes in service use and client outcomes. Mixed model linear regression is a statistical technique used to analyze longitudinal outcome data in which follow-up observation data points are not independent (i.e. they are correlated because they pertain to the same individual/client). Thus, a distinctive feature of these models is that they adjust the standard errors, the spread of follow-up data around the group mean for the correlation of data within individual clients (Bryk & Raudenbush, 1992).

The mixed linear regression models used in this report generally included one or more class variables, representing time (e.g., baseline, 3-month, 6-month, 9-monthfollow-up) and client sub-groups of interest (e.g., housed at baseline vs. not housed, male vs. female, etc.). They also included one or more covariates representing potentially confounding baseline characteristics (e.g., site or client socio-demographic characteristics bivariately associated with service use and outcomes measures, including the baseline values of client outcome measures). We present least square means which are adjusted for these covariates. Some mixed regression models examined the main effect of time alone, while others examined the 2-way interaction effect of time and various client sub-groups. Main effect statistics representing time are used to determine whether there was significant change in a given measure over the period of the evaluation, while interaction statistics compare patterns of change over time between client sub-groups (e.g., whether clients housed at baseline or other subgroups showed a greater or lesser rate of improvement over time than other participants).

Two sets of mixed models were developed. The first set of models was developed to examine changes in the use of services and in client outcomes over the 12-month follow-up period for all evaluation subjects, without considering whether they were housed at baseline or not. For each measure, a single class variable represented the main effect of time and was coded 0 for the baseline observation, and 1 through 4 for 3-month, 6-month, 9-month, and 12-month observations, respectively. Covariates used included 10 site dummy codes to adjust for site effects for all service use and outcome measures. Covariates used with client outcome measures also included the baseline value of the measure, and 11 eligibility and baseline socio-demographic characteristics. Least square means, adjusted for covariates and site effects are reported for each time point. Statistics indicating whether significant trends (upward or downward) were observed among measures of service use and client outcomes over time are also reported.

Mixed models with weighted observations (marginal structural models) were used to examine the possible influence of attrition (loss to follow-up) on service use and outcomes (Robins & Finkelstein, 2000). Logistic regression was first used to model successful follow-up at each time point, using the set of 11 eligibility and baseline socio-demographic characteristics to estimate the predicted probability that each interview would be completed. Observations were then weighted by the inverse of the predicted probability that they would be completed so that interviews completed with people whose baseline characteristics suggest a low probability of follow up were given more weight than those more likely to be completed.

A second set of mixed regression models is presented to determine the extent to which housing status at baseline or other baseline characteristics were significantly associated with changes in service use and/or outcomes over time. In these analyses housing status at baseline was added as a second class variable, coded "1" for clients housed at baseline and "0" for others. Baseline housing status, group, and group x time interactions were both examined. Baseline differences between the housed and not housed groups identified as significant on bivariate analyses (using ANOVA analyses) were included as covariates. Least square means for each time point, for each housing status group, are reported, along with fixed effect statistics representing the significance of the interaction of housing group and time.

Site differences in service use and outcomes

Differences in patterns of service use and outcomes across sites were evaluated using mixed models that included main effect for site, for time, and a term presenting site by time interaction.

To determine whether the differences in patterns of service use account for the differences in outcome across sites, we re-tested the significance of differences in outcomes across sites including measures of service use as covariates.

Group differences in baseline characteristics, service use and outcomes between CICH clients and comparison group clients.

Bivariate differences on over 250 baseline characteristics between CICH client subjects (N=296) and comparison group (N=118) subjects at the five comparison group CICH sites were evaluated using independent samples t-tests. Significant differences between the two groups were found on 80 of the 253 baseline characteristics examined. Descriptive statistics were then run on these 80 baseline measures among the combined sample of CICH clients and comparison subjects (N=414) to identify measures having larger amounts of missing data. Eleven measures were found to have fewer than 380 observations, due largely to skip patterns within the baseline assessment form, and were therefore excluded from subsequent logistic regression analyses. Sixteen other baseline measures were also excluded due to redundancy with other variables. Logistic regression was then used to select a smaller, more parsimonious set of baseline covariates among the remaining 53 bivariately significant measures to be included in subsequent multivariate mixed model analyses. The 53 baseline measures were entered as a single block of independent variables, with group (coded as 1=CICH client, 2=comparison group subject) as the dependent variable. The stepwise method of selection was used, both forward and backward.

The stepwise forward selection method identified 10 measures in the final model. When stepwise backward selection method was used, the same 10 measures were identified along with four additional measures, for a total of 14 baseline covariates. These covariates included category of chronic homelessness (i.e., experiencing 4 or more episodes of homelessness during the previous 3 years); race/ethnicity (i.e., Asian/Pacific Islander); work history (i.e., primarily unemployed during the previous 3 years); access to healthcare (i.e., having health insurance and a usual source for medical care, and the total number of healthcare providers); mental health diagnoses (i.e., bipolar or anxiety disorder, and total number of mental health and substance abuse diagnoses); receiving supportive services (i.e., housing, vocational rehabilitation, case management visits in the community); number of days hospitalized; and, spirituality. On most measures comparison clients have less severe health problems and better community adjustment.

Multivariate mixed models were then used to evaluate differences between CICH and comparison clients in service use and client outcomes during the first 12-months of CICH program, as described earlier. The class variables in these mixed models included group, time, and the group*time interaction. Covariates included the baseline value of the dependent variables, four site dummy codes (with New York being the reference site), and the 14 baseline covariates identified above.

View full report


"report.pdf" (pdf, 1.49Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®