Given the paucity of current documentation of system change and its effects, we cannot reliably identify gaps in knowledge; instead, we must point to a broad array of important questions for which we have mostly anecdotal answers. With more than 300 communities around the country developing 10-year plans to end homelessness (see Interagency Council on Homelessness Web site (www.ich.gov), and at least 90 of them promulgating those plans and taking some steps toward implementing them (Cunningham, et al., 2006), and independent efforts to change systems in many locations, there is great need to evaluate the impact of these efforts and the factors that were most important in shaping (or blocking) that impact. We know of no plans to do so, beyond the self-assessments that are likely to show up in annual applications for continuum-of-care funding in response to HUD requirements. It would indeed be a great shame if, at the third National Symposium on Homelessness Research 10 years from now, there is no more systematic research evidence for the impacts of system change than we have been able to report here.
This is an area where practitioners and advisors abound, but hard evidence is elusive. In this paper we have tried to lay out some basic answers to a preliminary question: How will we know that systems have changed? We must go on from there, to design and fund research that answers the following questions:
- Which actions were pivotal in achieving change?
- What level of effort, staffing, and political will is required to implement change?
- How will we know we are making progress toward ending homelessness, and will we be able to say with confidence that systems change contributed to any observed reductions?
- Are some systems more important to change than others in community efforts to end homelessness?
- How much of any change we observe will we be able to attribute to the introduction of more effective program approaches (e.g., adoption of best practices), compared to streamlining system processes?
- How much of any change we observe will we be able to attribute to the structure of the change effort? As one reviewer noted, the communities currently organizing for ending homelessness or chronic homelessness are using very different structures. The motive for seeking an answer to this question would be to help communities just starting on the process to select the most effective structure. The suggested structures include:
- A centralized, coordinated, state-to-local model (Utah),
- A two-tier framework that aligns state with locals to implement state framework (Michigan, Missouri, North Carolina),
- Integrated CoC and 10-year plan efforts (District of Columbia, Contra Costa County, California) that plan to implement across multiple populations all at once, and
- An incubator approach (Montana).
Individual communities can and should implement formative and summative evaluations using the frameworks and methods described in the Evaluation Processes section of this paper. In addition, a systematic system change-oriented multisite, multi-year research project similar to the ACCESS evaluation, set up to measure a spectrum of change processes and their impacts over time, could be very fruitful.