[ Full Report | Contents of Full Report ]
The purpose of this study was to determine the evaluability of discharge planning as a strategy to prevent homelessness. In 1994 the Federal Interagency Council on Homelessness identified inadequate discharge planning as a significant factor contributing to homelessness among persons with mental illnesses and/or substance use disorders. The prevention of homelessness is a key goal in the U.S. Department of Health and Human Services (DHHS) action plan to end chronic homelessness.(1) This study is related to a strategy in the plan that recommends identifying and promoting the use of effective, evidence-based homelessness prevention interventions.
Past research has indicated that many people with severe mental illnesses and substance abuse problems who experience homelessness travel in “institutional circuits,” or move repeatedly through systems and institutions such as state psychiatric hospitals, jails and prisons, homeless shelters, and drug treatment programs. However, although discharge planning is often recommended as a strategy to prevent homelessness among people released from institutions or youth aging out of foster care, very few studies have examined this strategy. The Assistant Secretary for Planning and Evaluation of DHHS sponsored this study in order to build knowledge for researchers and policy makers in the field of homelessness regarding the evaluability of discharge planning in institutional and custodial settings. The four institutional and custodial settings listed below were included in this study because previous research has indicated that many of those entering shelters have recently come from one of these settings.
Some of the key research questions of this study are listed below; a more exhaustive list can be found in this report's Methods section.
[ Go to Contents ]
The study involved the following components:
The primary data sources for this study included the documentary analysis materials; staff discussions at the primary and affiliate agencies of the site visit programs; and a review of procedures, forms, and a limited number of medical records (with client consent). The study has a number of methodological limitations. Most important, findings are based on qualitative examination of a modestly sized convenience sample of program sites selected because experts thought that their discharge planning processes were superior to other programs. The sites studied are examples from four extremely different and diverse settings. We did not interview clients, nor did we systematically examine quantitative data to confirm staff reports.
[ Go to Contents ]
In this Executive Summary we present only the key findings that hold across
all four settings. Please refer to the Analytic Findings section of the report
for key findings specific to each setting.
The study team concluded that a rigorous summative evaluation (i.e., an outcome
or impact evaluation) of discharge planning as a strategy to prevent homelessness
in institutional and/or custodial settings is not justified at this time.
The recommendation against conducting a summative evaluation of discharge
planning as a strategy to prevent subsequent homelessness is based on the
findings that discharge planning is not readily separable from the broader
program, that it is not well defined or consistently implemented, and that
a summative study would be costly and is premature given the state of knowledge
in this area. However, we found that alternative study designs to evaluate
specific issues or activities related to discharge planning and homelessness
prevention are feasible and justifiable, and these designs are described
in this report.
A key evaluability question for each of the four settings is whether the
discharge planning process is separable from the broader program in which
it operates. Many discharge planning activities, such as client assessment,
are also critical to treatment planning and are often performed by the same
staff. While there are distinct, identifiable activities associated with
discharge planning, they take place within the context of the broader treatment
or service delivery process and cannot be clearly separated from that context.
For example, the primary goal of a residential substance abuse treatment
program is to reduce or eliminate a client's dependence on alcohol or other
drugs.(2) However, the intervention of interest
for the evaluability assessment is not the entire treatment (which includes
discharge planning) provided in the residential substance abuse treatment
program. Rather, the evaluability assessment focuses on the discharge planning
process alone; other activities that occur in the residential substance abuse
treatment program would be mediating variables in an evaluation of the discharge
planning process.
Few programs appear to have a well-designed and integrated model of the discharge planning process, nor have they implemented the process in a systematic manner likely to produce consistent results over time or across clients. Although most programs do have at least rudimentary discharge planning procedures and forms, few of the programs examined have a written protocol to ensure that staff members apply the interventions uniformly or document discharge planning processes well. None of the 19 programs studied used screening instruments to identify clients at risk of homelessness and in need of intensive discharge planning efforts.
In addition, most programs examined lack rigorous staff training and quality assurance activities in support of discharge planning. As a result, the discharge planning process is inconsistently applied within each program. Likewise, programs collect very little systematic postdischarge data that could create a feedback loop to improve the discharge planning process over time.
The discharge planning process consists of an imprecisely defined set of
activities. While some governmental and professional organizations have developed
consensus standards on what constitutes a model discharge planning
process,(3) we found little evidence that these
models have been effectively disseminated or widely implemented. No studies
have yet tested the effectiveness of these models in actual practice. Critics
have suggested the models were not attuned to “real world” scarcity
of housing and other resources, or to the tendency of organizations to pursue
self-interest rather than collaborate effectively.
The study team found that avoiding homelessness, the outcome of interest,
is determined as much or more by the availability of suitable housing and
support services in the community as by the discharge planning process. An
example of this was found in the most well-structured and best implemented
discharge planning process observed in this study. This model discharge planning
process was implemented in a rural community so lacking in housing options
that many clients were placed in large congregate semi-institutional conditions
upon discharge. The best discharge planning process cannot overcome a lack
of community housing and services.
The tremendous variability in the discharge planning process across clients,
programs, settings, and communities dictates that a summative evaluation
enroll thousands of clients across many programs. The discharge planning
process is highly complex and tightly bound to programmatic, client,
interorganizational, and community resource factors. Numerous mediating variables
affect the discharge planning process and its outcomes; some of these variables
lack well-formulated measures. A summative evaluation of the discharge planning
process as a strategy to prevent homelessness would be complex, lengthy,
costly, and could not be assured of producing clear and definitive findings.
Further preliminary and exploratory research is called for before undertaking
a study of such complexity and resource requirements.
Within a single residential or custodial program, there are several intervention
and discharge planning “tracks” depending on who pays for or oversees
a client's care and the community programs for which they are eligible. These
tracks often result in differences in clinical interventions, lengths of
stay, agencies involved in the discharge planning process, community housing
alternatives, and available community services. For example, the treatment
and discharge planning process a patient receives in a psychiatric hospital
will be determined in part by whether the person meets eligibility criteria
for Supplemental Security Income (SSI), Medicaid, a private insurer, the
state mental health department, or is self-pay and of limited means. For
youth in residential treatment centers, the discharge process is partially
determined by who has custody of the youth — the family, child
welfare, or juvenile justice. The availability of services in the patient's
community of residence is another important determinant. Each funding source
or community program may have its own complex eligibility standards and
application process that is time-consuming to negotiate. An evaluator of
the discharge planning process must address the complexity of multiple discharge
planning tracks that results from the involvement of these diverse payers
and community contexts.
In most cases the discharge planning process is funded as incidental to the treatment and custodial care, often as part of a bundled per diem rate. Any study will face challenges in determining financial expenditures for discharge planning activities since they are not billed separately and are usually carried out by the same program staff who provide clinical care.
[ Go to Contents ]
If a study of discharge planning is conducted, a number of key study design and measurement issues will need to be considered. The initial bullet points below address factors critical to designing a study, while later bullets discuss mediating variables that would need to be controlled for in a summative evaluation and other design considerations. In most cases, there are existing measures that could be used, although they will require some adaptation depending on the study context and setting. There are some key concepts, such as the availability of appropriate community housing, that will have to be carefully negotiated and for which no definitive measures are readily available. This section outlines some of these measurement, data collection, and other design issues.
The sample size needed will depend on the purpose of the study. A sample of several thousand will likely be necessary to achieve sufficient statistical power if a summative evaluation is the goal. A more modest sample size of 100 or fewer might suffice if the purpose of the evaluation is formative or exploratory. Although obvious, it is important to state that precise calculations of sample size and statistical power will require a clear articulation of study goals and design.
The study should recruit clients who are at significant risk of homelessness. Some programs serve only those who are homeless or at high risk of homelessness; however, most of the programs examined serve a significant proportion of clients who typically return to stable housing after the conclusion of their residential stays. Clients should be screened and included in an evaluation study only if they meet some risk threshold for homelessness.
Early terminators — those who leave programs after a brief stay, sometimes against professional advice — present particular challenges to any evaluation study of discharge planning. Some programs have high early termination rates (50 percent or higher) and followup data on these clients are often limited. These clients may be at the greatest risk of homelessness, yet are least likely to receive adequate treatment or discharge planning because of their early termination. They may also be more difficult to enroll and follow in a tracking study, but are critically important to include if the goal is to prevent homelessness.
If a summative evaluation study of discharge planning in any of the four settings is undertaken, the study team recommends a followup period of 1 year or longer. The rationale for this position is based on two observations. First, the short-term base rate of homelessness following discharge is relatively modest in many settings, even given flawed, “non-exemplary” discharge planning processes. This rate can be highly variable across programs depending on the characteristics of the program and the population it serves, the availability of housing and services in the community, and other factors. Unless the followup period is extended to a year or more to allow a longer period for measuring homelessness (since the risk of homelessness often increases with time), it may not be possible to distinguish the impact of “exemplary” discharge planning in further lowering that already modest rate of homelessness. The second rationale relates to the ability of a study design to differentiate the effects of the discharge planning process from the progression of a disorder or the course of maturation. In the case of an individual in acute care treatment for mental illness and/or substance abuse, a short-term followup study of discharge planning runs the risk of confounding “natural relapses” with the outcomes of an inadequate discharge planning process. In parallel fashion, for studies of youth “aging out” of care, a longer followup period is necessary because of the possible confounding of developmental changes with factors attributable to discharge planning. A longitudinal design of a year or more allows for examining multiple transitions across settings and levels of care and better distinguishes between factors associated with the natural course of the individual's disorder or maturation and those factors attributable to the discharge planning and transition process.
Conversely, if the evaluation is formative or exploratory in nature an argument can be made for conducting a study that examines only what happens at the immediate point of discharge. In that case, the housing measure is not residential stability, but only what setting the client is placed in on the day of discharge. The measure of service linkage is not attendance at scheduled appointments, but only that appointments are made and the client informed. These types of measures are clearly insufficient to assure residential stability in the community but could, in conjunction with a variety of other measures, provide rough indicators of the quality of a program's discharge planning process.
Any evaluation study examining homelessness outcomes will require a resource commitment to develop the infrastructure for data collection and client followup. Existing program data (hardcopy and electronic medical records) contain some but not all the information needed to conduct a discharge planning evaluation, and these data are of varying quality. Furthermore, most programs do not have the resources to follow up with clients after discharge or collect data on followup outcomes. The response rates for programs that do collect data on followup outcomes are inadequate for a rigorous evaluation. If a study is conducted it will be critical to tighten procedures and provide additional resources for program data collection and followup up; alternatively, the data collection activities could be contracted to external organizations with expertise in this area.
The four broad program settings addressed in this study actually represent many discrete types of programs. This observation suggests that care must be taken in determining which programs to compare in a discharge planning evaluation, and in generalizing evaluation findings from one program setting (or subtype within a setting) to another. For example, the psychiatric inpatient treatment settings category includes state psychiatric hospitals, psychiatric units of general hospitals, and free-standing private psychiatric hospitals. Yet these three types of psychiatric inpatient units differ in many important respects, including characteristics of clients served, length of stay, staffing patterns, risk of subsequent homelessness, and form and extent of linkage to community agencies, all of which bear upon the discharge planning process. Similar distinctions are apparent between subsets of programs within the other three settings.
Unlike the other three settings in the evaluability assessment, foster care independent living programs are not primarily “treatment” programs, but are fundamentally about assisting youth to make a transition to living independently in the community. Although these programs have processes that correspond to each element of exemplary discharge planning, they are unique in many respects and are subject to a range of particular influences and constraints. It would be particularly questionable to generalize findings from foster care independent living programs to other settings, or from the other settings to foster care.
Differences in client characteristics affect the discharge planning process, options available to discharge planners, and postdischarge outcomes. Key client characteristics that should be measured in a discharge planning evaluation include presence of mental illnesses, substance use disorders, physical disabilities, developmental disabilities, co-occurring disorders, current and historical involvement with criminal or juvenile justice, and past success in mental health or substance abuse treatment. Critical factors for youth in foster care include the age of emancipation and educational attainment, as well as the presence of serious emotional disturbances, substance use disorders, and developmental disabilities. These individual history, demographic, and clinical characteristics also affect a client's eligibility for entitlements and services upon discharge, and influence discharge planning outcomes.
Another critical variable is the availability of appropriate housing and supports in the communities the programs serve. The arrangement of stable housing and other needed services depends not only on the quality of the discharge planning, but also on the availability of appropriate resources in the community. Even in exemplary discharge planning programs, the outcomes achieved can be disappointing if the housing resources and services are not available.
Each program is defined in part by the larger policy context in which the program operates (e.g., contractual obligations; accreditation standards and requirements; and state laws, rules, and regulations). The regulatory and accrediting bodies, like the payers, influence the conditions in the program and the discharge planning process. Their policies help determine which services are provided, how discharge planning activities are implemented, and who provides oversight.
The program's relationship to other organizations is also an important factor. If the treatment or custodial program is part of an umbrella agency that also provides outpatient care or housing, it may be easier to link clients to those intramural services. Similarly, if the program has invested in strong and trustful working relationships with community partner agencies, the housing and services provided by those agencies may be more easily accessible.
[ Go to Contents ]
We have identified at least four possible study designs, detailed in the full report, that would advance the field's understanding of discharge planning as an intervention to prevent subsequent homelessness. These studies are:
[ Go to Contents ]
1. U.S. Department of Health and Human Services (DHHS). (2003). Ending chronic homelessness — Strategies for action: A report from the Secretary's Workgroup on Ending Chronic Homelessness. Washington, DC: Author.
2. Center for Substance Abuse Treatment (CSAT). (1995). Detoxification from alcohol and other drugs, Treatment Improvement Protocol (TIP) #19. Department of Health and Human Services (DHHS Pub. No. BKD172). Substance Abuse and Mental Health Services Administration (SAMHSA).
3. Substance Abuse and Mental Health Services Administration (SAMHSA, 1997). Exemplary practices in discharge planning. Rockville, MD: Author.; American Association of Community Psychiatrists (AACP, 2001a). Continuity of care guidelines: Best practices for managing transitions between levels of care. Dallas: Author.; American Association of Community Psychiatrists (2001b). Continuity of care guidelines for addictions and co-occurring disorders. Dallas: Author.; Osher, F., Steadman, H. J., & Barr, H. (2002). A best practice approach to community re-entry from jails for inmates with co-occurring disorders: The APIC model. Delmar, NY: The National GAINS Center.
Main Page of Report | Contents of Report
Home Pages:
Human Services Policy
(HSP)
Assistant Secretary for Planning and Evaluation
ASPE)
U.S. Department of Health and Human Services
(HHS)