Findings from a Study of the SSI/SSDI Outreach, Access, and Recovery (SOAR) Initiative

Final Report to:
Office of the Assistant Secretary for Planning and Evaluation (ASPE)
Substance Abuse and Mental Health Services Administration (SAMHSA)
Health Resources and Services Administration (HRSA)
U.S. Department of Health and Human Services (HHS)

Prepared by:
Jacqueline Kauff, Jonathan Brown, Norma Altshuler, Noelle Denny-Brown, and Emily Sama Martin
Mathematica Policy Research, Inc.

Under Contract No:
223-02-0086/HHSP233200700007T

Additional information regarding this project can be obtained from the Federal Project Officers:
Flavio MenascГ© (202-260-0384, Flavio.Menasce@hhs.gov),
Lynnette Araki (301-443-6204, LAraki@hrsa.gov), and
Mattie Cheek (240-276-1745, Mattie.Cheek@samhsa.hhs.gov).

"

Introduction

The Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI) programs provide critical income support for those who meet eligibility requirements. For individuals or families who are homeless, receiving SSI or SSDI is often an important first step in improving their life circumstances. SSI/SSDI cash assistance provides financial resources for housing and facilitates eligibility for Medicaid, which enables beneficiaries to access critical medical and mental health services. SSI/SSDI recipients typically have access to employment assistance programs offered through agencies such as vocational rehabilitation or alternative providers and the Ticket to Work program. However, the combination of disabilities that often include mental health and/or substance abuse problems, and the precariousness of the living situations of individuals who are homeless make it difficult for them to successfully complete the SSI/SSDI application process.

The SSI/SSDI Outreach, Access, and Recovery (SOAR) initiative aims to improve access to SSI/SSDI benefits for individuals who are homeless through a multi-pronged strategy designed to mitigate the challenges this population faces when navigating the SSI/SSDI application process. One main aspect of the SOAR initiative is the provision of technical and strategic planning assistance to help states develop policies and procedures that will aid individuals who are homeless obtain SSI/SSDI. Such assistance aims to bring social service providers, advocates for the homeless, and other state and local agencies together to determine how to create an effective system for obtaining SSI/SSDI. A second important aspect of SOAR is training case managers, social workers, and other staff in Stepping Stones to Recovery (SSTR)  a curriculum designed explicitly to provide case managers and other advocates with the skills and information needed to support individuals who are homeless through the SSI/SSDI application process. SOAR employs a train-the-trainer (TTT) model in which a technical assistance (TA) contractor trains a few staff in interested states who then train others in their respective states. Finally, the initiative includes an ongoing TA function whereby the contractor helps states strengthen their efforts and gathers and disseminates information on promising practices.

SOAR has the potential to help end and prevent chronic homelessness through two avenues. First, the technical and strategic planning assistance to states may promote the development of new cross-agency collaborations and change how case managers perform their work. Second, increased SSI/SSDI receipt among those who are homeless translates into the availability of greater cash and medical resources, ultimately improving quality of life. Facilitating access to SSI/SSDI and associated Medicaid benefits among this target group can also lead to outcomes that benefit the state, such as reduced reliance on and expenditures from state-funded general assistance programs, reduced expenditures by state-only health or mental health services, and reduced incidence of uncompensated, emergency medical care.

To determine how and the extent to which SOAR is achieving its goals, the Assistant Secretary for Planning and Evaluation at the Department of Health and Human Services contracted with Mathematica Policy Research (Mathematica) in 2007 to conduct an evaluation of SOAR. The goals of the evaluation were to (1) provide a comprehensive description of SOAR processes, (2) examine the outputs and some of the short- and long-term outcomes that may be associated with these processes, (3) assess the factors that appear to be associated with successful implementation of the initiative, and (4) describe ways in which the initiative might be improved at either the state or federal level.

This report summarizes the findings from the evaluation, which are especially timely as initial federal funding for SOAR has expired and federal government agencies, as well as other funders, are considering how much and in what ways to best invest additional resources in the initiative. The findings in this report may help policymakers and program administrators determine the extent to which investments in the initiative to date have been beneficial and how consideration of additional investments could be worthwhile. The remainder of this introductory chapter of the report describes the policy context for the initiative, the key components of the SOAR model, and the study objectives and research questions. Chapter II describes the study methodology. Chapter III discusses outcomes of the initiative and Chapters IV and V describe the activities and outputs that contributed to the outcomes. Chapter VI contains concluding remarks and a discussion of considerations for the future.

The Policy Context

SSI/SSDI eligibility is contingent on having a medical condition that meets Social Security's definition of disability. To be eligible for SSI benefits, individuals must pass an income and resource test and be deemed disabled. To be eligible for SSDI, individuals must have worked in jobs covered by Social Security and be deemed disabled. According to the uniform definition of disability applied in all states, individuals are disabled if they have a medically determinable physical or mental impairment that (1) is expected to last at least 12 continuous months or to result in death; (2) prevents them from performing substantial gainful activity (that is, earning about $900 per month); and (3) results in functional impairment in at least two of four areas. Applicants bear responsibility for submitting the information necessary to document their medical conditions, and each state's Disability Determination Service (DDS) decides whether applicants meet the Social Security Administration's (SSA) criteria.

Given that disability determination is based on several factors, applicants often face difficulty compiling needed information or judging whether they meet SSA's eligibility criteria. Providing enough documentation that verifies functional and medical disability requires that contact information for treating providers in the application must be complete, that treating providers must respond to requests for information, and that the information is adequate to permit claims adjudicators to make a medical determination of disability. Some medical providers lack experience with SSA's medical criteria for determining disabilities and therefore sometimes do not provide the information needed for a favorable decision. Medical examiners at DDS do not generally meet with claimants and must instead rely exclusively on written documentation to make their determinations. A lack of medical evidence may require SSA to request the applicant to undergo a consultative examination (CE) with an SSA-contracted medical provider. Satisfaction of such a requirement and the need to provide or remedy other missing or inadequate information in the application can delay the claim adjudication process and/or result in a denial, especially if the applicant neglects to respond to requests for additional information in a timely manner or fails to keep scheduled CE appointments. One study attributed 25 percent of denied claims among individuals who are homeless to failure to complete the CE, which is nearly three times higher than among non-homeless persons (9 percent) (O'Connell et al. 2004).

Aside from the general complexity of the application process, several other barriers can prevent homeless individuals with disabilities from being awarded SSI/SSDI benefits, including poor health status, lack of a stable address, fragile social support networks, and inadequate relationships with health and social service providers. In addition, mental health and substance abuse problems experienced by those in this group compromise their ability to navigate the complex and lengthy SSI/SSDI application process. As many as 77 percent of adults who are homeless report a chronic health condition (O'Toole et al. 2002), 22 percent report a serious and persistent mental illness, and 67 percent report having received a mental health diagnosis during their lifetime (U.S. Conference of Mayors 2004; Goering et al. 2002; North et al. 2004). As many as 68 percent of homeless adults report substance abuse problems, and many have both substance abuse and mental health disorders (Goering et al. 2002). These problems limit physical and cognitive functioning and impair an individual's ability to make decisions and keep appointments (Macnee and Forrest 1997), both of which are necessary to complete the SSI/SSDI application process.

Many adults who are homeless have no usual source of medical care and lack trusting relationships with providers who can document their disability for the application (O'Toole et al. 2002). In particular, few of these individuals have a relationship with a provider (Zima et al. 1996; Bird et al. 2002), often reflecting the homeless community's distrust of health and social service professionals (Bhui et al. 2006). In addition, women who are homeless report relatively less support from social service agencies (Stovall and Flaherty 1994). At the same time, the stigma associated with homelessness and mental health problems may prevent these individuals from seeking professional help (Bird et al. 2002). In addition, as many as 23 percent of individuals who are homeless have been incarcerated, and 34 percent report legal troubles while homeless. These individuals may be particularly reluctant to sustain relationships with social service agencies during the SSI/SSDI application process (Kushel et al. 2005; Goering et al. 2002).

Finally, even when these individuals do receive medical care, their providers and family members may not be familiar with the disability determination process and the criteria used to determine disability. Indeed, a professional and social support network versed in the application process is essential for managing it. In practice, it is often family members and friends who provide nonmedical information for the application, yet individuals who are homeless frequently function within strained social networks (Meadows-Oliver 2005) and lack family supports. Individuals who are homeless with the least family support remain homeless longer (Caton et al. 2005) while those who receive SSI/SSDI tend to have larger social networks (Segal et al. 1997), perhaps suggesting that friends and family members offered much-needed support during the application process.

For all of these reasons, the approval rate among individuals who are homeless is much lower than among the broader population. In 2004, 28.9 percent of SSI claims submitted by persons over age 18 were approved following the initial application, with 26.2 percent of denials related to problems with the application process (Social Security Advisory Board 2006). The approval rate for those who are homeless, however, is only 10 to 15 percent (Rosen et. al. 2001). Even among those approved, the processing time can take several months or longer. The average processing time for an initial claim is about 120 days. For claims that are denied and appealed, processing time exceeds 400 days on average (Social Security Advisory Board 2006). Given the typically long wait for a hearing before an adjudicative law judge at appeal, it is extremely important to ensure that information is as complete and relevant as possible in the initial disability application.

The SOAR Initiative

SOAR is a multi-agency federal initiative that provides training and TA to states to develop SSI/SSDI advocacy initiatives. Its ultimate goal is to help case managers (1) develop the means to expedite processing of SSI/SSDI applications for homeless individuals and (2) improve the quality of the information submitted with the application in order to increase the likelihood of disability benefit award at initial application and to substantially reduce the processing time associated with the application. States do not receive direct financial resources from the federal government to implement the initiative, but do receive training and TA.^[1]

States were initially invited to apply for SOAR training and TA through the Homeless Policy Academies sponsored by the Health Resources Services Administration (HRSA). The Homeless Policy Academy Initiative was designed to help state and local policymakers address the issues of chronic homelessness and the needs of families with children experiencing homelessness. Those states interested in receiving SOAR training and TA were required to submit an application that described their service system, leadership, and commitment to SOAR. A small business contractor provides the TA with funding from the Substance Abuse and Mental Health Services Administration (SAMHSA) and HRSA as well as from the Department of Housing and Urban Development (HUD). The TA contractor and federal partners together selected the states to receive the SOAR training and TA from those that applied. The TA that the contractor provides consists of three core activities:

1. Strategic Planning.  The TA contractor assists states and communities with developing a strategic action plan to ensure that SOAR's critical program elements will be in place to maximize the award of SSI/SSDI benefits to individuals who are homeless. Up to 30 stakeholders participate in a two-day, professionally facilitated forum intended to lay the groundwork for the SSI/SSDI outreach initiative. Stakeholders include state mental health and substance abuse agencies, local homeless assistance providers, local SSA field offices, the state DDS, public hospitals and clinics, and a variety of other service providers that assist individuals who are homeless. The strategic action plan is intended to establish who and how many staff are to be devoted to the effort; how it will be implemented, funded, and sustained; and the nature of cross-agency relationships that will be developed or strengthened to facilitate the success of the initiative.
2. Train-the-Trainer (TTT) Program.  After the state has developed a strategic action plan, it may send up to four staff members to attend a four-day SSTR training program. The SSTR curriculum is designed to make case managers more knowledgeable about the disability application process by providing information and tools needed to effectively guide applicants through the process. After completing the intensive training, state trainers are expected to return to their states and train state and local program staff who serve individuals who are homeless. State trainers receive assistance from the TA contractor as they plan their initial training sessions as well as feedback on session content and their training techniques based on observations by the TA contractor.
3. Ongoing TA and Monitoring.  Each state receives ongoing TA and monitoring of action plan implementation for one year following its initial development. In addition, the TA contractor hosts a website with materials and tools for use by case managers.

The SOAR approach emphasizes several key components that hold promise for increasing and expediting initial SSI/SSDI application approvals, such as obtaining consent for case managers to become an applicant's representative during the application process; taking all possible steps to avoid the need for a CE; working closely with health care providers, SSA, and the state DDS; and drafting an effective summary report for inclusion with the application to help point the DDS medical examiner to the relevant evidence to support an applicant's claim. SOAR's approach also emphasizes the need for adequate staffing levels, staff training, and collaboration among stakeholders as well as the need to track and report on outcomes achieved by organizations implementing the SSTR curriculum.

Figure I.1 illustrates the inputs, activities, and potential outputs and outcomes of the SOAR initiative. Many of the relationships depicted in this model are reciprocal and less linear than depicted. Several inputs facilitate the implementation of SOAR. These include support from several federal agencies and departments including HUD, ASPE, HRSA, and SAMHSA. These federal entities have a long history of supporting the development of the SSTR curriculum and other homelessness initiatives. For example, SAMHSA supports the Projects for Assistance in Transition from Homelessness (PATH) program, which provides grants to states for community-based outreach, substance abuse prevention and treatment and mental health services, case management, as well as a limited set of housing services for people experiencing serious mental illness and are experiencing homelessness or at risk of becoming homeless. HRSA supports Healthcare for the Homeless, a grant program that provides primary care (including primary health care and substance abuse services, emergency care and referrals, outreach and assistance in qualifying for entitlement programs and housing) to homeless people. As another input for SOAR, staff of the TA contractor, who are specialists in the fields of homeless services, mental health, program development, and training, lend their expertise to states and local communities. Finally, participation by a diverse group of state departments and local service providers, many of which work with homeless populations, facilitates the entry of the SOAR initiative into their communities through their participation in the strategic planning process and training.

Several activities result from the inputs. In the logic model presented below, SOAR activities are those things the TA contractor does to introduce SOAR into a community. First, the TA contractor facilitates state planning forums during which all stakeholders plan for implementing SOAR in their community. During and after this process, the contractor works with state and local staff to identify the critical components of SOAR that will require the most attention in their communities. These components are listed below in Figure I.1. The contractor also works to facilitate dialogue and collaboration among SOAR stakeholders so that the critical components are implemented with minimal burden. This contractor then trains leaders from each state in the SSTR curriculum, provides ongoing TA support as needed, and serves as a resource to state agencies.

There are at least three observable outputs that results from the SOAR activities facilitated by the TA contractor. In the logic model below, outputs reflect states' efforts to implement SOAR and are what enable SOAR to achieve its intended outcomes (improve access to SSI/SSDI benefits for and quality of life among individuals who are homeless). First, state leaders are able to use the knowledge and skills they have learned from the TA contractor to train other case managers to implement the critical components. Second, communities in which case managers receive training build cross-agency collaborative relationships. Finally, state team leaders provide ongoing support and assistance to in-state trainers and case managers in providing application assistance to the target population.

Implementation of SOAR critical components may lead to several short-term outcomes. For example, SOAR communities may begin to develop an infrastructure to track administrative and client outcomes related to SOAR. (One of SOAR's critical components is assessment of results.)

Increased awareness of the need to assist those who are homeless obtain SSI/SSDI may encourage case managers to submit a higher number of SSI/SSDI applications. Possibly, case managers may have acquired the necessary techniques to help individuals submit timely and more complete applications, a change that may improve the SSI/SSDI approval rate by reducing the share of technical denials as well as decrease DDS processing time. Finally, case managers may have developed a greater understanding of the advantage to clients of following up with SSI/SSDI when appropriate (for example, to obtain more information if an application is denied).

Long-term outcomes of supplying SOAR clients with a steady income and improving access to housing and health care may follow the short-term outcomes. To date, long-term outcomes have been extremely difficult to measure, both because the data do not exist in many states and because such outcomes take time to realize. Thus, another long-term goal of SOAR is the increased monitoring of client and system outcomes. As interagency partnerships develop due to SOAR and positive short-term and long-term outcomes accrue, participating communities may sustain SOAR, and other communities may seek to replicate the initiative. In addition, states may begin to realize cost savings as they recoup General Assistance and uncompensated state Medicaid expenditures from SSA for those who are approved for SSI.

The short- and long-term outcomes may be mediated or moderated by both the existing infrastructure of SOAR stakeholders in each community and the existing relationships between those stakeholders. That is, the extent to which SOAR may result in positive outcomes depends, in part, on the starting point of each community. These mediators and moderators are especially relevant for determining the extent to which SOAR can be or has been sustained in communities.

Study Objectives and Research Questions

The objectives of the SOAR evaluation were to (1) provide a comprehensive description of SOAR activities, (2) examine the outputs and some of the short-term outcomes that may be associated with SOAR processes, (3) assess the factors that appear to be associated with successful implementation of the initiative, and (4) describe ways in which the initiative might be improved at either the state or federal level. To meet the objectives, we addressed the following specific research questions:

• How and how well does the strategic planning process work?
• How and how well are the TTT sessions conducted?
• What kinds of ongoing TA have states requested and received?
• How and how well are the in-state trainings conducted? How many trainings have been conducted? What are the goals and objectives of the trainings? When and where have the trainings been held? Who has participated in the trainings? How successful have the efforts been to train case managers with the SSTR curriculum?
• To what extent have state and local stakeholders contributed to SOAR since the strategic planning forum?
• To what extent are in-state training participants implementing SOAR on the ground?
• How and how well do states/localities track SOAR outcomes?
• To what extent and are individuals who are homeless and states benefitting from SOAR?
• What does the future hold for SOAR?

Data Sources and Methodology

To answer the evaluation questions, we examined existing data from all states that implemented SOAR and collected in-depth data from select states. This approach enabled us to obtain some breadth in the analysis while achieving substantial depth in selected states. We obtained (1) data on all states from documentation the TA contractor maintains on the results of its TA efforts and (2) detailed data on a subset of states through in-depth interviews and focus groups. In addition, we conducted in-person observations of some key SOAR activities. These data sources are described in detail below.

Some states that have implemented SOAR, however, were excluded from the evaluation. SOAR implementation occurred in three rounds. The first two rounds of TA were supported with funds from SAMHSA and HRSA while the third was supported with funds from HUD. TA began for Round One states in August 2005, Round Two states in August 2006, and Round Three states in August 2007. Because Round Three states were still in the initial stages of program development when we were collecting data for the evaluation, their contribution to the evaluation was likely to be minimal. In addition, a few states have implemented SOAR outside of the federal initiative, compensating the TA contractor directly with state funds. These states may have received somewhat different TA than the states participating in the federal initiative. Thus, Round Three states that received non-federally funded TA were excluded from the evaluation. The analysis of TA contractor data focused on the 25 Round One and Round Two states only and the case study states were a subset of these 25. Figure II.1 identifies Rounds One, Two, and Three states.

Figure II.1
SOAR Round by State

Rounds and states:

1. Washington, Oregon, Montana, California, Nevada, Utah, Arizona, Oklahoma, Ohio, Kentucky, Virginia, Florida, Georgia, Hawaii
2. Colorado, Minnesota, Indiana, Pennsylvania, Maryland, New Jersey, Connecticut, Massachusetts, Tennessee, North Carolina, Washington, DC
3. Alaska, Texas, Louisiana, Alabama, Michigan, New York, New Hampshire, West Virginia, Delaware, Rhode Island

The rest of the states were not SOAR states.

Analysis of TA Contractor Data

The SOAR TA contractor collects substantial information from SOAR states about the successes, challenges, and preliminary outcomes of their efforts. It uses the information to inform its TA efforts and to promote best practices in individual states. Recognizing that when examined through a research lens some of these data could contribute to the evaluation, we compiled and conducted secondary analysis of the data. Below we describe the data sources and the specific evaluation questions each source addressed.

Participant evaluations of the TTT sessions.  After each TTT session, the TA contractor requests that each participant complete an evaluation form, consisting primarily of close-ended questions. The contractor has hard copies of each evaluation form it received from the initial two TTT sessions it conducted. The evaluation forms helped us answer the following questions:

• How satisfied were participants with the TTT session?
• What did participants find most and least helpful?
• What did participants feel they learned from the session?

Documentation from initial in-state trainings (observation, participant evaluations, and participant pre/post test results).  The TA contractor observes each initial in-state training to (1) assess the extent to which the training was provided with fidelity to the model and (2) provide feedback to trainers on how to improve their efforts. Observers summarize their impressions and rate the skills of the trainers and their presentation of the SSTR material on a five-point Likert scale. These summaries and ratings are available for each state, and the quality of the data is relatively consistent because the same four individuals who provide the TA conduct all of the observations. Also available for each state are (1) evaluation forms that participants in the initial in-state trainings complete and (2) results from a pre-post test designed to measure the extent to which participants' knowledge increased following training.^[2] In addition to feedback on the training, the evaluation form provides basic background data for each participant (for example, the type of agency where the participant works, position at the agency, and experience with SSI/SSDI applications prior to training). Data from these documents helped us answer the following questions:

• How did the TA contractor rate the initial in-state trainings?
• In what areas did trainers perform well and not perform well?
• Who participated in the initial in-state trainings?
• How satisfied were participants with the in-state training?
• What did training participants find most and least helpful?
• How much and what did participants learn from the training?

Interviews with TA contractor staff.  These interviews were conducted to gather their perceptions of SOAR's successes and challenges. Specifically, we spoke with the SOAR team project director, the SOAR project coordinator, and the senior consultant who authored the SSTR curriculum and provides the bulk of the TA to states.

Observations of Key SOAR Activities

To gain first-hand perspective on how SOAR activities are implemented in practice, we observed a Round Three strategic planning forum and TTT session. We observed the strategic planning forum for New York City in its entirety in November 2007, and observed segments of the TTT session that took place in Baltimore in January 2008. Specifically, we observed the portion of the training that introduced SOAR and described the critical components as well the TA contractor's introduction of the training techniques they use. We also observed how the contractor encouraged participants to use in their in-state trainings, including hands-on practice and role-playing. In addition, we looked at the content-based training for roughly half of the critical components, paying close attention to how the contractor mixed the introduction of new content with giving information about methods participants could use to teach that content to others.

Case Studies

To gather more qualitative information on how and how well states and localities are implementing SOAR, we conducted in-depth case studies in a subset of six states. The case study approach is useful for explaining the links between real-life interventions and outcomes that are too complex to capture with quantitative measures (Yin 1989). To provide the most complete picture of the initiative as possible, we purposively selected six states for the case studies. Within each, we gathered information from a broad range of agencies and staff directly involved in the SOAR initiative, as well as from other stakeholders. Below we describe our process for selecting the states and briefly describe the six states included in the case studies.

1.  Site Selection

We attempted to select states for the case studies that were diverse on a range of characteristics. First, we selected a mix of Rounds One and Two states, assuming that they would be at different stages of implementation. Second, we attempted to select states that were diverse in terms of geography, income, and race/ethnicity. And third, we attempted to select states that were diverse with respect to implementation of SOAR on the basis of non-subjective data collected by the TA contractor as of April 2008. For example, we considered the number of in-state trainings states reported conducting as of April 2008 (and selected 3 that conducted 5 or more and 3 that conducted less than 5), the number of participants that had been invited to in-state trainings (and selected 3 that had invited 100 or more and 3 that had invited less than 100), whether the lead SOAR agency was a state government or local agency (and selected 3 that were state government agencies and 3 that were local agencies), whether the state had conducted any organized SSI/SSDI advocacy work before SOAR (and selected 3 that had and 3 that had not), the number of initial SOAR pilot sites  that is, a community that attended the state's strategic planning forum and/or was among the first in the state to implement SOAR  (and selected 3 that had implemented SOAR in 10 or more local sites and 3 that had implemented SOAR in less than 10 local sites), and the application approval rate (and selected 3 that had approval rates of 50 percent or higher and 3 that had an approval rate of less than 50 percent or that had not tracked outcome data). Table II.1 presents these key characteristics of the six selected sites based on data collected by the TA contractor.

Our hope was that the six states selected for the case studies would reflect the range of experiences states have had implementing SOAR. Without detailed information from states that were not selected for the case studies, though, it is not possible to assess the extent to which the selected states are representative of all Rounds One and Two SOAR states and thus the extent to which the findings from the case studies are generalizable.

2.  Data Collection

Within each of the selected case study states, we visited one local community in which SOAR was implemented. We visited pilot communities for three key reasons: (1) presumably, pilot communities had been implementing SOAR the longest and thus have the most experiences to share; (2) most of the data the TA contractor has gathered from states pertains to the pilot communities (providing multiple data sources to compare and contrast in our analysis); and (3) most of the pilot communities participated in the initial planning activities (that is, the strategic planning forum) and thus could provide a more complete perspective on the TA process and how it has evolved at the local level than sites that did not participate in this process. In most cases, we selected the most urban among the pilot communities because the concentration of individuals who are homeless in these areas was the highest. We talked about SOAR rollout with state team leads and trainers during our on-site interviews to gain perspective on the implementation of SOAR in other communities. Table II.2 presents the local communities we visited in each state.

We conducted in-depth, in-person two- or three-day visits to each of the six states in the winter and spring of 2009. Using semi-structured guided discussion techniques, we conducted one-hour interviews with each of the following:

• State team lead
• Local team lead
• SSA administrators and staff
• DDS administrators and staff
• Key players from all other state and local agencies participating in the state planning forum, including administrators and staff from state mental health agencies and community-based organizations (CBOs), housing and other public assistance agencies, and homeless services providers
• State trainers

In addition, to gather data on how SOAR is implemented at the ground level, we conducted small group interviews with case managers in each site who had participated in the SSTR training. The interviews addressed questions regarding which of SOAR's critical components case managers do and do not use in practice (as well as why and how), the challenges they have faced in implementing what they learned in training, factors that have facilitated their implementation of SOAR, and outstanding needs they have in supporting homeless SSI/SSDI applicants.

Small group interviews generally lasted approximately 90 minutes. During our initial calls with the state or local SOAR team lead to arrange the site visit, we requested a list of individuals who participated in the initial training in the locality. We used this list to recruit five to ten case managers. One member of the site visit team facilitated the group interview (using a semi-structured discussion guide) while the other took detailed notes. Mathematica provided lunch for case managers as incentive to participate.

3.  Description of States

Although each state included in the case studies was trained in SOAR using a single model, each implemented the program slightly differently. In addition, the initiative operated in very different environments in each site, both structurally and politically. This section provides a brief description of each site, focusing on the evolution of SOAR and the key stakeholders as context for the remaining chapters of the report.

Utah.  The director of Utah's Homeless Task Force believed that SOAR could help the state advance its ten year-plan to end homelessness. His leadership of SOAR and persistent outreach and follow-up efforts were essential to the formation and continuation of SOAR in the state. Initially, in-state trainings reached case managers at a range of state agencies and CBOs. One year into SOAR, however, the leadership of the Department of Workforce Services (DWS), the state agency that provides employment and support services to improve customers' economic opportunities, created a team of SOAR-trained specialists charged with helping General Assistance clients in the most populous regions of the state apply for SSI and Medicaid. DWS is in the process of extending that assistance to clients statewide. While current SOAR trainings and support focus on DWS staff, some case managers from other organizations continue to draw on prior SOAR trainings to help clients access SSI. All individuals who have received SOAR training are invited to quarterly SOAR summits that provide a forum for case managers to ask technical questions; SSA and DDS attend these meetings as well as in-state trainings.

Pennsylvania.  Pennsylvania had an unusual introduction to SOAR. The state submitted an application for Round One TA and was unsuccessful, but was later invited by the federal agencies supporting SOAR to participate in Round Two, with the specific condition that SOAR be implemented in Philadelphia. This city had recently lost other sources of federal funding in the area of homelessness and the implementation of SOAR there was perceived as a way to mitigate the ramifications of that loss. Thus, all SOAR activity in Pennsylvania is focused in Philadelphia. Two state agencies (the Department of Community and Economic Development and the Department of Public Welfare) have a degree of responsibility for clients who are disabled and homeless, but aside from some involvement during the strategic planning phase, they have had minimal involvement in SOAR. In Philadelphia, all SOAR activities are managed by the Homeless Advocacy Project (HAP), a non-profit organization that offers free legal services to such clients. Their SOAR-related activities are funded under a contract from the city Office of Supportive Housing (OSH). In response to concerns that SOAR would become an unfunded mandate, OSH identified a source of funding for SOAR through a pre-existing contract it had with HAP in a different issue area. The first year of funding supported HAP in conducting SSTR training for 40 local agency case managers, hosting two local SOAR strategic planning forums, coordinating and overseeing the submission of 40 SSI/SSDI applications and related documents by those who received training, and collecting outcome data. The second year of funding supported HAP in continuing to roll out SOAR and oversee the submission of an additional 29 applications. HAP had experience conducting SSI advocacy and appeals before SOAR, and existing personal relationships between staff at HAP and other agencies involved in SOAR facilitated ongoing and open communication about how the initiative would be implemented. Funding from OSH for SOAR will expire in June 2009, and a key challenge in the continuation of the initiative in its current form and to expanding the initiative to other parts of the state will be identifying alternative sources of funding.

Ohio.  In Ohio, SOAR was rolled out to six communities that the state identified through a request for proposal process. Leadership for SOAR originated in the state Interagency Council on Homelessness. Bidding communities (10 areas submitted proposals to participate in SOAR through their continuums of care) were required to discuss in their applications how they would provide the staff and resources required to execute the initiative given no financial or human resources were available, aside from SOAR training, to assist staff. Although some staff had a strong leadership role in the state at the outset, lack of financial and human resources combined with staff turnover left the initiative without clear direction, leadership, or enthusiastic support from any upper-level staff in the state. At the local level, a combination of staff turnover and what the state perceived as the wrong audience at the in-state trainings (supervisors rather than case managers) meant that a great deal of information on the application process and key SOAR practices was lost before the initiative was implemented in the field. Despite challenges in leadership, motivated case managers and their supervisors have kept the initiative moving forward in several communities and continue to use the skills acquired through SOAR.

Massachusetts.  The SOAR initiative in Massachusetts originated with the Department of Transitional Assistance (DTA), which funds homeless shelters and homeless services statewide and also provides individuals and families with food assistance, job assistance, cash assistance and emergency shelter.^[3] While the DTA initially agreed to serve as the state agency home for SOAR, its involvement in SOAR has been minimal since the initiative's inception. Prior to SOAR, the Department of Mental Health (DMH), Healthcare for the Homeless (HCH), and the state DDS were involved in their own efforts to provide SSI/SSDI benefit assistance though these were not necessarily coordinated in any formal way. Realizing that a coordinated effort to provide SSI/SSDI benefit assistance to homeless populations was a critical factor to ending chronic homelessness and addressing unmet medical needs, DMH and its PATH contractor, Eliot Community Services, attempted to implement SOAR statewide. Three training sessions were planned  one in Boston, one in central MA, and one in western MA  though the training in western MA never materialized due to competing priorities among the trainers. Although the state and partnering organizations believed in the intent of the SOAR initiative, efforts to sustain the SOAR model as designed diffused for several reasons. First, local agency staff were not committed to submitting applications; staff were already overworked and unable to dedicate time to SOAR and there was no buy-in or incentive among agency directors or supervisors to implement SOAR. Second, state agencies were reluctant to dedicate staff time to SOAR in the absence of funding and did not have a cohesive plan for moving the initiative forward. Trying a different tactic, in 2008, DMH awarded excess PATH funds to Eliot to hire six dedicated benefit specialists. The funding is for federal fiscal year 2009 only; the state is actively trying to identify other sources of funding to sustain the PATH benefit specialist positions beyond 2009. Each benefit specialist is deployed to one of six regions across the state to assist individuals who are homeless apply for SSI/SSDI benefits. They regularly visit homeless shelters and receive referrals from shelter case managers and tenant preservation programs. While the specialists did not go through SOAR's SSTR training, they did receive the training manuals and they use strategies consistent with the SOAR model.

New Jersey.  The SOAR initiative in NJ was implemented along two distinct tracks. First, the state Department of Mental Health (DMH) applied for and received PATH funds to support one full-time equivalent staff member, whose responsibility would be to take a leadership role in navigating the SSA system, linking individuals who are homeless to SSA resources, and training state PATH agencies to do the same. This position was created within Project Live, the PATH provider for the city of Newark. Since January 2007, Project Live staff have been providing training to other PATH program staff statewide as well as to other interested community agencies upon request. Second, the state Veteran's Administration (VA) sent two of its staff members to a SOAR TTT session with the intention of providing training to VA social workers and case managers. Three of four planned trainings were held for VA staff; the fourth training was canceled due to lack of VA staff interest and the VA has no plans to continue with SOAR. The initiative in NJ began to flounder when both the PATH Coordinator at DMH and the Homeless Services Coordinator at the VA left their respective positions; their departures left these two key state agencies with no one to claim ownership of SOAR. Though Project Live continues to conduct at least four trainings per year, as it is required to do by contract, there are no other SOAR-related activities occurring in the state and no coordinated effort to track SOAR activities and application outcomes. While current leadership within both DHM and the VA want their own case managers to receive SOAR training, neither feels that they have time to serve as the SOAR state lead or that doing so is within their purview.

Virginia.  Virginia PATH leaders applied for SOAR because they saw the program as a natural extension of existing efforts to assist individuals who are homeless obtain benefits. Prior to its involvement in SOAR, the state PATH director had collaborated with the SOAR TA contractor on a benefits acquisition training program. The PATH director was designated as the SOAR state lead and planned to use SOAR to strengthen communities with existing initiatives and implement the training in two new sites. However, the SOAR sites varied considerably in their ability to use the training. In some localities, SOAR leaders were unable to convince stakeholders to dedicate time or other resources to SOAR, and as a result collaboration between stakeholders stagnated. In other sites, SSA and DDS staff work closely with case managers, and agency leaders reported that the number and quality of SOAR applications has increased. A large number of case managers were trained across the state, but many of these case managers do not prepare SOAR applications. Currently, PATH funds support several SOAR specialists (one full-time and several part-time) who prepare SOAR applications regularly in their localities. Staff turnover within these positions, however, has been problematic. In addition, the state PATH director's position was vacant for several months, leaving Virginia without state leadership or coordination for SOAR. The state has recently hired a new PATH director who is making considerable efforts to engage SOAR stakeholders and re-invigorate the initiative with the assistance of local leadership.

SOAR Individual and System Level Outcomes

SOAR is intended to improve the processes associated with obtaining SSI/SSDI benefits for individuals who both have disabilities and are homeless, thereby improving their quality of life. System-level measures are required to assess the extent to which SOAR indeed improves processes, and individual-level measures are required to assess the extent to which SOAR is meaningful to this target population. SOAR is not intended to prompt temporary changes during the period of its implementation, but rather to effect lasting systems change. Thus, at the system level, a key outcome measure is the institutionalization and proliferation of SOAR practices and principles. Individual-level outcomes can be delineated into two categories: short-term and long-term outcomes (see Figure I.1 in Chapter 1). Short-term outcomes center on the immediate results of the application process. Examples include the application submission and approval rate and the time between initial application and approval. Longer-term outcomes center on the implications of application outcomes for SSI/SSDI applicants' quality of life. Examples include applicants' personal income, housing status, access to healthcare, and overall health. Short-term application outcomes can also lead to financial benefits for states and service delivery systems. For example, states may be able to recoup from SSA General Assistance benefits paid to individuals who are approved for SSI/SSDI. In addition, medical providers may be able to recoup the cost of uncompensated care provided to uninsured individuals who become enrolled in Medicaid as a result of SSI/SSDI approval.

Data on SOAR's outcomes can provide critical information to federal agencies about the relative costs and benefits of the initiative and the extent to which it is worthy of an additional investment of resources. Data can also be beneficial to states and local communities by enabling them to assess the results of their efforts with an eye toward continuous improvement, and to market the initiative to new partners and potential financial supporters. Yet, very little data on individual- and system-level outcomes exist. Most state and local communities have not systematically collected outcome data, and the data they have collected is not rigorous enough to use for evaluation purposes.

This chapter describes the challenges states face in collecting outcome data and the practices that some have put in place. It also presents information on outcomes that we obtained during in-person visits to the case study states. The data are based only on reports from stakeholders in those states and on all applications for SOAR TA from Rounds One and Two states. Mathematica did not conduct primary data collection on application submissions and approvals or on related quantitative measures because states lacked appropriate data infrastructures to do so. While the study was primarily focused on processes, we present data on outcomes as described by states because of their importance to current policy decisions.

Outcome data in this chapter are presented solely for the six case study states. However, according to information gathered by the TA contractor from stakeholders elsewhere, other states have realized some important individual- and system-level outcomes that should not be disregarded, as they are indicative of SOAR's potential. Brief descriptions of some of these outcomes are included in Appendix A.

Systems and Procedures for Tracking Outcomes

In their applications for SOAR TA, states were expected to document availability of outcome data or provide a specific plan to collect, such data to assess the effectiveness of the state's plan to increase access to SSA disability benefits. Yet, once applications were approved, there was never an ongoing mandate for states to actually collect outcome data. Most states planned to use the Homeless Management Information System (HMIS) that was in development at the time of their application or to develop a new tool designed specifically to collect SOAR data, but few states actually did so. This section describes the challenges states faced in setting up systems to track outcomes and the practices they have been able to put in place.

1.  Challenges in Tracking Outcomes

As the gatekeeper of information on all SSI/SSDI applications, SSA could theoretically track SOAR outcomes. To track outcomes, SSA would need (1) a way to identify applications from homeless and/or SOAR clients and (2) resources to maintain the information. Neither currently exists. In some states, SSA does flag paper applications submitted by homeless or SOAR clients. However, there is no flag in SSA's electronic application to identify homeless or SOAR applicants and creating one would require changes to the SSA electronic filing software. Thus, states that wish to track SOAR outcome data must devise other ways of doing so.

As noted above, many of the 25 Rounds One and Two states planned to use the HMIS to collect SOAR outcome data, as documented in their initial applications for TA. Implementation of an HMIS is a requirement for Continuums of Care (CoC) that receive Department of Housing and Urban Development (HUD) McKinney-Vento Homeless Assistance Act funding. HMIS is a data tool designed to collect system-wide, client-level information about the characteristics and service needs of those who are homeless. HUD has set standards for a universal set of data elements to be collected for all people who utilize homeless services, including basic demographic characteristics, residence information prior to program entry, and service entry and exit dates. Also included are program-specific elements such as income sources and amounts (including SSI and SSDI), receipt of non-cash benefits, in-depth disability information, education and employment, and housing at program exit (including information on type of housing, tenure, and receipt of subsidies).

In general, states found it difficult to use the HMIS to collect SOAR data. Case study states provided several reasons why this was so. First, not all SOAR partners receive McKinney-Vento funding and thus do not use or have direct access to the HMIS or serve clients who come through the CoC system. Second, the HMIS was not designed to track the outcomes of particular interest under SOAR, such as application submission and approval rates and dates (and length of time between the two). States agreed that modifying the HMIS to track these types of data was theoretically feasible, but that they had no incentive to spend scarce resources to do so. No federal funding was provided to states to modify existing or develop new systems to track SOAR outcomes, and neither SAMHSA nor any other federal agency required states to report any type of SOAR data.

Other states planned to develop new SOAR-specific systems for tracking outcomes, but had difficulty doing so in the absence of resources and federal-level guidance on the type and amount of data to collect. The SOAR TA contractor offered some guidance early on about the type of data states should consider collecting, but states had primary responsibility to develop systems. One case study state created an online data tool for case managers, but it was designed to be a caseload management tool rather than an outcome measurement tool and was not widely used. Utilization was limited by confidentiality concerns and lack of sufficient resources for cross-agency training. Recently, the TA contractor developed SOAR outcome data collection software and related tools and documentation, and plans to make them easily accessible and to market them to states. These materials can be invaluable in helping states document the results of their efforts, but state and local staff will likely need training and ongoing monitoring to use them effectively.

2.  State and Local Practices

Our case studies suggest that states attempting to track SOAR outcomes have done so in two ways: through DDS or by developing informal, customized spreadsheets. In some states, DDS appends a flag sheet to paper applications to identify SOAR clients and then maintains records on application disposition electronically. This typically occurred in states in which DDS was already tracking applications from clients who are homeless and could simply add a checkbox for SOAR to an existing flag sheet. However, this effort failed to capture the ultimate result of the application process (that is, SSI/SSDI application approval or disapproval from SSA) because DDS maintains data only on its own allowance rates, processing times, and denial reasons. DDS attempted to track SOAR outcome data in five of the six case study states, though efforts in some of those states waned over time.

Some state leads (or their designees) developed customized spreadsheets to track SOAR outcomes. An advantage of this approach is that they collected data most relevant to assessing SOAR in their states in an easily accessible format. A disadvantage is that it was somewhat burdensome, as it required state leads (or their designees) to regularly collect data to enter into the spreadsheet from all of the case managers or agencies providing SOAR application assistance. The quality of the data is also dependent on the amount and quality of data case managers or agencies maintain on applications, which can vary from agency to agency. State leads (or their designees) tracked SOAR outcomes through customized spreadsheets in four of the six case study states.

Outcome Data for Case Study States

To the extent they attempted to track individual outcome data, states focused on short-term outcomes. Obtaining longer-term outcomes on applicants' quality of life would require either a longitudinal survey of applicants to observe changes over time in their income, housing status, and health and health care status, or that a case manager have a sufficiently long-term relationship with the applicant to record such outcomes in a management information system. Lack of resources precludes the former, and the latter is not typically characteristic of case management among homeless populations. Thus, in this section we describe the data case study states had available on short-term outcomes pertaining to the disposition of SSI/SSDI applications and associated data on contributions of application outcomes to financial benefits to states and service delivery systems. The data are not necessarily consistent with those used to select the case study states, as described in Chapter 1, because they pertain to a different time period and are reported directly by states (rather than through the TA contractor). We also discuss in this section examples of the institutionalization and proliferation of SOAR practices and procedures that can be used to assess more system-level outcomes of the initiative.

1.  SOAR Applications

Pennsylvania.  Data collected by the organization that manages all SOAR activities in Philadelphia (i.e., the local lead and organization that supplies the in-state trainers) indicate that the SSI/SSDI approval rate has been higher after SOAR than it had been before. The organization uses a spreadsheet to track all applications submitted by in-state training participants (including members of its own staff and case managers from partner CBOs) on behalf of homeless clients. The vast majority of applications to date have been submitted by staff within the organization rather than case managers from partner CBOs. However, among the 50 SOAR applications submitted in the first year of SOAR operations in Philadelphia, 49 were approved, and one was denied. Nine months into the second year, 41 out of 47 applications submitted were approved, one was denied, and 5 were still pending. This organization had been providing SSI/SSDI benefit assistance to clients before SOAR using strategies substantially less intensive than those inherent in the SOAR model. Prior to implementing SOAR, the organization had experienced a 50 to 60 percent approval rate on the SSI/SSDI applications it submitted on behalf of clients (as reported anecdotally by staff). While the organization does not track application processing time, staff report that DDS decisions on SOAR applications are often made within 10 days. The organization's goal for the future is shifting the proportion of applications submitted by its own staff and by case managers from partner CBOs so that the latter outweighs the former.

Utah.  In Utah, the State Department of Workforce Services (DWS) maintains data on all SSI/SSDI applications submitted by their own case managers on behalf of homeless clients. They are not representative of all SOAR applications filed in the state because they do not capture applications submitted by case managers in CBOs who received SSTR training early in the initiative. The data indicate that in 2008, of the 412 SSI applications submitted during the year, 338 (82 percent) were approved and 203 of those (49 percent overall) were approved upon initial application. In comparison, the approval rate for all SSI/SSDI applications in Utah as reported by DDS was 44 percent in 2008. Average time between application submission and initial approval was 3.6 months after SOAR compared to 15 months before SOAR.

Massachusetts.  Data that the state DDS tracked on the outcomes of SOAR applications indicated that both the number of SOAR applications filed statewide and the application approval rate were low in the early phase of the initiative. Over a two-year period from 2007 through 2008, 138 applications SOAR were filed and about one-quarter were approved. The number of SOAR applications was low, in part because SOAR did not initially take hold well on the ground level and case managers did not submit a large volume of applications, and because difficulties tracking SOAR applications within DDS led to some applications going uncounted. In state fiscal year 2009, PATH funds were designated to support six SSI/SSDI benefit specialists housed in the state's sole PATH contractor. Since then, the PATH contractor has assumed responsibility for tracking outcomes SSI/SSDI application outcomes submitted by the benefit specialists on behalf of homeless individuals. While the benefit specialists did not receive formal SSTR training, they received the training manuals and use strategies consistent with the SOAR model. Data are maintained in a spreadsheet and indicate that designating staff whose sole responsibility is SSI/SSDI benefit assistance has facilitated an increase in application submissions, though it is too early to assess the effect on the application approval rate. As of February 2009, decisions were made in 141 (51 percent) of the 275 applications that the specialists had submitted between June 2008 and February 2009 and that 70 of those 141 applications (50 percent) were approved; the remainder were denied or in appeal. Among approved applications for which data on both the date of application and approval existed, average time between application and approval was 90.5 days (or about three months).

Virginia.  Outcome data for the Tidewater area in Virginia are maintained by DDS and based on a flag for SOAR on applications. They indicate that SOAR was at its strongest in 20052006, when the allowance rate was 52 percent.^[4] In April 2006, at the peak of SOAR, average DDS analyst decision time for a SOAR application was 34 days, and the average total time from application to SSA award was 52.6 days. Unfortunately, no comparison data on processing time for all applications was available. In 2006 the state team lead took a new position and SOAR was without leadership for some time. In addition, there was substantial local staff turnover at that time. In 20062007, the allowance rate dropped to 25 percent and in 20072008 to 24 percent. The allowance rate for 20082009 to date is 29 percent.

Ohio and New Jersey.  Stakeholders in Ohio and New Jersey were unable to report any information on outcomes as there is no sustained effort to collect data in these states. In both states, the number of applications submitted is likely extremely low as the lack of attention to outcomes is due to a general lack of focus on SOAR.

2.  Resultant Financial Benefits to States and Service Delivery Systems

As noted in Chapter I, facilitating access to SSI/SSDI among homeless individuals can also lead to outcomes that financially benefit the state or service delivery systems. SSA may enter into agreements under which states or local governments are reimbursed for basic needs assistance, such as General Assistance, provided during the period that an eligible individual's SSI application for benefits was pending. Currently, 39 states have interim assistance reimbursement agreements with SSA (http://www.ssa.gov/OACT/ssir/SSI09/). In addition, medical providers may recover from Medicaid the costs of uncompensated care they provided to individuals who enroll in the program. Providers are able to recover all costs incurred while the application was pending as well as during a retroactive period of three months prior to the effective filing date. As of the time of our site visits, none of the case study states had attempted to track whether and how much providers had been reimbursed by Medicaid, but several had data regarding recoupment of General Assistance expenditures from SSA.

In Massachusetts, the state recovers Emergency Aid to Elders, Disabled, and Children (EAEDC) dollars when individuals are approved for SSI. EAEDC is a Massachusetts state-funded program that provides cash and medical assistance to needy families and individuals who are not receiving TANF, SSI, or other similar benefits  that is, it is the state's General Assistance program. The state maintains cost recovery information on all EAEDC recipients who were awarded SSI, but does not distinguish between homeless versus non-homeless recipients. In fiscal year 2008, the state had recovered $12,326,520 from SSA in EAEDC. The state has little incentive to track cost recovery information specifically for homeless EAEDC recipients or to market SOAR as a major cost recovery mechanism because of the EAEDC payment structure. Individuals receive $92 per month in EAEDC when they are in shelters but $363 per month when they live in the community, so there is more financial incentive to recover EAEDC costs for individuals who are housed.

In Utah, DWS tracks in an Excel spreadsheet the number of SOAR clients statewide who have been approved for SSI/SSDI as well as the number of months they were on General Assistance prior to approval. In 2008, DWS recovered about $800,000 from SSA for General Assistance payments made on behalf of these clients.

3.  Systems Change and Outgrowths of SOAR

SOAR has helped engender systemic changes in communities that will remain in place even if all SOAR-specific activities in the state were to cease. In some states, for instance, SOAR has contributed to an institutionalization of agency roles regarding benefits acquisition. In Utah, DWS created a team of specialists charged with helping General Assistance clients in the most populous regions of the state apply for SSI and Medicaid. Other states have incorporated the concept of benefits assistance (and specifically SOAR) into their 10-year plan to end homelessness. And, SOAR has contributed to the development of personal and organizational relationships that can transcend a breakdown in agency- or system-level activity related to the initiative. Specifically, in many localities there is more interaction than there has ever been between SSA/DDS staff and CBO staff who work directly with applicants who are homeless. These and other changes that have occurred in communities in relation to SOAR are described in more detail in Chapter V.

Another measure of the institutionalization of SOAR practices and principles is the extent to which states are using SOAR as a springboard for developing related initiatives for other low-income populations. At the time of our site visit, Pennsylvania was in the process of exploring the possibility of expanding the SOAR model to work with single people on cash assistance and with Temporary Assistance for Needy Families (TANF) parents. The concept was motivated by a DPW-sponsored analysis which merged TANF caseload and HMIS data. The analysis indicated that 30 percent of TANF families in which someone had a disability also had a history of homelessness. Ohio was also in the process of expanding SOAR's reach to a broader population. The Ohio Benefit Bank (OBB) is the state's online screening and application tool through which public assistance applicants can learn about programs for which they are eligible and begin the application process. At a single location, clients can access information about and applications for public benefits such as food stamps, TANF, health care coverage, home energy assistance, and child care subsidies. At the time of our site visit, the OBB was undergoing an upgrade. In addition to many other elements, the state was considering integrating the SSI application process and components of the SOAR approach into the new version. Staff in 10 communities across the state will be trained (using components of the SSTR curriculum) to assist clients with the electronic application.

Looking Ahead

The data presented in this chapter can be of some use in decision making related to the potential value of future investments in SOAR. To systematically measure SOAR outcomes and make more definitive conclusions in the future, however, it is necessary to establish rigorous quantitative data collection mechanisms at both the state and local levels and to design a study that maximizes the ability to attribute outcomes to the initiative rather than to other environmental factors. Rigorous data collection mechanisms could be established through collective use of a SOAR specific management information system (such as the one the TA contractor has recently developed) or purposeful and formal integration of SOAR measures into existing cross-state data systems (such as the HMIS, the Government Performance and Results Act system, and/or the web-based system in which PATH measures are reported). While random assignment experiments are the gold standard for evaluating the effect of social programs, an experimental evaluation of SOAR may be unrealistic given likely logistical and ethical challenges as well as resource constraints. More practical designs include a comparison site study, pre-post study, or study that combines the two (that is, a difference-in-difference approach, which compares changes in outcomes observed in SOAR sites before and after the initiative with changes in outcomes in a set of otherwise comparable non-SOAR sites before and after the initiative). Such a study would be of tremendous value in documenting investments in SOAR, assessing its effectiveness, and justifying its costs.

SOAR Activities

While states do not receive any financial resources to implement SOAR, they do receive substantial TA during initial planning and implementation and to sustain and expand their efforts. This chapter provides an overview of the three key components of the TA  (1) strategic planning, (2) the TTT program, and (3) ongoing TA and monitoring  and describes how each was implemented in the six case study states. Information presented in this chapter is drawn from a variety of sources, including site visit interviews with key informants in each state; state applications to receive SOAR TA; summaries of strategic planning forums; and state-level lists of attendees at each TTT session, as well as evaluations of these sessions.

Strategic Planning Forums

SOAR states participate in a two-day professionally facilitated strategic planning forum to strategize how SOAR critical components will be implemented locally. Up to 30 stakeholders participate in the forum, which is intended to establish buy-in among stakeholders and lay the groundwork for inter-agency collaboration. The forum is the first opportunity for stakeholders to come together to collectively strategize how SOAR can be implemented in the context of existing local efforts to support homeless individuals. During the forum, stakeholders develop a strategic action plan that specifies the steps participants will take toward implementation. Below we describe the TA contractor's role in the forums, the myriad of forum participants, the emergence of in-state leadership, and the resultant action plans.

1.  The Role of the TA Contractor

In some states, the TA contractor's influence was critical to bringing various partners to the table. The TA contractor assists each state in preparing for its strategic planning forum. In advance of the forum, the contractor convenes a planning call to help state leads identify which stakeholders to invite to the forum so that the appropriate agencies and individuals are represented. Engaging the appropriate stakeholders during the planning process is important to foster early buy-in for the initiative so stakeholders are motivated to serve as change agents in implementing SOAR. Establishing early buy-in among stakeholders also helps to mitigate inter-agency implementation barriers that could threaten the success of the initiative. Without the contractor's influence and recruitment efforts, some state leads were skeptical that they would have been able to assemble all key players, particularly SSA. They believed that they did not have sufficient prior relationships with the key players to draw on and were not yet savvy in marketing the initiative to potential partners.

Forum participants uniformly praised the TA contractor for its facilitation of the in-state forums and the extent to which it engaged participants. The TA contractor serves two key roles at the strategic planning forums. First, the contractor introduces SOAR to stakeholders by describing the program's critical components and how it can help stakeholders achieve systemic changes. Informing stakeholders how their agencies can be instrumental in launching as well as directly benefit from SOAR is crucial to establish broad buy-in and support for the initiative. Second, the TA contractor assists states and communities with developing a strategic action plan to ensure that critical program elements will be in place to maximize the award of SSI/SSDI benefits to individuals who are homeless. Specifically, the contractor helps stakeholders assess their current environment, consider possibilities for change to better support individuals who are homeless and have disabilities, and think concretely about how such change can be implemented. During the forum, the contractor provides structure and facilitation for state-driven decision making. The contractor's initial intention is not to be prescriptive about the roles stakeholders should assume or the specific steps states should take to implement SOAR. As a result, the content strategic action plans varied from state to state and the process did not necessarily result in a strong action plan (see Section A.4 below for more discussion).

2.  Forum Participants

Stakeholders participating in the forums typically include state PATH contacts, local homeless service providers, state mental health agencies, local and/or regional SSA staff, state DDS representatives, and a variety of other service providers that assist individuals who are homeless. In most places, prior relationships between SSA/DDS and other stakeholders did not exist and there was little awareness among other stakeholders of SSA and DDS processes. The strategic planning forum was often the first time SSA and DDS participated in a cross-agency dialogue of how to better support individuals who are homeless and have disabilities.

At least some CBOs that directly serve members of the target group were present at all strategic planning forums in the case study states. While there was considerable variability across states in the extent to which case managers from these organizations had experience providing SSI/SSDI application assistance to clients, all organizations acknowledged at the forums that additional training for case managers was necessary because SSA eligibility requirements and application processes are complex and difficult to understand.

Some potential partners were absent from most strategic planning forums  most notably, large medical providers, such as hospitals, state mental health treatment providers, and representatives from HCH. Formal agency-level relationships typically did not exist between large medical providers and other key stakeholders, in part because most states sent forum invitations to only those agencies they deemed most central to SOAR's objects. Medical providers, however, could be instrumental partners in SOAR because they provide medical documentation and can perform CEs for SOAR claims. These providers also have incentive to support SOAR (financially or otherwise) because it could be a mechanism through which they could recover the costs of providing care to uninsured individuals with disabilities. Organizations that provide legal advocacy were also absent from most strategic planning forums. Such organizations have extensive expertise with SSA eligibility requirements and application processes and routinely interface with SSA and DDS. A legal advocacy organization serves as lead agency in one SOAR locale; such organizations could also be effective partners in other SOAR sites.

3.  Emergence of In-State Leadership

In most states, the agencies that applied for SOAR did not emerge as the operational lead for SOAR activities. States applied for SOAR TA on behalf of their homeless policy academy teams. In only one of the six case study states, the policy academy team lead who applied to participate in SOAR later became the SOAR state lead and was instrumental in bringing stakeholders together to implement the initiative. In other states, member agencies of the homeless policy academy team applied for SOAR and named themselves as the lead agency either as a formality (never intending to assume the responsibilities required of the state lead) or without a firm grasp of what SOAR leadership would entail.

Organizations that emerged as the operational lead for SOAR did so because its activities were a natural extension to their ongoing work assisting clients who are homeless access SSI/SSDI benefits. These organizations were experienced in providing outreach and benefits acquisition assistance to homeless populations and had existing service models that could be adapted to accommodate SOAR activities. However, not all organizations that had this experience had the time or wherewithal to take on SOAR, and in some states, an operational lead never emerged.

Agencies were further motivated to continue acting as the operational lead for SOAR when they recognized that their organization or state could benefit financially from SOAR  either through direct funding to implement SOAR components or through cost recovery mechanisms. In half of the case study states, lead organizations were PATH providers and received PATH funds to carry out SOAR activities. In two of these organizations, funds were directed to hiring staff to complete SSI/SSDI applications and collect data on SOAR outcomes. The third organization received funds to conduct four SOAR trainings per year targeted to PATH staff throughout the state. In one other state, SOAR was viewed as an effective cost recovery mechanism, enabling the state to recoup the cost of providing General Assistance benefits to adults with mental or physical impairments. To assist in this effort, the lead state agency hired staff to complete SOAR applications for individuals receiving state benefits, such as General Assistance, TANF, or refugee benefits. The operational lead in another state received municipal funds to conduct SOAR trainings, complete SOAR applications, collect data on outcomes, and convene stakeholders in an annual SOAR forum.

4.  Resultant Strategic Action Plans

The most tangible result of the strategic planning forum is an action plan for propelling SOAR forward. This plan is intended to establish who and how many staff will contribute to SOAR; how the initiative will be implemented, funded, and sustained; and the nature of cross-agency relationships that will be developed or strengthened to facilitate the success of the initiative. Across the six states, strategic action plans focused on establishing collaborative relationships with organizations that are involved with SSI/SSDI applications. Many stakeholders sought to identify a clear contact at SSA or DDS for homeless providers to use as a resource. States with experience and existing service models for providing benefit assistance strategized how to develop stronger relationships with stakeholders not represented at the forum or in existing efforts. For many, this meant enlisting support from the medical community. Some aimed to establish relationships with medical records departments to improve procedures for obtaining medical evidence, train medical doctors on the importance of providing high-quality medical evidence, or to enlist nurse practitioners, medical residents, and medical doctors to perform medical evaluations. One state also sought to identify a contact at Medicare and the Veterans Administration.

The more specific stakeholders were in identifying timelines, activities, and staff roles, the more they were able to leave the forum with a concrete, actionable plan. Some plans described how and where the trainings would be implemented, identified trainers and CBO's that would be invited to receive SSTR training (and who would recruit them), and approaches for fostering buy-in among local agencies for their staff to implement SOAR. Stakeholders also identified potential solutions to implementation barriers, many of which hindered existing efforts to provide SSI/SSDI application assistance to homeless populations. For example, some states strategized how to assist people who are homeless obtain identification, birth certificates, and a stable mailing address. In states with a scarcity of representative payees, stakeholders strategized ways to increase the availability of payees to better serve the members of the target group.

Train-the-Trainer Sessions

The TTT sessions teach a select number of representatives from each SOAR state how to train case managers in the state in the SSTR curriculum. The sessions are held at strategic locations throughout the country and typically accommodate up to 40 people. Each state participating in SOAR for the first time may send up to four people (two for free and two at their own cost) to a TTT session facilitated by the TA contractor, and may send additional people to a subsequent TTT session if space allows. These individuals are then expected to return to their states and train state and local program staff who serve individuals who are homeless in the SSTR curriculum. The processes to apply for SSI/SSDI and receive a disability determination are complex and difficult to navigate. The SSTR curriculum is designed to make case managers more knowledgeable about the disability application process by delivering the information and tools needed to effectively guide those who are homeless through this process. The SSTR curriculum consists of sixteen modules, each of which provides an in-depth explanation of a unique aspect of the process and pointers on assisting clients through it.

The focus of the four-day TTT sessions is split between the content of the SSTR curriculum and preparing the trainers to train others to be able to implement the curriculum when they return to their states. The TA contractor sets a tone for the sessions that is centered on learning, welcoming audience questions and experiences, and developing trainer buy-in to the importance of understanding and implementing each of the SOAR critical components. Incorporated into the TTT sessions are training activities and strategies that include icebreaker activities, small prizes to reward participation and collaboration, small group activities, role playing to allow participants to gain experience training on a given module, and opportunities for trainer and peer feedback. At the end of each day, the TA contractor reviews with participants areas they found most useful as well as areas where they needed additional instruction. The TA contractor encourages the trainers to adapt these activities and strategies for use during the in-state trainings in their home state.

The TTT model used in SOAR offers several benefits. First, it offers a way for states to sustain their efforts if federally funded TA is no longer available. Second, it offers a way for states to tailor trainings to their individual communities and to take ownership of SOAR. And third, it enables training to reach a volume of local case managers that would not be feasible if the TA contractor were to conduct all of the trainings. However, the efficacy of the TTT model depends on the qualifications of the in-state trainers, the type and amount of support trainers receive from the state and the TA contractor, and the extent to which there are systems in place to handle turnover among trainers. Below we describe how the trainers were selected and their general qualifications as well as trainers' experiences with the TTT sessions.

1.  Selection of Trainers

The TA contractor conducted a total of 12 TTT sessions, the first of which was a pilot training session. In total, 202 individuals from the 25 Rounds One and Two states attended a TTT session. States generally sent trainers from a mix of organizations  often one or more state agencies and one or more CBOs. Approximately 60 percent of training participants were from CBOs and the majority of the others were state, county, or municipal staff. In a handful of states, staff from DDS attended the TTT session. From the six states we visited, 33 individuals attended a TTT session, about a third of whom were from lead agencies that were instrumental in implementing SOAR. A few state or local leads became trainers and conducted regular trainings. Although this had the advantage of helping state leads engage with case managers and get a fuller picture of the initiative, some talented and dedicated state leads did not have the time or background necessary to become successful trainers.

Attendees at the TTT sessions had mixed professional backgrounds. About two-thirds were social workers, case managers, and benefit specialists, and most of the remaining attendees were supervisors and program managers. Generally, trainers who were case managers or benefit specialists from active stakeholders were more likely to have ongoing involvement in SOAR because they were invested in the initiative, had agency-level support to implement SOAR critical components, and could identify personally with the challenges and potential returns of integrating SOAR concepts into their jobs. However, some case study states intentionally sent a mix of case managers and supervisors to the session to provide some continuity to the initiative, in light of high case manager turnover.

Attendees at these sessions also had varying degrees of knowledge of SSI/SSDI processes and experience as trainers. In one state, a PATH clinician felt confident delivering SOAR trainings because she had conducted SSI/SSDI benefit assistance trainings for both shelter staff and advocates, using training materials she had designed prior to SOAR. However, some trainers were initially overwhelmed because they did not have a strong background in homelessness or SSI/SSDI processes. One such trainer leaned heavily on the TA contractor before the first in-state SOAR training until she gained confidence as a trainer. In another state, a trainer with substantive knowledge of SSI/SSDI processes re-trained her colleagues after the TTT session because they were overwhelmed by the material. These findings suggest that prior knowledge of SSI/SSDI processes and experience conducting trainings could be instrumental in ensuring the quality of subsequent in-state trainings. They also suggest that trainers who do not have strong backgrounds in homelessness or SSI/SSDI process could benefit from more intensive TA before conducting the first in-state training.

Turnover among trainers and resource constraints sometimes stymied in-state trainings even before they ever materialized. For example, one state sent only two individuals to the TTT session because it could not afford to send others with state funds; both vacated their positions within a year after SOAR was implemented. In another state, the two original trainers were not able to conduct any in-state trainings because their agencies could not afford them the time to do so.

2.  Trainers' Experiences with the TTT Sessions and Follow-Up Support

TA contractor staff were praised for their enthusiasm, facilitation skills, and in-depth knowledge during the TTT sessions. Overall, 95 percent of in-state trainers reported they were either satisfied or very satisfied with the way in which the TTT session was organized and presented. Moreover, 99 percent of in-state trainers agreed or strongly agreed that the training information in the SSTR curriculum was presented clearly, and 97 percent reported that the training was organized in a way that was conducive to delivery and learning.^[5]

Trainers' perception of the SSTR training material varied depending on their professional background. Overall, 99 percent of trainers agreed or strongly agreed that the SSTR training materials were comprehensive and well-designed. Some trainers with extensive experience working with those who are homeless, however, did not need as much training on strategies for working with  this population, individuals with mental health issues, or on SSI in general. Others who did not have strong backgrounds in either social security or homelessness were overwhelmed by the content. These findings underscore the difficulty in providing training to a diverse audience and suggest that the TTT sessions may need to be tailored to accommodate the information needs of the audience, that different levels of training may need to be offered, or that the TA contractor may need to require states to select trainers with certain prerequisites.

The content covered during the TTT sessions provided valuable information on the SSI/SSDI application process. A total of 99 percent of the trainers agreed or strongly agreed that they have a better understanding of the process and their role as a trainer teaching case managers about the processes. All trainers completed a 12-item questionnaire at the start and end of the session to assess gains in knowledge along several key dimensions. The questionnaire is included as Appendix B. An analysis of the questionnaires from nine TTT sessions indicates that the TTT sessions indeed resulted in substantial gains in knowledge among trainers about SSA policies and SSI/SSDI application processes. At the start of the training sessions, the mean number of correct responses across the 287 trainers was 69 percent. The mean number of correct responses at the conclusion of the training sessions was 89 percent, a statistically significant 20 percentage point increase relative to the start of the session.

Ongoing Technical Assistance

The TA contractor works with SOAR states after the strategic planning forum for up to 18 months to support their implementation efforts. In addition to observations of the first in-state training and provision of feedback to trainers to ensure adherence to the SSTR curriculum, the TA contractor provides a minimum of three activities for all states: (1) regular teleconferences to share resources and information on important SOAR issues; (2) periodic communication with SOAR state leads to check in and assist in addressing any challenges encountered; and (3) dissemination of information and resources (through a website and written material) on promising practices and SOAR outcomes.

The TA contractor is available and eager to provide additional tailored services to SOAR states as needed, but many states have not reached out to take advantage of this opportunity. For example, one state lead admittedly could have used, but did not request, TA on how to reengage state partners to assist local communities that were struggling with implementing SOAR. Reasons for states' reluctance to request assistance are unclear. It could be that states are not aware of the potential for ongoing TA or do not have the time or motivation to seek it out. Also, some states may not have requested assistance from the TA contractor because they felt they had sufficient supports in place at the local level.

States that are proactive in accessing TA have found that assistance very useful in helping them implement SOAR and progress toward effecting systematic changes. One state has had quarterly teleconference calls with the TA contractor to assist in their efforts to obtain a waiver so that state hospital staff can submit SSI/SSDI applications more than 30 days prior to release from the facility. In another state, the TA contractor has provided feedback on how the SOAR model can be used with the TANF population. In yet another state, follow-up contact between the TA contractor, the state lead, and other stakeholders was instrumental in addressing early implementation issues and simply getting the initiative off the ground.

Looking Ahead

The TA contractor puts substantial effort into paving the way for SOAR to take shape in state and local communities through three key activities: strategic planning, the TTT program, and ongoing implementation support. However, to fully exploit the activities and the contractor's expertise states must be active participants in the TA process. For example, they need to enlist the appropriate stakeholders from the outset, purposefully and carefully identifying leads that will become actively engaged in SOAR and trainers with the appropriate background, qualifications, and commitment. And, states must not only develop thoughtful and specific action plans detailing the implementation process, but take ownership of those plans and seek ongoing TA offered by the contractor to assist in carrying out implementation plans. Various tools could assist states in these efforts in the future, such as job descriptions for state leads and trainers, memoranda of understanding specifying the roles of each stakeholder, and a schedule for ongoing communication between the state and the TA contractor. A broader focus on strategic planning activities  for instance, by conducting several forums within a state  could also help states engage the appropriate stakeholders and devise and then implement realistic and incremental action steps toward implementation. Whether and how the case study states were able to capitalize on the current TA activities is the focus of the next chapter.

SOAR Outputs

The SOAR activities facilitated by the TA contractor lead to outputs that enable communities to achieve the desired outcomes of the initiative. This chapter focuses on the three key outputs identified in the SOAR logic model in Chapter 1. Section A describes factors that influenced organizations' commitment to SOAR and the nature of cross-agency relationships and collaborations that developed in SOAR communities. Section B describes the in-state SOAR trainings. Section C describes the implementation of SOAR's critical components within each state.

Stakeholder Commitment and Collaboration

The success of SOAR depended on the extent to which stakeholders committed time and resources to the initiative, and on their willingness to collaborate. We identified at least four factors that influenced stakeholders' commitment to SOAR:

1. Encouragement and support from state leads and partner organizations. Strong leaders marketed SOAR as an opportunity for stakeholders to advance organizational goals and worked persistently to engage reluctant partners. Dedicated staff sometimes supported state leads with this work; for example, stakeholders in one state credited a case manager's persistent legwork with building a strong regional partnership with DDS. Strong state leads looked at SOAR efforts holistically and helped troubleshot problems and identify ways to strengthen the initiative.
2. Supervisory support for SOAR participation.  Organizational leaders were more likely to encourage staff to dedicate resources to SOAR if it fit into an ongoing initiative (for example, a state's ten-year plan to end homelessness), and if the organization was not overburdened with other projects. Staff at all levels were more likely to participate when their supervisors were excited about the initiative.
3. Correspondence between SOAR activities and stakeholders' core job functions. Improving the quality of SSI/SSDI applications is a natural goal for DDS and SSA, and helping clients get benefits is often part of comprehensive case management, particularly among those who specialize in benefits acquisition. For other stakeholders, making SOAR successful required work beyond their core responsibilities. State leads and trainers needed to dedicate time and energy to the initiative's success in organizations other than their own, with state leads working to build and maintain cross-agency relationships and trainers reaching staff from an array of organizations. Organizations and individuals often made an initial commitment to SOAR but then were forced to turn their attention to other more urgent organizational priorities. Some continued to contribute to SOAR even though it was not integral to their job; these staff typically had support from their supervisors and/or were personally enthusiastic about SOAR.
4. Staff turnover.  Across participating organizations, staff at all levels changed frequently, and turnover often hindered inter-agency collaboration. Turnover often meant that organizations lost expertise, enthusiasm for SOAR, and institutional knowledge. Collaboration was particularly stymied when state or local leads or staff that had used personal relationships to build collaboration with other organizations changed jobs. When champions of SOAR within a particular institution left, their replacements did not necessarily advocate for continued organizational support for SOAR. Turnover sometimes had a ripple effect, with partners becoming frustrated when another organization's involvement decreased.

At the strategic planning forums, which laid the foundation for future collaboration, stakeholders developed a work plan and agreed to communicate regularly about the initiative. Follow-up meetings and communication after the strategic planning forum, however, required leadership, either from the state lead or someone else. Some state leads did not clearly understand their role as the facilitator of continued communication between stakeholders. In most states, the TA contractor organized and played a role in recruiting stakeholders to participate in the strategic planning forum. Thereafter, state leads may not have realized that they had to take the reins in fostering ongoing collaboration, or that doing so would be timing-consuming and challenging. Without ongoing communication facilitated by a leader, partners expressed confusion over next steps and relationships had little opportunity to blossom.

Regular meetings between stakeholders were essential to develop lines of formal and informal communication, build on relationships, and overcome challenges. States that had mechanisms for ongoing structured communication were better able to keep partners engaged. For example, some states had quarterly meetings that brought together case managers, SSA and DDS staff, and senior leadership to share their experiences with SOAR and develop strategies for overcoming any challenges. These meetings provided an opportunity to hear positive feedback and solidify interagency relationships.

States in which stakeholders communicated regularly in a structured manner generally had more positive outcomes than others. Communication came most readily when stakeholders had partnerships prior to SOAR implementation, but successful state initiatives helped stakeholders strengthen existing relationships and build new partnerships.

Figures V.1 through V.4 illustrate the importance of collaboration to the utility of the SOAR model. They illustrate communication patterns before and after implementation of SOAR in, respectively, one community that was unable to demonstrate any application outcomes and one that demonstrated positive application outcomes. As the legend delineates, line types represent how frequently entities reported communicating with one another. Sometimes stakeholders' perceptions of how frequently they communicated differed, so we provide separate lines for each perspective; arrows show the direction of communication.

In the state depicted in Figures V.1 and V.2, most stakeholders reported a minimal or moderate level of communication before and after SOAR implementation. The state lead remained largely disconnected from other organizations. After SOAR implementation many stakeholders reported more frequent contact with SSA, but SSA did not report increased contact with any other entity.

In contrast, in the state depicted in Figures V.3 and V.4, communication among stakeholders increased noticeably after SOAR. Each stakeholder reported some level of communication with all other entities before SOAR implementation, but many stakeholders communicated more frequently with these groups after the initiative was in place. The state lead, who previously had a moderate amount of communication with all other parties, reported frequent communication with all stakeholders after SOAR implementation, and SSA and the in-state trainers also reported communicating more frequently with other participants. Neither figure includes case managers because members of this diverse group had varying levels of communication with other stakeholders. In the state depicted in Figure Y, however, case managers meet regularly with other stakeholders, including trainers, the state lead, SSA, and DDS.

Figure V.1
Communication Before SOAR in State with No Outcomes

Figure V.2
Communication After SOAR in State with No Outcomes

Figure V.3
Communication Before SOAR in State with Positive Outcomes

Figure V.4
Communication After SOAR in State with Positive Outcomes

The success of SOAR depends particularly on relationship building among state and local leads, SSA, and DDS field offices, medical providers, trainers, and case managers. In the following section we describe how and the extent to which each of these stakeholders developed relationships with the others and the contributions that each made to SOAR.

1.  State and Local Leads

The role of the state lead in the SOAR model is multifaceted. Ideally, state leads work to (1) encourage initial and ongoing participation of stakeholders; (2) help stakeholders identify resources for SOAR; (3) facilitate ongoing stakeholder communication, informally and/or through regular meetings; (4) help organizations identify and overcome potential challenges of staff turnover; and (5) coordinate outcomes tracking and submit data to the TA contractor. Formal or informal local leaders could supplement this work, drawing on their local contacts and knowledge of the service system.

In practice, state leads varied considerably in the time and energy they dedicated to SOAR, and in the extent to which they were able to provide effective leadership. Fostering participation and collaboration was easier when strong partnerships already existed, and when the state lead was a respected community leader or held a relatively senior position within a prominent organization, particularly a state agency. For example, one state lead was the director of the state's Homelessness Task Force, and consequently had strong connections to a range of stakeholders, the stature to recruit new partners, time and energy to dedicate to strategic planning and partnerships building, and insights about how different partners could enhance SOAR. In contrast, some state leads lacked the time, resources, or sense of empowerment necessary to facilitate SOAR's success outside of their own organization. For example, one lead from the state PATH organization encouraged PATH grantees to dedicate staff time to SOAR and build collaboration between with SSA and DDS, but did little to engage other organizations, in part because the lead felt that she did not have the authority or clout to influence the activities of a different state agency. Some state leads made initial efforts to engage other entities, but did not follow-up after their initial efforts floundered.

While state-wide SOAR efforts generally struggled without a strong state lead, official or unofficial local leads were sometimes able to propel the initiative forward in their communities. For example, one state did not have state-level leadership for several months due to staff turnover but a motivated case manager supervisor in one community kept the initiative active during that time. While the SOAR model calls for a state and local lead in each pilot community during initial implementation, no state we visited had both consistent strong local and state leadership. However, if SOAR expands to multiple communities within more states, complementary state and local leadership may become more essential. Local leads would be responsible for community-specific partnerships and challenges, and state leads would coordinate efforts, identify and troubleshoot challenges, and facilitate relationships at the state or regional level (for instance, with SSA and DDS).

2.  SSA and DDS

Cooperation between case managers and SSA and DDS is a SOAR critical component. Ideally, SSA and DDS staff will understand the challenges SOAR applicants face, accommodate those difficulties to the extent possible, and develop relationships with case managers to whom they can turn for additional information about applicants. Better access to applicants and case managers, along with improved application quality, can increase the number of applications approved and decrease processing times.

While SSA and DDS are large, bureaucratic entities, regional and local offices have some autonomy over practices and procedures, and have the potential to adapt them to facilitate SOAR's objectives. One way that SSA and DDS offices modified procedures was by flagging SOAR applications. A flag or notation on the application serves as a signal to agency staff that they can contact the SOAR case manager with questions, and helps them contextualize the applications. Flags also facilitate the tracking of SOAR outcome data. Some states developed a flag simply by using a cover sheet on paper applications or through an electronic data system. SSA's online application system does not have a field to record that the application is completed by a SOAR worker, and creating such a field would require changes to SSA's software at the federal level. However, states were creative about developing workarounds (for example, flagging a case as homeless and writing SOAR in a comments field). Stakeholders sometimes had different understandings of what homeless or SOAR flags meant. For instance, sometimes case managers defined anyone at risk of being unsheltered as homeless, but DDS and SSA had a more stringent definition of homelessness as being currently unsheltered. Miscommunication about what flags meant sometimes undermined partnerships. Strong leaders, however, were generally able to work with stakeholders to clarify definitions.

Another way SSA and DDS offices modified procedures was by designating specific staff to process SOAR applications. This provided case managers with a specific contact person who developed an in-depth appreciation for the challenges involved in navigating the SSI/SSDI application process for individuals who are homeless. Some SSA and DDS staff reported that designating workers to process SOAR or homeless applications improved processing time. But, this practice is not without challenges. In particular, it was difficult in some locations to sustain designated staff due to turnover. In addition, some SSA and DDS offices perceived that the volume of SOAR applications was too low to justify specialized staff. Some dealt with this challenge by designating staff who would be responsible for all SOAR cases in addition to a reduced regular caseload.

The extent to which SSA and DDS were willing to modify procedures by flagging applications or designating staff varied, often based on how worthwhile leaders perceived that SOAR would be to their agencies. While SOAR cases are a small percentage of all SSA applications, some office managers recognized that SOAR can help decrease overall workloads (if case managers submit higher quality applications and facilitate access to applicants) and were therefore eager to make small changes to accommodate SOAR. Generally, engaging SSA and DDS required the persistence of SOAR leadership and the marketing of SOAR's potential benefits, but state or local SOAR leaders did not always have the time to invest in these activities. SSA and DDS leaders were most receptive to SOAR when they saw the initiative as congruent with an existing organizational priority or initiative, or when they already had relationships with other SOAR partners. Generally, field offices were more proactive in making changes when regional offices communicated their support for SOAR.

Even when SSA and DDS offices did not formally modify procedures, SOAR often fostered more frequent contact between field office staff and case managers. When communication increased, field office staff reported receiving more complete applications, and stated that by connecting them to case managers, SOAR made the process of obtaining additional information easier. Case managers reported that they submitted better applications when they could contact SSA and DDS staff with questions, and when they received feedback on their applications.

3.  Medical Providers

Collaboration with medical providers has the potential to enhance SOAR. Case managers and DDS rely on the medical community to provide records and perform CEs for SSI/SSDI applications. However, many medical providers are not aware of the criteria DDS uses to evaluate cases or of DDS's terminology, which is often different from providers' standard medical language. With greater awareness of the SSI/SSDI application process, medical providers could tailor their existing work and strengthen SOAR applications. In addition, application processing times have the potential to improve when DDS locates and receives medical records quickly. Further, medical providers could encourage their patients to apply for SSA benefits, not only because benefits would have a positive impact on the lives of their patients, but also because those who are approved for SSI/SSDI qualify for Medicaid, which can then reimburse medical providers for the cost of uncompensated care. The TA contractor reported that several states that were not included in our case studies developed partnerships with medical providers that led to cost recovery (see Appendix A).

Despite the potential benefits of engaging medical providers in the SOAR initiative, many states have not integrated the medical community into their efforts. Only two of the six case study states formally integrated medical providers into their SOAR initiatives. In one, the Veteran's Administration (VA) sent several case managers to a TTT session, and conducted three trainings for VA staff. In another, stakeholders have made extensive use of Healthcare for the Homeless (HCH), a grant program funded by the HRSA to provide primary care (including primary health care and substance abuse services, emergency care and referrals, outreach and assistance in qualifying for entitlement programs and housing) to those who are homeless. In this state, HCH directs all SSI application issues to one doctorand-nurse team, which also conducts a benefits clinic once a week. DDS had recently begun using HCH doctors to perform CEs. This arrangement was expected to better serve homeless applicants because they frequently access care at the clinics and many have an established relationship with the medical doctors. The VA and HCH are two entities that could be engaged in SOAR at the national level; the relationships described here illustrate the potential benefits of closer collaboration. In some states, case managers routinely request copies of medical records from providers, but SOAR leaders have not tried to streamline this process or build more formal relationships. Providers often have different forms and releases required before medical records may be accessed and the process of completing these can be time-consuming. Developing a common form could streamline this process, but would require buy-in from a diverse range of providers. Case managers in other states do not pursue copies of medical records because providers give copies to DDS for free but require CBOs to pay for copies. In addition, they perceive DDS as the agency responsible for obtaining medical records and as being able to do so most efficiently. In many states, SOAR leaders are still concentrating on getting the more basic components of the initiative in place, such as arranging for in-state trainings and building relationships with SSA and DDS, and therefore have not yet focused on developing medical provider relationships.

4.  Trainers

The intent within the SOAR model is for state leads and trainers to collaborate on when, where, and to whom the in-state SSTR trainings should be delivered. For SOAR to be successful, stakeholders need to target trainings towards the appropriate case managers. Sometimes, however, decision making around in-state trainings was not collaborative. Rather, state leads dictated decisions around in-state trainings to the trainers, or trainers assumed sole responsibility for the in-state trainings without consulting the state lead. The former typically happened when stakeholder collaboration was weak, and the latter typically happened when state leadership was weak. A variety of models can foster collaboration between the state lead and trainers. In some states, the state lead was a trainer, or active trainers were part of the state or local lead's organization. When the state lead was a trainer or communicated closely with the trainers, the lead was aware of on-the-ground implementation issues and connected to case managers.

Another intent within the SOAR model is for trainers to provide ongoing assistance to case mangers after the in-state trainings to submit SOAR applications and work through early implementation challenges. Trainers varied in their ability to provide this type of assistance. Generally when followup did occur, case managers initiated contact and did so only when they had a prior relationship with the trainer. After the in-state trainings, case managers rarely contacted trainers with whom they did not already have a relationship. Trainers who were employed by the lead agency were more likely to develop relationships with other stakeholders because they were invested in the initiative and tended to have agency support. Trainers had more difficulty providing follow-up assistance and conducting on-going trainings when they were not employed by the lead agency, in part because of other demands on their time.

5.  Case Managers

Case managers use the SOAR training to help individuals complete SSI/SSDI applications and navigate the application process. Peer support can bolster case managers' efforts to implement SOAR critical components and provide ideas and encouragement for addressing challenges. This support was most feasible when case managers were already connected through existing networks (for example, when case managers in different organizations were supported by the same funding streams or part of a larger umbrella organization). Some states held quarterly meetings that brought together case managers and SSA and DDS staff along with state leaders to share promising practices and troubleshoot any challenges. These meetings provided feedback on the application process and application outcomes and served as an opportunity to share in their success, which encouraged case managers and SSA and DDS to continue their work.

One of the most important relationships that can facilitate access to benefits for individuals who are homeless is that between case managers providing application assistance and SSA/DDS staff processing the applications. Case managers often had to take substantial initiative to develop relationships with SSA and DDS staff. Within a single locality or even a single organization, some case managers took the initiative to reach out to SSA and DDS staff and some did not. Case managers found it easier to develop relationships with SSA and DDS employees who were designated as SOAR contacts, or who attended in-state trainings. In most states, agency representatives regularly attended trainings, and case managers reported that learning more about DDS and SSA procedures from an agency representative was invaluable. In some localities, DDS and SSA representatives gave presentations at the trainings on how to prepare quality applications and foster ongoing communication with their respective agencies. In other localities they participated minimally, introducing themselves and adding a few comments about their agency's work.

In-state Trainings

The in-state trainings provided an opportunity for case managers to learn specific skills to assist homeless individuals complete SSI/SSDI applications. This section describes the recruitment and participation of case managers in the trainings, the extent to which in-state trainings adhered to the SOAR curriculum, and case managers' perceptions of the trainings.

1.  Outreach to and Recruitment of Case Managers

For in-state trainings to be useful, it is critical that stakeholders identify the right set of individuals to be trained. Some states initially tried to train as many case managers as possible. To recruit participants, training organizers leveraged existing networks by reaching out to organizations that had already agreed to participate in SOAR as well as other partners. In many communities, information about trainings also spread by word of mouth. States often recruited supervisors along with case managers for initial in-state trainings. Some states included supervisors to build enthusiasm for SOAR, and others thought that since case managers turned over rapidly, educating supervisors would provide continuity. Later, many state leads and trainers agreed that supervisors did not benefit from learning all of the details in the full two-day training.

Recruitment practices tended to change as the initiative matured, becoming more narrowly targeted. As states refined SOAR efforts, they targeted trainings to organizations that were participating in SOAR, particularly to case managers who would actually be responsible for assisting homeless individuals. And, among those, in order for training to have the most impact, states began targeting training to case managers who had the most time and resources to implement SOAR and who had the support of their supervisors. Because of staff turnover, however, it was often not enough to offer training to case managers from a particular agency just once. The availability of training to new case managers who replaced those who left varied based on organizational commitments and the availability of trainers.

For in-state trainings to be useful, participants and their supervisors must also understand the commitment involved in and expectations associated with attending training. In some communities, the state lead or trainers did little to build attendees' enthusiasm for SOAR or enlist their supervisors' support before the training sessions. Case managers were also not obligated to commit to using SOAR prior to attending the training. As a result, they and their supervisors sometimes viewed SOAR training as one of many staff development programs and not as an opportunity to develop new skills or a mandate to put those skills to use.

2.  Extent of In-State Trainings

The number of trainings in states we visited ranged from two to twelve. The number of participants at those training varied substantially  from fewer than ten participants to as many as sixty per training. Formal contracts helped to ensure that trainings were conducted as planned. For example, in one state, a local organization had a contract from a city government office to conduct at least one SOAR training per year, complete SOAR applications, convene stakeholders, and track data. Another had a contract with the state PATH agency to conduct 4 trainings per year. In each state, trainings occurred as planned. Other states were able to conduct trainings regularly without issuing formal contracts, but still others conducted trainings more haphazardly and not all that were intended to occur actually did.

The number of in-state trainings, however, is not necessarily a good indicator of much effort states have made to implement SOAR or how well states are implementing it. Some states conducted a large number of trainings but completed few SOAR applications and do not have infrastructure to support trained case managers. Other states held few trainings but have completed a relatively large number of applications and/or have achieved notable application approval rates. More important was whether states targeted trainings to case managers with the time and resources to implement SOAR, and whether trainers had support from their supervisors to spend time and resources providing ongoing assistance to case managers and other stakeholders.

3.  Fidelity to the SSTR Curriculum

The TA contractor provided states with assistance designed to ensure fidelity to the SSTR curriculum. The TA contractor observed the first in-state training and provided trainers with detailed feedback. The tool that the TA contractor used to provide feedback is included in Appendix C. This tool provided numerical ratings and comments on trainers' fidelity to several elements of each module of the SSTR curriculum. We analyzed these fidelity assessments from 24 of the 25 Rounds One and Two states and found that, overall, most trainers covered all of the curriculum's modules and taught the critical components described in Chapter I. While the TA contractor offered constructive criticism to each trainer to improve their approach, this feedback rarely changed how trainers conducted the trainings.

As the initiative progressed, trainers tailored the SSTR to their local community and the perceived needs of trainees. Some trainers conducted abbreviated trainings and omitted content regarding medical summary reports. The SSTR curriculum covers a great deal of material in a short amount of time, and some trainers saw a tension between providing comprehensive information and not overwhelming case managers. Many trainers abbreviated the curriculum to focus on the essentials of the application process rather than the relationship-building components of the initiative or the sections on engaging and empathizing with homeless clients. They deemed the latter unnecessary since most case managers came to the in-state trainings with substantial experience working with homeless individuals The extent to which trainers provided instruction on completing medical summary reports varied according to DDS staff feedback on how useful they deemed the reports. Some DDS staff reported that they did not use the medical summary (preferring strictly medical evidence without a third party's observation of functionality) while others found it useful.

In several states, informal or substantially modified trainings sometimes supplemented or replaced in-depth SOAR trainings. One organization designed special trainings for benefits specialists, which were conducted by a DDS examiner and DDS psychological consultant; these three and a half hour trainings focused on DDS procedures and requirements. These specialists did not receive the complete SSTR training, but had SOAR manuals available for their reference. Organizational leaders thought that these procedural trainings better met the needs of the benefit specialists than the SOAR training, which they perceived to be designed for clinical staff. In another state, the state lead gave brief presentations to social service organizations regarding the SOAR initiative to raise awareness about SOAR practices, and conducted six-hour trainings for case managers that excluded the community-building components of the SSTR curriculum.

4.  Participants' Experiences with Trainings

Case managers' feedback about the in-state trainings was overwhelmingly positive. Almost all participants interviewed during our site visits agreed that the training helped them navigate the benefit application process and that the trainers were knowledgeable and engaging. An analysis of data from 470 participant evaluation forms from initial trainings in 23 of the 25 Rounds One and Two states confirmed these positive perceptions. Overall, 97 percent of the respondents agreed or strongly agreed that the training improved their understanding of the disability determination process and how to develop medical evidence, and 96 percent agreed or strongly agreed that the training would help them assist clients with SSI/SSDI applications. Respondents also praised the presenters and the curriculum content. The evaluation form is attached in Appendix D.

In the first round of in-state trainings, participants demonstrated a statistically significant increase in knowledge about the SSI/SSDI application process. We analyzed the results of a pre-post true-false questionnaire administered by the TA contractor during the training (the questionnaire is attached in Appendix B). Because the test changed slightly between Rounds One and Two, we analyzed these data separately. Pre-post data were available for six of the fourteen Round One states and eight of the eleven Round Two states. Table V.1 presents the results.

Despite the positive feedback and short-term knowledge gains, case managers, particularly those with little prior experience with the SSI/SSDI application process, often left the trainings overwhelmed at the prospect of having to put the training information into practice. On questionnaires administrated after the first round of in-state trainings, 46 percent of respondents agreed and another 30 percent strongly agreed that the pace of the training was just right not too fast and not too slow. However, during our site visit, some trainers and case managers complained that the training covered too much material too quickly, and that it was unrealistic to expect case managers to obtain a level of knowledge they could apply in practice in two days. Some thought it beneficial for case managers to attend multiple refresher trainings. Others, particularly case managers with prior background in SSI/SSDI applications, said that the training was intense but they appreciated the comprehensive curriculum.

Implementation of SOAR's Critical Components

After training, some case managers were able to integrate SOAR into their regular practice and prepare high-quality SSI/SSDI applications. Generally, these case managers had time and organizational support to prepare SOAR applications, were enthusiastic about the initiative, or both. They found some SOAR critical components more or less useful than others. Case managers almost universally became clients' authorized representatives and found this practice extremely useful. One of the most contended components was the medical summary report. Case managers expressed confusion about the purpose of the medical summaries and anxiety and misunderstanding about how to write them. In light of this, and in response to time constraints and/or feedback they received from DDS, many case managers did not complete them. In one state, DDS encouraged case managers who lacked the time or ability to write full medical summary reports to call the assigned DDS contact to give an oral report, to fill out a short functional observation form that DDS developed, or both. The two-page focuses on activities of daily living (ADLs) that are relevant for homeless applicants; the standard thirteen-page form that DDS often uses for the general pool of SSI/SSDI applicants was too long and not entirely applicable.

More often, case managers found in-state trainings informational, but not transformational; many never or rarely implemented SOAR in practice. Simply obtaining information about the SSI/SSDI application process has value in and of itself, but the intent of SOAR is to alter case managers' behavior. Many case managers felt that they did not have time to prepare SOAR applications. Case management staffs were often overburdened with other responsibilities and faced budget cuts that affected client to staff ratios. In some sites, time spent on paperwork (including benefit applications) that did not entail face-to-face time between case managers and clients was not billable. Moreover, if case managers did not implement SOAR soon and frequently after training, the skills they learned diminished over time.

Despite the wealth of information it contained, many case managers did not use the SSTR manual as a reference and felt the need for refresher training. One trainer did offer follow-up sessions regarding submitting online applications, which she described as an opportunity to give participants a booster shot regarding SOAR critical components; other trainers encouraged participants to attend multiple training sessions designed for case managers who are new to SOAR.

Given time and resource constraints, assisting homeless individual with filing SSI/SSDI applications using the SOAR model had the most potential when CBOs dedicated one or more SOAR specialists to provide application assistance and enabled other case managers to maintain their current responsibilities. Five states were able to dedicate benefit assistance specialists either by shifting responsibilities around within agencies' current organizational framework and resources or by obtaining funding for newly created staff positions. Three states used PATH monies to fund full- or part-time benefits acquisitions specialists, because state leaders saw SOAR as congruent with PATH's goals and because SAMSHA supported using PATH funds for this purpose. One state welfare agency created new positions focused exclusively on benefit assistance, hoping that these positions would ultimately allow the state to recoup General Assistance expenditures. In the fifth state, a city agency redirected funds, awarding a contract to implement SOAR to a CBO with which they had partnered on a previous anti-homeless initiative. Having specialized staff, however, did not guarantee the collaboration and commitment necessary to yield positive application outcomes. In two states, during periods of staff turnover SOAR specialists received little supervisory support and had weak relationships with other stakeholders; the frequency and quality of their SOAR applications diminished during this time.

Looking Ahead

The extent to which states achieved SOAR outcomes depended on how much time and resources stakeholders invested in the initiative. Ongoing success required sustained leadership and structured communication between all stakeholders. Strategically targeting partners with continued commitment to SOAR, however, was equally important; states with fewer, committed partners had more success than those with numerous partners whose contributions were minimal and/or faded over time. The TA contractor provided states with solid training and support, but to a large degree, the extent to which states succeeded in implementing SOAR depended on factors outside of the TA contractor's control. Nonetheless, federal funders and the TA contractor could spark improved in-state commitment and collaboration. The next chapter discusses considerations for facilitating such improvements.

Conclusions and Discussion

SOAR provides states with an opportunity to engage in strategic planning, develop and nurture inter-agency relationships, and receive training and TA to improve the SSI/SSDI application process for individuals who are homeless, particularly those who have mental health conditions. When implemented with adequate staff resources and commitment, SOAR improves the capacity of case managers to navigate the SSI/SSDI application process and has the potential to increase the SSI/SSDI application approval rate while decreasing processing time. The ultimate benefits of the initiative to states and their homeless populations are directly proportional to states' investment of time and resources. This chapter identifies some common elements that facilitated successful implementation of SOAR and provides considerations for the initiative's future.

Factors That Facilitate Successful Implementation of SOAR

States varied widely in the extent to which SOAR was implemented with fidelity to the model and to which their efforts were sustained after the initial in-state planning forums and trainings. Based on the interviews, focus groups, and analysis of data collected in the states selected for the case studies, several factors appeared to contribute to the successful implementation of SOAR across communities. As illustrated in Table VI.1, many of these factors were missing in states that struggled to implement SOAR.

• Strong and Consistent Leadership.  Consistent leadership was essential at all phases of the project. At the state level, leadership was needed to coordinate the in-state trainings and empower and facilitate on-going communication among key stakeholders. Effective state leads had a genuine commitment to SOAR and an understanding of SOAR's goals and potential benefits, both to individuals who are homeless and to their own agencies.
• Agency-level Support for SOAR.  Support from their agency management and direct supervisors empowered state leaders to dedicate the time and energy needed to coordinate trainings, troubleshoot challenges, and facilitate regular communication among stakeholders to sustain the initiative. Without such buy-in, particularly in the absence of financial support for SOAR, state leads were less able to treat the initiative as a priority. In states where the state level leadership of SOAR changed due to staff turnover, buy-in from the state leader's agency and direct supervisor helped to ensure that the initiative did not flounder during the transition to new leadership. In addition to the state lead having support, developing buy-in at the highest possible levels of other agencies that participated in SOAR, including those agencies in which case managers worked, was essential to ensuring that the initiative was sustained. In several states, support for SOAR came from the same agency that administered PATH funding. There may be potential for states to develop greater synergy between the activities of the PATH and SOAR initiatives.
• Engagement of SSA and DDS.  Active participation of SSA and DDS in all phases of the initiative helped to ensure that the development of SOAR-specific application processes and procedures was well-received and fit well within standard local SSA and DDS office practices. The engagement of SSA and DDS allowed communities to better tailor SOAR to overcome the particular challenges to obtaining SSI/SSDI specific to their communities. The more deep-rooted engagement among SSA and DDS was, the more beneficial it was to the initiative. Communities in which SSA and DDS were most engaged had the commitment of regional and local office staff, including front-line staff, their supervisors, and senior management. These staff were often engaged in the planning of activities but also participated in the in-state trainings and provided ongoing feedback on application outcomes to case managers and the state lead.
• Structured Inter-Agency Communication.  The success of SOAR is largely dependent on developing and maintaining strong interagency collaborations to develop application procedures, monitor the progress of the initiative, and continuously improve the application process. Consistent communication among SSA, DDS, case managers, and the state and/or local lead was essential to solidify inter-agency collaborations, advance SOAR, and overcome challenges to implementation. While informal frequent communication was often facilitated through existing relationships, several states conducted formal monthly or quarterly meetings among these stakeholders to foster regular structured communication. These meetings provided an opportunity to discuss any challenges to completing application, gather feedback on the quality of applications, and provide ongoing training. Further, they provided stakeholders with motivation to continue the initiative and were a forum to celebrate the successes of SOAR.
• Identification of Qualified Trainers.  In-state trainers who had some existing familiarity with the SSA application process and support from their supervisors were well-positioned to train case managers in the SOAR curriculum. Trainers who had no previous experience with the SSA application process required more support to become competent trainers, and in some states, never conducted any trainings.
• Supervisory Support for Case Managers and/or Dedicated Benefit Specialists.  Case managers were empowered to use the critical components of the SOAR curriculum when they had support from their supervisors to dedicate the time required to assist individuals who are homeless with applications. To the extreme, some states dedicated staff exclusively to providing SSI/SSDI application assistance to these clients. States that employed this model were best able to submit a volume of high-quality applications.
• Targeted Implementation.  States that piloted SOAR in a small number of local sites were able to hone the SOAR approach and troubleshoot challenges, learn from their early experiences, and then spread the initiative to other areas of the state. Those that attempted to implement SOAR in more than three communities directly following the in-state planning forums, and particularly those that attempted to implement SOAR statewide, experienced greater difficulties coordinating activities and engaging in ongoing communication.
• Outcome Data Collection.  States that made an effort to systematically track outcomes at the beginning of the initiative were better able to monitor the progress of activities, troubleshoot challenges, and overcome barriers to success. The tracking of outcomes developed a sense of accountability and provided the data necessary to encourage stakeholders to sustain and propel the initiative forward. Without the systematic collection of data, case managers, SSA, and DDS staff were unable to determine whether SOAR was having a positive impact on their clients and were therefore less encouraged to continue to dedicate the time and energy necessary to implement the initiative.

Considerations for the Future

In the final section of this report, we offer considerations for future SOAR efforts. We first discuss ways in which federal agencies can continue to support SOAR and ensure that states have the resources and support necessary to implement and sustain the initiative in a manner that is most likely to facilitate positive impacts on the lives on homeless individuals. We then discuss strategies that can enhance the future TA that states receive. A major strength of the SOAR model is that it can be tailored to each community while implementing a common strategy of strengthening interagency relationships and teaching a standard curriculum that provides case managers with skills to navigate the SSI/SSDI application process. Thus, options for future TA efforts might be considered in the context of each state and community. The broad considerations described below will need to be developed further as part of the strategic planning within each state and community.

1.  Considerations for Federal Agencies Supporting SOAR

• Resources to Support State Leadership.  In the absence of financial support to conduct SOAR activities, state leads found it difficult amid their many other responsibilities to dedicate the time necessary to develop SOAR to its potential. Many states are currently struggling with budget deficits and are searching for ways to reduce expenditures on health and human services, which may jeopardize the ability of states to allocate resources for SOAR. Federal agencies supporting SOAR might consider providing states with resources to fund (either fully with federal dollars or through a federal-state match) a full-time position for a SOAR project coordinator. Alternatively, federal agencies might consider encouraging or providing incentives to states to dedicate a portion of their PATH, Mental Health Transformation, or Continuum of Care funds to support SOAR leadership. This financial support would enable the state lead to fully commit a portion of his or her time to the initiative. The ideal agency in which the state lead resides and the mechanisms of financially supporting that individual may differ by state depending on the existing service structure and interagency relationships.
• Ensure the Commitment of Participating State Agencies and Organizations. Implementing and sustaining SOAR requires that each stakeholder commit time and resources to achieve the maximum benefits for individuals who are homeless. In addition to the commitment of a state lead, SOAR requires the institutionalization of the initiative within the state lead's agency. While many state and community organizations have fully committed to SOAR, federal agencies might consider developing a formal mechanism to solidify their commitments. For example, a memorandum of understanding (MOU) between the head of the state team lead's agency, the TA contractor and, to the extent feasible, federal agency representatives may help to encourage states to continue to support the initiative throughout the time that they receiving TA and to build momentum to sustain SOAR after federal support for the initiative has ended. An MOU might outline the responsibilities of each party, stipulate the channels of communication, and specify clear milestones, target dates, and consequences for inaction. State agencies might use the same template to develop MOUs with partner agencies in the state.
• Work with SSA at the Federal and Regional Levels.  SSA functions in a manner that allows, within some boundaries, local offices to tailor policies and procedures to meet the needs of the community and service system. While having some local autonomy allows SSA to better meet the needs of applicants, some SSA staff reported the need to gain support from their managers and/or regional director before proceeding with SOAR. Gaining that support was often difficult and time consuming for local SSA staff. Federal agencies supporting SOAR might consider developing a strategy for SOAR at the national and/or regional levels while balancing the need for local SSA offices to maintain their autonomy. Staff from the federal agencies that support SOAR, in partnership with the TA contractor, may wish to reach out to SSA national and regional directors and other senior management within SSA to provide them with preliminary evidence of the initiative's success at reducing backlogs and improving the lives of applicants, as well as tested methods of altering local SSA office procedures to support the initiative. Gaining such support could be useful in getting the message down to local SSA offices and creating efficiencies in decision making related to SOAR within SSA.
• Remove Limits on the Time States are allowed to Receive TA.  All states involved in SOAR can likely benefit from additional TA, but their needs vary according to their stage of implementation. Launching a new initiative (or resurrecting a failing one) can take a long time and sustaining an existing initiative can require unique expertise as new challenges arise. Federal agencies supporting SOAR might consider removing any limits on the amount of time states are able to receive federally funded SOAR TA to ensure that states receive a level of support proportionate to their level of need and consistent with the realities of the timeline for implementation. This would allow states that experience setbacks an opportunity to follow through with implementation. The key would be not to create additional lag time between the TTT session and the initial in-state training, but to provide more assistance to states upfront in establishing relationships and processes as well as during and after the in-state trainings to facilitate the use of SOAR in practice. The TA contractor could conduct an interim scan of states' needs to determine which states may require TA beyond 18 months and develop state-specific plans accordingly. Alternatively, states could request TA beyond 18 months but be required to document their challenges and develop a brief plan that both describes which activities require TA in order to meet their goals and proposes a timeline for the activities. The TA would then be tailored to the specified goals and activities and conducted within the proposed timeline.
• Support the Reporting of Performance Indicators.  Currently, states have little accountability for their efforts related to SOAR. States are not required to report any data to the TA contractor or federal agencies. The lack of accountability may contribute to less interest toward implementing the model with fidelity and sustaining the initiative over time. Federal agencies supporting SOAR might consider requiring states to report performance indicators and/or collect outcome data as a condition of receiving TA or other resources associated with SOAR. However, data collection and reporting may be difficult with tight budget constraints. States are likely to need resources and TA to establish or use existing data tracking and reporting systems. Performance indicators may include number of in-state trainings, number of SOAR applications completed, application approval rates, time from application submission or protective filing date to approval or denial, and reasons for denials.
• Fund a Rigorous Evaluation to Examine the Impact of SOAR.  This evaluation was able to document the extent to which selected states have implemented SOAR activities and developed the policies, procedures, and infrastructure necessary to achieve the short-term outcomes. Because SOAR states have not been required to collect or report systematic data, it is not possible at this time to make any definitive statements about the short- or long-term impacts of SOAR in terms of application outcomes or the housing stability, income, and health of individuals who are homeless. There is limited evidence to suggest that SOAR may have helped some communities recoup General Assistance or other costs, but it is not possible to attribute cost savings to SOAR at this time. A rigorous evaluation of the short- and long-term outcomes of SOAR involving the collection of high-quality quantitative and qualitative data from states could gather the evidence necessary to determine the impacts of SOAR and assess the benefits of SOAR relative to the investment in the initiative. Ideally, such an evaluation would examine whether SOAR has an impact on the number of applications submitted, application approval rates, time to benefit determination, and cost savings. While an ideal evaluation would also assess the impact of SOAR on quality of life among applicants who are homeless (for instance, by measuring outcomes such as income, housing, health, and health care), doing so would likely require careful longitudinal data collection and thus a substantial investment of resources.

2.  Considerations for Future TA Efforts

• Systematic Outreach to the Supervisors of Case Managers.  Case managers who had full support from their direct supervisor and management of their organization were empowered to take the time to participate in SOAR training and complete applications for individuals who are homeless. While some supervisors attend the in-state trainings, there may be a need to reach out to supervisory staff and senior management at CBOs to educate them on the benefits of SOAR for their clients, staff, and agencies as well as the level of effort that will be required of their staff to implement SOAR. Supervisors may benefit from a brief introduction to SOAR and be given an overview of the training that their staff will receive.
• Expand Strategic Planning.  The in-state planning forum is an essential first step in developing collaborative relationships and gaining stakeholder buy-in, but these relationships take time to develop. Some states sent staff to a TTT session before there was sufficient stakeholder buy-in and sometimes even before the planning forum itself. States currently develop a strategic action plan as a result of the forums but may require additional forums facilitated by the TA contractor or other structured opportunities to refine the plan and begin the process of nurturing relationships before any training occurs.
• Make More Explicit the Role of Ongoing TA.  Ongoing, customized TA is available to help states implement SOAR and overcome any challenges, but states have underutilized this resource. States and the TA contractor may wish to incorporate into the strategic action plans that results from the planning forum (or any new documents developed, such as MOUs) a mechanism for consistent ongoing and structured communication among the TA contractor and the state lead, SSA, DDS, and other partners to troubleshoot any challenges and maintain momentum for the initiative. States may benefit from more regularly scheduled followup from the TA contractor.
• Provide More Guidance on Expectations of Participating Entities.  To foster buy-in among state and local agencies, tailor the initiative to the community context, and ensure the initiative could be sustained in the absence of federally funded TA, SOAR TA was designed to support state-driven decision making. Given the struggles that many states have had propelling the initiative forward, however, more prescriptive TA with respect to the expectations of participating organizations may be useful. For instance, applications for TA (or other mechanisms) might require states to demonstrate that the designated lead has a certain minimum percentage of time available to dedicate to SOAR. They also might require that in-state trainers possess certain minimum qualifications and that states designate back-up trainers who will be available in the event of turnover. Applications for the first three rounds of TA required that states identify one or more pilot sites, but afforded states the flexibility to define a site. Future applications might specify that a site entails a city, county, or specific jurisdiction and require that the states identify only one in which SOAR implementation will begin and others targeted for early rollout.
• Identify Sources of Sustainable Funding for Communities.  Implementation of SOAR's critical components had the most potential when agencies dedicated one or more employees exclusively to providing benefit assistance through SOAR. Communities identified various sources of funding extemporaneously to support dedicated staff positions, but generally funding was temporary. Communities should begin developing strategies for sustaining the initiative and obtaining funding at the outset of SOAR, perhaps as part of the strategic planning forum. Some communities may not necessarily need new funding to sustain the initiative, but may be able to shift responsibilities to dedicate staff to SOAR. The TA contractor could work closely with state agencies and CBOs to determine how best to do this given their current staff structures.
• Market Data Tracking Systems.  Many states and communities have not attempted to track SOAR outcomes while others have tracked outcomes on an ad hoc basis. The TA contractor has developed software that states and communities can use to track outcomes. Communities will likely need assistance to learn to use the software and will require continuous encouragement and quality reviews to sustain any data collection efforts they undertake. However, extensive marketing and TA specific to the implementation of this system may prompt more standardized and comprehensive collection of data necessary to facilitate a quantitative analysis of outcomes and assess the ultimate impact of SOAR.

References

Bhui, K., L. Shanahan, and G. Harding. Homelessness and Mental Illness: A Literature Review and a Qualitative Study of Perceptions of the Adequacy of Care. The International Journal of Social Psychiatry, vol. 52, no. 2, 2006, pp. 152-165.

Bird, C.E., K.J. Jinnett,  M.A. Burnam, P. Koegel, G. Sullivan, S.L. Wenzel, M.S. Ridgely, S.C. Morton, and A. Miu. Predictors of Contact with Public Service Sectors Among Homeless Adults with and Without Alcohol and Other Drug Disorders. Journal of Studies on Alcohol, vol. 63, no. 6, 2002, pp. 716-725.

Caton, C.L., B. Dominguez, B. Schanzer,  D. S.Hasin, P.E. Shrout, A. Felix, H. McQuistion, L. A. Opler, and E. Hsu. Risk Factors for Long-Term Homelessness: Findings from a Longitudinal Study of First-Time Homeless Single Adults. American Journal of Public Health, vol. 95, no. 10, 2005, pp. 1753-1759.

Goering, P., G. Tolomiczenko, T. Sheldon K. Boydell, and D. Wasylenki. Characteristics of Persons Who Are Homeless for the First Time. Psychiatric Services, vol. 53, no. 11, 2002, pp. 1472-1474.

Kushel, M.B., J.A. Hahn, J.L. Evans, D.R. Bangsberg, and A.R. Moss. Revolving Doors:  Imprisonment Among the Homeless and Marginally Housed Population. American Journal of Public Health, vol. 95, no. 10, 2005, pp. 1747-1752.

Macnee, C. L. and L. J. Forrest. Factors Associated with Return Visits to a Homeless Clinic. Journal of Health Care for the Poor and Underserved, vol. 8, no. 4, 1997, pp.437-445.

Meadows-Oliver, M. Social Support Among Homeless and Housed Mothers: An Integrative Review. Journal of Psychosocial Nursing and Mental Health Services, vol. 43, 2005, no. 2, pp. 40-47.

North, C.S., K.M. Eyrich, D.E. Pollio, E.L. Spitznagel. Are Rates of Psychiatric Disorders in the Homeless Population Changing? American Journal of Public Health, vol. 94, no. 1, 2004, pp. 103-108.

O'Connell, J., P. Quick, and B. Zevin. Documenting Disability: Simple Strategies for Medical Providers. Nashville, TN: National Health Care for the Homeless Council, 2004.

O'Toole T.P., J. Arbelaez, C. Haggerty, and the Baltimore Community Health Consortium. The Urban Safety Net: Can It Keep People Healthy and Out of the Hospital? Journal of Urban Health: Bulletin of the New York Academy of Medicine, vol. 81, no. 2, 2004, pp. 179-190.

Rosen, J., R. Hoey, and T. Steed. Food Stamp and SSI Benefits: Removing Access Barriers for Homeless People. Journal of Poverty Law and Policy, March-April 2001, pp. 679-696.

Segal S.P., C. Silverman, and T. Tempkin. Social Networks and Psychological Disability Among Housed and Homeless Users of Self-Help Agencies. Social Work in Health Care, vol. 2, no. 3, 1997, pp. 49-61.

Social Security Advisory Board. Disability Decision Making: Data and Materials. Washington, DC: Social Security Advisory Board, 2006.

Stovall, J. and J. Flaherty. Homeless Women, Disaffiliation and Social Agencies. The International Journal of Social Psychiatry, vol. 40, no. 2, 1994, pp. 135-140.

U.S. Conference of Mayors. A Status Report on Hunger and Homelessness in America's Cities: 2004. Available at [www.usmayors.org]. Accessed July 12, 2007.

Yin, Robert. Case Study Research: Design and Methods. Newbury Park, CA: Sage Publications, 1989.

Zima B.T., K.B. Wells, B. Benjamin, and N. Duan. Mental Health Problems Among Homeless Mothers: Relationship to Service Use and Child Mental Health Problems. Archives of General Psychiatry, vol. 53, no. 4, 1996, pp. 332-338.

Endnotes

Appendix A: Outcomes in Other SOAR Communities

The data presented in the body of this report pertain only to the six case study states. However, according to information gathered from other SOAR communities by the TA contractor, other states have achieved notable individual- and system-level outcomes. Below, we describe outcomes regarding SOAR application approval rates and processing times and cost recovery efforts in a few communities to demonstrate more comprehensively SOAR's potential. All information in this appendix was provided by the TA contractor; Mathematica has not evaluated the strengths and weaknesses of the underlying data sources or otherwise verified these data.

A. SOAR Applications

Georgia.  Georgia's Department of Human Resources Mental Health Division used PATH funds to employee 1.5 FTE SOAR coordinators; in addition to coordinating statewide SOAR efforts, these staff conduct trainings and help clients prepare SOAR applications. The department also signed MOUs with state hospitals to improve access to medical records and worked to incorporate SOAR into these hospitals' discharge planning. The TA contractor reported in May 2008 that SSA had approved 42 of 63, or 67 percent, of resolved SOAR applications in Atlanta; on average, approvals took 75 days.

Tennessee.  In May 2008, the TA contractor reported that 98 percent of Nashville's first 87 SOAR applications were approved; time from application to approval averaged 56 days. Facilitating access to benefits, including SSI/SSDI, was one the guiding principles underlying Nashville's 10-year plan to end homelessness. A homeless service provider, Park Center, employed four staff focused on preparing SSI/SSDI applications.

Michigan.  Unlike many SOAR communities, Michigan immediately rolled out SOAR statewide; the state PATH program coordinated the initiative across eight regions. In May 2008, the TA contractor reported that 78 percent of the 55 resolved SOAR applications were approved; on average, approvals took 53 days.

New York.  Before New York State received federally funded TA for SOAR, a SOAR-trained case manager implemented a pre-release and re-entry application program in Sing Sing prison. In May 2008, the TA contractor reported that 88 percent of the 99 pre-release cases for which decisions had been issued were approved; on average, approvals took 59 days

B. Reimbursement of Medical Costs

Kentucky.  St. Elizabeth Medical Center in Covington spent $18,000 to fund a part-time SOAR position at a local homeless center; this staff member assists clients, including those referred to the homeless center from this hospital, with SSI/SSDI applications. The TA contractor reported that within one year, 21 of the hospital's clients were awarded benefits. The hospital recouped more than $182,000 from Medicaid and Medicare toward the $482,638 it incurred in uncompensated care to these clients. Based on the program's initial success, the hospital increased its funding, turning the part-time position into a full-time one.

Rhode Island.  In Providence, Butler Hospital invested $30,000 in ACCESS-RI, a statewide program designed to serve individuals who are chronically homeless and have been diagnosed with mental disabilities or substance abuse. The program utilizes the SOAR model. The hospital reports that it was able to recoup its initial investment plus additional expenses for providing 146 days of hospital care to four clients who were homeless after those clients were approved for SSI and Medicaid. Butler received a total of $197,000 in reimbursement for their care.

Oregon.  In Multnomah County (Portland) a team of specialists assists individuals who are homeless and who have disabilities to expedite acquisition of federal benefits and entitlements (SSI/DI and Medicaid/Medicare). The Benefits and Entitlement Specialist Team (BEST) consists of benefits specialists, a records/data coordinator, and a physician who conducts the CEs required by DDS. The team reported to the TA contractor that 16 medical providers have received significant retroactive Medicaid reimbursements after three clients were awarded benefits. These reimbursements totaled roughly $110,000.

Appendix B: Stepping Stones to Recovery, Pre-Post Test

These are the HTML versions of the Pre-Test and Post-Test forms,
see apb.pdf for the actual forms.

Appendix C: Stepping Stones to Recovery, Training Module Fidelity Assessment

This is the HTML version of the form,
see apc.pdf for the actual form.

Appendix D: Stepping Stone to Recovery, Participant Evaluation Form

Training Evaluation Form
Stepping Stones to Recovery

I. Participant Background

1. I work at an agency that is primarily a... (circle one)
   1. Mental health agency
   2. Homeless service agency
   3. SSA or DDS office
   4. Other (please specify) __________
2. At the above agency, my current job is ... (circle one)
   1. Outreach worker
   2. Case Manager
   3. Shelter Worker
   4. Benefits Specialist
   5. Program Coordinator/Supervisor
   6. Other (specify) ______________
3. My position is PATH-funded?   ___ Yes     ___ No     ___ Don't Know
4. About how many adults did you assist with SSI or SSDI applications in the past year?
   1. None
   2. Less than 1 per month
   3. About 1 per month
   4. Between 1 and 3 per month
   5. About 1 per week
   6. More than 1 per week
5. About what proportion of adults who you have assisted with SSI or SSDI applications are typically approved on their initial applications?
   1. None
   2. 1-25 percent
   3. 26-50 percent
   4. 51-75 percent
   5. 76-100 percent
   6. Not applicable
6. For applications where you have assisted people, what is the approximate length of time between application and initial decision?
   1. 3 months or less
   2. 4-6 months
   3. 7-12 months
   4. More than 12 months
   5. Not applicable

II. Training Evaluation Questions

II. Training Evaluation Questions To what extent do you agree disagree with each statement below as it relates to this training?	Circle One
	Strongly Disagree	Disagree	Agree	Strongly Agree
1. I have a better understanding of the differences between SSI and SSDI, including the health insurance offered and eligibility requirements	1	2	3	4
2. I am better able to identify the non-medical criteria for SSI/SSDI eligibility	1	2	3	4
3. I have a better understanding of the disability determination process and how to develop medical evidence to support a disability claim	1	2	3	4
4. I have a clearer understanding of the role of functional information in the determination of disability.	1	2	3	4
5. I feel more equipped to thoroughly interview a client and assess his/her functioning.	1	2	3	4
6. I will be able to write a comprehensive medical summary to be submitted for disability determination	1	2	3	4
7. Overall, I feel this training will help me do a better job assisting individuals with SSI/SSDI applications	1	2	3	4
8. The trainer was knowledgeable and well-prepared	1	2	3	4
9. The trainer was interesting and help my attention	1	2	3	4
10. The trainer presented the information clearly	1	2	3	4
11. The training was organized in a way that was conducive to learning	1	2	3	4
12. Attendees were given numerous opportunities to ask questions and discuss the material	1	2	3	4
13. The trainer provided helpful answers to our questions.	1	2	3	4
14. The pace of the training program was just right — not too fast and not too slow.	1	2	3	4
15. The interactive role play and/or video helped me explore how I will use the information in my own work.	1	2	3	4
16. The written materials supported the presentation.	1	2	3	4
17. The PowerPoint materials supported the presentation	1	2	3	4
18. The balance of information among written materials, slides, and presentation was effective.	1	2	3	4
19. The exercises helped me to better understand the material being covered.	1	2	3	4
20. The worksheets will be useful tools for my work in the field.	1	2	3	4

21. Overall, how satisfied were you with the information provided during the training program? (Circle the box that most closely expresses your opinion)

Very Dissatisfied     1

Dissatisfied     2

Neutral     3

Satisfied     4

Very Satisfied     5

22. Overall, how satisfied were you wit the way in which the training session was organized and presented?

Very Dissatisfied     1

Dissatisfied     2

Neutral     3

Satisfied     4

Very Satisfied     5

23. What were two things you liked most about the training?

1. _______________________________
2. _______________________________

24. What were two things you disliked most about the training?

1. _______________________________
2. _______________________________

25. What specific suggestions would you make to improve the training?

_______________________________________________________

_______________________________________________________