In their applications for SOAR TA, states were expected to document availability of outcome data or provide a specific plan to collect, such data to assess the effectiveness of the state's plan to increase access to SSA disability benefits. Yet, once applications were approved, there was never an ongoing mandate for states to actually collect outcome data. Most states planned to use the Homeless Management Information System (HMIS) that was in development at the time of their application or to develop a new tool designed specifically to collect SOAR data, but few states actually did so. This section describes the challenges states faced in setting up systems to track outcomes and the practices they have been able to put in place.
As the gatekeeper of information on all SSI/SSDI applications, SSA could theoretically track SOAR outcomes. To track outcomes, SSA would need (1) a way to identify applications from homeless and/or SOAR clients and (2) resources to maintain the information. Neither currently exists. In some states, SSA does flag paper applications submitted by homeless or SOAR clients. However, there is no flag in SSA's electronic application to identify homeless or SOAR applicants and creating one would require changes to the SSA electronic filing software. Thus, states that wish to track SOAR outcome data must devise other ways of doing so.
As noted above, many of the 25 Rounds One and Two states planned to use the HMIS to collect SOAR outcome data, as documented in their initial applications for TA. Implementation of an HMIS is a requirement for Continuums of Care (CoC) that receive Department of Housing and Urban Development (HUD) McKinney-Vento Homeless Assistance Act funding. HMIS is a data tool designed to collect system-wide, client-level information about the characteristics and service needs of those who are homeless. HUD has set standards for a universal set of data elements to be collected for all people who utilize homeless services, including basic demographic characteristics, residence information prior to program entry, and service entry and exit dates. Also included are program-specific elements such as income sources and amounts (including SSI and SSDI), receipt of non-cash benefits, in-depth disability information, education and employment, and housing at program exit (including information on type of housing, tenure, and receipt of subsidies).
In general, states found it difficult to use the HMIS to collect SOAR data. Case study states provided several reasons why this was so. First, not all SOAR partners receive McKinney-Vento funding and thus do not use or have direct access to the HMIS or serve clients who come through the CoC system. Second, the HMIS was not designed to track the outcomes of particular interest under SOAR, such as application submission and approval rates and dates (and length of time between the two). States agreed that modifying the HMIS to track these types of data was theoretically feasible, but that they had no incentive to spend scarce resources to do so. No federal funding was provided to states to modify existing or develop new systems to track SOAR outcomes, and neither SAMHSA nor any other federal agency required states to report any type of SOAR data.
Other states planned to develop new SOAR-specific systems for tracking outcomes, but had difficulty doing so in the absence of resources and federal-level guidance on the type and amount of data to collect. The SOAR TA contractor offered some guidance early on about the type of data states should consider collecting, but states had primary responsibility to develop systems. One case study state created an online data tool for case managers, but it was designed to be a caseload management tool rather than an outcome measurement tool and was not widely used. Utilization was limited by confidentiality concerns and lack of sufficient resources for cross-agency training. Recently, the TA contractor developed SOAR outcome data collection software and related tools and documentation, and plans to make them easily accessible and to market them to states. These materials can be invaluable in helping states document the results of their efforts, but state and local staff will likely need training and ongoing monitoring to use them effectively.
Our case studies suggest that states attempting to track SOAR outcomes have done so in two ways: through DDS or by developing informal, customized spreadsheets. In some states, DDS appends a flag sheet to paper applications to identify SOAR clients and then maintains records on application disposition electronically. This typically occurred in states in which DDS was already tracking applications from clients who are homeless and could simply add a checkbox for SOAR to an existing flag sheet. However, this effort failed to capture the ultimate result of the application process (that is, SSI/SSDI application approval or disapproval from SSA) because DDS maintains data only on its own allowance rates, processing times, and denial reasons. DDS attempted to track SOAR outcome data in five of the six case study states, though efforts in some of those states waned over time.
Some state leads (or their designees) developed customized spreadsheets to track SOAR outcomes. An advantage of this approach is that they collected data most relevant to assessing SOAR in their states in an easily accessible format. A disadvantage is that it was somewhat burdensome, as it required state leads (or their designees) to regularly collect data to enter into the spreadsheet from all of the case managers or agencies providing SOAR application assistance. The quality of the data is also dependent on the amount and quality of data case managers or agencies maintain on applications, which can vary from agency to agency. State leads (or their designees) tracked SOAR outcomes through customized spreadsheets in four of the six case study states.
"index.pdf" (pdf, 1.23Mb)
"apb.pdf" (pdf, 330.32Kb)