The data presented in this chapter can be of some use in decision making related to the potential value of future investments in SOAR. To systematically measure SOAR outcomes and make more definitive conclusions in the future, however, it is necessary to establish rigorous quantitative data collection mechanisms at both the state and local levels and to design a study that maximizes the ability to attribute outcomes to the initiative rather than to other environmental factors. Rigorous data collection mechanisms could be established through collective use of a SOAR specific management information system (such as the one the TA contractor has recently developed) or purposeful and formal integration of SOAR measures into existing cross-state data systems (such as the HMIS, the Government Performance and Results Act system, and/or the web-based system in which PATH measures are reported). While random assignment experiments are the gold standard for evaluating the effect of social programs, an experimental evaluation of SOAR may be unrealistic given likely logistical and ethical challenges as well as resource constraints. More practical designs include a comparison site study, pre-post study, or study that combines the two (that is, a difference-in-difference approach, which compares changes in outcomes observed in SOAR sites before and after the initiative with changes in outcomes in a set of otherwise comparable non-SOAR sites before and after the initiative). Such a study would be of tremendous value in documenting investments in SOAR, assessing its effectiveness, and justifying its costs.
"index.pdf" (pdf, 1.23Mb)
"apb.pdf" (pdf, 330.32Kb)