Westat was awarded a contract from ASPE with funding from SAMHSA to evaluate the implementation of the initiative in all three communities. The objectives of the evaluation were threefold: First, to describe the characteristics of grantee implementation processes, including social marketing efforts, screening procedures, provision of brief interventions, and referrals to more intensive services, as needed. This information was to come primarily from annual site visits to the grantee communities during which time Westat evaluators would interview project staff, Steering Committee members, community partners, and enrolled clients. The second evaluation objective was to report on the individual-level outcomes achieved by each of the grantees. As noted previously, grantees were required to record information about enrolled clients' characteristics and services in the SAIS data system. SAMHSA analysts cleaned these data and submitted them annually to Westat for subsequent analysis. Finally, Westat was to assess the extent to which the CRRI initiative was able to improve community-level resilience in the face of adverse economic circumstances. Data for this third objective were to be collected by grantees through local contacts (e.g., local emergency room data) and/or administrative data, and through the annual administration of a community-level survey.
Assessing the extent to which the CRRI initiative was able to improve community-level resilience proved problematic. The primary challenge that prevented Westat from meeting this objective was the requirement spelled out in the RFA that grantees were responsible for collecting the needed community-level data elements. Although grantees were required to hire their own evaluators to assess their program implementation and progress, evaluators had different ideas about what their responsibilities were for the community-level component. This challenge is discussed in more detail in Section 4.5.