This project sought to use the experience of HCAP to learn about the long-term sustainability of federally-funded coalitions and answer three key research questions:
- How many community coalitions funded under HCAP are still in existence?
- What coalition characteristics are associated with sustainability and what factors promote or hinder community coalition sustainability?
- What are the impacts of HCAP coalitions post-federal funding and what are the types of outcomes achieved? To what extent have the coalitions evolved to address the needs in the community?
To examine these important issues, NORC conducted a multi-method assessment using qualitative and quantitative research methods, including a review of literature on coalition sustainability; a survey of the 260 former HCAP grantees; key informant interviews with sustained and not sustained HCAP coalitions; and case studies involving site visits with sustained coalitions.
The first component of the study consisted of a literature review and the development of the conceptual framework. The literature review focused on community coalition sustainability. Specifically, the literature review explored the ways in which researchers, policy makers, and practitioners have defined and measured sustainability for community coalitions. The literature review also explored the facilitators of and barriers to sustainability in community coalitions. Findings from the literature review were synthesized to develop operational definitions for key concepts, such as what it means for a coalition to be "sustained" versus "not sustained." With a working definition of sustainability in the context of community coalitions and their activities, NORC developed a conceptual framework for assessing the sustainability of community coalitions after their initial federal funding has expired. In the second component of the project, the literature review and conceptual framework supported the development of a survey to assess the HCAP coalitions based on the sustainability definition and the factors driving sustainability included in the conceptual framework. A mailed, self-administered questionnaire was sent to all 260 HCAP grantees between March 10 and May 31, 2011 and achieved a 63% response rate. The questionnaire included a screening question to determine the coalition’s current operational status; other questions focused on the coalition’s structure, mission and goals, funding sources, activities, evaluation methods, sustainability plans, and overall impact.
From the pool of grantees that completed the survey (113 (68%) sustained and 52 (32%) not sustained), a total of 25 (15%) coalitions were invited to participate in telephone interviews. Telephone interviews with key informants of these coalitions were conducted in August and September 2011. The purpose of the interviews was to gather more detailed information and confirm survey responses regarding the coalitions’ experience and strategies for trying to sustain the coalition after initial federal funding ended, coalition outcomes, and future plans. In the final phase of the project, NORC conducted site visits with six high performing sustained coalitions. Two members from the NORC team traveled to each location to conduct site visits. All site visits occurred during October 2011. The site visits involved in-depth interviews with the coalition directors and lead staff and representatives from the coalition’s partner organizations. NORC staff facilitated the interviews using a semi-structured interview protocol that focused on facilitators and obstacles of sustainability, the coalition’s structure and dynamics, and lessons learned.
Results from the surveys, key-informant interviews, and site visits were analyzed to identify overarching themes and differences among sustained coalitions and not sustained coalitions. Survey data analyses were conducted in STATA 10, and focused on the calculation of descriptive statistics, such as frequencies, central tendencies, and distributions for all questions. Standard difference of means and difference of proportions tests were used for making statistical comparisons between sustained and not sustained coalitions. Logistic regression models were employed to assess specific hypotheses derived from the conceptual framework. Qualitative data analysis of the 25 phone interviews was conducted using QSR NVivo 9 (NVivo). NVivo facilitated the identification of common themes across community coalitions in addition to major differences between sustained and not sustained coalitions. NORC also analyzed the site visit data by validating and confirming key information and themes; reviewing notes from each site visit; identifying significant quotes and comments; and conducting debrief conversations with each grantee director either at the end of the site visit or shortly thereafter by telephone to confirm the interpretation of key findings and ensure that important information was not omitted.