Measures are necessary to assess sustainability and to monitor the progress toward sustainability (Shediac-Rizkallah & Bone, 1998). One of the challenges of measuring sustainability is that the analysis or assessment captures each community coalition at a point in time—and cannot take into account the changes that are occurring within each coalition over time, or that may occur in the future (Rog et al., 2004). Despite these difficulties, a variety of measures and indices have been used to measure and evaluate sustainability in community coalitions. These measures and indices should not be confused with the conceptual models of sustainability presented earlier. Rather, these are tools that can be used to evaluate the impacts of specific coalition characteristics on sustainability. They include the following:
Program Sustainability Index. Mancini and Marek (2004) developed a 53-item Program Sustainability Index (PSI) that was used in a study of sustainability in community-based programs. The PSI includes 53 items reflecting seven sustainability elements: leadership competence, effective collaboration, understanding the community, demonstrating program results, strategic funding, staff involvement and integration, and program responsivity.
Framework to Track Collaboration and its Development. Weiss, Coffman, and Bohan-Baker (2002) tracked collaboration and its development over the course of a five-year evaluation of the Kellogg Foundation’s Devolution Initiative. The Devolution Initiative funded research, policy, and advocacy organizations as well as scholars and community organizers to work together to create an information base about the impact of welfare reform and health care devolution, and to disseminate that information to policymakers. Based on semiannual interviews with grantees and review of questionnaires, the researchers evaluated the type of collaborations occurring. Specifically, grantee collaboration (i.e., the collaboration occurring within each funded collaborative) was measured according to four levels: collaboration, coordination, contribution, and communication occurring amongst the grantees.
The “collaboration” level was considered the most extensive form of collaboration, whereby entities within each grantee collaborative were working together to prepare reports and conduct joint meetings. A step down from collaboration was the “coordination” level, whereby grantees were coordinating data collection and messages. A step down from coordination was the “contribution” level, whereby grantee entities were providing input on each other’s reports and responding to information requests. The lowest level of collaboration was “communication,” whereby entities were simply on a mailing list, communicating informally.
Measures of Adherence to Program Framework. Feinberg et al. (2008) examined the predictors of sustainability among 110 prevention coalitions that participated in Communities That Care (CTC) between 1998 and 2008. CTC is a program of the Substance Abuse and Mental Health Services Administration that strives to reduce adolescent problem behaviors. The researchers categorized each CTC coalition as active or defunct after the seed funding was withdrawn. On an annual basis, several measures were used to determine whether the CTC sites were continuing to adhere to the CTC program framework. Measures included: “continuing to conceptualize the need for prevention programming in relation to local risk-factor profiles, promoting evidence-based programs, maintaining a community collaborative approach (versus program implementation by a single agency), and maintaining contact with the state technical assistance framework” (p. 498). These measures were highly specific to the program’s framework.
Coalition Board Functioning Index. Feinberg et al. (2008) also created a board functioning index. This index was formulated based on the mean of four factors: 1) board work (measured by board directedness, board efficiency, leadership style, and leadership competence), 2) organizational resources (turnover of board membership, recruitment of new members, and barriers), 3) staff-board communication, and 4) board relations (board cohesion and conflict). Elements of this index can also be used to assess collaboration among coalition members.
Evaluations provide feedback and build the capacity of the coalition to measure its progress, readjust, and adapt to the changing needs of the community. Evaluation can also help to identify the factors that facilitate or mitigate sustainability in different types of community coalitions. The next section explores some of the key predictors of sustainability in the extant literature on community coalitions.