Speaking at this session were Ada Skyles of Chapin Hall, Arlene Andrews of the Institute for Families and Society of the University of South Carolina, Michael Bennett of the Egan Urban Center of DePaul University, Jennifer Jewiss of the University of Vermont, David Murphey of the Vermont Agency of Human Services, and Mary Nelson of Bethel New Life Inc. Skyles then recapitulated the previous two sessions, listing a number of key points.
- Public engagement is an active process and you never quite arrive, but must continually engage.
- Different strategies are required for different audiences, but even within each audience, there are a variety of actors.
- It is important to be realistic about public engagement. For example, it requires money.
- Think continually about what is useful. Different audiences may provide different opportunities.
- When you engage the public, you are dealing with a power shift.
- It takes time to build up an infrastructure that allows for community input, but such an infrastructure is well worth the investment if done well.
- Among the concerns presented by states during previous meetings included minimizing the potential of data to stigmatize communities by emphasizing positive change and data that highlights strengths.
Arlene Andrews said, noting that it was obvious, that the only reason to collect data is to use it, particularly in the communities, where children and families live and where change takes place. She said that Institute for Families and Society serves as a broker between providers of statistical data and community users of statistical data. Much of the work of the Institute is with community-based organizations (CBOs). Currently they are involved with 130 organizations.
Andrews said that the Institute has a community assessment program that is largely focused on rural areas. She characterized rural poverty in the south as exhibiting racial segregation and the rural community-based organizations as being generally grounded in African-American culture.
Discussion moved to the value of statistical information and gaps in statistical and mathematical literacy and technological expertise among stakeholders. Andrews used a series of overheads to illustrate how statistical data must fit with the needs of the community and address what communities know about themselves in order to be valuable. She also covered the differences in state and local perceptions of the meaning of particular data and the ways in which data can be used to reinforce bias or become a part of larger issues of power and control.
Andrews and the participants stressed the importance of statistical literacy among people seeking to use data on communities. It was noted that CBOs often lack the resources to purchase up-to-date computers, making it difficult for them to effectively use statistical information.
Andrews offered a number of observations on the use of statistical indicators.
Indicators need to be considered in the context of community life and in terms of specific change processes already underway. (Assume that the community is already working on the problem even if they don't describe the problem using statistics.)
Take into account the ongoing change process in a community. Most communities are organized in some way and those community structures are frequently working with other organizations. Statistical information becomes part of this process, along with qualitative data.
Once you have the data--talk about it, analyze it, try not to jump to any conclusions about it. What are the stories behind the data? (Or what is the data behind the story.)
When you go into a community, get a good sense of where they are and also work to understand what an indicator says (and doesn't say or doesn't capture) for a community perspective.
Issues of power are hypersensitive at the community level. It is critical to be prepared to process data in ways that take into account community sensitivity.
There are multiple venues for the use of indicators.
Data can cause bias. One way it does this is because it seems to create priorities that may be at odds with a community's perceptions of its needs.
One of the challenges to community planning is that people don't know how to use statistics.
Andrew's handout is reproduced below. She said that the set of questions in the handout are all intended to prompt a group reflection and consensus building or, if there is dissention in the group, then get that documented about what these data could be saying.
From Indicators to Outcomes: Considerations and Strategies for Communities
ASSUME: An inclusive community planning group has a comprehensive set of statistical indicators about children and their families. They have qualitative information to enrich the data. They have measures of dispersion as well as central tendencies, information for subpopulations, and time trend analyses. They have a pretty clear picture of how children in their community look.
What is a community to DO with this information?
A critical principle in the use of community needs and resource assessments is that groups should hesitate to jump to solutions. Groups should carefully analyze and interpret the data through extended discussion. The story behind the data becomes the foundation for exploring relevant and effective solutions.
Community groups who review and discuss the data should consider a number of focused questions to guide their deliberations, such as these:
1. What is the purpose of the data review? Why are we doing this? What does the group hope to accomplish by examining the information?
2. Is the group satisfied that the data is accurate, inclusive, reliable, and valid?
3. Item by item, consider these issues:
- Examine detailed reports regarding how many children are in the categories of response to this item: how severe is this issue?
- What does the data indicate about children's strengths? Vulnerabilities?
- What are the standards by which the group assesses the indicator (e.g. comparison to another community or normative group; community opinion; trends over time)? Standards may be relative to an absolute level (e.g. no drug use is acceptable) or relative to other groups (i.e. this seems typical and appropriate for children this age).
- When current data is compared to past data and to, trend analysis over time, is the indicator suggesting child well-being is getting better or worse? What are the possible reasons for the status and trend of this indicator?
- If the indicator or set of indicators suggest an asset that needs to be enhanced or a problem that needs to be reduced, how can the issue be clearly defined?
4. What are the linkages among the various indicators? How are they "woven" together?
5. What is missing in the picture portrayed by the data? What can be done, if anything, to clarify the picture?
6. Taken as a group, which indicators suggest the need for community action? Which is highest priority? Why?
7. Based on this data analysis and interpretation, what courses of community action seem to be appropriate? What is the logic behind the alternative courses? What is known about effective practices?
8. Which course of action is most likely to lead to the group's vision of child well-being in the community?
9. What is the consensus about the most appropriate and feasible course of action?
After communities review their child well being indicators, the next step is to develop an action plan to promote community resources so that children's well-being can be maintained or enhanced. The plan leads to implementation, evaluation, and reassessment.
Andrews and the audience discussed how issues in a community defined, whether by the community itself, by the state or other actors, or both, and participants outlined ways in which they helped explain data to communities and make data useful. These included
- Providing technical assistance or money to buy computers
- Being careful to never put out data without also putting out an explanation.
These and other steps were thought to guard against misinterpretation of data. One participant advised convening a focus group before data release in order to increase the possibility that all possible reactions would be anticipated.
Using the World Wide Web
Andrews asked participants how they encouraged community-based organizations to use the World Wide Web. Answers ranged from walking them through the use of the web in tandem with making applications and other useful materials available thorough the web, and providing an 800 number that organizations could call to get information otherwise available on the web.
Ways of countering inappropriate leaps of logic by data users were discussed. One recommendation to avoid this was in careful framing of data and careful statement of the definitions behind the data. Andrews and participants volunteered examples of differences between the way data elements might be defined by those who collect them and by communities.
David Murphey said that the Vermont Agency for Human Services has helped to establish 12 regional partnerships in the state. These tend to be collaborative partnerships involving the education, health, and other agencies along with nonprofit organizations. These collaboratives meet on a regular basis. They are responsible for implementing outcomes at the community level. These partnerships are at different stages of development.
Vermont's approach embodies these steps.
- Share data
- After outcomes are agreed upon, put out the data related to those outcomes.
- Communities want to know what works and part of the state's job is to take on the task of showing communities what programs and strategies have evidence of effectiveness. Murphey noted that responses from communities might be that "nothing works" or that "what we've been doing for a long time is working fine," such assertions may lack the confirming data. Part of the state's mission is to help show communities the programs likely to be successful and which ones don't yet have confirming evidence.
- In a process akin to professional development, increasing people's comfort with an outcomes-based approach, including increasing statistical literacy through the use of training sessions. Sessions of that type are described below.
Murphey displayed the overhead below and commented on it, calling it a work in progress.
Vermont: An Evolution of State-Regional Partnerships
Trainings to-date sponsored by the Agency of Human Services in statewide meetings:
- "Growing Great Groups:" Dynamics of group-building and sustaining (leadership, decision-making, uniting diverse partners and similar tasks.)
- Families as Partners (family-centered practices, treating families as partners with expertise).
- "From Risk to Resiliency." This is a training piece related to risk and protective factors, including the idea of what counts for individual and community-level well-being, with a focus on the more resilient or more positive definitions.
- "From Promising Approaches to Best Practices." This is an attempt to summarize in a clearly digestible format, without pretending to be an encyclopedia or the last word, what works.
- Using Community Data. This includes how to do a community assessment, how to find off-the-shelf existing data, and the difficulties of generating or collecting new data (particularly by surveys). The training also includes some basic statistical concepts, including the problems of small cell size, the role of chance, and how to use data to set some strategic priorities.
- Strategic Planning, including how to use data as part of a more comprehensive process and how to use them to set priorities.
- State-Local Negotiations. If funds are saved through the use of community programs, some of those funds can be retained by the communities and reinvested in programs the community thinks successful.
- Results-Based Budgeting. This relies on the model developed by Mark Friedman.
- The "Patch" Concept of Service-Delivery (with Gerry Smale). This concept was developed in the U.K. and it is an integrated service-delivery model, based in a community, that relies on paraprofessionals and people who are trained in multiple disciplines so that community members have one-stop shopping when it comes to receiving services. (This tends to work best, in their experience, at the intersection of two service districts in which participants need to choose one or the other.)
- Asset Based Community Development. John McKnight (Northwestern University) presented on his work with a national research and training program focused on neighborhood revitalization, as part of a day-long conference on "Building Community Through Citizen Participation" (Nov. 12, 1998).
- Living Systems. Margaret Wheatley (author of Leadership and the NewScience, and others) presented on her work with organizations aiming to achieve coherence in the midst of chaotic environments, as part of a day-long conference on "Getting the System to Move" (Jan. 25, 1999).
- Local Economic Development. Michael Shuman, consultant, and former co-director of the Institute for Policy Studies, presented on achieving local self-reliance in a global economy, as part of a day long conference on "Partnering for Sustainable Development" (April 12, 1999).
Murphey concluded by saying that his presentation was designed to make clear the range of things that Vermont has shared with community partners. Most recently, with the help of the Casey Foundation, they have provided funds to the partnerships for technical assistance, and provided a menu of training options from which to choose.
Jewiss was involved in a qualitative study of two of the community partnerships in Vermont mentioned by David Murphey. The designers of the study chose to examine two partnerships that showed progress on their indicators, without saying necessarily that the partnerships were causing this improvement. The study yielded two findings. One was that data become the most useful when viewed as revealing the situations people experience. The second was that it is important to work with the media to help them understand what the data say about positive change.
The Data & The Stories Behind the Data
The communities involved in the study reported that having the quantitative data on child well-being was very important to them and that the fact of having the data had a powerful impact in discussions facilitated by local leaders.
However, in addition the research team heard that the local communities needed to find other effective ways to relay information to other community members. One asked, "How do you really make the story come alive?" Repeatedly, the research team heard about the need to augment the data with reports of personal experiences. One strategy for augmenting data was to get local media to run stories of successes. Many partnership members were substantially interested in this and some were quite good at it. Positive media reports were thought to fuel the ongoing effort required to sustain outcomes.
One of the partnerships benefited from the acquisition of the local newspaper by a more community-oriented owner.
Survey of the Media
The Vermont Research Partnership, composed of the education and human service agencies as well as the University of Vermont, surveyed state media and talked with media leaders. They concluded that both the agencies and media each feel that the other fails to understand their role in the community. From the media's point of view, data on public service paid for by tax money is public information and communities need to understand that it is the public's right to have it reported to them in the paper. The agencies felt that the media care only about negative stories.
Things that would improve the working relationship between the media and agencies included
- Agency and community leaders need to better understand the media's context and working habits. "News basics" needs to be part of their professional development. (Including understanding that most of the information journalists get they have to discard and they work in this torrent of information on very tight deadlines.) As part of this, agencies and communities need to better understand how to write a good press release that will catch the reporter's attention and explain things in a clear and compelling way.
- Communities and agencies need to develop thoughtful communications plans that will relay to the public what they are doing.
- Communities and agencies need to build good relationships with the members of the media who cover the beats of interest.
Based on these findings, the Research partnership is developing an action research project for which they will invite teams from five parts of the state to come to monthly workshops with media and public relations experts focused on positive youth development and working with the press.
A panel composed of Michael Bennett, Mary Nelson, and Arlene Andrews followed Jennifer Jewiss. Ada Skyles introduced the panel.
Michael Bennett began by identifying three experiences that shape his comments. First, he noted that Agriculture Secretary Dan Glickman had released a report that indicated that a family with a child born in 1999 could expect to spend about $160,000 raising that child to age 18 years (with inflation factored in, the total rises to $237,000). Bennett said that these figures indicate to him that more children will have inadequate resources for their growth and development.
Second, he noted that the preceding day, while he attended a Council on Foundations meeting in Los Angeles, he went to a breakfast in which a panel of celebrities talked about how they can be useful for particular causes. Rob Reiner impressed Bennett in his description of how he had dedicated himself to the Clinton-Gore goal of having children ready to learn by the time they get to school and how he was trying to think comprehensively about the components of school readiness--such as health, families, and communities.
Third, Bennett, a graduate of Kent State University, pointed out that 30 years ago to the day four Kent State students (5) were shot to death, and others injured, by the Ohio National Guard. Bennett said that the demonstration at which the students were killed was: "about people taking risks because they thought they had the ability to change, about often unmovable forces of government, and about unintended consequences that impact our lives."
Definitions of Community
Bennett offered three definitions of community as it related to the work of supporting children and families. These were
- A geographic definition, something that, on the surface, seems clear but has the potential to be more complex as there are neighborhoods within communities.
- A definition by ascribed characteristics, such as youth, older people, the poor, female-headed households, etc.
- A definition based on a state of being, such as communities in transition, in which immigration is substantial and there is a visible influx of a new population.
Communities and Indicators
Bennett had four points to make related to communities and indicators.
- Communities and community-based organizations have undergone development and have become very sophisticated. But there are some whose public relations image is better than their actual work. It is important to separate fame from accomplishment. (He contrasted having "30 years of experience" with having "one year of experience, thirty times.")
- A lot of communities are not aware of the existence of state data that they could use. They are sometimes more familiar with federal data, such as Census data, but not with possibly more relevant state datasets.
- A number of communities are undergoing or have undergone community-planning processes and have been funded by larger foundations to look at a wide variety of issues. Involvement with such processes could be rewarding for states.
- A number of relationships can be developed that prove useful, such as university-community outreach partnerships, although the barriers to such relationships are formidable.
Bennett said that communities can be very important in helping data gatherers interpret and use data. Meanings may not be obvious to outsiders.
Mary Nelson began her portion of the discussion by describing Bethel New Life, a faith-based organization involved with approximately 10,000 individuals who live on Chicago's West Side. Bethel New Life is a real estate development organization (including commercial and industrial development) but, in addition, it is focused on strengthening and supporting families in a community context. That latter part of their mission includes involvement with the Chicago Case Management and WIC (Women, Infants, and Children) programs, programs for the elderly, welfare-to-work programs, child development programs, daycare homes, and other activities.
Bethel New Life has been involved in a variety of programs, including a state infant mortality reduction program in the early 90s that required the organization to keep track of 100 process objectives. They are currently in the fourth year of a five-year demonstration project involving the American Hospital Association called the Outcomes Tool Kit. In most of the other sites in which this project is under way, it is housed in big institutions, hospitals, or government departments.
Bethel's Own Indicators
Nelson said that Bethel New Life had developed its own set of indicators in order to assess whether things are getting better for families in their area of activities. They developed their indicators using this definition of a healthy, sustainable community. A healthy, sustainable community has:
- Economic security
- Ecological integrity
- High quality of life for all
- Public participation in decision making
They are committed to tracking their measures over five years. They have analyzed their findings, and produced easy-to-understand reports that are designed to be useful to those who live in the community. Bethel New Life is very interested in developing approaches focused on outcomes. Nelson notes that the thinking of many community-based organizations is focused on outputs--that is, measures of services provided--and that changing this thinking to an outcome-oriented approach is challenging.
Nelson had four points she wanted to make with the meeting participants. First, if state agencies and other players want community-based agencies to be involved, there must be funds to support this work beyond the per-child service funds. Second, communities must be real partners. They must lead change, not follow the state's agenda. Third, the work must start with assets, not needs, with communities identifying their own strengths and assets. She also said that efforts to work with communities should start not with needs, but with assets. Efforts that focus first on outcomes, then on capacities, help avoid the trap of becoming overwhelmed by needs. Finally, she said that the training process must work both ways and there must be an appreciation of what communities have to share.
Moving to data, Nelson said that the data must be sliced in ways easily accessible. It should be organized by community area, not by zip code, for example. And it should be put into public libraries where it is easily available to communities.
5. Allison Krause, Jeffrey Miller, Sandra Scheuer, and William Schroeder.