(May 4, 2000)
[ Main Page of Report | Contents of Report ]
Speaking at this session were Ada Skyles of Chapin Hall, Arlene Andrews of the Institute for Families and Society of the University of South Carolina, Michael Bennett of the Egan Urban Center of DePaul University, Jennifer Jewiss of the University of Vermont, David Murphey of the Vermont Agency of Human Services, and Mary Nelson of Bethel New Life Inc. Skyles then recapitulated the previous two sessions, listing a number of key points.
Arlene Andrews said, noting that it was obvious, that the only reason to collect data is to use it, particularly in the communities, where children and families live and where change takes place. She said that Institute for Families and Society serves as a broker between providers of statistical data and community users of statistical data. Much of the work of the Institute is with community-based organizations (CBOs). Currently they are involved with 130 organizations.
Andrews said that the Institute has a community assessment program that is largely focused on rural areas. She characterized rural poverty in the south as exhibiting racial segregation and the rural community-based organizations as being generally grounded in African-American culture.
Discussion moved to the value of statistical information and gaps in statistical and mathematical literacy and technological expertise among stakeholders. Andrews used a series of overheads to illustrate how statistical data must fit with the needs of the community and address what communities know about themselves in order to be valuable. She also covered the differences in state and local perceptions of the meaning of particular data and the ways in which data can be used to reinforce bias or become a part of larger issues of power and control.
Andrews and the participants stressed the importance of statistical literacy among people seeking to use data on communities. It was noted that CBOs often lack the resources to purchase up-to-date computers, making it difficult for them to effectively use statistical information.
Andrews offered a number of observations on the use of statistical indicators.
Indicators need to be considered in the context of community life and in terms of specific change processes already underway. (Assume that the community is already working on the problem even if they don't describe the problem using statistics.)Take into account the ongoing change process in a community. Most communities are organized in some way and those community structures are frequently working with other organizations. Statistical information becomes part of this process, along with qualitative data.
Once you have the data--talk about it, analyze it, try not to jump to any conclusions about it. What are the stories behind the data? (Or what is the data behind the story.)
When you go into a community, get a good sense of where they are and also work to understand what an indicator says (and doesn't say or doesn't capture) for a community perspective.
Issues of power are hypersensitive at the community level. It is critical to be prepared to process data in ways that take into account community sensitivity.
There are multiple venues for the use of indicators.
Data can cause bias. One way it does this is because it seems to create priorities that may be at odds with a community's perceptions of its needs.
One of the challenges to community planning is that people don't know how to use statistics.
Andrew's handout is reproduced below. She said that the set of questions in the handout are all intended to prompt a group reflection and consensus building or, if there is dissention in the group, then get that documented about what these data could be saying.
ASSUME: An inclusive community planning group has a comprehensive set of statistical indicators about children and their families. They have qualitative information to enrich the data. They have measures of dispersion as well as central tendencies, information for subpopulations, and time trend analyses. They have a pretty clear picture of how children in their community look.
What is a community to DO with this information?
A critical principle in the use of community needs and resource assessments is that groups should hesitate to jump to solutions. Groups should carefully analyze and interpret the data through extended discussion. The story behind the data becomes the foundation for exploring relevant and effective solutions.
Community groups who review and discuss the data should consider a number of focused questions to guide their deliberations, such as these:
1. What is the purpose of the data review? Why are we doing this? What does the group hope to accomplish by examining the information?2. Is the group satisfied that the data is accurate, inclusive, reliable, and valid?
3. Item by item, consider these issues:
- Examine detailed reports regarding how many children are in the categories of response to this item: how severe is this issue?
- What does the data indicate about children's strengths? Vulnerabilities?
- What are the standards by which the group assesses the indicator (e.g. comparison to another community or normative group; community opinion; trends over time)? Standards may be relative to an absolute level (e.g. no drug use is acceptable) or relative to other groups (i.e. this seems typical and appropriate for children this age).
- When current data is compared to past data and to, trend analysis over time, is the indicator suggesting child well-being is getting better or worse? What are the possible reasons for the status and trend of this indicator?
- If the indicator or set of indicators suggest an asset that needs to be enhanced or a problem that needs to be reduced, how can the issue be clearly defined?
4. What are the linkages among the various indicators? How are they "woven" together?
5. What is missing in the picture portrayed by the data? What can be done, if anything, to clarify the picture?
6. Taken as a group, which indicators suggest the need for community action? Which is highest priority? Why?
7. Based on this data analysis and interpretation, what courses of community action seem to be appropriate? What is the logic behind the alternative courses? What is known about effective practices?
8. Which course of action is most likely to lead to the group's vision of child well-being in the community?
9. What is the consensus about the most appropriate and feasible course of action?
After communities review their child well being indicators, the next step is to develop an action plan to promote community resources so that children's well-being can be maintained or enhanced. The plan leads to implementation, evaluation, and reassessment.
Andrews and the audience discussed how issues in a community defined, whether by the community itself, by the state or other actors, or both, and participants outlined ways in which they helped explain data to communities and make data useful. These included
These and other steps were thought to guard against misinterpretation of data. One participant advised convening a focus group before data release in order to increase the possibility that all possible reactions would be anticipated.
Andrews asked participants how they encouraged community-based organizations to use the World Wide Web. Answers ranged from walking them through the use of the web in tandem with making applications and other useful materials available thorough the web, and providing an 800 number that organizations could call to get information otherwise available on the web.
Ways of countering inappropriate leaps of logic by data users were discussed. One recommendation to avoid this was in careful framing of data and careful statement of the definitions behind the data. Andrews and participants volunteered examples of differences between the way data elements might be defined by those who collect them and by communities.
David Murphey said that the Vermont Agency for Human Services has helped to establish 12 regional partnerships in the state. These tend to be collaborative partnerships involving the education, health, and other agencies along with nonprofit organizations. These collaboratives meet on a regular basis. They are responsible for implementing outcomes at the community level. These partnerships are at different stages of development.
Vermont's approach embodies these steps.
Murphey displayed the overhead below and commented on it, calling it a work in progress.
Trainings to-date sponsored by the Agency of Human Services in statewide meetings:
Murphey concluded by saying that his presentation was designed to make clear the range of things that Vermont has shared with community partners. Most recently, with the help of the Casey Foundation, they have provided funds to the partnerships for technical assistance, and provided a menu of training options from which to choose.
Jewiss was involved in a qualitative study of two of the community partnerships in Vermont mentioned by David Murphey. The designers of the study chose to examine two partnerships that showed progress on their indicators, without saying necessarily that the partnerships were causing this improvement. The study yielded two findings. One was that data become the most useful when viewed as revealing the situations people experience. The second was that it is important to work with the media to help them understand what the data say about positive change.
The communities involved in the study reported that having the quantitative data on child well-being was very important to them and that the fact of having the data had a powerful impact in discussions facilitated by local leaders.
However, in addition the research team heard that the local communities needed to find other effective ways to relay information to other community members. One asked, "How do you really make the story come alive?" Repeatedly, the research team heard about the need to augment the data with reports of personal experiences. One strategy for augmenting data was to get local media to run stories of successes. Many partnership members were substantially interested in this and some were quite good at it. Positive media reports were thought to fuel the ongoing effort required to sustain outcomes.
One of the partnerships benefited from the acquisition of the local newspaper by a more community-oriented owner.
The Vermont Research Partnership, composed of the education and human service agencies as well as the University of Vermont, surveyed state media and talked with media leaders. They concluded that both the agencies and media each feel that the other fails to understand their role in the community. From the media's point of view, data on public service paid for by tax money is public information and communities need to understand that it is the public's right to have it reported to them in the paper. The agencies felt that the media care only about negative stories.
Things that would improve the working relationship between the media and agencies included
Based on these findings, the Research partnership is developing an action research project for which they will invite teams from five parts of the state to come to monthly workshops with media and public relations experts focused on positive youth development and working with the press.
A panel composed of Michael Bennett, Mary Nelson, and Arlene Andrews followed Jennifer Jewiss. Ada Skyles introduced the panel.
Context
Michael Bennett began by identifying three experiences that shape his comments. First, he noted that Agriculture Secretary Dan Glickman had released a report that indicated that a family with a child born in 1999 could expect to spend about $160,000 raising that child to age 18 years (with inflation factored in, the total rises to $237,000). Bennett said that these figures indicate to him that more children will have inadequate resources for their growth and development.
Second, he noted that the preceding day, while he attended a Council on Foundations meeting in Los Angeles, he went to a breakfast in which a panel of celebrities talked about how they can be useful for particular causes. Rob Reiner impressed Bennett in his description of how he had dedicated himself to the Clinton-Gore goal of having children ready to learn by the time they get to school and how he was trying to think comprehensively about the components of school readiness--such as health, families, and communities.
Third, Bennett, a graduate of Kent State University, pointed out that 30 years ago to the day four Kent State students (5) were shot to death, and others injured, by the Ohio National Guard. Bennett said that the demonstration at which the students were killed was: "about people taking risks because they thought they had the ability to change, about often unmovable forces of government, and about unintended consequences that impact our lives."
Definitions of Community
Bennett offered three definitions of community as it related to the work of supporting children and families. These were
Communities and Indicators
Bennett had four points to make related to communities and indicators.
Bennett said that communities can be very important in helping data gatherers interpret and use data. Meanings may not be obvious to outsiders.
Mary Nelson began her portion of the discussion by describing Bethel New Life, a faith-based organization involved with approximately 10,000 individuals who live on Chicago's West Side. Bethel New Life is a real estate development organization (including commercial and industrial development) but, in addition, it is focused on strengthening and supporting families in a community context. That latter part of their mission includes involvement with the Chicago Case Management and WIC (Women, Infants, and Children) programs, programs for the elderly, welfare-to-work programs, child development programs, daycare homes, and other activities.
Bethel New Life has been involved in a variety of programs, including a state infant mortality reduction program in the early 90s that required the organization to keep track of 100 process objectives. They are currently in the fourth year of a five-year demonstration project involving the American Hospital Association called the Outcomes Tool Kit. In most of the other sites in which this project is under way, it is housed in big institutions, hospitals, or government departments.
Bethel's Own Indicators
Nelson said that Bethel New Life had developed its own set of indicators in order to assess whether things are getting better for families in their area of activities. They developed their indicators using this definition of a healthy, sustainable community. A healthy, sustainable community has:
They are committed to tracking their measures over five years. They have analyzed their findings, and produced easy-to-understand reports that are designed to be useful to those who live in the community. Bethel New Life is very interested in developing approaches focused on outcomes. Nelson notes that the thinking of many community-based organizations is focused on outputs--that is, measures of services provided--and that changing this thinking to an outcome-oriented approach is challenging.
Nelson had four points she wanted to make with the meeting participants. First, if state agencies and other players want community-based agencies to be involved, there must be funds to support this work beyond the per-child service funds. Second, communities must be real partners. They must lead change, not follow the state's agenda. Third, the work must start with assets, not needs, with communities identifying their own strengths and assets. She also said that efforts to work with communities should start not with needs, but with assets. Efforts that focus first on outcomes, then on capacities, help avoid the trap of becoming overwhelmed by needs. Finally, she said that the training process must work both ways and there must be an appreciation of what communities have to share.
Moving to data, Nelson said that the data must be sliced in ways easily accessible. It should be organized by community area, not by zip code, for example. And it should be put into public libraries where it is easily available to communities.
5. Allison Krause, Jeffrey Miller, Sandra Scheuer, and William Schroeder.
Main Page of Report | Contents of Report
Home Pages:
Advancing States' Child Indicators Initiatives
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)