Advancing States' Child Indicators Initiatives:

Key Themes: Reflections from the Child Indicators Projects

Guiding Principles for Selecting Child Indicators: Reflections from the Child Indicators Project

Mairéad Reidy. Ph.D.,
Senior Research Associate
Chapin Hall Center for Children
University of Chicago,
(773) 256 5174 (phone)
reidy-mairead@chmail.spc.uchicago.edu

This short paper is based on discussions between the fourteen states participating in the ASPE Child Indicators Project. It focuses on state reflections on the factors that are important in the selection of indicators at the state and local levels.

Sponsored by the U.S. Department of Health and Human Services (HHS) Office of the Assistant Secretary for Planning and Evaluation (ASPE), with additional support from the Administration for Children and Families (ACF) and The David and Lucile Packard Foundation, the Child Indicators project has aimed over the past 3 years to promote state efforts to develop and monitor indicators of health and well-being of children during this era of shifting policy. The fourteen participating states are Alaska, California, Delaware, Florida, Georgia, Hawaii, Maine, Maryland, Minnesota, New York, Rhode Island, Utah, Vermont, and West Virginia.Chapin Hall Center for Children provided technical assistance to grantees. Grantees typically exchanged knowledge and expertise through a series of technical assistance workshops coordinated by and held at Chapin Hall Center for Children. The workshops encouraged peer leadership and collaboration among states, and provided states with an opportunity to work with and learn from one another on areas of common interest. This short paper draws on the discussions of these meetings as well as individual consultation with states. I am grateful to participants for sharing their insights.

• When monitoring goals for children or measuring outcomes from child initiatives, the experience of the participating states in the ASPE Child Indicators Initiative suggests that the choice of indicators is typically driven by the policy priorities and energies of policymakers, the audience for the indicators, the availability of data, and by the strongest predictors of desired outcomes.
  The priorities of policymakers can change often. When policymakers need to justify expenditures to the legislature to secure continued funding for an initiative, the priorities of the legislature will take precedence and measures will be selected to cater to those priorities. Advocates, service providers, and researchers may all demand and require different indicators than those useful to legislators. What satisfies one may be confusing to another, and all need different levels of detail and explanation. The choice of indicators will also be determined by available data. Typically, health data aside, early childhood data is scarce. Initiatives will understandably focus at least initially on what is measurable. Choice is further guided by the outcomes of interest and the research on legitimate predictors of outcomes and interim measures of expected change, as articulated in the theory of change.

• It was widely believed that communities need to own the indicators. The choice of indicators at the community level will also be determined by the needs of the community, where the energies of a community lie, and the availability of trend data.
  Communities need to feel invested in indicator selection and use. For the community to own these indicators, many states believed that the community must participate in the selection process. Community residents must not feel that the indicator choice has been foisted on them. As at the state level, the choice of indicators at the community level will be determined by the needs of the community, where the energies of a community lie, and the availability of data. Some states, such as Vermont, that provide available trend data on outcomes to communities have found that this data can be an important catalyst for community engagement and ownership of indicators.

• It is critically important to select measures that are appropriate for diverse cultural, racial/ethnic, and economic groups and are adaptable to local circumstances.

  Cultural differences may mean that certain indicators, useful in some states or in some communities, may be irrelevant in others. For example, whether a child is read to everyday may have less meaning in a state such as Alaska in which some cultural groups rely more on an oral tradition.

• At both the state and community levels, it is critical to choose measures that have high-quality data that will be available for a period of years.
  There are groups of indicators that fall more easily in to this category that have their origin in ongoing vital statistics, Census data, and administrative data. This is not to preclude indicators built from sample surveys, but we need to acknowledge that samples of sufficient size are needed to disseminate reliable state and regional data, and that these surveys need to be repeated to build trend data.
  
• At both the state and community levels, measures selected should be clear in interpretation over time, across localities and subgroups.
  Trends in an indicator should ideally represent unambiguously whether conditions are improving. It should be clear that when an indicator moves in a particular direction that it represents an improvement (or deterioration) in well-being overall. School achievement test data in Florida has, at times, not included certain children including those with low attendance throughout the year, making it very difficult to use the indicator to say reliably whether schools are getting any better or worse over time. Sometimes we can improve clarity by mapping sets of related indicators. A decline in the percentage of children in special education programs can be considered an improving picture for children if we also show that fewer children need services. Some indicators are particularly sensitive to variation in practice at the district level such as child abuse and neglect and foster care placement rates and juvenile crime rates. These rates are sensitive to child protection agency practice and the criminal justice system practice at the local level (who gets reported and into the system, the rate referral of juvenile offenders to courts, etc.). When measures are sensitive to variation in practice over time or variation at the regional or local level, every effort should be made to acknowledge these differences.

• Measures selected should also be shown to be robust and comparable for the socioeconomic and demographic groups involved in the initiative.
  Many measures have been developed using white middle class families, and can fail to pick up important dimensions of the lives of particular cultures or income groups.

Where to?

Top of Page
Back to Key Themes main pages

Home Pages:
Advancing States' Child Indicators Initiatives
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)