Advancing States' Child Indicators Initiatives:

Key Themes: Reflections from the Child Indicators Projects

General Uses of Child Indicator Studies

Mairéad Reidy. Ph.D.,
Senior Research Associate
Chapin Hall Center for Children
University of Chicago,
(773) 256 5174 (phone)
reidy-mairead@chmail.spc.uchicago.edu

This short paper discusses the general usefulness of indicator studies, and is based, in part, on discussions among the fourteen states participating in the ASPE Child Indicators Project.

Sponsored by the U.S. Department of Health and Human Services (HHS) Office of the Assistant Secretary for Planning and Evaluation (ASPE), with additional support from the Administration for Children and Families (ACF) and The David and Lucile Packard Foundation, the Child Indicators project has aimed over the past 3 years to promote state efforts to develop and monitor indicators of health and well-being of children during this era of shifting policy. The fourteen participating states are Alaska, California, Delaware, Florida, Georgia, Hawaii, Maine, Maryland, Minnesota, New York, Rhode Island, Utah, Vermont, and West Virginia. Chapin Hall Center for Children provided technical assistance to grantees. Grantees typically exchanged knowledge and expertise through a series of technical assistance workshops coordinated by and held at Chapin Hall Center for Children. The workshops encouraged peer leadership and collaboration among states, and provided states with an opportunity to work with and learn from one another on areas of common interest. This short paper draws on the discussions of these meetings as well as individual consultation with states. I am grateful to participants for sharing their insights.

• Indicator studies can highlight generally that things may not be working as planned, and can guide decisions o the types of in-depth evaluations that might be helpful.
• Using indicator studies, policymakers and researchers can examine broad trends over time. We track indicators over time for a range of purposes including the following:
  • to describe child, family, community conditions
  • to inform state and local community planning and policymaking
  • to improve programs for children and families (e.g., increase access, improve quality)
  • to measure progress in improving child outcomes
  • to monitor changes for children in relation to investments and policy choices
• We can enhance the power of indicators to monitor broad trends and to inform policy when we analyze how sets of indicators vary across socioeconomic and demographic subgroups, counties, regions, etc., and when we cluster indicators and examine whether sets of related indicators move in the same direction. It is important to remember that a change in one indicator may disguise movement in another area. Without looking at a series of indicators, what appears to be a clear improvement may in fact be deterioration. For example, improvements in kindergarten retention rates (where fewer children are being retained) imply that children are faring better. However, without looking at other indicators for example, later success rates of children not retained, it is difficult to interpret whether a change in the retention rate in fact shows that children are doing better. Likewise, to be sure that a decline in substitute care placement rates is a positive outcome for children, the indicator should be accompanied by other indicators such as a declining rates of child abuse and neglect
• Although indicator studies cannot establish causal relationships between initiatives and outcomes, they can monitor progress toward outcomes across time. Indicator studies can thus play critical roles in monitoring progress towards goals and in documenting whether changes in outcomes are occurring in desired directions.
• Indicator studies can warn that things are not working as planned. Such warning can precipitate in-depth evaluations (Prosser and Stagner, 1997).
• The power of indicators to monitor policy and program outcomes can be enhanced if they are used in conjunction with a logic model. The logic model can guide decisions about what indicators should be measured and in what order the measurement should take place. The logic model enables us to measure short-or intermediate-term outcome indicators with some confidence that observable change on those outcomes will be followed later by changes in longer-term outcomes.
• Indicators can complement the data collected from impact studies by placing results in the context of broader social and economic trends.
• The conditions necessary for implementing impact studies do not always exist. Governments are sometimes reluctant or unable to prevent the exposure of families to a promising new initiative, making it impossible to create a control group. Furthermore, when policy changes occur quickly and in multiple programs, as is the case with many statewide early childhood initiatives, it is often extremely difficult to isolate the effects of a single policy or initiative using impact studies (Child Trends 2000). Under these circumstances, indicators may serve as the only source of information on the general direction of change.

References:

Child Trends (2000) "Children and Welfare reform: A Guide to Evaluating the Effects of State welfare Policies on Children", Child Trends.

Where to?

Top of Page
Back to Key Themes main pages

Home Pages:
Advancing States' Child Indicators Initiatives
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)