Hosted by Rhode Island KIDS COUNT
Providence, December 2 & 3, 1999
This meeting was sponsored by the Advancing States’ Child Indicators Initiatives project of the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, in partnership with the Carnegie Corporation of New York Starting Points initiative and the Finance Project. Martha Moorehouse of ASPE is the Project Officer for the Advancing States’ Child Indicators Initiative project.
These proceedings were prepared by the Chapin Hall Center for Children at the University of Chicago. Harold Richman is the Principal Investigator for the Advancing States’ Child Indicator Initiatives project and Mairéad Reidy is its Project Director.
"Goal
To assist states and communities participating in the ASPE/HHS Child Indicators Project and the Carnegie Starting Points Initiative to develop practical indicators that can be used to track children’s readiness at school entry and to monitor child care quality, availability, and affordability.
Strategies:
- To share information on indicators currently being used by states to track children's readiness for school and to monitor the child care infrastructure.
- To discuss conceptual frameworks that can inform state efforts to track children's readiness for school and to monitor child care quality, availability, and affordability.
- To develop a working list of practical indicators of children’s readiness for school and child care quality, availability, and affordability.
Thursday, December 2
Welcoming Remarks
The meeting participants gathered for lunch and were welcomed by Jay Lindgren, the Chair of the Rhode Island Children’s Cabinet; Ann Segal of the Office of the Assistant Secretary for Planning and Evaluation; Michael Levine of the Carnegie Corporation of New York; and Elizabeth Burke Bryant who heads Rhode Island Kids Count.
Luncheon Address: Christine C. Ferguson, Director Rhode Island Department of Human Services
DHS Director Christine Ferguson ‘s remarks focused on the urgent need to select a small number of clear, understandable indicators that can be used to show that investments in early childhood education yield positive results for children. She said that legislatures around the country, including Rhode Island’s General Assembly, are making unprecedented investments in early childhood programs. They have been told that participation in high quality early childhood education programs increases children’s readiness for school, and that the long- term cost/benefit is very positive because dollars invested in early childhood programs save much larger expenditures for more costly interventions later in life.
Director Ferguson gave the example of RIte Care, Rhode Island’s Medicaid Managed Care program. Just before a major expansion of the program occurred four years ago, the Department of Human Services identified several clear, understandable, and compelling measures to gauge the effectiveness of the program, such as birth intervals, the number and timing of prenatal care visits, and smoking during pregnancy. These indicators have been tracked every year, and the results have been very positive. She is able to take simple trend graphs depicting these results around the state to her meetings with legislators and business leaders. She said it has been clear to her that when you can show compelling data on the impact of a program, elected officials will happily support continued or expanded funding because they can see that it is making a difference.
Director Ferguson said that in the area of early childhood education and child care, there is no one set of perfect indicators and encouraged the participants to work during the course of the conference to agree on a small number of indicators so that we can all start using them. She warned that unless we immediately identify and start tracking a few measures that are compelling, and that make sense to the average person, people in her position and others around the country will have a difficult time convincing elected officials that they should continue making large appropriations for child care and early childhood education at the expense of other competing priorities.
Keynote Address: Sharon Lynn Kagan
Judith Jones, Clinical Professor of Public Health at Columbia University, introduced Sharon Lynn Kagan, who is Senior Associate at the Yale University Bush Center in Child Development and Social Policy and President of the National Association for the Education of Young Children. Kagan delivered the keynote address, "Assessing Young Children: Issues, Definitions, and Options." A summary of that address follows.
Introduction
Assessment of young children is a high-stakes issue. Parents, legislators, and others want answers now.
Where Are We Now?
We are now marking the tenth anniversary of the National Educational Goals. One of these goals calls for a school-readiness standard. A decade after the call was issued an agreed-upon standard is not yet in place. Even without an agreed-upon readiness standard, assessment has been a part of the early childhood field for a decade and is increasing. Testing of preschoolers is in place in some areas and is coming to others. Test makers are in the process of developing instruments. Although early childhood professionals understand age and cultural appropriateness issues related to assessments, this understanding is not universal outside the field. There is a danger that data secured for one purpose may be applied to other, inappropriate purposes.
What Do We Need?
We currently lack clear definitions of what is appropriately assessed and what are appropriate assessment tools and measures.
Purposes of Assessment
There are many purposes for assessment. It can be used as a tool to
- Improve learning and to identify what children know and are able to do
- Identify children with special needs and provide services to them
- Evaluate programs and monitor progress
- Provide accountability by supplying information to make decisions and by demonstrating that investments have made improvements
Types of Assessment
Different types of assessments will measure different qualities. We should think about these types of assessment as we choose different indicators and try to include some that fit in each of the following four categories or "buckets." (Kagan’s bucket metaphor was adopted by meeting participants and employed frequently in subsequent discussions.)
Bucket 1: Child outcome assessments that measure things children know and can do. Children should be measured in five areas: physical well-being, social and emotional development, approaches toward learning, language development, and cognition and general knowledge.
Bucket 2: Conditions that surround children. Such aggregate indicators of child and family well-being might include information on parents’ ages, substance-abuse histories, or income levels.
Bucket 3: Delivery/program standards. This type of assessment is related to the availability, quality, and accessibility of services for children. Examples include the prevalence of high-quality childcare programs or the level of access pregnant woman have to prenatal care.
Bucket 4: Systemic performance assessment. Assessments of systems of services can address such issues as service efficiency, provider compensation, or the availability of training across programs.
Understanding the Importance of Child Outcomes
Although establishment of program standards and measurement of systemic performance provide important information, they tell us little about the progress of children and do not provide information to help improve programs in ways that result in better performance for children. For a long time, program input standards and Head Start standards have been used in place of measures of child results. Relying on these standards does not contribute to the improvement of program quality. Data around what children can be expected to know and do — the bucket 1 assessments — can be an "elixir of quality" for preschool programs. (There may be greater application for programs for older preschoolers than for infant and toddler programs.)
What is Our Strategy?
We need to develop consensus on a strategy for assessments that reflect the views of early childhood professionals. It is not essential for states to have a definition of school readiness. Instead, chronological age could be the definition of whether or not a child is ready for school if the goal is to make schools ready for all children.
We need to develop instruments that collect information on children over time and are appropriate for use by multiple providers to collect this information. If localities have too much input in the development of instruments, data may not be comparable across sites. We need to think systemically.
States and localities are in the forefront of this work. Connecticut and California are among the states that have developed statewide child indicators. They are holding localities accountable to these standards. Florida has recently passed legislation that aggregates funding streams at the state level and then gives the control of these resources to local governance. Florida is now developing an assessment system for accountability.
There is a tremendous need to assess what preschool kids can know and do. There is a need for a continuum of data as well; data collected in schools should include information on children’s preschool experiences.
Discussion
Following her talk, Kagan entertained questions. Regarding assessment, Kagan said that making the case for assessment was critical to sustaining related enterprises, such as family preservation and family support programs. She noted the need to protect assessment data from misuse and to ensure that appropriate measures of context were incorporated into assessments.
Response Panel
A response panel composed of Deborah Benson of the New York State Council in Children and Families, David Murphey of Vermont Agency of Human Services, and Fredericka Wolman, M.D., of the Maine Department of Human Services, commented on the keynote address. Murphey addressed the issue of focusing on assessments of what children can know and do at particular developmental stages (bucket 1). Wolman noted Maine’s difficulty in creating simple indicators to describe very complex situations. Benson said that she was unconvinced the public has embraced the notion that society has responsibility for children’s education before they enter school.
Discussion
Catherine Walsh of Rhode Island Kids Count summarized Kagan’s talk and noted that both short- and long-term goals can be supported by school-readiness and child care indicators. Then Mairéad Reidy and Sara Watson sketched the indicators under development by the ASPE-supported and Starting Points projects.
The ASPE-Supported Sites
Mairéad Reidy of Chapin Hall reviewed the indicators under development in each ASPE-supported project. Those indicators were summarized on a handout that is reproduced below.
School Readiness
Health-Related Indicators
Immunization
- Percentage of kindergartners fully immunized (Department of Health immunization program) [Vermont]
- Percentage of properly immunized two-year-olds (Kids Count) [Hawaii]
- Percentage of children entering school immunized (Department of Health) [New York]
- Percentage of children who are immunized on appropriate schedule (early childhood screening) [Minnesota]
Abuse & Neglect
- Rate of child reported abuse & neglect (Kids Count) [Hawaii]
- Rate of confirmed child abuse & neglect (Kids Count) [Hawaii]
- Indicated reports of child abuse and maltreatment (Office of Children and Family Services) [New York]
Injuries/Hospitalization
- Rate of unintentional injuries receiving emergency room treatment (Kids Count) [Hawaii]
- Rates of hospitalization resulting from asthma, ages 0–4 years (Department of Health) [New York]
Birth Related
- Teen birthrate (Kids Count) [Hawaii]
- Percentage of low-birthweight births (Department of Health) [New York]
- Percentage of women with late or no prenatal care (Department of Health) [Rhode Island]
- Percentage of low-birthweight infants (Department of Health) [Rhode Island]
- Infant mortality rate (Department of Health) [Rhode Island]
Health Status
- Percentage of children in group care receiving nutritious breakfasts (Department of Education, USDA) [Hawaii]
- Percentage of children with health insurance (Kids Count) [Hawaii, Rhode Island]
- Percentage of children with history of lead poisoning (Department of Health, lead screening data) [Rhode Island]
Social/Economic Related Indicators
- Percentage of children/youth in poverty (Office of Temporary & Disability Assistance) [New York]
- Child poverty rate (Kids Count) [Hawaii]
- Percentage of children (ages 0 - 4) in families receiving welfare (Department of Social Welfare) [Vermont]
- Percentage of children receiving public assistance (Office of Temporary & Disability Assistance) [New York]
- Percentage of children receiving food stamps (Office of Temporary & Disability Assistance) [New York, Rhode Island]
- Number of centers participating in USDA food programs (Department of Education, USDA) [Hawaii]
Educational Program Participation and Developmental Indicators
Participation in Early Interventions
- Percentage of children (ages 0-3) receiving early intervention (Department of Health and Social Services) [Delaware]
- Participation rate of eligible 4-year-olds in Head Start & early child assistance programs (Department of Education) [Delaware]
- Number/percentage of children enrolled in Head Start (Head Start) [Maryland]
- Number/percentage of children enrolled in infants & toddlers program (Department of Education) [Maryland]
- Percentage of children in early intervention (Department of Health) [New York]
- Percentage of children enrolled in preschool special education (Department of Education) [New York]
- Percentage of 4-year-olds enrolled in licensed/registered early childhood programs (Office Children & Family Services) [New York]
- Percentage of children in full-day kindergarten programs (Department of Education) [Rhode Island]
Family Involvement
- Percentage of parents actively involved in children’s learning/education (ECFE, School Readiness, Head Start) [Minnesota]
- Percentage of families with parenting knowledge and skills to anticipate and meet a child’s developmental needs (ECFE, School Readiness, Head Start) [Minnesota]
Testing and Grade Retention
- Percentage of third graders scoring in 4th quartile on basic skills math, reading, & language (Department of Education) [West Virginia]
- Percentage of third graders scoring in 1st quartile on basic skills math, reading, & language (Department of Education) [West Virginia]
- Number/percentage of students scoring B average or better across 6 domains combined on the pre-kindergarten assessment (evaluation & assessment) [Utah]
- Percentage of children "ready" for kindergarten, as judged by teachers (kindergarten teacher survey) [Vermont]
- Percentage of children retained by the end of first grade (Department of Education) [West Virginia]
- Percentage of children retained at least once prior to grade 4 (Department of Education) [West Virginia]
Physical Development
- Percentage of children with hearing and vision problems at time of entry into kindergarten (early childhood screening) [Minnesota]
Social Development/Language/Communication
- Percentage of children showing individual developmentally appropriate progress in all areas of their development (early childhood screening, IFSP) [Minnesota]
Cognitive Development
- Percentage of children showing individual developmentally appropriate progress in all areas of their development (early childhood screening, IFSP) [Minnesota]
Childcare
Quality of Care Indicators
Licensing, Accreditation, and Regulation
- Number of licensed centers (Childcare Services Division) [Vermont]
- Number of registered family day care homes (Childcare Services Division) [Vermont]
- Number of nationally accredited early childcare and education sites (National Association for the Education of Young Children) [Delaware]
- Number of nationally accredited family childcare homes (NAEYC) [Delaware]
- Number of accredited preschools (NAEYC) [Hawaii]
- Number of regulated childcare centers and family care homes (licensing files) [Hawaii]
- Number of accredited facilities (Office of Children & Family Services) [New York]
- Number of regulated/licensed programs (Office of Children & Family Services) [New York]
- Number of licensed slots compared to the number of children with working parents, planned (Census Bureau) [Utah]
- Percentage of childcare center slots in accredited programs (Department of Human Services) [Rhode Island]
- Percentage of family childcare slots in accredited programs (Department of Human Services) [Rhode Island]
- Percentage of Head Start slots meeting quality standards (Head Start) [Rhode Island]
- Number of family childcare associations (Governor’s Cabinet on Children & Families) [West Virginia]
- Number of nationally accredited early childhood programs (Governor’s Cabinet on Children & Families) [West Virginia]
- Number of nationally accredited family childcare homes & facilities (Governor’s Cabinet on Children & Families) [West Virginia]
Staffing Patterns and Characteristics
- Percentage of parents using quality childcare arrangements as defined by training levels, experience & accreditation (CCR&R) [Minnesota]
- Percentage of childcare slots in programs with low staff turnover rates (rate survey) [Rhode Island]
- Percentage of childcare center staff with early childhood education degree (rate survey) [Rhode Island]
- Percentage of family childcare staff with early childhood training (rate survey) [Rhode Island]
Safety
- Percentage of early care and education slots in programs without health and safety violations (Department of Children, Youth, Families, childcare licensing) [Rhode Island]
- Number of serious injuries (abuse) in childcare (Childcare Services Division) [Vermont]
Accessibility of Care Indicators
Availability
- Number of slots in regulated childcare centers and family care homes (licensing files) [Hawaii]
- Number of licensed childcare spaces available for every 100 children of working parents (CCR&R) [Minnesota]
- Number of early care and education slots per 100 children from birth to age 3 (Department of Human Services) [Rhode Island]
- Number of early care and education slots per 100 children ages 3-5 years (Department of Human Services) [Rhode Island]
- Number of early parent education programs (birth though 2 years) (Governor’s Cabinet on Children & Families) [West Virginia]
- Number of starting points family resource centers (birth through 8 years) (Governor’s Cabinet on Children & Families) [West Virginia]
- Number and rate of before/after school programs per 1,00 low income children with working parents (Census Bureau) [Utah]
- Number of children on Head Start waiting list (Head Start) [Hawaii]
- Estimated percentage that need childcare (Childcare Services Division) [Vermont]
Participation
- Percentage of low-income children enrolled in comprehensive childcare programs or Head Start (Department of Human Services) [Rhode Island]
- Percentage of children enrolled in early care and education programs the year prior to school entry (SALT survey) [Rhode Island]
- Percentage of "at-risk" children enrolled in early care and education programs prior to school entry (administrative databases) [Rhode Island]
- Number of children in Head Start programs (Head Start) [Hawaii]
Affordability of Care Indicators
General Costs
- Average weekly rate (annual market rate survey) [Vermont]
- Subsidized weekly rate (Childcare Services Division) [Vermont]
- Percentage of families paying more than 10 percent of their income for childcare (CCR&R) [Minnesota]
Subsidy Related
- Number of subsidized slots (Office of Children & Family Services) [New York]
- Percentage of families receiving childcare subsidy support (Department of Health and Social Services) [Delaware]
- Number of children receiving subsidized care in regulated homes (Department of Human Services) [Hawaii]
- Number of children receiving subsidized care in family childcare homes (Department of Human Services) [Hawaii]
- Percentage of families on childcare subsidies (Office of Children & Family Services) [New York]
- Percentage of licensed childcare providers who are willing to care for children receiving childcare assistance (CCR&R) [Minnesota]
- Percentage of families receiving childcare assistance (CCR&R) [Minnesota]
- Percentage income-eligible families using childcare subsidies (Department of Human Services) [Rhode Island]
The Starting Points Sites
Sara Watson of the Finance Project described the Carnegie Corporation of New York’s Starting Points sites. She began by describing the Starting Points initiative, which aims to support the implementation of programs that address some of the needs identified in Carnegie’s Starting Points report on early childhood issues.
Watson distributed a handout, reproduced below, that provided examples of some of the indicators that are under consideration in the Starting Points sites. (In supplying a copy for this summary, Watson stressed the exemplary quality of the indicators listed, cautioning that some sites have more extensive activities than are revealed by this list and also that, because indicators are in development, some on the list may not be realized.)
Watson noted that these indicators fall into all four of Kagan’s buckets, but in some cases, Starting Points sites take a different approach than do the ASPE-supported projects. Watson commented on the importance of positive measures.
| Indicator Description (actual or proposed) | Data (or Indicator) Source |
|---|---|
| Pittsburgh | |
| The percentage of children, ages zero to five years, enrolled in family centers in Allegheny County meeting the appropriate developmental targets for their chronological ages. | Early Learning Accomplishment Profile/LAP and Denver II Screening Tool |
| San Francisco | |
| Increase childcare space availability | San Francisco Starting Points Initiative, "Children Are Ready and Able to Learn" |
| Increase eligible children receiving childcare subsidies | Same |
| Increase proportion of kindergartners who attended childcare | Same |
| North Carolina | |
| Smart Start Evaluation Indicators | |
| Status of childcare arrangements, child health, family satisfaction with childcare | Smart Start Family Questionnaire |
| Children’s motor, language, social and cognitive skills | Kindergarten Teacher Checklist |
| Child health data | Kindergarten Health Assessment |
| Childcare professionals’ perceptions of skills and need for training in child development, childcare environment, curriculum, learning and professionalism | Self-Assessment for Child Care Professionals |
| Childcare center director, teacher and child characteristics | Childcare Center Director Interview |
| Classroom practices | Childcare Center Provider Form |
| Childcare teacher demographics | Childcare Center Provider Form |
| Childcare provider interactions with children, measuring sensitivity, harshness, detachment and permissiveness | Caregiver Interaction Scale |
| Family childcare provider status | Family Child Care Provider Interview |
| Demographic, education and work information for families with children in family childcare | Survey of Families with Children in NC Family Childcare Homes |
| Smart Start Program Performance Standards | |
| All teachers working in early childhood programs have an associates or bachelors degree in early childhood education or child development, or they are enrolled in a degree program leading towards the attainment of such a degree. | Workforce Study, TEACH |
| Every early childhood program has a three-star rated license and/or is nationally accredited or is progressing towards the attainment of a three star rated license and/or national accreditation. | State agencies, TEACH |
| Teachers working directly with children in early childhood programs are compensated at a rate that is comparable to teaching staff with comparable education in public schools (measured at the county median). | Workforce Study, TEACH |
| A sufficient supply of childcare exists that is appropriate for the individual child and accessible to families who need and want it. Families are able to find and access early childhood programs that are needed and appropriate for the individual family’s need. | State agencies |
| At least 75 percent of young children (0-5) living in families needing childcare and earning less than 75 percent of the county median income will receive subsidized early care and education services. | State agencies |
| No family will have to pay more than 10 percent of their gross income for childcare. | State agencies, parent and other surveys |
| All young children receiving Smart Start services will have a source of primary medical and dental care, with access to comprehensive, integrated, specialized care (including mental health services) as necessary. All care, including preventive screenings, will be coordinated with the child’s family and primary healthcare provider. | Direct service provider |
| Childcare environments are safe and healthy for all children in care. | State agencies |
| To meet the diverse needs of families, a variety of family support services appropriate for individual children and families are available and accessible. | State agencies, schools, surveys |
| Family support services are coordinated with and support the development of health and quality early childhood education services. | Direct service provider |
| Five Star Childcare License | |
| Quality assessments of childcare centers | Rating scale using
Infant/Toddler Environment Rating Scale |
| Baltimore (Maryland) | |
| School readiness | |
| Maryland Model for School Readiness: ongoing assessment (proposed) | Work Sampling System examining (1) social and personal; (2) language and literacy; (3) mathematical thinking; (4) scientific thinking; (5) social studies; (6) the arts; and (7) physical development domains |
| Number of children entering kindergarten that attended preschool | Proposed – Recommended Results Indicators of Child and Family Well-Being Report |
| Number of children enrolled in early intervention programs | Same |
| Number of low-income children in Head Start or pre-kindergarten programs | Same |
| Childcare | |
| Child population | Maryland Committee for Children/U.S. Bureau of the Census, etc. |
| Childcare costs as compared to other major household expenses | MCC/Maryland state agencies |
| Average weekly cost of full-time childcare | Same |
| Children’s programs by type with capacity/ enrollment | Same |
| Percentage of family care providers and childcare centers willing to care for subsidized children | Same |
| Number of family providers and centers who serve/have served children with special needs | Same |
| Annual wage rate information for public school and childcare staff | Same |
| Density of family providers and center programs by jurisdiction, census tract | Same |
| Number of children 0-5 years of age per regulated childcare space | Same |
| Past and anticipated growth patterns for family/center providers | Same |
| Children served by age | Same |
| Children served by locational preference for care | Same |
| Children served by type of care preferred | Same |
| Reason childcare is needed | Same |
| Full-time or part-time care needs of children served | Same |
Dividing into Work Groups
The participants then broke into two groups; one focused on childcare and the other on school readiness. Discussion in these two groups helped frame the issues to be covered during Friday’s sessions.
Friday, December 3
Measuring Children’s Readiness for School: From the Ideal to the Practical
Introduction
Martha Moorehouse of the Office of the Assistant Secretary for Planning and Evaluation of the U.S. Department of Health and Human Services introduced the morning’s speaker, John Love of Mathematica Policy Research. Among Love’s research activities is the evaluation of the federal Early Head Start program, for which he serves as principal investigator.
John Love
John Love began by noting that readiness is of interest to the public, taxpayers, politicians, and courts. He cited a recent New Jersey Supreme Court ruling that the state must provide services to low-income children. By way of introduction, he said that program evaluation is composed of two factors, what programs can do for kids and what programs are doing.
In moving from the idea to the practical, we need to consider
- What is the ideal? (What is its definition, how is it measured? What are our assessment strategies? How do we interpret what we measure?)
- What are the practicalities?
- What limits our vision?
- Where do we go from here?
This consideration will help us create a process that involves the whole community.
What Are Appropriate Conditions of School Success?
Love asked what are the appropriate standards for success in school. Kindergarten teachers value children who are physically healthy, rested, and well nourished; children who are able to communicate their needs, wants, and thoughts verbally; and who are enthusiastic and curious in approaching new activities. Teachers don’t necessarily judge success by how high a child can count or how well he or she knows the alphabet.
Community Supports for Readiness
Love asked, "how do communities support readiness?" He touched on dimensions of readiness and on community attributes that support readiness, such as
- The presence of high-quality, developmentally appropriate preschool programs
- The presence of programs that help ensure that children receive nutrition and health care
- Programs to ensure that parents are children’s first teachers
- The presence of accessible training and support programs for parents that support readiness.
Love sees a value in efforts to look at such support programs and capture other circumstances (such as birthweight) that are not themselves measures of readiness, but can predict readiness.
Achieving Readiness
Love asked, "how do we achieve readiness for all children?" He noted that there are so many differences in experiences and inequities before school and there are substantial differences in children’s needs. He asked that we seek reasonable and appropriate expectations for children’s learning. Drawing on an NAEYC position statement, he suggested a commitment to
- Addressing the inequities in early life experiences so that all children have access to the opportunities that promote school success
- Recognizing and supporting individual differences among children
- Establishing reasonable and appropriate expectations of children’s capabilities on school entry
Defining and Measuring Readiness
Love presented his ideas on defining and measuring readiness. Appropriate definitions of readiness take into account that
- Readiness is not a characteristic of the child
- Readiness describes a relationship
- Readiness includes levels of development and learning appropriate for success in school
Love said that there are degrees of readiness. He urged a broad definition of readiness extending beyond academics. Such a definition could include
- Physical well-being and motor development of children
- Children’s social and emotional development
- Children’s approaches to learning
- Children’s language development
- Children’s cognitive and general knowledge
Readiness might include child’s strategies to cope with a range of different circumstances.
Love called for measures that are appropriate for all cultural, racial, ethnic, and linguistic groups in the community; balance positive and negative indicators; and are adaptable to local circumstances. He urged that measures be flexible, expandable, and of types that can be implemented now.
Assessment Strategies and Interpreting Data
Love recommended employing multiple modes of assessment and multiple perspectives. Multiple modes might include direct assessments, judgmental ratings, and classroom observations (which he cautioned tend to be expensive). Multiple perspectives would include those of teachers, parents, children, and others.
In interpreting measures, Love suggested
- A focus on the collective or aggregate status of entering kindergartners
- The use of community-wide estimates
- The development of estimates of readiness for all the important subgroups within the community
- The application of findings to continuous community improvement efforts
He noted that many times it is too expensive to gather individual-level data but that survey sampling can allow a relatively small number of cases to reflect the diversity in the community. We need to think creatively. It is important: to think about why we are doing this. We should be able to address policy questions: Are the investments we are making paying off? Are they related to any changes in the readiness of children?
Practicality
Love asked, "what are the practicalities?" As an answer, he said that we should look at community commitment
- Of parents’ and teachers’ time
- To coordination and supervision
- To quality control
- To data management
Next Steps
Love closed by sketching next steps.
- Determining the process for deciding the outcomes
- Selecting measures
- Deciding on assessment procedures
- Determine how to sample the children
- Decide on levels of aggregation for analysis
- Analyze and interpret the data
- Reporting to stakeholders and creating a feedback loop
Next
Following Love’s talk, the meeting divided into two work groups—one focused on school readiness and the other on childcare. The school readiness group further divided into three subgroups.
Work Group I: Measuring Children’s Readiness for School
Subgroup 1: Tracking Trends in Five Domains of Early Learning
Subgroup 1 was composed of representatives from Hawaii, Maine, Maryland, Rhode Island, New York, and Chapin Hall. It was charged with identifying school readiness indicators that could be used to track trends using the five domains of early learning, development, and abilities and the four assessment buckets plus a fifth category related to the readiness of schools for children. Although the initial task was to identify indicators that are currently in use, as well as future indicators, the subgroup recognized that each state was at a different developmental stage, and consensus could not be reached on what is available now or what will be developed. Therefore, the subgroup members, pulling from their respective state’s practices, compiled one list of current and proposed indicators.
Cath Burns from the University of Vermont mentioned Vermont’s three-pronged assessment that uses three surveys. One survey examines whether schools are ready for kids, a second collects data from parents of kindergartners, and the third collects data from nurses. In addition, Burns shared a draft kindergarten teacher questionnaire designed by the University of Vermont. Since the questionnaire is based and organized on the five domains and has been drafted, reviewed and revised, the subgroup agreed to incorporate the majority of its questions into the first bucket, things children know and can do. Because its primary task was to identify school readiness indicators that could be tracked over time, the group opted not to focus on bucket four, system issues. The readiness of schools had been added to the subgroup’s list and it recognized the responsibility of the school to provide an environment for promoting learning and development.
The following table is organized by the five domains and by the four buckets plus indicators of the readiness of schools. While the subgroup identified specific bucket 1 indicators for each domain, these indicators are not mutually exclusive. Indicators identified for buckets 2, 3, and 5 were identified as generally having an impact across the domains and have been grouped and listed once.
| Buckets | Domains | ||||
|---|---|---|---|---|---|
| Physical Well-Being and Motor Development | Social and Emotional Development | Approaches Toward Learning | Communication Skills | General Knowledge | |
| What Children Know and Can Do |
|
|
|
|
|
| Child and Family Conditions |
|
|
|||
| Services Provided |
|
|
|||
| Understanding the Success of the System | Not applicable to the focus of this subgroup | ||||
| School Ready |
|
|
|||
Subgroup 2: Political Indicators/Indicators that Communicate the Results of Investments in School Readiness
The recommendations made for proposed indicators by the second school readiness group included
- Adopt a "low-hanging fruit" strategy. (That is, use what you can get.)
- Link existing data sources (e.g., birth certificates and lead-screening records) to show something different that the sources show individually.
- Correlate data with the different "buckets" and research insights (e.g., mother's education level on birth certificate with research linking cognitive development and mother's education level).
- Identify periodic data contact after birth (e.g., lead screening, well-child visits) that can be used to create a picture of child over time.
- Recognize that changemakers may need 100 charts; have different datasets that could be used to address your changing needs and theirs.
- Sustain a core of indicators to be tracked consistently over time.
- Identify priority groups (pediatricians who may touch most young children periodically between birth and five) that can ask "cross-service/discipline" questions (e.g., about childcare).
- Note that many indicators apply to two or more "buckets."
- Use a marketing approach to present data. Couple data with human presentation (e.g., if reporting results of survey of kindergarten teachers, have a panel of teachers share their stories at the same time). Explain data in terms that resonate with the audience (e.g., turnover rates in childcare to impact that turnover has on the business community).
| BUCKET | NOW | OVER TIME |
|---|---|---|
|
1 |
Mother’s education level
Parental stress index Kindergarten assessment Kindergarten registration information |
Survey of children’s function (all domains)
Kindergarten teachers’ survey of children’s abilities Children’s ability to communicate Reduction in inappropriate special education placements |
| 2 | Children receive nutritious breakfast
Birth certificate information (mother’s education, family income) Snapshot information (Kids Count core data elements) |
Parents of young children receive substance-abuse treatment
Parents spend time with children Presence of active father/father figure in child’s life Mobility of families |
|
3 |
Parent preference as shown by use
Staff compensation levels Access to CHIP and sufficiency of providers |
Pediatricians collect information on systems’ linkages/transitions
Staff turnover Parent survey (what did they get, did they like it, was it culturally sensitive) Program accreditation |
|
4 |
Joint professional development
Staff compensation Transition protocol between early childhood programs and elementary schools Parents supported during transition |
Presence of community collaboration across services and domains
Staff turnover Provider perception of their place in "system" and value to system Funding leveraged |
|
5 |
Full-day kindergarten
After-school activities Mobility of families Contents of teacher contracts |
Teacher training and accreditation
Teacher competency Classroom environment Principals’ survey of whether schools are ready Parental involvement Kindergarten teachers’ expectations of what children should know and be able to do |
Subgroup 3: Access, Quality, and Whole Systems Accountability
Subgroup 3 developed a list of short- and long-term indicators for quality and for ready schools.
Short-term indicators for systems quality included:
- Supply of licensed and accredited programs relative to demand by age and geographical area
- Staff salaries and benefits
- Number of children screened for vision, hearing, and lead poisoning or who are up to date with their immunizations
Short-term indicators for ready schools included:
- Existing school policies on transition into kindergarten
- Existing school policies on parent/family involvement
Long-term indicators for quality and school readiness included:
- Number of children who participate in licensed or accredited programs
- Percentage of programs that meet quality standards (e.g., High Scope, ECERS)
- Quality of staff as indicated by such measures as the number of teachers with bachelor’s degrees and staff turnover rate
- Percentage of staff that participate in professional development opportunities
- Percentage of family income being spent on early care and education program
- Percentage of children on waiting list for subsidized childcare
- Substantiated cases of abuse and neglect
- Percentage of schools that follow a full-service school model
- Percentage of schools that retain students in kindergarten
- Percentage of schools that provide full-day kindergarten
- Percentage of schools that have a policy on school readiness
Work Group II: Measuring Childcare Quality, Availability, and Affordability
The moderator for this session was Sara Watson of the Finance Project. The resource persons were Jason Sachs of the Massachusetts Department of Education and Kathleen Bernier of the Frank Porter Graham Center for Child Development. At the beginning of the session, participants laid out three purposes to which indicators might be applied. These were:
- To describe, map, and monitor trends
- To communicate results of investment
- To improve program access and quality
The group decided to meet as whole, rather than subdivide itself into smaller groups, each focused a single purpose. They further decided that the focus of the discussion would be on the first two purposes, with the first purpose taking center stage. The group then constructed a table listing indicators of quality, affordability, and accessibility. Once the table was constructed, participants voted on which indicators were most important. The table below presents those indicators, not in the order in which they were suggested, but with those receiving the most votes listed first. Also, in the middle of the session, Jason Sachs proposed four indicators of his own and these are presented following the table.
| Quality (each participant had 5 votes) | Affordability (each participant had 3 votes) | Access (each participant had 5 votes) |
|---|---|---|
| Education level of teachers (12 votes) | The percentage of families spending more than an identified percentage of their income on childcare (15 votes) | A cluster of concerns relating to use, including:
|
| Percentage of accredited programs (7 votes) | The value of childcare subsidies provided in relation to the market rate for care (9 votes) | Number of licensed childcare slots per 100 children who need care (12 votes) |
| Staff turnover rate (7 votes) | The percentage of eligible families who receive subsidies (as opposed to those who, for a variety of reasons, are eligible for subsidies but do not receive them) (7 votes) | Number of care opportunities available from relatives and other individuals close to the family needing care; number of unregulated (but legal) opportunities (11 votes) |
| Childcare environment. (This could include such measures as the number of books per child. It might also include the relationship between child and teacher, an indicator also discussed separately.) (7 votes) | The true cost of childcare (7 votes) | Number of programs that accept subsidies (8 votes) |
| Parent involvement (5 votes) | The amount that families receiving subsidies need to pay the provider (4 votes) | Use patterns of childcare by ethnic group (7 votes) |
| Relationship between child and teacher (5 votes) | Childcare market rates (4 votes) | Parent satisfaction with type and quality of care (6 votes) |
| The number of transitions a child makes per day or per year (5 votes) | Parents estimation of what constitutes affordable care (no votes) | Size of the illegal childcare market (4 votes) |
| Child manifestations of happiness or contentment (4 votes) | Can parents avail themselves of a sliding fee scale in addition to any applicable subsidies (no votes) | Map of available care in relation to the location of low-wage employment (4 votes) |
| Staff salaries or compensation (4 votes) | Amount of time families spend on waiting lists by income level (3 votes) | |
| Childcare financing (3 votes) | Percentage of children who need care who are in Head Start or other approved preschool programs (3 votes) | |
| Teacher-child ratio (2 votes) | Percentage of TANF clients who cannot go to work because childcare is inaccessible (2 votes) | |
| Parent satisfaction or parent activities to address lack of satisfaction, such as moving the child to a new program (2 votes) | Number of hours each day that a child spends in unsupervised settings (1 vote) | |
| Group size (1 vote) | Number of transitions a child makes each year (no votes) | |
| Provider staff benefits (1 vote) | Reasons that people chose childcare (no votes) | |
| Support services linked to the childcare center (1 vote) | ||
| Provider violations of licensing requirements (1 vote) | ||
| Injuries suffered to children in facilities (1 vote) | ||
| Availability of staff training (no votes) | ||
| Number of unannounced visits for inspection (no votes) |
Jason Sachs offered four other indicators,
- How many more people have gone to work under welfare reform?
- How large a portion of the economy is the childcare sector?
- How many businesses are using local childcare?
- What percentage of childcare facilities were designed (either newly built or redone) to be childcare facilities?
To these, a meeting participant added
- How many families would move if they could?
Lois Haggard of Utah mentioned research indicating that women with a history of sexual abuse have difficulty leaving their children with strangers.
Once the list was created and priorities selected, discussion continued. One participant observed that a focus on child outcomes is a key to political acceptance. Another asked if research supports a relationship between quality and child outcomes, would it be enough to measure quality?
Jason then challenged the group, asking, of these measures, which could be examined with existing data today.
In the area of quality, three indicators were suggested
- Percentage of programs accredited (general agreement)
- Teacher turnover (one state)
- Education level of teachers (one state)
In the area of affordability
- Percentage of families spending an identified portion of their incomes on childcare (2 states)
- Value of subsidies in comparison with the market rate (2 states)
In the area of accessibility
- Number of subsidized slots (some states)
- Number of licensed slots (some states)
The indicator judged to be the most valuable of the access measures, the cluster of factors that included the availability of shift, special needs, and other care and also transportation issues, was not thought to be measurable with existing resources.
Concluding Statements
There were few indicators of child status, and yet a focus on child outcomes (not just environment) is key politically. If research supports the relationship between quality and child outcomes, it may be sufficient, in some cases, to measure quality indicators. However, representatives from four states pointed out that they are required now to measure child outcomes for children in childcare settings.
Accreditation is a powerful political tool, as it is a proxy for many aspects of quality care. But with so few programs currently accredited, there may be a need for an overall indicator between licensing (minimal standards) and accreditation (often a very high standard).
Priorities for Indicator Development
Introduction
This whole-group session was facilitated by Martha Moorehouse of ASPE and Catherine Walsh of Rhode Island Kids Count. Moorehouse invited each state and municipality represented to respond to two questions: What will you do next? What kind of help do you need? Catherine Walsh added that it would be helpful if those speaking could specify indicators they might add to their projects as a result of discussions at this meeting.
Summaries by Participants
Alaska. Margaret Thomas of the Alaska Department of Education and Early Development said that she expected to drop from her list of childcare indicators a count of the number of licensed slots. She said that Alaska would continue to look at the audiences they hope to reach with the indicators and what they can expect to accomplish through their use.
Delaware. Jim Lesko of the Delaware Department of Education said that the Delaware project would be looking at how the indicators across all buckets mesh well together. He said that the state needs to continue looking at addressing the concerns of multiple parties. (In conjunction, he noted the help that Sharon Lynn Kagan had provided on the state’s Not By Chance proposals). He went on to say that Delaware had not yet figured out how to reach consensus on accountability and would like to discuss accountability concerns with other states.
Georgia. Monica Herk of Community Partners said that she came to the meeting with a goal of addressing ways to measure increases of affordable, accessible, quality childcare and she will take back to Georgia the list of indicators generated by the childcare working group. She had a particular interest in looking at the uses of a books-per-child indicator, feeling that such a measure would be valuable to legislators. She said that Georgia already has its school readiness measures, but she expects that they will reexamine them in light of the school readiness ideas expressed by other states and municipalities.
Hawaii. Both representatives of Hawaii, Liz Chun of the Good Beginnings Alliance and Betsy Moneymaker of the state Department of Education, made comments. Chun said that she was pleased to see the people thinking about school readiness indicators paying attention to making schools ready for kids as well as making kids ready for schools. She said that Hawaii is preparing to look at children’s transitions and is working on a staffing study and is still looking for pieces to put into bucket one. Moneymaker said that she is puzzled by some of the questions she is hearing about why states are focusing on readiness indicators. She asked, "Are too many kids being sent to special education because they are not ready for schools? Are too many kids not meeting SAT standards? Are too many kids not finishing high school?"
Maryland. Charlene Hughins Uhl of Baltimore’s Ready at Five Partnership was one of two speakers for Maryland. She said that she was interested in exploring the capacity of communities to collect and use data. She called the current set of indicators developed for Baltimore a "starter set" and looked forward to the development of a richer set to bring to the legislature.
Minnesota. Debbykay Peterson of the Minnesota Department of Children, Families, and Learning said that she came to the meeting to look at other states’ contextual indicators--such as poverty. In an effort to encompass multiple contexts, Minnesota is looking at linking the data in large state agency databases, a step that can support a partnership among agencies. Ms. Peterson noted that Minnesota faces a legislative mandate to present in 2001 a plan to integrate childcare and Head Start. Ms. Peterson went on to say that these meetings had helped her focus on using indicators to express what they want to do for children, what services they need to provide, and in what settings. She also noted that a major goal for Minnesota is to take data and convert those data into information for use in communities.
New York. Deborah Benson of the New York State Council on Children and Families said that there has been an absence of discussion of school readiness indicators in New York State, but that the state has new money and new standards for prekindergarten and, in time, people will realize that more emphasis on school readiness will mean less remediation. Benson said that she expected to see whether some of the indicators discussed at this meeting can be added to those envisioned in New York. She said that, in particular, subsidies for childcare and Head Start funding had both risen, but New York has no real status report on childcare. She also expressed her intention to engage the state Department of Education in addressing some of these issues.
Rhode Island. Sherry Campanelli of the Rhode Island Department of Human Services said that she found a number of ideas raised at the meeting helpful. These included Sharon Lynn Kagan’s grouping of areas to be studied into four categories (see above) or buckets. She also noted her agreement with Martha Moorehouse that indicators should not stand alone, but a balanced set of indicators should be employed to assess a complex system and to avert misuse of data.
Campanelli said that she was eager to do more with administrative data and she has identified some agency data that could be useful to her. She remarked that linking those data might accomplish the same result as a universal identification system for services recipients, in that it would ease the understanding of the service use careers of individuals, but might be easier to accomplish. Campanelli also looks forward to the possibility that Rhode Island may conduct a large-scale survey to supplement its administrative data.
Massachusetts. Three individuals spoke for Massachusetts and the recorder, alas, missed some names. Deb Laughlin said that she found the opportunity to look at indicators across the buckets very helpful. She said that she would like help drafting a set of indicators useful in political circles that would have application to agencies across 15 Massachusetts commonwealth agencies. She sees a challenge in figuring out how to use existing data in ways that are not threatening to agencies, and said that hearing what other states are doing in this area is very helpful. Christine Johnson-Staub stressed the importance of thinking across buckets and datasets. Jason Sachs of the Commonwealth Department of Education added that he sees all the states doing the same kinds of things and moving forward to do the appropriate benchmarking and using data as a common voice.
Vermont. Jim Squires of Vermont said that the challenges states are facing in school readiness is validating in that they help him feel that all states are on the right track. In particular, he is concerned about Vermont’s impending kindergarten survey as Vermont seeks ways to secure the expertise necessary to analyze data, to communicate the results to communities, and to sustain the survey over time. He also suggested that indicators projects think about ways to give parents an ongoing role in designing the transition between pre-kindergarten and kindergarten settings. In Vermont, they need to articulate a set of standards for getting children ready for school that fit naturally with their K through 12 standards. They would also like more technical assistance on helping community members turn data into action.
West Virginia. Commenting for West Virginia was Kimberly Veraas of the Governor’s Cabinet on Children and Families. She said that West Virginia was looking to post some indicators data on their website by early 2000 as part of their process of using data for action. Veraas said that West Virginia may add some indicators of school readiness based on the discussions at the meeting. In the area of childcare, they are already looking at accessibility of home visiting and other programs, how to make programs more widely available and to sustain them, and at quality issues, such as a multistep-accreditation process. She noted West Virginia’s interest in the innovations underway in Florida.
San Francisco. Carol Stevenson of the Mayor’s Office of Children, Youth, and their Families, said that the meeting gave her confidence that San Francisco is not far off the mark in what it is seeking to measure. She said that San Francisco has a new initiative to enhance salaries for childcare workers and that it was important to gather data on the effect of this effort right off the mark.
Colorado. Susan Geisser of Colorado Bright Beginnings said that she was glad to see that some of the challenges faced by Colorado are found in other states. She is particularly interested in public awareness.
Utah. Lois Haggard of Utah’s Department of Health, Office of Public Health Assessment, commented on both school readiness and childcare. In terms of school readiness, she expressed interest in the idea of schools being ready for kids and in the idea of training teachers to observe behavior.
In terms of childcare, she said that Utah has a good interagency council but, because public education staff are the driving force, childcare has fallen through the cracks. She hopes that dialogue on childcare can be initiated when she returns to Utah. Such dialogue might take up the responsibilities for childcare, mapping childcare to low paying jobs, talking to the business community about staff turnover problems, and other issues.
Subsequent Discussion
Martha Moorehouse noted the many references to needs for data, in particular survey data, and suggested that some measures have been developed, including measures for Head Start programs and measures to capture parent and teacher perspectives on school readiness. She asked if conference participants have been able to find federal resources, such as America’s Children: Key National Indicators of Child Well-Being, available from the Federal Interagency Forum on Child and Family Well-Being, www.childstats.gov.
Allen Harden of Chapin Hall said that what distinguished this meeting is that participants laid out their work so far and, in doing so, acquainted everyone with what had been done and what was pending—very useful information. He mentioned his responsibility to help design a track for the next Chicago meeting focused on moving from concept to indicators and asked for ideas. He also said that another track at that meeting will address how to deal with indicators in the policy environment.
Mairéad Reidy amplified Allen’s point, noting that Chapin Hall would be contacting states to assess their technical assistance needs. She also noted that a Minnesota meeting on assets-based indicators has been moved to early February. Reidy also said that that projects operate at multiple levels, noting that the Vermont approach, which encompasses surveys of teachers, nurses, principals, and the possibility of parents, is encouraging. She urged states to share their survey instruments with each other, using the child indicators list serve.
Sara Watson of the Finance Project said that she was struck by the resistance by some participants to reporting indicators on child outcomes. She referred participants to a Center for Law and Social Policy report, published by Child Trends, on data sources or child outcome indicators.
Kathleen Bernier of the Frank Porter Graham Center for Child Development said that North Carolina is working on a statewide measure of kindergarten readiness and she will take some ideas from this meeting back to that work.
Karen Tvedt of the Child Care Bureau said that the bureau has a mandate to develop performance measures around the childcare development fund, explaining that the infusion of money into childcare that accompanied the implementation of TANF makes it necessary to better understand what is happening in childcare provision. She asked that those with thoughts about what measures would be of interest contact her at ktvedt@acf.dhhs.gov.
Lesko warned that events and policies are developing swiftly and the indicators work needs to keep pace. "This is rocket science," he said, "but the people want to know that you got to the moon, nothing else."
Joyce Butler said that the National Child Care Information Center has been working on outcomes in some states and has felt that providers are not engaged in the dialogue. She has been concerned that those to be held accountable — the providers — would not be kept fully in mind. She now feels like this group is keeping them in mind.
Catherine Walsh made a number of observations,
- Projects need to accelerate progress on child outcomes indicators. Vermont and Rhode Island are among the states that have some tools in this area.
- Quality is important and we need to find a way to use indicators to drive quality improvements.
- Now that states know what they want to achieve by linking data across agencies, they need more knowledge of the technical assistance available to them to support this.
Debbykay Peterson of Minnesota’s Department of Children, Families, and Learning underlined the need to continue making information on individual project efforts widely available. She was echoed in the sentiment by Catherine Walsh.
This Summary
This summary is based on the notes of Barbara Burgess, Jeff Hackett, Allen Harden, Ann-Marie Harrington, Toni Lang, Mairéad Reidy, Charlene Uhl, Sara Watson, and Cathie Walsh.