Minutes of the Technical Assistance Workshop, May 3-5, 2000. Topic 4: Revisiting School Readiness and Promotional Indicators

05/05/2000

Speaking at this session were

  • Mairéad Reidy, Senior Research Associate at Chapin Hall
  • Elizabeth Burke Bryant, Executive Director of Rhode Island KIDS COUNT
  • Reeva Sullivan Murphy, Childcare Administrator in the Rhode Island Department of Human Services
  • David Murphey, Senior Policy Analyst in the Vermont Agency of Human Services
  • Beatrice Colón of the Illinois State Board of Education
  • David Ayer, Director of Research, Evaluation, and MIS in the Maryland Governor's Office for Children, Youth, and Families
  • Monica Herk, Project Coordinator of the Georgia Child Indicators Project
  • Janel Harris, Research Scientist in Minnesota's Department of Health, Division of Family Health/MCH
  • Steve Heasley, Consultant to the West Virginia Governor's Cabinet on Children and Families

Mairéad Reidy

Reidy began the session by recapitulating some of the previous discussions on school readiness indicators, including some themes of the December 1999 meetings on school readiness measures held in Providence. These included the ambiguity of the term "school readiness" and the need for consensus definitions of what children should be ready for and what schools need to do to be ready for children. She showed a series of slides:

New England Meeting of the Child Indicators Projects: Forum on School Readiness and Childcare Indicators

Hosted by Rhode Island KIDS COUNT

Providence, December 2 & 3, 1999

This meeting was sponsored by the Advancing States' Child Indicators Initiatives project of the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, in partnership with the Carnegie Corporation of New York Starting Points Initiative.

Meeting Objective

To assist states and communities participating in the ASPE Child Indicators Project and the Carnegie Starting Points Initiative to develop practical indicators that can be used:

  • To track children's readiness for school
  • To monitor the childcare infrastructure in terms of quality, affordability, and accessibility

School Readiness

  • What children know and can do
  • Conditions that surround children
  • Availability, quality, and accessibility of services
  • Whether schools are ready for children

What Children Know and Can Do

  • Physical well-being and motor development
  • Social and emotional development
  • Approaches toward learning
  • Communication skills, language development and cognition
  • General knowledge

Physical Well-Being and Motor Development

  • Nutrition
  • Vision
  • Hearing
  • Asthma
  • Allergies
  • Asthma hospitalization rates

Social and Emotional Development

  • Can meet and play with different children appropriately for his/her age
  • Adapts to changes in routines at school
  • Expresses basic emotions appropriately for age
  • Develops and maintains friendships to an age-appropriate level
  • Separates easily from caregiver
  • Uses problem-solving skills to address social dilemmas with peers

Child and Family Connections

  • Can meet and play with different children appropriately for his/her age
  • Adapts to changes in routines at school
  • Expresses basic emotions appropriately for age
  • Develops and maintains friendships to an age-appropriate level
  • Separates easily from caregiver
  • Uses problem-solving skills to address social dilemmas with peers
  • Percent of out of home placement
  • Percent of children experiencing domestic violence
  • Continuity of care givers
  • Number of residential moves
  • Parents at risk (e.g., alcohol, substance abuse, depression)
  • Percent of children with elevated lead levels
  • Access to multiple adults

Services Provided

  • Access to primary care physician
  • Access to breakfast program
  • Access to school lunch program
  • Access to early childhood programs
  • Access to libraries
  • Access to preschool programs
  • Access to family services
  • Access to community resources

Ready Schools

  • Offer school-based health clinics
  • Offer breakfast programs
  • Offer lunch programs
  • Offer professional training to teachers
  • Offer health screening
  • Offer counseling
  • Monitor health insurance
  • Monitor immunization
  • Parent involvement in schools
  • School facility is used for non-school activities (e.g., before- and after-school programs; family resource center)
  • Teachers have access to school services

Data Collection

Speaker John Love said that:

  • Individual-level data is expensive to gather.
  • Survey sampling can allow a relatively small number of cases to reflect the diversity in the community.
  • It is important to identify priority groups that can be asked "cross-service"/"cross-discipline" questions about children.

Communications Strategies

Love recommended that we:

  • Use a marketing approach to present data. Couple data with human presentation (e.g., if reporting results of survey of kindergarten teachers, have a panel of teachers share their stories at the same time). Explain data in terms that resonate with the audience (e.g., link turnover rates in childcare to impact that turnover has on the business community).
  • Recognize that change makers may need 100 charts; have different datasets that could be used to address your changing needs and theirs.
  • Sustain a core of indicators to be tracked consistently over time.
  • Apply the findings to continuous community improvement efforts.

Data Interpretation

At the New England meetings, Love suggested states:

  • Focus on the collective or aggregate status of entering kindergartners.
  • Use community-wide estimates of school readiness.
  • Develop estimates of readiness for all important subgroups within the community

Childcare

  • Quality
  • Affordability
  • Accessibility

Childcare Measures that Could be Examined Using Existing Data

Quality

  • Percentage of programs accredited
  • Teacher turnover
  • Education level of teachers

Affordability

  • Percentage of families spending an identified portion of their incomes on childcare
  • Value of subsidies in comparison with market rate

Accessibility

  • Number of subsidized slots
  • Number of licensed slots

Elizabeth Burke Bryant

Bryant said that the state of Rhode Island is very serious about developing school readiness indicators because of the major investments the state has made in childcare during the past two years. Legislative leaders will continue to support the expansion and improvement of Rhode Island's early care and education system if they have evidence that these investments are having a positive impact in terms of children arriving at school ready to learn. Because time was of the essence, the Rhode Island Child Indicators team decided that there is no perfect set of measures, and we moved ahead in selecting a logical set of indicators that are a good reflection of a young child's health, cognitive ability, and social competency. Some of these indicators were already being tracked by Rhode Island Kids Count; others had to be developed and put in the field. Rhode Island added childcare quality questions to our market rate survey of childcare providers (now in the field), and we are adding school-readiness questions to our survey of kindergarten teachers. A complete listing of Rhode Island's school readiness indicators was distributed as a handout for the workshop participants.

Reeva Sullivan Murphy

Murphy said that her background as an early childhood educator and kindergarten teacher served her well in her new position as state Childcare Administrator. As part of her work with Rhode Island's Child Indicators Team, she has been developing indicators that, taken together, paint the full picture of a child's readiness for school, and the school's readiness for each child. It is the combination of good health, and a comfort level in interacting with other children that make some children more ready than others in a kindergarten teacher's view. Murphy said that using the market rate survey every two years will be a great way to obtain data on school readiness, and that the willingness of the people who administer the comprehensive School Accountability for Learning and Teaching (SALT) (2) survey to every public school student will provide the kindergarten data that we need. She encouraged other states to select a list of indicators that cover the key areas and that will make sense to policy makers when it comes time to report on whether the state's early childhood investments are making a difference.

David Murphey

David Murphey of Vermont picked up on the idea of consensus in the area of school readiness. He said that over the past year, a degree of consensus has emerged among those working on school readiness indicators and discussed some of the different approaches he sees states taking in understanding school readiness. Murphey saw consensus around these ideas:

  • Multiple perspectives should be acknowledged and multiple measures are appropriate
  • Even when focusing on the child, school readiness is a multidimensional construct
  • School readiness is not a test for children or teachers
  • Readiness is an outcome and is conditioned by prior experiences and serves as a predictor of how well a child is going to do
  • There is value in both reporting on readiness in the aggregate and using individual-level data for some purposes, as long as the appropriate protections are in place

Murphey identified these as different paths states are taking:

A

  • A checkpoint at a key developmental transition and thus an important point for analysis
  • A profile of community performance
  • A tool for community systems improvements

B

  • A continuous assessment process
  • A profile of student performance
  • A tool for improvement of instruction

Murphey discussed Vermont's approach to assessing school readiness.

Overhead (3)  A Chronology of Vermont's Strategy: A 3-Part Assessment of School Readiness

  • Brief, child-focused questionnaire completed by the kindergarten teacher (5 minutes).
  • A school-focused principal questionnaire on ready schools covering such issues as parent involvement, the quality of the relationship between the school and the early childhood provider, what supports and services are available to kindergarten teachers. It is six pages in length and is expected to take about ten minutes.
  • A very brief child-focused health checklist. This instrument has eight questions on such topics as hearing, vision, immunization, dental health care, developmental problems, and health insurance.

Vermont has used focus groups of interested parties to receive input on this plan and to obtain information on ways to revise their instruments. This fall, Vermont plans to use this assessment statewide. This survey will replace an earlier survey that the state had found less useful than it hopes the new assessment will be. Murphey made draft instruments available to the audience. In response to questions from the audience, Murphey said that they plan on asking teachers questions on their own experiences and history. He also described the anticipated ways in which the Vermont data collection would be conducted.

Before Beatrice Colón began, Martha Moorehouse said that Rhode Island's initiatives in the area of school readiness are motivated in part because they are making big investments in this area. She asked if the work in Illinois and Vermont was taking place in the context of changing investments and the desire to track those investments. Murphey said that there is strong support in Vermont for standards-based assessment but that it remains to be seen if there will be a funding investment.

Beatrice Colón

Beatrice Colón said that about six years ago, the State Board of Education decided that teachers needed a systematic way to collect and assess data on what children should know and what they do indeed know. A small group of teachers and administrators developed a system to use work sampling as an alternative to testing. The system has three components: developmental checklists, portfolios, and summary reports done three times a year. These reports provide a chance for the teacher to reflect on what the child has gained. They can be shared with the parents, too. Colón pointed out that Illinois has 450 prekindergarten programs. Not every one of those programs is using work sampling. Each district can choose what kind of assessment they will use. At most levels, the work sampling system is being implemented using only one to two domains as a start. A few systems are using all of the domains.

The developmental checklist follows the domains. Those domains are:

  • Personal and social development
  • Language and literacy
  • Mathematical thinking
  • Scientific thinking
  • Social and cultural awareness
  • Art and music
  • Physical development

Some of the systems that tried to implement work sampling using all domains had a difficult time doing so.

Colón said that an integral part of the system is continuous training and staff development. It is her view that continuous staff development must be built in to the process and school administration must support it. Without those steps, the system will fail. Because there are 450 prekindergarten programs (excluding Chicago) and more than 400 classroom programs in Chicago alone, Illinois State Board of Education (ISBE) contracts out some of this staff development work. A challenge to work sampling is the variation in the ways in which principals implement it and support, or fail to support, staff development.

As a result of the work sampling, and of questions and suggestions from the school district, ISBE has developed and is further developing indicators, outcomes, and standards for early childhood programs. Currently, Colón has a draft document that is being reviewed. They are also developing program standards for birth-to-three programs.

Colón said that Illinois does not collect data, per se, on work sampling. But ISBE does collect data on children in the prekindergarten program. There is a record for each child and family that contains health, safety, education, and other data. There is also a longitudinal study that follows a sample of children through 12th grade, but that the population of interest is very transient.

David Ayer

David Ayer said that, in the mid-1990s, Maryland developed its Maryland Model for School Readiness. They wanted a way of assessing children by the time they reach the end of their kindergarten year. The work sampling system, which covers a great deal of ground, was chosen to do that. Since 1997, there has been a pilot implementation underway. All of the twenty-four school systems in Maryland will use this in kindergarten in the coming school year or the one following. They will look at all of the domain areas available in the work sampling system. Maryland will examine the aggregate percentages of children at the end of kindergarten reaching a proficient level. This information will be incorporated into Maryland's indicators.

Monica Herk

Monica Herk identified four Georgia benchmarks. These are:

  • Percentage of low-income students in Head Start or prekindergarten programs.
  • Percentage of kindergarten students who attend preschool or childcare programs. (Georgia has universal, state-funded prekindergarten program. This eases data collection because one of the mandates accompanying funding is that the preschool programs, even if they are in private settings, help children make connections with public schools.)
  • Percentage of kindergarten students passing the Georgia Kindergarten Assessment Program (GKAP).
  • Percentage of students who are two or more years overage in third grade.

Herk focused mostly on the GKAP, created by legislative mandate to assess the readiness of kindergarten students to enter first grade. Herk provided eight pages of background information on the Georgia Benchmarks and on GKAP. Summarizing the GKAP findings over the past four years, she said that the numbers have been relatively unchanged. (A handout put this number between 87.3 and 88 percent over the period.) She said that the Georgia example shows that, although the state has made a huge investment in prekindergarten education, the measure doesn't show a change. However, evaluation studies that have tracked children are showing that children who go through the state-funded prekindergarten program seem to score better on the kindergarten assessment.

With the most recent revision of the GKAP (GKAP-R) some of the earlier data will not be comparable to later data. One of the changes made in this revision aligned the GKAP-R more closely with the Georgia Quality Core Curriculum (4) content standards. This alignment will also mean that Georgia's scores will not be comparable to data from other states. There have also been changes in methods of administration in an effort to reach a form of administration best suited to children of this age.

The GKAP-R has three windows of assessment, one at the beginning, one at the middle, and one at the end of the school year. This supports the use of the GKAP-R as a diagnostic tool for the teacher.

Janel Harris

Janel Harris said that the February 2000 meeting on promotional indicators held in St. Paul had tried to address three goals:

  • To explore the conceptual framework for promotional indicators;
  • To obtain consensus among the states, especially the four convening states--New York, Georgia, West Virginia, and Minnesota--on particular promotional indicators that they could agree upon as being important and also possibly useful to collect data on; and,
  • To get information on the federal perspective on opportunities to incorporate promotional indicators into the federally mandated indicators.

Harris felt that they made progress on all of these goals, but that the promotional indicators effort remains very much a work in progress. One conclusion they drew was that they needed to further examine the research that underlies some of the areas in which indicators are needed so that they could see what research relates to those things that seem intuitively to have value as promotional indicators. Harris also commented on two talks on indicators by federal agency staff--Martha Moorehouse of ASPE and Casey Hannon of the CDC. Harris felt that most of this federal work was deficit-oriented and she found very little in the way of positive or promotional indicators. She said that, none the less, she felt it was important to push forward the promotional indicators agenda, in part because people are "really tired of being viewed as a collection of problems." Promotional indicators can provide a more balanced picture of the situation being examined.

Steve Heasley

Steve Heasley went over what has happened with promotional indicators since the Minnesota meetings. He said that it is important to point out that a lot of the interest in promotional indicators stems from frustration with traditional indicators that are deficit oriented. He said that it is interesting that frameworks are developed "to measure, 'child well-being,' but, because of the limitations of the data, what we are measuring is the absence of well-being." The philosophy before promotional indicators is to provide measures that do capture well-being--those that are related to strengths and assets.

Heasley said that he was struck by Ben-Arieh's presentation and the work on the indicators worldwide.

Some of the areas in which promotional indicators could be developed include:

  • Are children being read to?
  • Do children have caring adults in their lives?
  • Measures of parenting skills
  • Volunteering in the community
  • Supporting family life

Heasley said that the cross-state email group is open to anyone who wants to participate, not just to the four states that convened the February meeting.