|
|
|
October 21 – 22, 2008
Washington, DC
By:
National Center for Children in Poverty
Mailman School of Public Health
Columbia University
and
Abt Associates Inc.
The meeting was sponsored by the
U.S. Department of Health and Human Services, Office of the Assistant Secretary
for Planning and Evaluation (ASPE) and the
Administration for Children and Families, Office of Planning, Research, and
Evaluation (OPRE)
This report is available on the Internet at:
http://aspe.hhs.gov/hsp/10/SchoolReadiness/
Printer Friendly (PDF) Version, excluding appendices
(21 pages)
[Free PDF reader
]
Appendices
This document presents the proceedings from “A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research,” which was held on October 21-22, 2008 in Washington, D.C. The meeting was sponsored by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Office of Planning, Research, and Evaluation (OPRE), Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services. Abt Associates Inc. and the National Center for Children in Poverty (NCCP) convened the meeting.
The purposes of the meeting were to:
The “school readiness gap” has received increasing attention over the last two decades. Evidence suggests that children from low-income homes are entering school significantly behind their peers from more resourced homes. Research consistently shows that children’s readiness for school when they enter kindergarten is associated with socioeconomic status (Lee and Burkam, 2002; Magnuson and Duncan, 2005; Hart and Risley, 1995). Furthermore, children who enter school behind their peers rarely catch up. The Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K), a nationally representative study of children in kindergarten in 1998-1999, has documented the school readiness gap at kindergarten entry; cognitive scores among children in the highest SES group are 60% higher than those of children in the lowest SES group (Lee & Burkam, 2002).
Concerns about the readiness gap have led to efforts to develop strategies for enhancing children’s development in ways that ready them for school. Educators have turned to early care and education as a potential developmental intervention. Evidence of the potential benefits of early care and education programs is based primarily on a few small-scale, carefully controlled experimental studies of educational interventions that have suggested the ability of high-quality preschool interventions to enhance the school readiness of low-income children. The evidence of the school readiness gap, combined with descriptive data indicating that children in early childhood care and education programs can make cognitive and socioemotional gains has motivated the federal government to sponsor a number of early childhood education programs that focus on serving children from low-income families, including Head Start, Early Head Start, and Even Start.
The federal government has also conducted studies examining the effectiveness of these programs for promoting school readiness. In the 1990’s, the Federal government conducted a series of rigorously-designed Congressionally-mandated evaluations of early childhood programs, such as the Head Start Impact Study, the National Evaluation of the Even Start Program, and the Early Head Start Research and Evaluation Project. Additionally, the government funded a substantial body of research focused on expanding knowledge about the specific practices, interventions, and curricula that can successfully improve school readiness among children from low-income families and can do so across diverse settings. This research focused primarily on evaluations of quality enhancements to existing programs. The research included:
By the fall of 2008, final results from PCER and Project Upgrade had been released and preliminary findings from the ISRC studies were beginning to emerge.
Synthesizing the findings from this now large body of evaluation research is critical for identifying its contributions to our existing knowledge base and informing a future research agenda. In an effort to begin this synthesis and examine the body of evidence emerging from these studies, a meeting of experts was convened entitled, “A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research.”
In preparation for the meeting, a series of working papers were commissioned by ASPE and OPRE and were prepared by experts in the field. The papers focus on the findings from the most recent set of federally funded studies. The meeting was organized around these papers; authors of the papers presented their findings, with prepared commentary by respondents. The meeting was designed to use the papers as the jumping-off point for open discussion among meeting attendees about the state of the science, how it may inform early childhood programs, gaps in our knowledge, and directions for future research.
An overview of the proceedings, summary of key themes identified during the meeting, and identified next steps are presented below.
The one-and-a-half-day meeting was divided into four panels. Each panel focused on a topic that was also the subject of one or more of the invited papers. For each panel, highlights from the relevant working papers were provided by the author, panel respondents provided commentary, and meeting attendees participated in discussion about the issues raised and future work that might be successful in addressing those issues.
The four panels and their corresponding key topics were:
1. Examining Children’s School Readiness Outcomes: Effects of Enhancements to Early Childhood Programs. This panel examined evidence on the effectiveness of various instructional practices, interventions, and curricula in early care and education programs in promoting young children’s development. The panel looked at effects on children in three broad domains— language and literacy, mathematics, and socioemotional development.
Barbara Goodson moderated this session. Three working papers were presented, each one corresponding to one of the domains of school readiness. Goodson also authored a paper which synthesized key themes across the other three papers and across domains of school readiness (see Appendix B.1). Carolyn Layzer presented a paper on language and literacy outcomes (Caswell & He, 2008; see Appendix B.2), Herbert Ginsburg and Margaret Clements presented their paper on mathematics outcomes (Ginsburg, Lewis, & Clements, 2008; see Appendix B.3), and Cybele Raver presented her paper on socioemotional outcomes (Raver, 2008; see Appendix B.4).
Following the presentations of the working papers, Jeanne Brooks-Gunn provided remarks as the panel respondent, linking the findings from the three papers in a discussion about what it takes to get children ready for school and the effectiveness of different interventions for narrowing the school readiness gap.
2. Perspectives on Using Research to Improve Programs. This panel focused on the bridge between research and practice and how research can contribute to early childhood program quality improvement and, ultimately, promote young children’s school readiness. Deborah Leong moderated the session, in which presenters drew on their expertise in utilizing research to inform practice and policy to discuss how well the set of studies that were the focus of this meeting address questions of current interest to programs and where the research needs to go to next to answer those questions more clearly.
Beth Rous began the session with a presentation in which she highlighted key lessons from the latest research seen through the lens of statewide implementation and policy development. Graciela Italiano-Thomas then discussed her experiences with the intersection of research and policy/practice as the Los Angeles Universal Preschool Executive Director and President and CEO of Washington’s Thrive by Five. Finally, Thomas Schultz gave a presentation focusing on how research has been used to improve programs in the past, challenges in doing so, and next steps for research and program improvement implicated by the emerging research focused on at this meeting.
3. Strategies for Professional Development of the Early Childhood Workforce. This panel examined evidence on the effectiveness of professional development at changing teacher practice and instructional skills. Further, the panel looked at evidence on how changes in teaching relate to young children’s school readiness outcomes. Ivelisse Martinez-Beck moderated the session in which presenters discussed issues of intervention implementation fidelity, workplace and teacher characteristics that may moderate changes in teacher behavior, quality improvement approaches, and strategies to train coaches and mentors.
First, Lisa Klein presented highlights of the working paper on early childhood professional development approaches (Klein & Gomby, 2008; see Appendix C). Robert Pianta served as a respondent, emphasizing the importance of supporting teachers and the classroom practices we know promote children’s development, in addition to the particular curriculum or intervention model through which those practices are implemented. As the second respondent on the panel, Kathryn Tout discussed her research implicating the important role of teachers’ readiness for change and features of providers of professional development in moderating changes in teachers’ behavior.
4. Approaches to Measuring and Narrowing the School Readiness Gap. This panel explored conceptual and methodological issues related to the ways the school readiness gap is represented, measured, and used to judge the effectiveness of early childhood care and education programs. Stephanie Jones served as the moderator for the panel and provided summary comments.
Jean Layzer began the session by presenting highlights of her paper (Layzer & Price, 2008; see Appendix D) which describes the school readiness gap, outlines progress being made to narrow the gap, and discusses alternative approaches to judging the effectiveness of interventions. As first respondent on the panel, John Love raised a series of critical questions for the field to consider in trying to better understand the school readiness gap and meet the goal of raising low-income children’s level of proficiency and skill in various domains of school readiness. Next, respondent Margaret Burchinal discussed some of the areas where we can focus our attention in the future, such as on infants and toddlers, determining the skills children need and how to teach them (as opposed to focusing on “the gap”), and better understanding how we take early childhood programs to scale effectively.
To close the meeting, Martha Zaslow provided a synthesis of themes and next steps, emphasizing how the studies focused on for this meeting offer cause for celebration and contribute numerous encouraging developments to the knowledge base, while also pointing to challenges and gaps where there is need for further work.
A number of cross-cutting themes emerged from the presentations and commentary, and the rich discussion about the body of evidence emerging from the school readiness studies that were the focus of this meeting.
One of the challenges in the research is describing the school readiness gap in a developmental metric that allows clear communication with policy makers, programs, and families. In much existing literature, achievement differences are defined in terms of an effect size that is calculated as the proportion of a standard deviation on a standardized, normed test score. These effect sizes are used to describe the magnitude of the gap and the impacts of interventions aimed at narrowing the gap. Although this way of describing the gap provides researchers with a common metric to describe effects, it can be abstract for policymakers. Therefore, researchers (e.g., Layzer & Price, 2008) have investigated ways to describe the gap between the performance of low income samples and the norming samples in terms of the difference in months of development. Using this metric, in their examination of several large samples of low-income children, Layzer and Price found that at age 4, the gap in cognitive and language development between low-income children and norms for their age group is approximately one year of development. In other words, at age 4, low-income children are, on average, one year behind in development, based on the average norm score for their same-age peers. Describing the gap as reflecting one year of development rather than as one standard deviation offers an interpretation of the difference that is more easily understood. It is by no means the only way to accomplish this goal. It will be important to continue this work to translate differences from abstract effect sizes into meaningful developmental metrics.
Determining a metric for describing the achievement gap between children from low-income and better-resourced homes is an important methodological challenge. Substantively, the challenge for the early childhood field is to identify interventions that reduce the gap, however it is measured. That is, we want to design interventions that are effective at “decoupling” family income and child achievement. If we are successful, there will continue to be individual variation in children’s achievement, but it will not be a function of family income.
Many studies have shown that school readiness interventions make progress toward narrowing the gap, but do not close it all the way. A wide range of hypotheses have been posited to explain the size and persistence of the gap. One hypothesis is that early childhood interventions are not intensive enough; many last only nine months and involve no more than half a day of exposure period, which may not be a sufficient amount of time to make up a one-year gap. Further, many interventions last only one year and occur in the year before children enter school. Some researchers suggest that the gap is so wide by the time children are age 4 that intervention needs to begin earlier as well as last longer. Questions also have been raised about the quality of the instructional practices in early childhood settings and whether most children are receiving the kinds of experiences that are most effective for improving their school readiness outcomes. Similarly, questions arise about approaches to improving the level at which providers implement interventions. In addition, researchers have questioned whether interventions provided in early care and education settings can be expected to close the gap entirely, when other environmental factors related to the gap are still at play.
Despite expectations that early childhood interventions will provide a boost that enables children to be successful in school, there is mixed evidence that impacts are sustained. Again, a number of hypotheses have been proposed to explain why impacts are not sustained. Some researchers suggest that, because students still remain below grade level despite making gains, the boost obtained from preschool may not be sufficient to enable students to keep pace at the next grade level. As a result, some researchers argue that students continue to require enhanced instructional supports in elementary school in order for impacts to be sustained. Another hypothesis is that gains are not sustained because many low-income children enter under-resourced kindergarten and elementary school classrooms that provide little support for earlier gains. Data suggest that regardless of children’s achievement level at the start of kindergarten, their growth rates during elementary school are similar, suggesting that children who start behind, stay behind (Layzer & Price, 2008). In fact, some research suggests the achievement gap widens over time. As students are expected to learn new skills, which build on foundational skills that may have not been established, students may fall further behind their on-grade-level peers.
Some researchers suggest that early childhood experts should work to develop a consistent definition of what constitutes school readiness (see Goodson, 2008 for a discussion). In theory, a definition of school readiness should identify the foundational skills, content knowledge, and concepts that children need when they enter school in order to achieve academic success in early elementary school and beyond. Clearly defined expectations for children’s development at school entry would provide specific objectives for children’s development. Before we can identify the interventions, curricula, and teacher practices that foster children’s school readiness, early childhood experts must identify the foundational skills essential for school success. However, the early childhood field has not developed a consistent definition of school readiness. Furthermore, connections drawn between preschool skills and later academic outcomes are based largely on correlational research. We need longitudinal research to explore the developmental trajectory of foundational skills. We need to better understand how foundational skills are learned, how they interrelate, and how they develop over time to support academic success.
Specifically, what experiences contribute to students learning specific skills in each school readiness domain? For example, what experiences contribute to students learning to regulate their attention, to understand word meanings, or to recognize patterns? Furthermore, how do the development of skills in one domain relate to the development of skills in another domain? School readiness domains are viewed as potentially synergistic. For example, socioemotional development may serve as a foundation for a focus on instruction in language and literacy or early math, or perhaps vocabulary development may provide the foundation for growth in other areas (e.g., socioemotional, math). Many questions about the interrelationship of skills across domains need to be addressed, such as: To what extent does children’s cognitive self-regulation support their learning academic content and skills in a classroom setting? To what extent are language skills the mechanism for understanding concepts in other domains, such as math and science, and the basis for internalized self-regulation? In addition, questions about the relationship between early skill development and later school outcomes are critical. For example, do children entering school at a particular skill level move more quickly onto the next set of skills? Building a strong body of research that establishes links between early skills and later school achievement will be an important basis for developing a definition of school readiness.
A measurable definition of school readiness within and across developmental domains (e.g., language, literacy, cognitive development, mathematics, socioemotional, approaches to learning, and health) is needed as a foundation for initiatives to promote school readiness. With clearly defined expectations for children’s outcomes, early childhood interventions can target those specific objectives. A goal for future research will be to identify the interventions, curricula, and teacher practices that effectively foster those outcomes. Likewise, the field will benefit from research that can identify the approaches for training and supporting teachers to use those practices.
One of the critical issues in the area of professional development involves the tension between professional development that is focused implementing a specific curriculum and professional development that focuses on effective practices more generally. Currently, most professional development is about a curriculum that may or may not encompass instructional practices that address all domains of development or that reflect state-of-the art knowledge about effective practices. An alternative approach to professional development is to start with the outcomes that we would like children to achieve, then identify teacher practices that have been shown to foster those outcomes, and finally build systems that support those practices. This series of steps places less emphasis on particular curricula, and instead focuses on developing supports for teachers around classroom practices, regardless of curriculum.
The challenge is to determine how to build systems that scaffold implementation in real world settings. All of the research suggests that professional development on an approach or curriculum does not guarantee that practitioners will consistently be on model when using the approach in their classrooms. One possibility discussed is to design professional development that is staged, to provide new layers of support as teachers become more skilled at implementing practices.
In general, despite increasing knowledge about effective instructional practices, this knowledge needs to be absorbed and reflected in the professional development provided to teachers. Continued work is needed to identify approaches for teaching teachers those skills and how to use them. In the field, the varied strategies have been used in teacher training and professional development, including one-time workshops, formal coursework, teacher-accessed web-based support, individualized web-mediated coaching, and intensive in-person coaching. The research shows that all of these methods have achieved mixed success in impacting teaching practices and child outcomes.
In their review, Klein and Gomby (2008) reported that training to implement curricula is associated with improvements in implementation and teacher’s classroom instructional practices. Specifically, they concluded from their review that coaching and mentoring were associated with improved implementation. Furthermore, some studies found that teachers who received professional development had improved classroom practice and also had children with improved outcomes. Based on their review, the Klein and Gomby suggested that, in order for professional development to impact child outcomes, teachers must deliver the curriculum as intended, the whole curriculum must be delivered, and children must attend the program with enough regularity to benefit from the intervention.
Coaching appears to be an especially promising approach for producing positive changes in teachers and improvements in child outcomes; however, a series of questions remain about coaching: 1) How are coaches trained?; 2) What is the coach doing that impacts the teacher?; 3) How is the teacher changing his/her practice as a result of the coaching?; and 4) How does this change in practice lead to child outcomes? The QUINCE studies, for which findings had not yet been released at the time of this review, will begin to answer some of these questions. Other newly funded efforts (e.g., the Head Start University Partnership Research Grants funded in 2008 examine strategies for developing teacher effectiveness) will also contribute to this body of literature.
Among the studies that were the focus of this meeting, a number of challenges were articulated that limit the capacity of the field to identify effective professional development approaches. For example, there is no common vocabulary or set of expectations for describing professional development. Many studies lack basic descriptive information about the professional development activities that were implemented, which makes it difficult to compare approaches across studies and offers little information on how to move the field forward. Additionally, delivering multiple professional development strategies as part of a single package, such as combining group training sessions with one-on-one coaching, without adequate descriptions of the components or a planned variation approach to their study makes it impossible to disentangle the components or isolate the effects of any one strategy.
There are a wide range of programmatic, classroom, teacher, and home/family factors that provide influence the implementation of interventions, the effectiveness of professional development approaches, and the effectiveness of interventions for improving children’s outcomes. There is a need to better understand the role of these contextual factors in translating high quality curricula, intervention models, or professional development experiences into improved teacher practices and children’s school readiness outcomes. However, contextual factors, such as teacher and workplace characteristics, are often not measured or included in evaluations of professional development.
Program context and administrators, instructional supervisors, and program structure play an important, yet relatively unexplored, role in supporting interventions. Furthermore, the role of teacher aides, as well as the relationship between the teacher and aide, in contributing to the success of professional development efforts, intervention implementation, classroom instruction, and program quality requires further attention in research. In a study of the Head Start REDI intervention (Domitrovich, Gest, Gill, Bierman, Welsh and Jones, 2009), researchers provided similar professional development to both the teacher and the aide, treating the teaching team as an intervention unit, and found positive effects of coaching for both the teacher and the aide.
Classroom characteristics and composition were also highlighted as an important context that can impact implementation. For example, in classrooms with high numbers of children with challenging behaviors, socioemotional curricula that were too cognitively oriented were difficult to implement. Another study, which examined the effectiveness of individualized, web-based consultation with a coach combined with teacher-accessed web-based support compared to teacher-accessed web-based support only, found the greatest impact of individualized, web-based consultation on teacher-child interactions in the highest poverty classrooms (Pianta et al, 2008), which. In contrast, teachers in high poverty classrooms who received only teacher-accessed web-based support without individualized web-based consultation with a coach showed a decline in the quality of teacher-child interactions over the course of the study. The results of this study suggest the need for greater support in under-resourced classrooms.
Teacher characteristics also play an important role in mediating outcomes. Specifically, there was discussion about the need to consider the psychological well-being of educators. For example, in the QUINCE study, much of the coaching relationship focuses on dealing with issues of depression and isolation. Other studies reported by Pianta and his colleagues have found that about 15 percent of teachers report depressive symptoms (Hamre & Pianta, 2004). There is a general consensus that it will be important for future professional development efforts to address teachers’ mental health. Furthermore, we need a better understanding of how interventions fit with teachers’ personal and professional goals, which will help to determine teachers’ readiness for change. This point was supported by the finding in several studies that teachers implemented curricula with greater fidelity when they had a high level of dedication to and positive attitudes towards the curriculum.
In addition to programmatic and staff characteristics, the role of parents and the home environment was emphasized as an important context. Research suggests that parents and the home environment have a larger impact on child outcomes than early childhood programs do (e.g., NICHD ECCRN, 2002). This finding suggests that interventions should involve and target parents as well as children. However, there is a question over what this involvement entails: (1) getting parents invested in early childhood program so that they view the program as an opportunity for their child, or (2) intervening with parents as well as with children. The rationale for the latter approach is to intensify the impact on children by improving the enriching experiences at home as well as providing high-quality experiences in the early childhood program. From this perspective, enhancing children’s experiences in multiple contexts offers greater opportunity to impact children’s outcomes and suggests the need to identify more ways to involve families. Many researchers view the literature as providing support for interventions with parents as well as children, while others point to research on such two-way intervention models that have not demonstrated impacts on parenting (e.g., Abecedarian and CARE). Interventions that target a wide range of parenting behaviors may be too overwhelming for parents, thus preventing improvement in any area. Interventions designed to target specific parenting outcomes may be an alternative to broader programs. Specifically, interventions to improve parents’ vocabulary may be one appropriate target that may help children as well.
Most of the curricula and interventions evaluated in recent randomized controlled trials targeted a range of school readiness domains (e.g., language and literacy, math, socioemotional development). The integrated focus of these interventions on multiple school readiness domains is driven by both the view that child outcome domains are interrelated and the need for early childhood program enhancements in all these areas of children’s development. A number of questions have arisen about curricula that focus on a range of school readiness domains. Although findings from the studies of integrated curricula are positive, effects were not as large in any given domain as those for interventions targeting a specific domain, such as in the PCER studies. One hypothesis for this difference in effects is that an integrated curriculum may result in less instructional time spent on any specific area. Therefore, the need to cover multiple developmental domains may dilute the impact of the curriculum on any one domain.
Another issue raised about integrated curricula is that it places a substantial training burden on teachers to learn how to implement many different instructional strategies. For example, Building Bridges is a comprehensive program designed to teach children socioemotional skills by implementing socioemotional lessons in language, communication, and math activities. The program is fully integrated thematically, but also integrates behavior management skills in teacher training and coaching. Only few and relatively small effects were found for a workshop-only group; however, no effects were found for a group receiving more intensive training and follow-on support. It may be that the intensive curriculum and more intensive professional development model were too demanding for an already overwhelmed teaching staff. It also might take longer for teachers become skilled at implementing all parts of an integrated curriculum. It may be important to consider how intervention models might be rolled out in stages and how to scaffold teachers’ learning of new curriculum and skills.
Furthermore, despite initial concerns that a more cognitively demanding intervention might provide a stressful learning environment and lead to worse behavioral outcomes for children, findings from the PCER studies provided little evidence for this relationship. In contrast, there was some evidence of a “spillover” effect resulting in improved classroom climate. As noted previously, understanding the interrelationship among school readiness domains is an area that warrants further investigation.
In addition to questions about the focus of integrated curricula on multiple domains, there is a need to further consider whether it makes sense to “build” an integrated curriculum from multiple curricula, each with a focus on a different school readiness domain (e.g., combining Big Math and Head Start REDI). A key component in the successful integration of multiple curricula is ensuring that there is a common underlying theoretical framework. Difficulties can arise if curricula are based on different theories of pedagogy or development. Explicit connections among various curricula need to be made, as this helps teachers to feel that there is continuity in what they are being asked to achieve in the classroom.
An important unanswered question concerns the question of intervention dosage– i.e., the extent to which the amount of intervention that children receive affects the outcomes. In particular, there is a question about whether there is a threshold of minimum amount of an intervention that is required to produce desired child outcomes. One of the problems with research in this area is that currently, researchers do not have a common approach for how to define and measure dosage (e.g., number of lessons, length of lessons, number of days, extent of individual versus group instruction). Dosage is typically conceptualized as the overall amount of instruction offered. However, some researchers suggest that dosage should be conceptualized as a function not only of what is offered but also what is received by children. Further, measuring what is “received” by children may go beyond the child’s exposure (i.e., time in a classroom) to their level of engagement with the intervention. Also, it is likely that the relationship of dosage to outcomes may be mediated by the quality of the intervention. The effect of dosage is likely to be different if the intervention being provided encompasses high-quality versus lower-quality practices. Future research will need to address the definitional issues surrounding dosage (offered or received) and the potentially complicated relationships among dosage, quality and features of the intervention being tested, and amount of dosage experienced by children.
The issue of dosage is a question in the area of professional development as well. Is there a dosage level of training to ensure sustained changes in teacher practices to ensure sustained changes in teacher behavior? Measuring dosage or intensity of professional development is not typically a part of evaluations. Even when descriptive data are provided, typically the data are about training sessions as opposed to the amount of coaching/mentoring, which is more difficult to measure. As a result, there is little evidence about the level of intensity of coaching that is beneficial and for whom. Future research should not only measure and report the intensity of interventions being studied, but also contrast varying intensities of instructional support provided to students to determine how much is necessary.
Further exploration of these dosage issues (e.g., definition, measurement, the intensity required to produce sustained outcomes) are necessary to further our understanding of how to ensure that all children are ready for and successful in school.
Although there has been progress in carrying out randomized controlled trials, and some important questions are being addressed by this research, there are still key design issues that need to be addressed. The randomized controlled evaluations of early childhood interventions reviewed for this meeting are generally designed to address global questions about whether specific intervention programs have impacts on children’s development and/or on teachers’ instruction compared to a business-as-usual condition.
“Compared to what?” It is important to note that the comparison of program enhancements to existing early childhood programs, as in recent studies, represents a shift from the earliest randomized studies of early childhood programs (e.g., Perry Preschool, Abecedarian). In the early studies, an intensive preschool program delivered under ideal conditions (e.g., delivered on a small scale by intervention developers) was compared to a “no intervention” control group rather than a “business-as-usual” control group. Given the different comparison being made, it is not surprising that the earlier programs were found to have larger impacts than the interventions evaluated in the more recent body of work. A key consideration in synthesizing and drawing conclusions from evaluations of early childhood interventions is the recognition of what comparison is being made.
Testing Intervention Components. The recent body of randomized studies is designed to provide more rigorous tests of whether the interventions are effective at improving child development or teacher practice. However, these studies are not designed to address questions about differential impacts of varied intervention components, delivery mechanisms, or professional development approaches. To address these questions, planned variation studies would be needed, which systematically compare multiple versions of an intervention or training approach. However, planned variation studies are complex to design and implement. In addition, they are expensive, since they require substantially larger sample sizes to test multiple treatment conditions. We either need to increase investments in large studies that allow for the testing of complex models, or lower the bar on what is considered acceptable statistical power and report confidence intervals for effect sizes.
Examining whether Impacts are Sustained. Given the positive short-term impacts of early childhood interventions that have been found in recent randomized studies, a next step might be to ask about whether initial impacts are sustained and the extent to which subsequent educational experiences mediate and moderate academic and socioemotional outcomes in early elementary school. However, addressing questions about the longer-term impacts of early childhood interventions presents substantial challenges. The extent to which subsequent child outcomes can be attributed to the early childhood intervention versus intervening educational experiences cannot easily be isolated. As students disperse into a wide range of kindergartens and elementary schools, collecting data on the quality of those educational experiences becomes a difficult task. Furthermore, randomized studies of early childhood interventions are generally not designed with sufficient power to examine how an intervention interacts with subsequent educational experiences, even if there were only a limited range of kindergarten and elementary experiences. Consensus over the value of conducting longitudinal studies of sustained intervention impacts has not been reached. Some researchers argue that longitudinal studies of sustained intervention impacts are still warranted despite the methodological challenges and some indications that effects of early childhood interventions dissipate in elementary school. However, other researchers argue that, given methodological limitations of doing so, such studies result in an attempt to attribute effects on child outcomes in later years (or the lack of effects) to the early childhood intervention without consideration of the role of intervening educational experiences.
Examining Impacts on Subgroups of Students. Questions also remain about key subgroups within samples. Some arguments have been made for including adequate sample sizes of key subgroups to answer the question, “what works for whom”. For example, the Early Head Start Research and Evaluation Project found differences between subgroups in the extent to which children benefit from the program (Early Head Start Research and Evaluation Project, 2003). However, some researchers do not view the subgroup question as a critical focus of early childhood evaluation research, especially considering the cost of conducting studies with sufficiently large sample sizes and the practical challenge of developing and implementing a different program model for different groups of children. How to improve early childhood programs so that the average child benefits has been posited as the critical question, rather than that of subgroup variation in impacts.
Limitations of Outcome Measures. Remaining questions about the reliability and validity of measures of child outcomes pose challenges in identifying whether interventions improve child outcomes. There is some indication that more academically-orientated domains may be measured with a higher degree of reliability than socioemotional domains. However, it should be noted that measures of children’s cognitive functioning tend to focus on simple aspects of performance rather than thinking or motivation. Research is needed to develop measures of children’s thought processes (rather than or in addition to measures of children’s mastery of skills), socioemotional development, and approaches to learning. Furthermore, as new measures are developed, it is important that they be normed to ease comparison across studies and provide information about the effectiveness of interventions. Additionally, there is a clear need for more consideration of how best to assess children who are English Language Learners (e.g., not just translating current measures into other languages).
Research limitations. There are also some limitations of what we can learn from randomized studies that are important to note. Given the nested structure of data in evaluations of early childhood interventions (i.e., children nested within classrooms within centers/schools), large samples of classrooms and/or centers are required to answer main effects questions about the impact of interventions. The investments required to design studies to answer questions about impacts for subgroups of children or to examine multiple variants of an intervention are an important consideration and can be prohibitive. Furthermore, randomized evaluation studies are not the best design for exploring some basic research questions that can inform the design of early childhood interventions.
The early childhood field needs more basic research about processes by which children develop (across domains, including math knowledge, cognitive knowledge, language, and their social and emotional functioning). As noted above, research is needed to better understand how foundational skills are learned, how they interrelate, and how they develop over time to support academic success. That knowledge can form the basis for studies of (1) interactions and behaviors teachers can use to facilitate or accelerate that development, and (2) approaches to training teachers that will help them learn approaches to fostering children’s skills and development.
The findings of the federally funded studies that were the focus of this meeting offer promise for valid conclusions regarding the effectiveness of early childhood interventions in improving classroom practices and promoting children’s school readiness. There is also a growing body of literature on the professional development approaches associated with teaching practice and children’s outcomes. However, there are still large gaps in our knowledge base about exactly the questions of greatest concern to the field—what works for which children, under which conditions, and for which outcomes. The conceptual and methodological challenges that remain suggest some strategic next steps to inform a future research agenda, which emerged from the meeting.
Drawing from the discussions of the four themes, a set of recommendations emerged for future research:
In sum, the recent body of federally-funded early childhood research represents an important advance in the use of more rigorous randomized designs to evaluate the effectiveness of early childhood program enhancements, curricula, and approaches to professional development. Despite this tremendous progress, many questions remain about how best to train early childhood providers, how to improve school readiness among low-income children, and how to narrow the school readiness gap. Future work must better define goals for children’s school readiness, including exploration of what skills are required to enable children to succeed in school rather than falling further behind. Defining these goals for children will also inform objectives for professional development and early childhood interventions. In addition, approaches to measuring the school readiness gap in months can provide a more intuitive way of interpreting and judging intervention impacts. Finally, future research must continue to explore substantive questions about training and implementation, other programmatic issues, and school readiness domains.
Caswell, L., & He, Y. (2008). Approaches to promoting children’s school readiness: A review of federally-funded research initiatives aimed at improving young children’s language and literacy skills in early education and care settings. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.
Comer, J.P., & Haynes, N.M. (1991). Parent involvement in schools: An ecological approach. The Elementary School Journal, 91, 271 – 277.
Domitrovich, C. E., Gest, S. D., Gill, S., Bierman, K. L., Welsh, J. A., & Jones, D. J. (2009). Fostering high quality teaching with an enriched curriculum and professional development support: The Head Start REDI program. American Educational Research Journal, 46 (2), 567-596.
Early Head Start Research and Evaluation Project (2003). Overall Findings and Implications for Programs from the Early Head Start Research and Evaluation Project, Long Version: PowerPoint Presentation. (http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/reports/dissemination/overall_long/overall_findings_talkingpts.pdf).
Ginsberg, H., Lewis, A., & Clements, M. (2008). School readiness and early childhood education: What can we learn from federal investments in research on mathematics programs? Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.
Goodson, B. D. (2008). What it means and what it takes to prepare children for school: A synthesis of evidence for the impacts of federally-funded research initiatives in early childhood education. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.
Hamre, B. & Pianta, R. (2004). Self-reported depression in non-familial caregivers: Prevalence and associations with caregiver behavior in child-care settings. Early Childhood Research Quarterly, 19 (2), 297-318.
Hart, B., & Risley, T. R. (1995). Meaningful Differences in the Everyday Experience of Young American Children. Baltimore, MD: Paul H Brookes.
Klein, L., & Gomby, D. (2008). A synthesis of federally-funded studies on school readiness: What are we learning about professional development? Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.
Kohl, G.O., Lengua, L.J., McMahon, R.J. (2000). Parent involvement in school conceptualizing multiple dimensions and their relations with family and demographic risk factors. Journal of School Psychology, 38, 501 - 523.
Layzer, J. & Price, C. (2008). Closing the gap in the school readiness of low-income children. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.
Lee, V. & Burkham, D. (2002). Inequality at the starting gate. Washington, DC: Economic Policy Institute.
Lee, V., & Loeb, S. (1995). Where do head start attendees end up? One reason why preschool effects fade out. Educational Evaluation and Policy Analysis, 17(1), 62-82.
Magnuson, K.A. & Duncan, G.J. (2005). Can family socioeconomic resources account for racial and ethnic test score gaps? School readiness: Closing racial and ethnic gaps. The Future of Children, 15 (1), 35-54.
NICHD Early Child Care Research Network. (2002). The interaction of child care and family risk in relation to child development at 24 and 36 months. Applied Developmental Science, 6(3), 144-156.
Pianta, R.C., Mashburn, A.J., Downer, J.T., Hamre, B.K., & Justice, L. (2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23 (4), 431-451.
Raver, C.C. (2008). Promoting children’s socioemotional development in context of early educational intervention and care: A review of the impact of federally-funded research initiatives on young children’s school readiness. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.
Ritblatt, S.N., Beatty, J.R., Cronan, T.A. & Ochoa, A.M. (2002). Relationships among perceptions of parent involvement, time allocation, and demographic characteristics: Implication for policy formation. Journal of Community Psychology, 30, 519 – 549.
Schultz, T., Halle, T., Forry, N., & Vick, J. (May, 2008). Trends, patterns, and disparities in early learning and development: Learning from the ECLS-B. Presentation at the 0-5 Policy Alliance, Washington, DC.
Snow, C. E., Barnes, W. S., Chandler, J., Goodman, I. F., & Hemphill, L., (1991). Unfulfilled Expectations: Home and School Influences on Literacy. Cambridge, MA: Harvard University Press. Journal of Early Adolescence, 14, 268 – 291.
Stipek, D. (2004). Teaching practices in kindergarten and first grade: Different strokes for different folks. Early Childhood Research Quarterly, 19(4), 548-568.
[1] Appendix A includes detailed descriptions of the set of studies that were the primary focus of this meeting.
[2] Curricula evaluated in the PCER studies included Creative Curriculum; High Scope; Bright Beginnings; Project Approach; Letter People; Doors to Discovery; Early Literacy and Learning Model; Open Court; Literacy Express; Ready, Set, Leap!; Curiosity Corner; Project Construct; Building Language for Literacy; and Language-Focused Curriculum.
To obtain a printed copy of this report, send the title and your mailing information to:
Human Services Policy, Room 404E
Fax: (202) 690-6562
Email: pic@hhs.gov
Home Pages:
Human Services Policy
(HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)
Last updated: 06/14/10