Proceedings from a Working Meeting on School Readiness Research: Guiding the Synthesis of Early Childhood Research

Washington, DC

By:
National Center for Children in Poverty
Mailman School of Public Health
Columbia University
and
Abt Associates Inc.

The meeting was sponsored by the
U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the
Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE)

This document presents the proceedings from A Working Meeting on Recent School Readiness Research:В  Guiding the Synthesis of Early Childhood Research, which was held on October 21-22, 2008 in Washington, D.C.В  The meeting was sponsored by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Office of Planning, Research, and Evaluation (OPRE), Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services. Abt Associates Inc. and the National Center for Children in Poverty (NCCP) convened the meeting.

The purposes of the meeting were to:

• Synthesize findings about impacts on children's school readiness outcomes and teachers' behavior emerging from a set of federally funded studies of early childhood programs, practices, interventions and curricula[1] and integrate new results with findings from previous research.
• Bring together early childhood research and policy experts to examine and evaluate the state of our knowledge about how to support the early development of young children, particularly those who are at-risk for poor outcomes because of poverty, and take stock of progress being made to understand how to narrow the school readiness gap.

"

Background

The school readiness gap has received increasing attention over the last two decades.  Evidence suggests that children from low-income homes are entering school significantly behind their peers from more resourced homes.  Research consistently shows that children's readiness for school when they enter kindergarten is associated with socioeconomic status (Lee and Burkam, 2002; Magnuson and Duncan, 2005; Hart and Risley, 1995).  Furthermore, children who enter school behind their peers rarely catch up. The Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K), a nationally representative study of children in kindergarten in 1998-1999, has documented the school readiness gap at kindergarten entry; cognitive scores among children in the highest SES group are 60% higher than those of children in the lowest SES group (Lee & Burkam, 2002).

Concerns about the readiness gap have led to efforts to develop strategies for enhancing children's development in ways that ready them for school.  Educators have turned to early care and education as a potential developmental intervention.  Evidence of the potential benefits of early care and education programs is based primarily on a few small-scale, carefully controlled experimental studies of educational interventions that have suggested the ability of high-quality preschool interventions to enhance the school readiness of low-income children.  The evidence of the school readiness gap, combined with descriptive data indicating that children in early childhood care and education programs can make cognitive and socioemotional gains has motivated the federal government to sponsor a number of early childhood education programs that focus on serving children from low-income families, including Head Start, Early Head Start, and Even Start.

The federal government has also conducted studies examining the effectiveness of these programs for promoting school readiness.  In the 1990's, the Federal government conducted a series of rigorously-designed Congressionally-mandated evaluations of early childhood programs, such as the Head Start Impact Study, the National Evaluation of the Even Start Program, and the Early Head Start Research and Evaluation Project. Additionally, the government funded a substantial body of research focused on expanding knowledge about the specific practices, interventions, and curricula that can successfully improve school readiness among children from low-income families and can do so across diverse settings. This research focused primarily on evaluations of quality enhancements to existing programs. The research included:

• The Preschool Curriculum Evaluation and Research (PCER) effort led by the Institute of Education Sciences in the US Department of Education, which involved 13 randomized studies of selected off-the-shelf curricula[2] as well as a cross-site evaluation.
• The Interagency School Readiness Consortium (ISRC), funded by the National Institute for Child Health and Human Development (NICHD), the Administration for Children and Families (ACF), and the Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the US Department of Health and Human Services (HHS) and the Office of Special Education and Rehabilitative Services in the U.S. Department of Education, which involved eight randomized studies of innovative, newly developed school readiness interventions that incorporate an integrated focus on cognitive, literacy, and socioemotional aspects of development.
• The Evaluation of Child Care Subsidy Strategies, funded by ACF, included two experimental evaluations of quality enhancement strategies one in child care centers (Project Upgrade) and the other in family child care settings (Massachusetts Family Child Care Study).
• The Quality Interventions for Early Care and Education (QUINCE) study, funded by ACF and ASPE, evaluated the effectiveness of two child care provider training models in enhancing the quality of family home or child care classrooms and promoting positive outcomes in children.

By the fall of 2008, final results from PCER and Project Upgrade had been released and preliminary findings from the ISRC studies were beginning to emerge.

Synthesizing the findings from this now large body of evaluation research is critical for identifying its contributions to our existing knowledge base and informing a future research agenda. In an effort to begin this synthesis and examine the body of evidence emerging from these studies, a meeting of experts was convened entitled, A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research.

In preparation for the meeting, a series of working papers were commissioned by ASPE and OPRE and were prepared by experts in the field. The papers focus on the findings from the most recent set of federally funded studies.  The meeting was organized around these papers; authors of the papers presented their findings, with prepared commentary by respondents.  The meeting was designed to use the papers as the jumping-off point for open discussion among meeting attendees about the state of the science, how it may inform early childhood programs, gaps in our knowledge, and directions for future research.

An overview of the proceedings, summary of key themes identified during the meeting, and identified next steps are presented below.

Overview of Proceedings

The one-and-a-half-day meeting was divided into four panels.  Each panel focused on a topic that was also the subject of one or more of the invited papers.  For each panel, highlights from the relevant working papers were provided by the author, panel respondents provided commentary, and meeting attendees participated in discussion about the issues raised and future work that might be successful in addressing those issues.

The four panels and their corresponding key topics were:

1.  Examining Children's School Readiness Outcomes: Effects of Enhancements to Early Childhood Programs.  This panel examined evidence on the effectiveness of various instructional practices, interventions, and curricula in early care and education programs in promoting young children's development. The panel looked at effects on children in three broad domains language and literacy, mathematics, and socioemotional development.

Barbara Goodson moderated this session. Three working papers were presented, each one corresponding to one of the domains of school readiness.  Goodson also authored a paper which synthesized key themes across the other three papers and across domains of school readiness (see Appendix B.1).  Carolyn Layzer presented a paper on language and literacy outcomes (Caswell & He, 2008; see Appendix B.2), Herbert Ginsburg and Margaret Clements presented their paper on mathematics outcomes (Ginsburg, Lewis, & Clements, 2008; see Appendix B.3), and Cybele Raver presented her paper on socioemotional outcomes (Raver, 2008; see Appendix B.4).

Following the presentations of the working papers, Jeanne Brooks-Gunn provided remarks as the panel respondent, linking the findings from the three papers in a discussion about what it takes to get children ready for school and the effectiveness of different interventions for narrowing the school readiness gap.

2.  Perspectives on Using Research to Improve Programs. This panel focused on the bridge between research and practice and how research can contribute to early childhood program quality improvement and, ultimately, promote young children's school readiness.  Deborah Leong moderated the session, in which presenters drew on their expertise in utilizing research to inform practice and policy to discuss how well the set of studies that were the focus of this meeting address questions of current interest to programs and where the research needs to go to next to answer those questions more clearly.

Beth Rous began the session with a presentation in which she highlighted key lessons from the latest research seen through the lens of statewide implementation and policy development.  Graciela Italiano-Thomas then discussed her experiences with the intersection of research and policy/practice as the Los Angeles Universal Preschool Executive Director and President and CEO of Washington's Thrive by Five.  Finally, Thomas Schultz gave a presentation focusing on how research has been used to improve programs in the past, challenges in doing so, and next steps for research and program improvement implicated by the emerging research focused on at this meeting.

3.  Strategies for Professional Development of the Early Childhood Workforce.  This panel examined evidence on the effectiveness of professional development at changing teacher practice and instructional skills.  Further, the panel looked at evidence on how changes in teaching relate to young children's school readiness outcomes.  Ivelisse Martinez-Beck moderated the session in which presenters discussed issues of intervention implementation fidelity, workplace and teacher characteristics that may moderate changes in teacher behavior, quality improvement approaches, and strategies to train coaches and mentors.

First, Lisa Klein presented highlights of the working paper on early childhood professional development approaches (Klein & Gomby, 2008; see Appendix C).  Robert Pianta served as a respondent, emphasizing the importance of supporting teachers and the classroom practices we know promote children's development, in addition to the particular curriculum or intervention model through which those practices are implemented.  As the second respondent on the panel, Kathryn Tout discussed her research implicating the important role of teachers' readiness for change and features of providers of professional development in moderating changes in teachers' behavior.

4.  Approaches to Measuring and Narrowing the School Readiness Gap.  This panel explored conceptual and methodological issues related to the ways the school readiness gap is represented, measured, and used to judge the effectiveness of early childhood care and education programs. Stephanie Jones served as the moderator for the panel and provided summary comments.

Jean Layzer began the session by presenting highlights of her paper (Layzer & Price, 2008; see Appendix D) which describes the school readiness gap, outlines progress being made to narrow the gap, and discusses alternative approaches to judging the effectiveness of interventions.  As first respondent on the panel, John Love raised a series of critical questions for the field to consider in trying to better understand the school readiness gap and meet the goal of raising low-income children's level of proficiency and skill in various domains of school readiness.  Next, respondent Margaret Burchinal discussed some of the areas where we can focus our attention in the future, such as on infants and toddlers, determining the skills children need and how to teach them (as opposed to focusing on the gap), and better understanding how we take early childhood programs to scale effectively.

To close the meeting, Martha Zaslow provided a synthesis of themes and next steps, emphasizing how the studies focused on for this meeting offer cause for celebration and contribute numerous encouraging developments to the knowledge base, while also pointing to challenges and gaps where there is need for further work.

Cross-Cutting Themes

A number of cross-cutting themes emerged from the presentations and commentary, and the rich discussion about the body of evidence emerging from the school readiness studies that were the focus of this meeting.

Measuring and Evaluating the School Readiness Gap

One of the challenges in the research is describing the school readiness gap in a developmental metric that allows clear communication with policy makers, programs, and families. In much existing literature, achievement differences are defined in terms of an effect size that is calculated as the proportion of a standard deviation on a standardized, normed test score.  These effect sizes are used to describe the magnitude of the gap and the impacts of interventions aimed at narrowing the gap.  Although this way of describing the gap provides researchers with a common metric to describe effects, it can be abstract for policymakers. Therefore, researchers (e.g., Layzer & Price, 2008) have investigated ways to describe the gap between the performance of low income samples and the norming samples in terms of the difference in months of development.  Using this metric, in their examination of several large samples of low-income children, Layzer and Price found that at age 4, the gap in cognitive and language development between low-income children and norms for their age group is approximately one year of development.  In other words, at age 4, low-income children are, on average, one year behind in development, based on the average norm score for their same-age peers.  Describing the gap as reflecting one year of development rather than as one standard deviation offers an interpretation of the difference that is more easily understood.  It is by no means the only way to accomplish this goal.  It will be important to continue this work to translate differences from abstract effect sizes into meaningful developmental metrics.

Determining a metric for describing the achievement gap between children from low-income and better-resourced homes is an important methodological challenge.  Substantively, the challenge for the early childhood field is to identify interventions that reduce the gap, however it is measured. That is, we want to design interventions that are effective at decoupling family income and child achievement.  If we are successful, there will continue to be individual variation in children's achievement, but it will not be a function of family income.

Many studies have shown that school readiness interventions make progress toward narrowing the gap, but do not close it all the way. A wide range of hypotheses have been posited to explain the size and persistence of the gap.  One hypothesis is that early childhood interventions are not intensive enough; many last only nine months and involve no more than half a day of exposure period, which may not be a sufficient amount of time to make up a one-year gap.  Further, many interventions last only one year and occur in the year before children enter school.  Some researchers suggest that the gap is so wide by the time children are age 4 that intervention needs to begin earlier as well as last longer.  Questions also have been raised about the quality of the instructional practices in early childhood settings and whether most children are receiving the kinds of experiences that are most effective for improving their school readiness outcomes.  Similarly, questions arise about approaches to improving the level at which providers implement interventions.  In addition, researchers have questioned whether interventions provided in early care and education settings can be expected to close the gap entirely, when other environmental factors related to the gap are still at play.

Despite expectations that early childhood interventions will provide a boost that enables children to be successful in school, there is mixed evidence that impacts are sustained.  Again, a number of hypotheses have been proposed to explain why impacts are not sustained.  Some researchers suggest that, because students still remain below grade level despite making gains, the boost obtained from preschool may not be sufficient to enable students to keep pace at the next grade level.  As a result, some researchers argue that students continue to require enhanced instructional supports in elementary school in order for impacts to be sustained. Another hypothesis is that gains are not sustained because many low-income children enter under-resourced kindergarten and elementary school classrooms that provide little support for earlier gains. Data suggest that regardless of children's achievement level at the start of kindergarten, their growth rates during elementary school are similar, suggesting that children who start behind, stay behind (Layzer & Price, 2008). In fact, some research suggests the achievement gap widens over time.  As students are expected to learn new skills, which build on foundational skills that may have not been established, students may fall further behind their on-grade-level peers.

The Importance of a Consensus on Defining School Readiness

Some researchers suggest that early childhood experts should work to develop a consistent definition of what constitutes school readiness (see Goodson, 2008 for a discussion).  In theory, a definition of school readiness should identify the foundational skills, content knowledge, and concepts that children need when they enter school in order to achieve academic success in early elementary school and beyond.  Clearly defined expectations for children's development at school entry would provide specific objectives for children's development. Before we can identify the interventions, curricula, and teacher practices that foster children's school readiness, early childhood experts must identify the foundational skills essential for school success.  However, the early childhood field has not developed a consistent definition of school readiness. Furthermore, connections drawn between preschool skills and later academic outcomes are based largely on correlational research.  We need longitudinal research to explore the developmental trajectory of foundational skills.  We need to better understand how foundational skills are learned, how they interrelate, and how they develop over time to support academic success.

Specifically, what experiences contribute to students learning specific skills in each school readiness domain?  For example, what experiences contribute to students learning to regulate their attention, to understand word meanings, or to recognize patterns?  Furthermore, how do the development of skills in one domain relate to the development of skills in another domain?  School readiness domains are viewed as potentially synergistic. For example, socioemotional development may serve as a foundation for a focus on instruction in language and literacy or early math, or perhaps vocabulary development may provide the foundation for growth in other areas (e.g., socioemotional, math).  Many questions about the interrelationship of skills across domains need to be addressed, such as:  To what extent does children's cognitive self-regulation support their learning academic content and skills in a classroom setting?  To what extent are language skills the mechanism for understanding concepts in other domains, such as math and science, and the basis for internalized self-regulation?  In addition, questions about the relationship between early skill development and later school outcomes are critical.  For example, do children entering school at a particular skill level move more quickly onto the next set of skills?  Building a strong body of research that establishes links between early skills and later school achievement will be an important basis for developing a definition of school readiness.

A measurable definition of school readiness within and across developmental domains (e.g., language, literacy, cognitive development, mathematics, socioemotional, approaches to learning, and health) is needed as a foundation for initiatives to promote school readiness.  With clearly defined expectations for children's outcomes, early childhood interventions can target those specific objectives.  A goal for future research will be to identify the interventions, curricula, and teacher practices that effectively foster those outcomes. Likewise, the field will benefit from research that can identify the approaches for training and supporting teachers to use those practices.

Ensuring Effective Professional Development to Promote Child Outcomes

One of the critical issues in the area of professional development involves the tension between professional development that is focused implementing a specific curriculum and professional development that focuses on effective practices more generally.  Currently, most professional development is about a curriculum that may or may not encompass instructional practices that address all domains of development or that reflect state-of-the art knowledge about effective practices.  An alternative approach to professional development is to start with the outcomes that we would like children to achieve, then identify teacher practices that have been shown to foster those outcomes, and finally build systems that support those practices. This series of steps places less emphasis on particular curricula, and instead focuses on developing supports for teachers around classroom practices, regardless of curriculum.

The challenge is to determine how to build systems that scaffold implementation in real world settings. All of the research suggests that professional development on an approach or curriculum does not guarantee that practitioners will consistently be on model when using the approach in their classrooms.  One possibility discussed is to design professional development that is staged, to provide new layers of support as teachers become more skilled at implementing practices.

In general, despite increasing knowledge about effective instructional practices, this knowledge needs to be absorbed and reflected in the professional development provided to teachers.  Continued work is needed to identify approaches for teaching teachers those skills and how to use them.   In the field, the varied strategies have been used in teacher training and professional development, including one-time workshops, formal coursework, teacher-accessed web-based support, individualized web-mediated coaching, and intensive in-person coaching.  The research shows that all of these methods have achieved mixed success in impacting teaching practices and child outcomes.

In their review, Klein and Gomby (2008) reported that training to implement curricula is associated with improvements in implementation and teacher's classroom instructional practices. Specifically, they concluded from their review that coaching and mentoring were associated with improved implementation. Furthermore, some studies found that teachers who received professional development had improved classroom practice and also had children with improved outcomes. Based on their review, the Klein and Gomby suggested that, in order for professional development to impact child outcomes, teachers must deliver the curriculum as intended, the whole curriculum must be delivered, and children must attend the program with enough regularity to benefit from the intervention.

Coaching appears to be an especially promising approach for producing positive changes in teachers and improvements in child outcomes; however, a series of questions remain about coaching: 1) How are coaches trained?; 2) What is the coach doing that impacts the teacher?; 3) How is the teacher changing his/her practice as a result of the coaching?; and 4) How does this change in practice lead to child outcomes?  The QUINCE studies, for which findings had not yet been released at the time of this review, will begin to answer some of these questions. Other newly funded efforts (e.g., the Head Start University Partnership Research Grants funded in 2008 examine strategies for developing teacher effectiveness) will also contribute to this body of literature.

Among the studies that were the focus of this meeting, a number of challenges were articulated that limit the capacity of the field to identify effective professional development approaches. For example, there is no common vocabulary or set of expectations for describing professional development. Many studies lack basic descriptive information about the professional development activities that were implemented, which makes it difficult to compare approaches across studies and offers little information on how to move the field forward. Additionally, delivering multiple professional development strategies as part of a single package, such as combining group training sessions with one-on-one coaching, without adequate descriptions of the components or a planned variation approach to their study makes it impossible to disentangle the components or isolate the effects of any one strategy.

Contextual Factors in the Effectiveness of Interventions

There are a wide range of programmatic, classroom, teacher, and home/family factors that provide influence the implementation of interventions, the effectiveness of professional development approaches, and the effectiveness of interventions for improving children's outcomes. There is a need to better understand the role of these contextual factors in translating high quality curricula, intervention models, or professional development experiences into improved teacher practices and children's school readiness outcomes. However, contextual factors, such as teacher and workplace characteristics, are often not measured or included in evaluations of professional development.

Program context and administrators, instructional supervisors, and program structure play an important, yet relatively unexplored, role in supporting interventions. Furthermore, the role of teacher aides, as well as the relationship between the teacher and aide, in contributing to the success of professional development efforts, intervention implementation, classroom instruction, and program quality requires further attention in research. In a study of the Head Start REDI intervention (Domitrovich, Gest, Gill, Bierman, Welsh and Jones, 2009), researchers provided similar professional development to both the teacher and the aide, treating the teaching team as an intervention unit, and found positive effects of coaching for both the teacher and the aide.

Classroom characteristics and composition were also highlighted as an important context that can impact implementation. For example, in classrooms with high numbers of children with challenging behaviors, socioemotional curricula that were too cognitively oriented were difficult to implement. Another study, which examined the effectiveness of individualized, web-based consultation with a coach combined with teacher-accessed web-based support compared to teacher-accessed web-based support only, found the greatest impact of individualized, web-based consultation on teacher-child interactions in the highest poverty classrooms (Pianta et al, 2008), which. In contrast, teachers in high poverty classrooms who received only teacher-accessed web-based support without individualized web-based consultation with a coach showed a decline in the quality of teacher-child interactions over the course of the study. The results of this study suggest the need for greater support in under-resourced classrooms.

Teacher characteristics also play an important role in mediating outcomes. Specifically, there was discussion about the need to consider the psychological well-being of educators. For example, in the QUINCE study, much of the coaching relationship focuses on dealing with issues of depression and isolation. Other studies reported by Pianta and his colleagues have found that about 15 percent of teachers report depressive symptoms (Hamre & Pianta, 2004). There is a general consensus that it will be important for future professional development efforts to address teachers' mental health. Furthermore, we need a better understanding of how interventions fit with teachers' personal and professional goals, which will help to determine teachers' readiness for change. This point was supported by the finding in several studies that teachers implemented curricula with greater fidelity when they had a high level of dedication to and positive attitudes towards the curriculum.

In addition to programmatic and staff characteristics, the role of parents and the home environment was emphasized as an important context. Research suggests that parents and the home environment have a larger impact on child outcomes than early childhood programs do (e.g., NICHD ECCRN, 2002).  This finding suggests that interventions should involve and target parents as well as children.  However, there is a question over what this involvement entails:  (1) getting parents invested in early childhood program so that they view the program as an opportunity for their child, or (2) intervening with parents as well as with children. The rationale for the latter approach is to intensify the impact on children by improving the enriching experiences at home as well as providing high-quality experiences in the early childhood program. From this perspective, enhancing children's experiences in multiple contexts offers greater opportunity to impact children's outcomes and suggests the need to identify more ways to involve families. Many researchers view the literature as providing support for interventions with parents as well as children, while others point to research on such two-way intervention models that have not demonstrated impacts on parenting (e.g., Abecedarian and CARE). Interventions that target a wide range of parenting behaviors may be too overwhelming for parents, thus preventing improvement in any area.  Interventions designed to target specific parenting outcomes may be an alternative to broader programs.  Specifically, interventions to improve parents' vocabulary may be one appropriate target that may help children as well.

Effectiveness of Integrated versus Targeted Curricula

Most of the curricula and interventions evaluated in recent randomized controlled trials targeted a range of school readiness domains (e.g., language and literacy, math, socioemotional development).  The integrated focus of these interventions on multiple school readiness domains is driven by both the view that child outcome domains are interrelated and the need for early childhood program enhancements in all these areas of children's development.  A number of questions have arisen about curricula that focus on a range of school readiness domains.  Although findings from the studies of integrated curricula are positive, effects were not as large in any given domain as those for interventions targeting a specific domain, such as in the PCER studies. One hypothesis for this difference in effects is that an integrated curriculum may result in less instructional time spent on any specific area.  Therefore, the need to cover multiple developmental domains may dilute the impact of the curriculum on any one domain.

Another issue raised about integrated curricula is that it places a substantial training burden on teachers to learn how to implement many different instructional strategies.  For example, Building Bridges is a comprehensive program designed to teach children socioemotional skills by implementing socioemotional lessons in language, communication, and math activities. The program is fully integrated thematically, but also integrates behavior management skills in teacher training and coaching. Only few and relatively small effects were found for a workshop-only group; however, no effects were found for a group receiving more intensive training and follow-on support.  It may be that the intensive curriculum and more intensive professional development model were too demanding for an already overwhelmed teaching staff. It also might take longer for teachers become skilled at implementing all parts of an integrated curriculum.  It may be important to consider how intervention models might be rolled out in stages and how to scaffold teachers' learning of new curriculum and skills.

Furthermore, despite initial concerns that a more cognitively demanding intervention might provide a stressful learning environment and lead to worse behavioral outcomes for children, findings from the PCER studies provided little evidence for this relationship.  In contrast, there was some evidence of a spillover effect resulting in improved classroom climate.  As noted previously, understanding the interrelationship among school readiness domains is an area that warrants further investigation.

In addition to questions about the focus of integrated curricula on multiple domains, there is a need to further consider whether it makes sense to build an integrated curriculum from multiple curricula, each with a focus on a different school readiness domain (e.g., combining Big Math and Head Start REDI). A key component in the successful integration of multiple curricula is ensuring that there is a common underlying theoretical framework. Difficulties can arise if curricula are based on different theories of pedagogy or development. Explicit connections among various curricula need to be made, as this helps teachers to feel that there is continuity in what they are being asked to achieve in the classroom.

Intervention Dosage

An important unanswered question concerns the question of intervention dosage i.e., the extent to which the amount of intervention that children receive affects the outcomes.  In particular, there is a question about whether there is a threshold of minimum amount of an intervention that is required to produce desired child outcomes. One of the problems with research in this area is that currently, researchers do not have a common approach for how to define and measure dosage (e.g., number of lessons, length of lessons, number of days, extent of individual versus group instruction).  Dosage is typically conceptualized as the overall amount of instruction offered.  However, some researchers suggest that dosage should be conceptualized as a function not only of what is offered but also what is received by children.  Further, measuring what is received by children may go beyond the child's exposure (i.e., time in a classroom) to their level of engagement with the intervention.  Also, it is likely that the relationship of dosage to outcomes may be mediated by the quality of the intervention.  The effect of dosage is likely to be different if the intervention being provided encompasses high-quality versus lower-quality practices.  Future research will need to address the definitional issues surrounding dosage (offered or received) and the potentially complicated relationships among dosage, quality and features of the intervention being tested, and amount of dosage experienced by children.

The issue of dosage is a question in the area of professional development as well.  Is there a dosage level of training to ensure sustained changes in teacher practices to ensure sustained changes in teacher behavior?  Measuring dosage or intensity of professional development is not typically a part of evaluations.  Even when descriptive data are provided, typically the data are about training sessions as opposed to the amount of coaching/mentoring, which is more difficult to measure.  As a result, there is little evidence about the level of intensity of coaching that is beneficial and for whom.  Future research should not only measure and report the intensity of interventions being studied, but also contrast varying intensities of instructional support provided to students to determine how much is necessary.

Further exploration of these dosage issues (e.g., definition, measurement, the intensity required to produce sustained outcomes) are necessary to further our understanding of how to ensure that all children are ready for and successful in school.

Research and Methodological Issues

Although there has been progress in carrying out randomized controlled trials, and some important questions are being addressed by this research, there are still key design issues that need to be addressed. The randomized controlled evaluations of early childhood interventions reviewed for this meeting are generally designed to address global questions about whether specific intervention programs have impacts on children's development and/or on teachers' instruction compared to a business-as-usual condition.

Compared to what? It is important to note that the comparison of program enhancements to existing early childhood programs, as in recent studies, represents a shift from the earliest randomized studies of early childhood programs (e.g., Perry Preschool, Abecedarian).  In the early studies, an intensive preschool program delivered under ideal conditions (e.g., delivered on a small scale by intervention developers) was compared to a no intervention control group rather than a business-as-usual control group.  Given the different comparison being made, it is not surprising that the earlier programs were found to have larger impacts than the interventions evaluated in the more recent body of work.  A key consideration in synthesizing and drawing conclusions from evaluations of early childhood interventions is the recognition of what comparison is being made.

Testing Intervention Components.  The recent body of randomized studies is designed to provide more rigorous tests of whether the interventions are effective at improving child development or teacher practice.  However, these studies are not designed to address questions about differential impacts of varied intervention components, delivery mechanisms, or professional development approaches.  To address these questions, planned variation studies would be needed, which systematically compare multiple versions of an intervention or training approach.  However, planned variation studies are complex to design and implement.  In addition, they are expensive, since they require substantially larger sample sizes to test multiple treatment conditions.  We either need to increase investments in large studies that allow for the testing of complex models, or lower the bar on what is considered acceptable statistical power and report confidence intervals for effect sizes.

Examining whether Impacts are Sustained.  Given the positive short-term impacts of early childhood interventions that have been found in recent randomized studies, a next step might be to ask about whether initial impacts are sustained and the extent to which subsequent educational experiences mediate and moderate academic and socioemotional outcomes in early elementary school.  However, addressing questions about the longer-term impacts of early childhood interventions presents substantial challenges.  The extent to which subsequent child outcomes can be attributed to the early childhood intervention versus intervening educational experiences cannot easily be isolated.  As students disperse into a wide range of kindergartens and elementary schools, collecting data on the quality of those educational experiences becomes a difficult task.  Furthermore, randomized studies of early childhood interventions are generally not designed with sufficient power to examine how an intervention interacts with subsequent educational experiences, even if there were only a limited range of kindergarten and elementary experiences.  Consensus over the value of conducting longitudinal studies of sustained intervention impacts has not been reached.  Some researchers argue that longitudinal studies of sustained intervention impacts are still warranted despite the methodological challenges and some indications that effects of early childhood interventions dissipate in elementary school.  However, other researchers argue that, given methodological limitations of doing so, such studies result in an attempt to attribute effects on child outcomes in later years (or the lack of effects) to the early childhood intervention without consideration of the role of intervening educational experiences.

Examining Impacts on Subgroups of Students.  Questions also remain about key subgroups within samples. Some arguments have been made for including adequate sample sizes of key subgroups to answer the question, what works for whom. For example, the Early Head Start Research and Evaluation Project found differences between subgroups in the extent to which children benefit from the program (Early Head Start Research and Evaluation Project, 2003). However, some researchers do not view the subgroup question as a critical focus of early childhood evaluation research, especially considering the cost of conducting studies with sufficiently large sample sizes and the practical challenge of developing and implementing a different program model for different groups of children. How to improve early childhood programs so that the average child benefits has been posited as the critical question, rather than that of subgroup variation in impacts.

Limitations of Outcome Measures. Remaining questions about the reliability and validity of measures of child outcomes pose challenges in identifying whether interventions improve child outcomes. There is some indication that more academically-orientated domains may be measured with a higher degree of reliability than socioemotional domains. However, it should be noted that measures of children's cognitive functioning tend to focus on simple aspects of performance rather than thinking or motivation. Research is needed to develop measures of children's thought processes (rather than or in addition to measures of children's mastery of skills), socioemotional development, and approaches to learning. Furthermore, as new measures are developed, it is important that they be normed to ease comparison across studies and provide information about the effectiveness of interventions. Additionally, there is a clear need for more consideration of how best to assess children who are English Language Learners (e.g., not just translating current measures into other languages).

Research limitations. There are also some limitations of what we can learn from randomized studies that are important to note.  Given the nested structure of data in evaluations of early childhood interventions (i.e., children nested within classrooms within centers/schools), large samples of classrooms and/or centers are required to answer main effects questions about the impact of interventions.  The investments required to design studies to answer questions about impacts for subgroups of children or to examine multiple variants of an intervention are an important consideration and can be prohibitive.  Furthermore, randomized evaluation studies are not the best design for exploring some basic research questions that can inform the design of early childhood interventions.

The early childhood field needs more basic research about processes by which children develop (across domains, including math knowledge, cognitive knowledge, language, and their social and emotional functioning). As noted above, research is needed to better understand how foundational skills are learned, how they interrelate, and how they develop over time to support academic success.  That knowledge can form the basis for studies of (1) interactions and behaviors teachers can use to facilitate or accelerate that development, and (2) approaches to training teachers that will help them learn approaches to fostering children's skills and development.

Strategic Next Steps

The findings of the federally funded studies that were the focus of this meeting offer promise for valid conclusions regarding the effectiveness of early childhood interventions in improving classroom practices and promoting children's school readiness. There is also a growing body of literature on the professional development approaches associated with teaching practice and children's outcomes. However, there are still large gaps in our knowledge base about exactly the questions of greatest concern to the fieldwhat works for which children, under which conditions, and for which outcomes.  The conceptual and methodological challenges that remain suggest some strategic next steps to inform a future research agenda, which emerged from the meeting.

Directions for Research

Drawing from the discussions of the four themes, a set of recommendations emerged for future research:

• Planned variation studies are important for starting to disentangle aspects of interventions and professional development approaches that lead to improvements in classroom environments and child outcomes. For example, planned variation studies could examine the differential impact of professional development in the form of workshops versus one-on-one coaching versus workshops plus coaching.  However, there are a number of issues and challenges related to conducting planned variation studies that have been noted and need to be weighed against what can be learned from them (e.g., cost and sample size requirements).
• Secondary analysis of existing large-scale, longitudinal databases, such as the ECLS-B or the ECLS-K, may be able to provide important data on the trajectory of children's development on the emergence of the school readiness gap in infancy and toddlerhood, the extent to which the gap persists or widens over time, and factors influencing the size and persistence of the gap.
• Current conceptualizations and operational definitions of dosage need to be examined.  Secondary analysis of existing databases needs to be explored as a basis for looking at dosage effects.  Ultimately, new studies need to be designed to evaluate the impact of dosage of the intervention or professional development on sustained outcomes in educator/caregiver behaviors and practices as well as child outcomes.
• Research needs to examine outcomes for the populations of children who are learning English as a second language and whether there are specific instructional practices that are effective with these students. Although some of the current set of studies include English Language Learning children as a subgroup in their analyses, it is important to expand this body of work, and also a need for more research on caregivers/educators who are learning English.

Improving Programs and Practices

• Professional development approaches should be developed to support classroom practices that have been shown to foster the developmental outcomes that we would like children to achieve.  Professional development should be based on objectives for children's development rather than on specific curriculum.
• Researchers and practitioners should explore how interventions for three- and four-year-olds can be integrated with interventions focusing on infants and toddlers. Research suggests that the achievement gap can be seen as early as 24 months (Schultz, Halle, Forry, & Vick, 2008), and children from low-income families are nearly nine months behind their more advantaged peers by the time they are three-years-old (Layzer & Price, 2008). Therefore, it is critical that we think about how to intervene earlier than ages three or four and identify effective birth to five models.
• Early childhood experiences should be aligned with expectations and goals for kindergarten and elementary school. Research shows that children from low-income families enter schools of lower quality than their more advantages counterparts (e.g., Lee & Loeb, 1995; Stipek, 2004). Therefore, even if children are given a boost by attending high-quality preschool programs, these effects may fade if children do not continue to be provided with high-quality care and learning experiences.
• Ways to involve parents more effectively in early childhood programs should be identified. Research suggests that parent involvement is critical to children's early learning (Comer & Haynes, 1991; Kohl, Lengua, & McMahon, 2000; Ritblatt, Beatty, Cronan, & Ochoa 2002; Snow, Barnes, Chandler, Goodman, & Hemphill, 1991). However, there is not yet clear evidence of how to integrate a parenting component to produce an effective, comprehensive early childhood model for preschool-aged children.
• Characteristics of teachers and providers should be considered in developing interventions and professional development approaches, and to include and consider nontraditional learners. There are rising expectations of early educators/caregivers. Some members of the early childhood workforce may find it difficult to implement new approaches, or may disagree with the recommendations coming from the research field. It will be important for future interventions/programs to include efforts to improve educators'/caregivers' human and/or social capital (e.g., consider strategies to support psychological well-being).

Connecting Research, Practice, and Policy

• Researchers, practitioners and policy makers should work together to develop a comprehensive, valid definition of school readiness, with delineation of thresholds in different outcome areas that could be applied across different programs or interventions.
• As research on intervention dosage emerges, findings can inform programmatic decisions about part-day versus full-day programs; school-year versus full-year programs; and one year versus multiple year programs.
• To to fully understand costs and educational benefits of early childhood interventions, studies should be designed to include follow-up of short-term effects into school.
• Criteria should be developed to determine when there is sufficient evidence to take an intervention to scale.

In sum, the recent body of federally-funded early childhood research represents an important advance in the use of more rigorous randomized designs to evaluate the effectiveness of early childhood program enhancements, curricula, and approaches to professional development.  Despite this tremendous progress, many questions remain about how best to train early childhood providers, how to improve school readiness among low-income children, and how to narrow the school readiness gap. Future work must better define goals for children's school readiness, including exploration of what skills are required to enable children to succeed in school rather than falling further behind.  Defining these goals for children will also inform objectives for professional development and early childhood interventions.  In addition, approaches to measuring the school readiness gap in months can provide a more intuitive way of interpreting and judging intervention impacts.  Finally, future research must continue to explore substantive questions about training and implementation, other programmatic issues, and school readiness domains.

References

Caswell, L., & He, Y. (2008). Approaches to promoting children's school readiness: A review of federally-funded research initiatives aimed at improving young children's language and literacy skills in early education and care settings. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.

Comer, J.P., & Haynes, N.M. (1991). Parent involvement in schools: An ecological approach. The Elementary School Journal, 91, 271 277.

Domitrovich, C. E., Gest, S. D., Gill, S., Bierman, K. L., Welsh, J. A., & Jones, D. J. (2009). Fostering high quality teaching with an enriched curriculum and professional development support: The Head Start REDI program. American Educational Research Journal, 46 (2), 567-596.

Early Head Start Research and Evaluation Project (2003).  Overall Findings and Implications for Programs from the Early Head Start Research and Evaluation Project, Long Version: PowerPoint Presentation. (http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/reports/dissemination/overall_long/overall_findings_talkingpts.pdf).

Ginsberg, H., Lewis, A., & Clements, M. (2008). School readiness and early childhood education: What can we learn from federal investments in research on mathematics programs? Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.

Goodson, B. D. (2008). What it means and what it takes to prepare children for school:  A synthesis of evidence for the impacts of federally-funded research initiatives in early childhood education. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.

Hamre, B. & Pianta, R. (2004). Self-reported depression in non-familial caregivers: Prevalence and associations with caregiver behavior in child-care settings. Early Childhood Research Quarterly, 19 (2), 297-318.

Hart, B., & Risley, T. R. (1995). Meaningful Differences in the Everyday Experience of Young American Children. Baltimore, MD: Paul H Brookes.

Klein, L., & Gomby, D. (2008). A synthesis of federally-funded studies on school readiness: What are we learning about professional development? Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.

Kohl, G.O., Lengua, L.J., McMahon, R.J. (2000). Parent involvement in school conceptualizing multiple dimensions and their relations with family and demographic risk factors. Journal of School Psychology, 38, 501 - 523.

Layzer, J. & Price, C. (2008). Closing the gap in the school readiness of low-income children. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.

Lee, V. & Burkham, D. (2002). Inequality at the starting gate. Washington, DC: Economic Policy Institute.

Lee, V., & Loeb, S. (1995). Where do head start attendees end up? One reason why preschool effects fade out. Educational Evaluation and Policy Analysis, 17(1), 62-82.

Magnuson, K.A. & Duncan, G.J. (2005).  Can family socioeconomic resources account for racial and ethnic test score gaps? School readiness: Closing racial and ethnic gaps.  The Future of Children, 15 (1), 35-54.

NICHD Early Child Care Research Network. (2002). The interaction of child care and family risk in relation to child development at 24 and 36 months. Applied Developmental Science, 6(3), 144-156.

Pianta, R.C., Mashburn, A.J., Downer, J.T., Hamre, B.K., & Justice, L. (2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms.  Early Childhood Research Quarterly, 23 (4), 431-451.

Raver, C.C. (2008). Promoting children's socioemotional development in context of early educational intervention and care: A review of the impact of federally-funded research initiatives on young children's school readiness. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the US Department of Health and Human Services, Washington, DC.

Ritblatt, S.N., Beatty, J.R., Cronan, T.A. & Ochoa, A.M. (2002). Relationships among perceptions of parent involvement, time allocation, and demographic characteristics: Implication for policy formation. Journal of Community Psychology, 30, 519 549.

Schultz, T., Halle, T., Forry, N., & Vick, J. (May, 2008). Trends, patterns, and disparities in early learning and development: Learning from the ECLS-B. Presentation at the 0-5 Policy Alliance, Washington, DC.

Snow, C. E., Barnes, W. S., Chandler, J., Goodman, I. F., & Hemphill, L., (1991). Unfulfilled Expectations: Home and School Influences on Literacy. Cambridge, MA: Harvard University Press. Journal of Early Adolescence, 14, 268 291.

Stipek, D. (2004). Teaching practices in kindergarten and first grade: Different strokes for different folks. Early Childhood Research Quarterly, 19(4), 548-568.

Endnotes

[1]       Appendix A includes detailed descriptions of the set of studies that were the primary focus of this meeting.

[2]       Curricula evaluated in the PCER studies included Creative Curriculum; High Scope; Bright Beginnings; Project Approach; Letter People; Doors to Discovery; Early Literacy and Learning Model; Open Court; Literacy Express; Ready, Set, Leap!; Curiosity Corner; Project Construct; Building Language for Literacy; and Language-Focused Curriculum.

How to Obtain a Printed Copy

To obtain a printed copy of this report, send the title and your mailing information to:

Human Services Policy, Room 404E
Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Ave, SW
Washington, DC 20201

Fax:  (202) 690-6562
Email:  pic@hhs.gov

Appendices

Selected Federally-Funded Early Childhood Intervention and Quality Improvement Studies

Endnote

1. This document represents a small subset of work funded by the federal government and included in the review conducted for the Working Meeting on School Readiness Research, October 21-22, 2008, Washington, DC.

Examining Children's School Readiness Outcomes: Effects of Enhancements to Early Childhood Programs

This paper is part of a series of working papers prepared for a meeting sponsored by the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE). Abt Associates Inc and the National Center for Children in Poverty (NCCP) were funded to convene the meeting. The views represented in this paper are those of the author(s) and do not necessarily reflect the opinions of the U.S. Department of Health and Human Services.

What It Means and What It Takes to Prepare Children for School: A Synthesis of Evidence for the Impacts of Federally-Funded Research Initiatives in Early Childhood Education

Introduction

In the last ten years, there has been a rapid increase in the number of research studies examining the impacts of preschool interventions on children's school readiness outcomes, with many of these studies using rigorous experimental methods that allow attribution of causal relationships. A substantial proportion of these studies have been supported through federal funding, as stand-alone evaluations of federal programs such as the Head Start Impact Study, as part of research initiatives such as the Preschool Curriculum Evaluation Research Initiative, or as individual studies funded through grants. State and local governments, as well as private funders, have also supported recent research studies of preschool interventions. One of the hallmarks of the current crop of research studies is the focus on the right hand side of the equation; that is, the studies are not simply concerned with demonstrating the size of the impacts on child outcomes but also with trying to understand the processes responsible for the impacts that are obtained.

The most recent meta-analyses of early childhood education programs (Jacob, Creps & Boulay, 2004; Nelson & Westhues, 2003; Gorey, 2001; Gilliam & Zigler, 2000) focus on the average size of the impacts of a range of early childhood education interventions. The meta-analyses either bypass the question of variation in instructional inputs as they relate to effect size or focus on programmatic features such as length of day, comprehensiveness of services or auspice rather than instructional methods. However, the research agenda in the past five to ten years has moved beyond proving that early childhood education can make a difference to children, especially at-risk children, to trying to build a body of knowledge about how to successfully intervene with at-risk children to improve their school readiness. The three summary papers discussed here are directly concerned with the most current evidence for instructional practices, interventions, curricula, and programs that have been shown to impact children's development in three domains: language and literacy, math, or socioemotional development.

The question being posed concerns the contributions of this emerging body of research as a source of new evidence or as an extension of what we know about effective interventions for school readiness. On the one hand, the three summary papers suggest that there are an increasing variety of types of early childhood education interventions and curricula that are effective at improving children's school readiness-related outcomes across domains. On the other hand, there are important limitations of the research. First, almost all of the interventions being tested encompass multiple components and the designs do not allow us to "unbundle" these components analytically to determine which programmatic factors make the biggest difference for children's outcomes. When the research is examined for lessons about variation in instruction, the interventions being compared differ on so many factors that it is impossible unable to link outcomes to specific characteristics of instruction or environmental changes. Just as in the past, this current research primarily consists of stand-alone studies, essentially unconnected to one another in any logical way nor connected to a systematic, integrated research plan. At the present time, the research does not go much further in helping us isolate the "potent" or "active" ingredients in instruction that are critical to different child outcomes.

The second limitation is that studies are not connected by a consistent definition of what in fact constitutes school readiness. Studies tend to use measures that align with the intervention and do not attempt to assess a more comprehensive set of outcomes across other domains. This limits our ability to compare the effects of different intervention strategies and to answer questions about whether focusing on one aspect of school readiness (e.g., self-regulation) has generalized impacts across other outcome domains.

Building a systematic knowledge base on effective practices, or a 'science of practice' for promoting school readiness will require an "infrastructure" to guide and link the research. More specifically, the infrastructure will be built on answers to two over-arching questions:

1. What constitutes school readiness?

1. Do we understand the foundational skills/content knowledge/understandings that children need to develop by the time they enter school for academic success in both early elementary grades and longer?
2. What is the developmental trajectory of these foundational skills?

A common definition of what is meant by "school readiness" will contribute to the ability to standardize the research on school readiness. A justifiable definition of school readiness will depend on evidence showing that skills developed during the preschool period have impacts on later school performance. While each of the three papers offers some rationale for linking the preschool outcomes in their domain to academic outcomes, the rationales are based on a mix of theory, opinions and correlational research. Even in the field of language and literacy, where the soon-to-be-released report from the National Early Literacy Panel will present a comprehensive summary of the research literature about the early or foundational skills/knowledge that are the strongest predictors of later reading achievement, the research base is correlational. Although some of the research reported in these papers will be able to test causal relationships between preschool and school outcomes, assuming long-term follow-up of children, for most of the interventions, it is too early to show long-term effects for children's academic performance and even longer-term social outcomes such as higher education and/or economic productivity. As such evidence is reported, it will be a basis for beginning to build a stronger research-based definition of school readiness. It is worth noting that there are other forces pushing us toward a measurable definition of school readiness. The large scale investments in early childhood through universal pre-kindergarten initiatives and quality improvement systems are being justified in terms of improvements in school readiness. In theory, a definition of school readiness should rest on research linking preschool skills/content knowledge/understandings to later school achievement.

Further, we need longitudinal evidence of the developmental trajectory of skills purported to be foundational.

2. What do we know about the contribution or influence of environmental factors in the development of the foundational skills, and can we build effective interventions based on this knowledge?

1. Is there evidence that the skills are learnable or modifiable and therefore susceptible to intervention?
2. Based on theory or basic research, can we develop effective interventions to enhance the development of these skills?
3. Can we show a causal link between specific instructional practices and student school readiness-related outcomes?

Interventions aimed at enhancing children's school readiness are based on two premises: (a) that the skills being taught or supported by the intervention are learnable, and (b) that there is research or theory to justify the intervention strategies for changing the early childhood education experience so as to alter the developmental trajectory. Even when there is clear agreement on objectives for children at the end of preschool, there are typically alternative theories about effective intervention approaches, as reflected in the variability across intervention designs. This brings us back to the question of which intervention strategies are most powerful in creating changes in children.

The current research is insufficient for understanding the process by which these interventions lead to child impacts. We don't know which of the changes being created in early childhood environments through multi-faceted interventions are the causal factors in changing student outcomes. Even the experimental studies being conducted can't, in fact, establish that the teachers or classrooms that changed the most as a result of the intervention are the same sites where the child outcomes changed the most.

One approach that has been used to begin to build this information base is planned variation research, where the research is designed to systematically test different intervention strategies with similar children and a common set of outcomes, to attempt to isolate which models have the largest impacts. However, unless this type of planned variation research varies and compares the impacts of intervention components rather than multi-dimensional models, it is not possible for the research to provide us with the information we want about mechanisms of change.^[1] Further, this kind of planned variation research focuses on the relative contribution of components of the instructional intervention. There are other aspects of the environment that are additional potential factors in the impact of instruction, such as how the classroom is managed (e.g., discipline, grouping), class size and heterogeneity of the child in terms of characteristics such as home language, special needs and the like. Research that allows us to disentangle the combined and individual effects of all of these factors will require complex designs and sample sizes that permit us to test multiple variations.

What follows is an overview of the conclusions from the three synthesis papers about where the current research stands in terms of the content and focus of the interventions being tested, what the findings tell us about effective instructional strategies, and what types of future research will be most informative. The two issues raised above  defining school readiness and the developmental trajectory of foundational skills and identifying the active ingredients in interventions  will resonate through the overview.

The outcomes of these interventions are discussed in the context of the following domains: socioemotional, language and literacy, and then math. The paper starts with the socioemotional domain because the constituent skills are hypothesized as constituting the platform underlying the child's ability to negotiate successfully all other learning tasks, including early literacy and early math understandings. The second domain discussed is language and literacy. Although language and literacy are often paired, in many respects language should be considered in conjunction with socioemotional development, because of the broad central role language plays in children's learning. For the developing child, the ability to understand and use language is the primary mode by which he builds knowledge of the world and communicates his own ideas and feelings. In this sense, most aspects of socioemotional development are completely intertwined with language development: Children's internalized regulatory mechanisms are language-based, their social understanding is language-based, and their ability to interact and engage with others is primarily negotiated through language. Language development can be labeled as an "engine" of development.

Early literacy and early math are the final outcome areas discussed. As opposed to socioemotional and oral language outcomes, early literacy and math both represent specific skills and understandings. Literacy, for example, includes print knowledge, the alphabetic code and phonological processing (phonological memory, access, and awareness); math includes number and operations with numbers, geometric shapes, spatial relations, and measurement.

Socioemotional Domain

As Raver reports, the current conceptualization of the socioemotional domain distinguishes three major mechanisms or processes that support children's development: self- regulation (emotional and cognitive), social cognitions, and prosocial skills. Raver also describes a fourth area of socioemotional development, behavior problems (externalizing and internalizing), which factors into children's ability to learn and relate to other people. The behavioral manifestations of these processes, taken together, form a picture of a child socially and emotionally ready for school. This child is able to:

• follow adult directions;
• control his/her own emotions, attention, and impulses independent of adult regulation;
• establish positive social relationships with peers and adults;
• successfully solve social problems without being disruptive or aggressive;
• attend in a sustained way to learning tasks in the environment;
• evaluate his/her own behavior and make corrections; and
• demonstrate "cognitive flexibility."

Raver's description of the intervention research in the socioemotional domain clusters studies based on which of the three underlying processes the interventions are designed to effect. The research on interventions in the socioemotional domain is most consistent in the area of self-regulation and social skills. Evidence of the ability to reduce aggressive behavior in the classroom is more mixed. Further, all of the data reported represents short-term findings, with no evidence to date of longer-term benefits for school performance. Further gaps include: evidence of whether and how the various components of socioemotional functioning are inter-connected; and evidence of the relationship of children's development of self-regulatory and social relationship skills in preschool to their oral language development or to the acquisition of early literacy or math skills. As discussed above, the fact that disparate intervention strategies all appear to have impacts raises the question of the mechanisms leading to child impacts.

Impacts on Children's Self-Regulation Skills

In the area of self-regulation, Raver cites evidence that children's attentional processes can be enhanced through a variety of intervention mechanisms. For example, Raver cites three interventions as having impacts on children's self-regulatory skills:

• Project REDI: uses small-group lessons focused on understanding emotions to help children regulate behavior and successfully negotiate social relationship; trains teachers on classroom management strategies that create a positive learning climate; uses instructional strategies in early literacy to build oral language skills and phonemic awareness that promote teacher/child interaction (scripted dialogic reading exercises to promote conversation and build vocabulary and small-group phonemic awareness activities to teach sounds and words).
• Chicago School Readiness Project (CSRP): focuses on improving the emotional climate of the class by providing teachers with training in behavior management and in-class coaching by mental health consultants on implementing positive behavior management strategies.
• Tools of the Mind: uses role play as a central mechanism to help children develop "self-regulatory scripts" to guide their own behavior; thematic dramatic play is the central type of role play, but roles also are used in children's work with peers in reading and other content areas.

All three of these interventions have reported impacts on children's levels of attention and focused effort and persistence, as measured through direct observations,^[2] despite the fact that the three interventions use very different approaches. REDI and CSRP use the teacher as the primary change agent for helping children develop self-regulation, while Tools uses children's own role play to help children develop their own self-regulatory scripts. The fact that all three interventions report impacts on children's development of self-regulation skills and all three use multiple avenues to affect these changes underlines the importance of systematic research to isolate the most important "levers." Further, data on the long-term effects of these curricula will be crucial for understanding whether the differences in the approaches of Tools versus REDI and CSRP have ramifications for the persistence of impacts over time, once children leave supportive early childhood environments. If the children in Tools of the Mind build internal self-regulatory structures while children in REDI or CSRP are more dependent on the actions of the teacher, then it is possible that Tools will have more robust long-term impacts.

Long-term follow-up data on differences in school performance for children with stronger or weaker self-regulation at the end of preschool will also provide important information to prove or disprove the contention that self-regulation encompasses a skill set that influences learning across content areas and across ages. For the same reason, it is important that the research on these interventions includes measures of children's acquisition of skills in other curriculum areas, such as early literacy or early math at the end of preschool. (For example, in the research on Tools, children not only develop stronger attentional processes, they also score higher on standardized tests of math at the end of preschool.)

In general, the maintenance of gains in preschool may depend not only on the types of behavioral and/or attentional changes that children experience in preschool but also on the characteristics of their subsequent classroom environments in elementary school. Gains in preschool may be maintained or even enhanced if children experience classroom environments in elementary school that continue to support positive, regulated behavior.

Impacts on Children's Social Cognitions and Prosocial Skills

A second area of intervention research described by Raver focuses on the social cognitive mechanisms underlying children's ability to form and sustain positive interpersonal relationships with peers and adults in the classroom and to solve problems in social relationships. The social cognitive mechanisms include: children's knowledge of emotions  their own and other people's; knowledge of prosocial behaviors (e.g. helping, sharing, and taking turns); and the ability to generate and use more effective social problem-solving skills. In this area, the child who is ready for school:

• Can develop a positive, engaged social relationship with the teacher;
• Can form positive friendships with peers;
• Can successfully solve problems that arise in social interactions with peers;
• Demonstrates prosocial behavior in the classroom, such as helping other children, sharing, and taking turns;
• Does not act aggressively with other children or adults.
• Does not act disruptively in the classroom.

In the same way that self-regulatory skills are correlated with children's learning across domains, children's social skills and the quality of their relationship with teachers have been found to be correlated to their later social and academic competence in early elementary school.

Raver focuses on the results from three interventions:

• Project REDI trains teachers to provide more emotional coaching and support in the classroom and includes a socioemotional curriculum that helps children develop emotional knowledge and accurate social attributions, and prosocial behavior strategies for interactions with peers. REDI reports significant differences for children's emotion understanding and interpersonal problem-solving, and significant gains in children's social competence (teacher rated aggression and observer-rated social competence). The project also reports significant changes in teachers' use of emotion coaching, positive classroom management and behavioral support.
• My Teaching Partner, a web-based teacher training curriculum developed by Pianta, focused on improving teacher/student relationships to be more responsive and supportive. Results showed that intervention teachers demonstrated significantly more sensitivity, language modeling, and quality of instructional support to students.
• CSRP also worked with teachers to establish more positive classroom environments, and there was a significant impact of the on positive classroom climate (d = .52 to d = .89). Although there was no child-focused curriculum on emotional language or self-awareness, the gains in children's behavioral self-regulation were attributed to the enhanced classroom environment.

Impacts on Children's Behavior Problems

Fewer studies have measured impacts of interventions on children's behavior problems. Project REDI, a socioemotional learning curriculum, reported significant reductions of children's aggression, as reported by teachers. Similarly, CSRP reports reductions in children's externalizing and internalizing problems as reported by teachers. Across the PCER studies, there were no effects on children's behavior problems as reported by teachers.

Language and Literacy

In many respects, the conceptualization of the critical foundational skills to be acquired during preschool has moved furthest along in the area of language and literacy. There has been a wealth of theoretical writings, professional opinions, and best practice documents proposing which skills are the precursors or foundational skills for reading achievement, and, it is only in the field of language and early literacy that we [soon] will have a systematic empirical summation of research demonstrating which early literacy skills predict later conventional literacy (via the National Early Literacy Panel). There is beginning to be a structure for understanding the developmental precursors to later reading and writing abilities. Further, in the challenge of defining school readiness, this domain has the advantage of the widely-shared criterion of the critical long-term academic outcome-becoming a skilled reader (with strong decoding and comprehension skills, a strong vocabulary, automaticity in reading).

Before the NELP, the field was driven in its thinking about "readiness" skills by two documents that provided consensus or narrative summaries of a portion of the research literature concerning the relation between early precursor skills and later conventional literacy skills: Whitehurst and Lonigan (1998) identified skills in the domains of oral language, print and letter knowledge, and phonological processing as encompassing two aspects (outside-in and inside-out skills) of emergent literacy that are related to later conventional forms of reading and writing; and Snow, Burns, and Griffin (1998), in their report of the National Research Council's panel on preventing reading difficulties in young children, identified weaknesses in oral language, phonological awareness, and alphabet knowledge as prime targets of intervention to prevent the occurrence of significant reading problems. Neither of these documents, however, was based on a comprehensive summary of the published literature.

The NELP provides an evidence base about early or foundational skills/knowledge that are the strongest predictors of reading achievement, as well as a summary of the average effects of the number of interventions to improve early literacy/language skills. In the ensuing discussion, we start with oral language and then move to early literacy, for the reasons spelled out above.

Oral Language

Oral language skills can be conceptualized as including productive language skills (forming sounds correctly, using the right forms of words, forming correct sentence syntax), language use (using words to express thoughts or ideas or to transmit information); and language content (understanding of vocabulary and narrative). In describing a child who is ready for school in terms of his/her oral language skills, the following skills are included:

• Ability to express thoughts, ideas into spoken words;
• Ability to understand other people when they talk;
• Ability to carry on a back-and-forth conversation with another person;
• Ability to use correct versions of plural, past and future tenses.
• Ability to understand narrative sequence (logical order of events);
• Expressive vocabulary that includes knowledge of words likely to be encountered in early readers; understanding of superordinate words for categories of objects (silverware, clothes, tools, etc).

As described in the synthesis paper by Caswell and He, numerous research studies have demonstrated a relationship between early, well-developed oral language skills and later reading abilities. Despite the primary of oral language skills in a child's cognitive readiness for school and, ultimately, for learning to read, the evidence for intervention effects is somewhat disappointing. Across the large number of interventions concerned with children's oral language outcomes, most show small to medium effects.

The synthesis paper describes some of the variety in the oral language activities used to promote children's understanding of vocabulary, comprehension of concepts, and language use. The problem with the research is that in most instances, the intervention being examined includes more than one type of oral language activity, as well as other literacy-related activities, so it is impossible to isolate the impact of the any one type of oral language activity. For example, a number of programs use dialogic reading to promote children's oral language skills. This includes dialogic reading as the sole intervention activity and dialogic reading that is integrated into a broader curriculum with additional activities and goals. There were inconsistent results of these interventions on children's outcomes, although most did find at least a small effect on children's vocabulary. Again, where dialogic reading was just one activity in the curriculum, we cannot know whether it was the dialogic reading was responsible for the impacts on vocabulary that were found.

Most of the research on oral language effects comes from studies of comprehensive or multi-dimensional curricula that included some oral language activities but were not focused on language specifically. The findings for impacts on oral language skills were inconsistent across studies.

Phonological Awareness

Phonological awareness is a component of the broader skill area of phonological processing, which includes not only the child's awareness of sounds, but also the ability to hold sounds in memory and to be able to access sounds from memory. Phonological awareness refers to the child's understanding that words are made up of smaller sounds that can be manipulated, combined and separated. This knowledge helps children understand the relationship between written language (letters) and spoken language (sounds). Research has established that phonological awareness develops in the preschool period starting with sensitivity to words and moving toward sensitivity to smaller and smaller units of sound (syllables, onset-rime, and phonemes).

Phonological awareness has been shown to be a strong predictor of reading success. At the same time, there is inconsistent evidence of our ability to impact children's phonological awareness skills. The strongest evidence comes from research on individual literacy curricula with some explicit attention to sounds in language, including two curricula studies in Project Upgrade and a couple of individual PCER studies of literacy curricula. In these evaluations, the children receiving the literacy curricula scored higher on a test of sound blending and elision. No evidence of an effect on phonological awareness was found in the Head Start Impact evaluation or the Early Reading First evaluation, possibly because local programs vary widely in the extent to which instruction incorporates an intentional and consistent focus on sounds.

Print Knowledge

Research indicates that print and letter knowledge are strongly related to later reading performance. Children's knowledge of the alphabet when they enter school is one of the single best predictors of later reading achievement, most likely because the ability to recognize and distinguish individual letters is a necessary precursor to learning the sounds that the letters represent. Overall, this component of early literacy is the one most often targeted by interventions.

The majority of the interventions reviewed targeted children's print knowledge as an essential skill and there was consistent evidence that the interventions were effective in improving children's print and letter knowledge. This included the large national early childhood studies, where there was substantial variation across sites in the programmatic activities and individual curricula.

Early Math

As argued by Ginsburg et. al, the fact that the intervention research on early math lags behind the research on early literacy can be explained at least partially by the long-held belief that young children are not able to understand mathematics in complex ways, and that even "everyday" mathematical skills cannot be cultivated in children as young as preschool. As research has built the case showing just the opposite, early math concepts are now central in early childhood education standards, and comprehensive early childhood curricula include deliberate, organized activities to promote understanding of mathematical concepts.

What are the early mathematical concepts that children should acquire in preparation for school? There does not appear to have been extensive conversation among math educators and researchers about what mathematical concepts constitute school readiness. Across the curricula that have been developed, there are similarities in the content areas, however, including:

• basic aspects of number and operations,
• geometric shapes,
• spatial relations,
• measurement, and
• patterns and logic.

The paper discusses six mathematics curricula for preschool on which impact research has been conducted in the United States and two with research from New Zealand. The paper also considers results from research on mathematics activities included as part of comprehensive curricula. As described in the paper, the curricula have different learning objectives and use a variety of materials and approaches, including games, story books, activities, manipulatives, and computer software; stand-alone activities and other activities. Across the various curricula and approaches, most had statistically significant impacts of at least moderate size. Since no two curricula studies used the same measure, it is difficult to compare effectiveness. Further, the research is not useful for determining which aspects of the instruction were most powerful in improving children's math knowledge. Long-term follow-up data also appear to be missing.

Final Thoughts/Future Research

The current set of research summarized in the three syntheses has moved the field forward in some respects. Until recently, there has been almost single-minded focus on language and literacy, which has conferred benefits in terms of the relative breadth and depth of knowledge we have for that domain. The current research reflects a new priority on socioemotional skills, especially self-regulation, and this has opened up new funding opportunities and new intervention designs, which are crucial for our ability to develop our knowledge base in this domain. Early math is also now receiving more scrutiny, although the research base is much more limited.

The current set of research studies does not address directly the critical over-arching issues of what constitutes school readiness, the developmental trajectory of the component skills in readiness, and the long-term benefits of early skill development in both the academic and social domains. The lack of a definition of readiness makes it difficult to summarize the findings from a large set of research studies, since different studies not only use different measures of the same construct but also assess a different set of constructs. Not only does this hinder comparisons, it also limits our ability to understand whether an intervention has broad or narrow effects on children's school readiness.

Nor is the research designed to yield supportable conclusions about the relationship between specific environmental inputs (intentional teaching, materials, technology) and child outcomes that go beyond simple correlations, for example, through systematic planned variation studies or through complex analyses such as instrumental variable analysis.

There also is a clear need for more longitudinal research on the development of children's early skills in all three domains, at least through preschool and into the early elementary grades.

All three synthesis papers note that future research will need to more clearly delineate the sources of variation in impact, as well as the overall impact. Potential factors include characteristics of students as well as of teachers and of the intervention itself.

The field is attempting to simultaneously develop effective, research-linked interventions, deliver them with high fidelity in a variety of education settings, use valid, reliable measures of what are often complex psychological constructs, and contribute to building a knowledge base on instructional practices. Despite the sometimes disappointing findings, we need to understand the difficulty of designing effective interventions to be implemented in real-life educational settings, with groups of at-risk children.

References

Angrist, J., Imbens, G, and Rubin, D. (1996). Identification of causal effects using instrumental variables." Journal of the American Statistical Association. 91(434) 444-55.

Gilliam, W.S. & Zigler, E.F. (2000). A critical meta-analysis of all evaluations of state-funded preschool from 1977 to 1998: implications for policy, service delivery and program evaluation. Early Childhood Research Quarterly, 15(4), winter 2000, 441-473.

Gorey, K.M. (2001) Early childhood education: A meta-analytic affirmation of the short- and long-term benefits of educational opportunity. School Psychology Quarterly, 16 (1), 9-30.

Jacob, R., C. Creps & B. Boulay (2004). Meta-analysis of research and evaluation studies in early childhood education. Abt Associates Inc., Cambridge, MA.

National Early Literacy Panel. (January 2008). Report of the National Early Literacy Panel. National Institute for Literacy & National Center for Family Literacy. Washington, DC.

Nelson, G. & A. Westhues (2003). A meta-analysis of longitudinal research on preschool prevention programs for children. Prevention & Treatment, 6, 31.

Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children. Washington, D.C.: National Academy Press.

Whitehurst, G. J. & Lonigan, C. J. (1998). Child development and emergent literacy. Child Development, 69, 848-872.

Endnotes

1. The traditional method used to link interventions to outcomes, even in the context of randomized designs, is based on OLS regression, in which variation in implementation is correlated with variation in outcomes. Another experimental approach uses instrumental variable analysis. This approach has been applied in many contexts by economists and is becoming increasingly popular for use with randomized experiments (for example, Angrist, Imbens and Rubin, 1996). It does not compare existing levels of student achievement and instructional practice. Instead it leverages the fact that a high-quality randomized experiment (or a well-executed regression discontinuity analysis) can produce unbiased estimates of program impacts on classroom instructional practices and on student test scores. The approach thus examines the association between outcomes that is implied by a pattern of program impacts. For example, in a randomized study, the analysis compares program-induced changes in student outcomes with program-induced changes in classroom instruction, where both changes are estimated using the randomized design. Under certain conditions, this analysis can provide internally valid (statistically consistent) estimates of the causal effect of classroom instruction on student performance. This methodology overcomes some of the problems in relational analysis (omitted variables and attenuation bias), although instrumental variable analysis depends on being able to show that there are no mediators additional to the classroom instruction that could account for the relationships with child outcomes.

2. The fact that parallel effects were not demonstrated on teacher ratings of children's attention and impulsivity may be related to power rather than to inconsistency in outcomes. Teacher ratings have been found to have higher correlations among children in the same classroom and center (ICCs) than do cognitive measures such as the PPVT. This means that only relatively large impacts can be detected for the teacher-reported outcomes,

Approaches to Promoting Children's School Readiness:  A Review of Federally-Funded Research Initiatives Aimed at Improving Young Children's Language and Literacy Skills in Early Education and Care Settings

This paper is part of a series of working papers prepared for a meeting sponsored by the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE). Abt Associates Inc and the National Center for Children in Poverty (NCCP) were funded to convene the meeting. The views represented in this paper are those of the author(s) and do not necessarily reflect the opinions of the U.S. Department of Health and Human Services.

Introduction

An examination of research in the field of early childhood language and literacy development reveals substantial changes over the past two decades. Initially, a shift in the conceptualization of what constitutes literacy and when literacy begins resulted in a burgeoning corpus of research that examined children's literate experiences before the beginning of formal schooling. This perspective, termed emergent literacy, brought a new and vigorous focus to the developmental precursors of formal reading that originate in children's early years, thus broadening the scope of research to the years prior to formal schooling, that is, into the early childhood years.

Although research in the field of emergent literacy has been diverse both in topic and methodology, there is currently consensus about the key elements that are foundational to learning to read: oral language, phonological processing, and print awareness (Whitehurst & Lonigan, 2001). Research has provided empirical evidence of the relationships between these early skills and later reading abilities. For example, numerous research studies have demonstrated that early, well-developed oral language skills are a strong predictor of later reading abilities (e.g., ECCRN, 2005; Hart & Risley, 1995; Walker, Greenwood, Hart, & Carta, 1994; Storch & Whitehurst, 2002; Dickinson & Porsche, 2008; Spira, Bracken, & Fischel, 2005; Tabors, Roach, & Snow, 2001). Similarly, children who are sensitive to the sounds in words and are able to manipulate and use them are more likely to be successful in learning to read (Snow, Burns & Griffin, 1998; Pullen & Justice, 2003; Whitehurst & Lonigan, 2001) because these abilities are strongly related to decoding abilities. Finally, in terms of print awareness, studies have shown that a child's knowledge of the alphabet when they enter school is one of the single best predictors of later reading achievement (Snow, Burns & Griffin, 1998; Whitehurst & Lonigan, 2001). The ability to recognize and distinguish individual letters, as well as knowing the sounds of the language, together form the foundation for learning the sound-symbol association.

The importance of successfully mastering these skills for young children cannot be underestimated since limited early literacy skills tend to translate into persistent deficits. For example, Tabors, Snow, & Dickinson (2001) found stability between relative levels of reading performance in kindergarten and seventh grade, while Cunnigham & Stanovich (1997) found the same stability between first grade and the end of high school. Therefore, the effect of poor language and literacy abilities in early childhood can be cumulative, such that children who are behind early on continue to fall further and further behind more skilled readers in reading as well as in other academic areas (Chall, Jacobs, & Baldwin, 1990). Furthermore, evidence indicates that it is very difficult to remedy children's language and literacy difficulties with compensatory programs (McGill-Franzen & Allington, 1991), particularly after third grade (Good, Simmons, & Smith, 1998). Of particular policy relevance is the fact that children of lower socio-economic status are at high risk for reading difficulties. These children tend to begin school with less-developed abilities in the three foundational skills of early literacy than their more economically advantaged peers. Thus, interest in effective interventions to improve children's early language and literacy skills is motivated in large part by the possibility of narrowing the school readiness gap.

One argument for focusing on providing comprehensive support for children's development of early language and literacy skills comes from economists such as Lynch (2004), who have conducted cost-benefit analyses that support the idea that the benefits of substantial investment in early interventions, in terms of increased educational attainment and income earnings outweigh the costs of these investments. Similarly, Reynolds (2005) found that early interventions are the most cost-effective method to make positive contributions to at-risk children's development.

Another argument comes from evidence that most children are able to achieve grade-level reading levels if they receive effective early reading instruction (Clay, 1985; Iverson & Tunmer, 1993; Pinnell, 1989; Snow, Burns, & Griffin, 1998; Wasik & Slavin, 1993). If this is indeed the case, then perhaps it is deficiencies in teachers' instruction, rather than in children's cognitive abilities that explains the large number of reading difficulties in U.S. schools (Dickinson, McCabe, & Clark-Chiarelli, 2004). Although parents are children's first and foremost teachers, more and more children are spending a large portion of their waking hours with adults in early childhood settings. Recent research has lent support for the idea that teachers' instructional practices can make a difference in children's outcomes. For example, Huttenlocher, Vasilyeva, Cymerman & Levine (2002) found a positive association between teachers' use of complex syntax and preschoolers' comprehension of complex syntax. More importantly, they found that classroom input made up for the lack of home input for children from disadvantaged backgrounds.

Thus, based on the benefits of attending to children's deficits in language and literacy before formal schooling, the lack of success remedying these difficulties after school entry, and the high cost of not doing so for later academic achievement, educators and policy makers have turned their attention to the possibilities of improving children's skills early on. Because over half of 3- to 5-year-old children in the United States - 57% in 2005 - spend time in early childhood care and education programs, including day care centers, Head Start programs, preschools, nursery schools, or prekindergartens (U.S. Department of Education, 2006), there has been a focus on reaching the many children who are in these settings.

However, despite substantial investments by federal and state governments in early childhood center-based programs such as Head Start, Even Start and public pre-kindergarten, until recently, little rigorous research had been conducted on the effectiveness of various curricula used to improve children's early language and literacy skills in these programs. It was against this backdrop that the federal government, through various agencies, funded rigorous evaluations of multiple curricula that focused on language and literacy, as well as other important school readiness skills. The Preschool Curriculum Evaluation Research (PCER) and the Interagency School Readiness Consortium (ISRC) consortia are two such federal sources that have provided funding for rigorous evaluations of curricula used in early childhood programs.

This review provides a synthesis of the emerging findings from this set of major federal research initiatives. We examine the evaluations of program enhancements funded through PCER, ISRC, and the Evaluation of Child Care Subsidy Strategies, as well as evaluations of federal early childhood programs  National Evaluation of Early Reading First and Head Start Impact Study  in terms of key issues in the field of young children's language and literacy development prior to formal schooling. For the PCER interventions, both the cross-site evaluation and individual papers (when available) were reviewed. For the ISRC interventions, the evaluations of which were funded later, there is no cross-site evaluation and most study teams had only reported initial findings in the form of conference presentations rather than journal articles. Therefore, the review of the ISRC interventions should be considered preliminary as findings are still emerging from this work. After synthesizing the set of studies, some possible directions for future research are suggested based on this body of research.

Key Issues in Early Childhood Language and Literacy

Below, we synthesize the findings from the studies reviewed, which examine the effects of different early care and education interventions on teacher and child outcomes. This paper focuses solely on child outcomes. We begin by discussing the evidence that federally-funded research on early childhood language and literacy-specific curricula has provided in terms of identifying effective interventions for improving young children's oral language, phonological sensitivity, and print knowledge skills  the three foundational skills upon which later literacy is based. We then discuss what this body of research has added to our understanding of some of the key factors that moderate the effectiveness of intervention programs. It should be noted that a challenge in reviewing this body of research was that most interventions were broad-based, encompassing many different components. This meant that, in this set of studies, when positive effects on child outcomes were found, it was often not possible to determine which of the many components was contributing to these effects. Fortunately, a more extensive review to be released soon  the National Evaluation of Early Literacy (NELP)  will be able to provide some insight into this question.

What Evidence Is Provided About Improving Children's Skills in Oral Language, Phonological Sensitivity, and Print Knowledge?

Oral Language

As more and more young children spend large portions of their time with teachers in early education settings, the quality of teacher language use plays a critical role in driving children's early language development. For example, studies have demonstrated that cognitively challenging conversations that address decontextualized or relatively abstract topics are particularly beneficial to children's language development (Dickinson, 2001a, 2001b; Dickinson & Smith, 1994). Unfortunately, not all teachers can provide high quality conversations, comments or questions. This is especially true with those underpaid or poorly prepared teachers serving low-income children in publicly funded programs. A descriptive study within the PCER initiative (Massey, Pence, and Justice, 2008) confirmed these prior findings about teacher talk by examining the quality and quantity of teacher questions in 14 preschool classrooms (both treatment  Language-Focused Curriculum  and control) serving economically disadvantaged 4-year-olds. They found that questions characterized one third of all teacher utterances, with management questions (e.g., "Are we ready?") occurring most frequently (44.8%), followed by more cognitively challenging questions (32.5%; e.g., "What do you think will happen next?") and less cognitively challenging questions (22.7%; e.g., "What was this called?") That is to say, more cognitively challenging questions represented only one tenth of all teacher utterances in the at-risk preschool classroom. They further examined the frequency of use for different types of questions across various classroom contexts and found that more cognitively challenging questions occurred most frequently in storybook reading. Unfortunately, according to Dickinson (2001a), only 1% to 4% of the total day is typically spent on storybook reading in early care and education settings.

The aforementioned findings naturally raise the following question: Are curricula that extend storybook reading time more effective in promoting children's language development? Several studies within the ISRC and PCER initiatives examined the effects of curricula that include interactive reading activities in the daily plan. The Head Start REDI (Research-Based Developmentally Informed) program developed a curriculum featuring interactive reading activities based on shared reading and dialogic reading, providing teachers with scripted books and targeted vocabulary and instructing teachers to elicit children's language more effectively and to be more responsive. A randomized control trial was employed to compare 4-year-olds in the intervention condition and a similar group in non-intervention Head Start classrooms. The post-intervention tests showed that, after being exposed to the intervention for seven to eight months (September to March/April), children in the treatment group outperformed the comparison group on both vocabulary and language use at home (with effect sizes of .15 and .25, respectively), but no effect emerged on measures of children's grammatical understanding. Similar results were obtained from other curricula that integrated interactive reading activities into the curriculum, such as Children's School Success (CSS) and Literacy Express (LE). HLM analyses demonstrated that CSS improved children's vocabulary and language use at home through changing teacher practice (e.g., more sensitive-responsive talk, richer talk, better instructional support). Teacher practice accounted for 53% and 67% respectively of the intervention effect on vocabulary and language use at home. LE was found to have a significant impact on expressive communication skills (ES = .30) and a potentially positive effect^[1] on vocabulary (ES = .45). Over all three measures used in the oral language domain, the average effect size was .36.

In addition, dialogic reading or reading-aloud was an important component of three other early childhood curricula  Breakthrough to Literacy (BTL), Ready, Set, Leap! (RSL), and Building Early Language & Literacy (BELL). Despite sharing common purposes, these three curricula differ in activity designs and implementation. Both BTL and RSL are comprehensive language and literacy programs that include activities throughout the day. BTL is built around a series of weekly books with a focus on interactive reading; while RSL utilizes interactive electronic technology and thematically-grouped children's trade books. In contrast, BELL, as an add-on pre-kindergarten literacy program, entails only two daily 15- to 20-minute lessons. The Project Upgrade study compared the curricula to each other and a business-as-usual control group. The results revealed that RSL and BTL had significant impacts on children's definitional vocabulary (ES = .30), even though the impacts were not large enough to reduce the gap (see below). On the other hand, BELL, the less intensive curriculum, yielded no significant impacts on any measures of early language and literacy. Taken together, findings from these studies may suggest that curricula with a focus on interactive reading activities do exert positive impacts on children's oral language development, given enough dosage of implementation.

Even though interactive reading seems to be an effective ingredient to improve oral language, not all curricula put an emphasis on interactive book reading. Instead, some PCER/ISCR curricula provide specific and explicit instructions to teachers to foster frequent and long high quality conversations that use complex syntax and address abstract concepts. For instance, the Language-Focused Curriculum (LFC), designed for preschoolers with language limitations, identified specific linguistic targets (e.g., verb phrase structures, adjective, pronouns, etc.) in daily lesson plans and instructed teachers to use a set of Language Stimulation Techniques (LSTs) to foster the delivery of linguistically-responsive conversations with children. In a study with a random-control trial design, Justice, Mashburn, Pence, Wiggins (under review) analyzed children's 10-minute language samples gathered in the fall and spring, with the amount of professional development that teachers received matched in the intervention group and comparison group. However, they found no impacts of LCF and LST exposure on children's expressive language skills. Instead, the results demonstrated that children who attended preschool more frequently benefited more from the LCF curriculum and LST exposure compared to those with low attendance (no effect size reported). This finding is not unexpected: if a child does not go to class frequently, how can s/he benefit from the curriculum? From another angle, this finding aligns with the result of the Project Upgrade study that showed no impact of the lower dosage curriculum, BELL.

Two other curricula, Let's Begin with the Letter People and Doors to Discovery, also provide teachers with a detailed plan of the scope and activities that are developmentally appropriate to enhance literacy development. This plan provides specific instructions to help teachers determine group size, sequence instructional goals, and match appropriate materials with learning objectives. Both curricula are thematically based and involve the use of learning centers in the classroom. Despite the similarity, Let's Begin with the Letter People has a particularly strong emphasis on letter knowledge and phonological awareness while Doors to Discovery (DD) puts a strong emphasis on language. In an experimental study funded under PCER, Assel and his team (Assel, Landry, Swank, & Cunnewig, 2006) examined the effectiveness of the two curricula across three different settings (Head Start, Title 1, and Universal pre-K classrooms) and included a control group in each setting, in which teachers used teacher-developed, nonspecific curricula. The results revealed that both of the intervention curricula demonstrated similar effectiveness. The auditory comprehension and vocabulary skills of children in classrooms using either of these two curricula grew more than children in control classrooms, but this effect was moderated by program site (Head Start versus Title I versus Universal pre-K).^[2] For auditory comprehension, children in Head Start showed the greatest gains compared to children in control classrooms, while for vocabulary, children in Head Start and Title I classrooms showed the greatest gains. Because their primary interest was to identify differences in the rates of growth of child skills over time, the authors acknowledge that their design did not control for differences in children's baseline scores. It was the case that universal pre-K children consistently showed higher initial scores than children in the other two programs, and Title I children outscored Head Start children. Therefore, differences in gains could be due to the fact that the Head Start children, who started with lower baseline scores, had more room to grow.

Two large national evaluations also demonstrated mixed results on children's oral language outcomes. The National Evaluation of Early Reading First, using a regression discontinuity (RD) design, evaluated the effect of additional funding for teacher professional development on teacher, classroom, and child outcomes. A variety of curricula were used in funded and non-funded early childhood sites, however, teachers in the funded sites received more professional development in all areas (language & literacy, assessments, and child development and behavior) than teachers in the non-funded sites. The program demonstrated positive impacts on teachers' language use and book reading practices in the funded classrooms. However, no significant impacts were found on children's oral language skills, as measured by the Expressive One-Word Picture Vocabulary Test or the Auditory Comprehension subscale of the Preschool Language Scale-IV. These findings mirror those from the recent Reading First Interim Study (Gamse, Bloom, Tepper, & Jacob, 2008), which also used an RD design to examine the effects of a federal funding stream at the K-3 level. Although the study found positive effects on teacher instructional practice, those effects did not translate into positive effects in student achievement. On the other hand, the National Head Start Impact Study found small positive impacts on 3-year old children's vocabulary scores (effects sizes in the .10 to .20 range).

Results of the PCER cross-site evaluation, however, were disappointing with respect to oral language. It should be noted that the lack of effects in the PCER cross-site evaluation could be due in part to small sample sizes, to the timing of the baseline testing, which sometimes occurred later than the baseline testing done by the individual evaluations (Assel et al., 2006), or to differences in measures (Justice et al., under review). Only two of twelve curricula were found to have positive impacts on children's oral language skills in either pre-K or kindergarten: DLM Early Childhood Express with Open Court Reading Pre-K (DLM) and the Early Language and Literacy Model (ELLM). For ELLM, effects were found only in kindergarten (not in pre-K), a surprising finding given that 11 of the 14 ELLM teachers were in their second year of implementation of the curriculum at the time of the evaluation. Effect sizes for both curricula were medium and similar in kindergarten for both curricula on the PPVT and the TOLD (Effect sizes range from .34 to .48), while in pre-K, DLM's effects were in the .40 range. One similarity across these curricula is the fact that both ELLM and DLM are implemented in combination with already comprehensive early childhood curricula and provide teachers with ongoing professional development and support, possibly indicating that the amount of curricular support to teachers needs to be fairly substantial in order to obtain effects on children's outcomes.

In sum, these recent federally-funded research initiatives, although far from conclusive, have provided some confirmatory evidence that children's oral language outcomes can be improved when teachers engage in and provide children with more complex language activities and opportunities. The fact that effect sizes for oral language were medium (according to Cohen, 1988), and not small, is a hopeful finding. The positive results, however, may be moderated by numerous factors, including the instructional support for the teacher, the dosage received by the child, and the program site, which in many cases serves as a proxy for other characteristics of children and teachers in those sites, including baseline test scores, poverty status, or teacher experience. Future research should focus on identifying more concretely the factors that need to be put in place to obtain consistent oral language gains, as well as the size of the effect that is needed to ensure success in reading comprehension.

Phonological Sensitivity

As stated above, the ability to distinguish and manipulate sounds is a strong predictor of reading success. Phonological awareness has been well documented for its critical role in learning to read (e.g., Gunning, 2000; Juel, 1994; Shu, Anderson, & Wu, 2000; Snow, Burns, & Griffin, 1998). Children who are more aware of the different sounds in words and are able to separate or combine sounds are more ready to learn to read and write. Studies have found that explicit instruction in phonological awareness can reduce the incidence of reading failure and thus improve the possibility of reading success (Adams, 1995; Stanovich, 1993; Snow et al., 1998).

In general, less evidence was found that the interventions studied through recent federally-funded research initiatives exerted positive impacts on children's phonological awareness skills than was found in terms of oral language. Neither of the two national evaluations included in this review, of Early Reading First and of Head Start, found effects on children's phonological awareness skills. Similarly, in the PCER cross-site study, 11 of 12 interventions showed no statistically significant effects in this domain (but note that possible limitations for the PCER cross-site evaluation listed above for the oral language domain apply for the phonological awareness domain as well). Only one intervention  DLM  was found to have positive effects in pre-K and kindergarten as measured by the Pre-CTOPPP (pre-K) or the CTOPP (kindergarten) with effect sizes ranging from .32 to .38.

In contrast to the PCER cross-site evaluation findings, however, the individual evaluations of several curricula indicated some positive effects on children's phonological awareness skills. As mentioned above, differences in findings between the cross-site evaluation and the individual evaluations could be due in part to small sample sizes, differences in the timing of baseline testing, or to differences in measures. For example, Literacy Express was found to have an average positive effect size of .63 in the phonological processing domain, as measured by the P-CTOPPP Blending and Elision subtests at the end of pre-K (Lonigan, 2006). Similarly, the Project Upgrade study demonstrated that Ready, Set, Leap! (RSL) had a significant impact on children's phonological awareness skills at the end of pre-K as measured by the TOPEL (ES = .39, when compared to the control group jointly with another intervention, Breakthrough to Literacy). In a study of Let's Begin with the Letter People and Doors to Discovery (Assel et al., 2006), children in classrooms receiving either curriculum showed greater gains in rhyming skill than those in control classrooms, as measured by the Woodcock-Johnson-3 Sound Awareness subtest (d = .26). Additionally, there were differences in rates of growth by curricula that were moderated by program site, such that universal pre-K classrooms using Let's Begin had higher rates of growth than those using DD by an effect size margin of .85. No differences, however, were found in children's rates of growth between the two curricula in Head Start and Title I classrooms (Assel et al., 2006). The same caveats mentioned above apply to these findings, that is, since Head Start and Title I children began with lower baseline scores than those in Universal pre-K, they may have been more likely to gain at a faster rate .

Some preliminary findings from the ISRC consortium are in line with the aforementioned findings. For example, in the Head Start REDI study, significant impacts on phonological awareness were found (ES =.39 for Blending subtest of the TOPEL, .35 for Elision subtest). This curriculum provided professional development for teachers that focused on implementing sound games (three times per week). The evaluation of Children's School Success found an interaction effect between pretest scores and quality of implementation on children's early literacy outcomes, including phonological awareness. The study found that children who scored lower on pretest measures benefited more from high implementation and less from low implementation of the curriculum.

In sum, recent federally-funded research initiatives have provided mixed evidence of the studied curricula's effectiveness to improve children's phonological sensitivity skills. This lack of consensus could be due to methodological issues such as statistical power or differences in measurement of these skills. Or, it also could be the case that gains in this area are difficult to effect. Future research needs to address these methodological issues so as to produce more conclusive results. In addition, as with oral language, moderating factors  such as dosage, children's pre-test scores, and program site  are cited in these studies. Planned variation studies would be an important addition to further clarify the role of these moderators of intervention effectiveness.

Print Knowledge

In line with the core research about the essential role of print and letter knowledge for later literacy success (e.g., Clarke, 1988; Clay, 1991; Torgeson & Davis, 1996; Whitehurst & Lonigan, 2001), the majority of the interventions reviewed targeted children's print knowledge as an essential skill. The goal of these interventions was to improve children's print and letter knowledge skills through training teachers how to a) explicitly teach these skills, and/or b) provide children with opportunities to practice these skills. Was there evidence that the interventions were effective in improving children's print and letter knowledge? Although not entirely consistent, the majority of interventions that targeted this area showed some evidence of positive effects. The national evaluations of ERF (U.S. Department of Education, 2007) and Head Start (U.S. Department of Health & Human Services, 2005) both had positive impacts on children's print knowledge. Head Start reduced, by almost half (47%) the gap in children's ability to recognize letters between Head Start children and the national average for all 3- and 4-year olds. Similarly, the impact of ERF on children's print and letter knowledge was 5.78 standard score points on the Pre-CTOPPP print awareness subtest (ES = .34).

The PCER cross-site evaluation conducted by Preschool Curriculum Evaluation Research Consortium (2008) indicated positive impacts for only two curricula of eleven that focused on children's language and literacy development  Curiosity Corner (CC) and DLM. The former curricula had an impact in kindergarten, while the latter had impacts in both pre-K and kindergarten. Of the three measures used, CC demonstrated positive impacts on the TERA and the WJ Letter Word Identification subtest (ES = .43 for both), while DLM had positive impacts on all three measures in pre-K (the TERA, the WJ Letter Word Identification subtest and the Spelling subtest) equaling effect sizes of .68, .51, and .46 respectively. In kindergarten, DLM had impacts only on the TERA and the WJ Letter Word Identification subtest (effect sizes equaled .76 and .50 respectively).

Of the nine remaining curricula that did not demonstrate statistically significant impacts in this domain in the cross-site evaluation, five were studied in individual evaluations and were found to have positive effects (ELLM, Let's Begin, DD, Literacy Express, and Ready, Set, Leap!). For example, the individual evaluation of ELLM suggests that the curriculum, which focuses on instructional strategies and learning materials for teachers to explicitly teach literacy skills and provide structured literacy experiences, had small, positive effects on measures of letter knowledge, print conventions, and meaning of print at the end of prekindergarten in favor of the intervention (effect sizes equaled .25, .28, and .26 respectively). By the end of kindergarten, positive effects were found only on letter knowledge (ES = .34). Similarly, Let's Begin with the Letter People and Doors to Discovery were both found to have positive effects on Head Start children's print knowledge skills, compared to children in Title I or Universal pre-K classrooms (ES = .53 for HS, versus .06 for Title I and .25 for Universal pre-K). The measure used in the study was the WJ-3 Letter Word Identification subtest. In the case of Literacy Express, the curriculum demonstrated statistically significant positive effects on children's skills in this domain, as measured by several assessments - the TERA-3 Alphabet subtest, the TERA-3 Meaning subtest, and the WJ-3 Spelling subtest (effect sizes equaled .57, .83 and .50 respectively). On two other measure - the P-CTOPPP Print Knowledge subtest and the TERA-3 Print Conventions subtest, impacts were not statistically significant, but were large enough by WWC standards to be substantively meaningful (effect sizes equaled .41 and .34 respectively). Finally, in the Project Upgrade study (U.S. Department of Health and Human Services, 2007), RSL, along with BTL had significant impacts on children's print knowledge skills, as measured by the Print Knowledge subtest of the TOPEL (ES = .63).

In summary, the majority of curricula evaluated seem to have been able to exert positive effects in the area of print knowledge across varied assessments and conditions, however there is much more to be done. The more extensive NELP review should provide more insight into common features across interventions that show effects on children's print knowledge. Future research would also benefit from moving beyond establishing the link, as done in the ERF evaluation, that more time spent by teachers on print awareness opportunities is related to children's higher print awareness scores, to identifying more effective ways to teach children alphabetic knowledge. For example, in one non-experimental descriptive study funded by PCER (Justice, Pence, Bowles & Wiggins, 2006), findings based on children in classrooms using either the Language-Focused Curriculum or High/Scope indicated that the order of letter learning was not random and that some letters hold an advantage over others to influence their order of learning. The authors suggest that perhaps early care and education teachers should teach more difficult, less known letters first, since children are more likely to know more common letters. Teachers should also account for individual differences since children know different letters, depending on both extrinsic and intrinsic influences.

What Evidence Is Provided About Factors that Moderate Intervention Effectiveness?

A review of some of the interventions evaluated for this review points to the range of activities/components that are often implemented with the goal of producing positive changes in children's early language and literacy outcomes. For example, ELLM includes five components: research- and standards-based literacy curriculum, family involvement, professional development, working partners, and practice-focused research and evaluation. The interrelationships among these components and their interdependence were prominent, and were discussed in almost every study that was reviewed for this paper. When these comprehensive curricular approaches are implemented in early childhood settings, which are dynamic and complex learning environments in themselves, it becomes difficult to tease out the critical features for success from the wide range of possible influences. Yet, is important to understand what factors might be moderating the effectiveness of interventions. Because variation in these factors was not a focus of this body of research  the aim of which was to provide evidence of effectiveness of the interventions studied, on average  researchers were not always able to address questions about moderating factors. In addition, most analyses of moderators were conducted outside an otherwise experimental design, and as such, cannot be considered causal. Despite these limitations, in the research reviewed in this paper, some, mostly non-experimental evidence was provided regarding three possible critical factors that the studies suggest may be important moderators of intervention effectiveness: professional development, dosage of implementation, and child background characteristics.

Professional Development/Coaching

Before implementing the specific curriculum, teachers (and sometimes other educational staff) usually received professional development or training on how to deliver the intervention. Some interventions also provided ongoing coaching to monitor or refresh ideas and to solve problems rising during ongoing implementation. Professional development may affect the impact of an intervention through changing teachers' practice and fidelity. Using non-experimental methods, the LFC study showed that treatment teachers exhibited strikingly high fidelity to the curriculum immediately following a professional development workshop (Pence, Justice, & Wiggins, in press). This aligns with the findings of an evaluation of Building Language and Literacy in Montgomery County public schools (Ramey, Ramey, Kleinman, Lee, Farneti, Timraz, Nielsen, et al., 2008 unpublished manuscript), which compared two coaching conditions: weekly versus monthly. It revealed that weekly work-embedded coaching significantly improved implementation levels of the curriculum and yielded significant positive impact on children's literacy skills (ES = .44). These contrasts were tested within the experimental design and indicate that sufficient professional development may be related to the success of an intervention.

Professional development can even compensate for the insufficiency in teachers' educational background. The Project Upgrade study (U.S. Department of Health and Human Services, 2007) analyzed (outside of the experimental design) the observational data from study classrooms, and, surprisingly, instead of finding an educational background effect, the results demonstrated that the interventions eliminated the differences between better-educated teachers and less-educated teachers. Teachers in the treatment group all looked remarkably similar, regardless of their educational levels, compared with the dramatic differences among control group teachers. In other words, the professional development that treatment group teachers received and the well-specified curricula diminished the differences in teaching instruction due to teacher educational background. Similarly, another group of researchers (Lieber, Goodman-Jansen, Horn, Palmer, Hanson, Czaja, Butera, et al., 2007) examined 30 Head Start teachers in implementing the CSS curriculum and found that coaching and teachers' motivation to change, rather than teaching experience or degree status, affected curriculum implementation. These analyses were correlational, and outside of the experimental framework.

Dosage of Implementation

Program dosage can be measured in days of children's attendance during the academic year. When measured in this way, greater program dosage has been found to be related to stronger program impact. For example, in a study of two state public pre-K programs, Ramey, Ramey, and Stokes (year not provided) found that children who received the full day and full school year LA4 program (Louisiana) gained nearly twice as much from the program as their peers who received only the half-year LA4 program (pilot year) or the half-day full year Montgomery County Public Schools program (Maryland). Similarly, in an experimental study of LFC, researchers found that children who attended early care and education regularly benefited more from the intervention than those with low attendance rates (Justice, Mashburn, Pence, Wiggins, under review). It can be inferred that children who attended school more regularly were exposed to a higher dosage of the intervention compared to those who attended school less regularly.

Program dosage can also be thought of in terms of the amount of time that has been allotted for the curriculum to be implemented. In the Project Upgrade study (U.S. Department of Health and Human Services, 2007), the three curricula being compared  RSL, BELL, and BTL  all focused on the development of early literacy skills and knowledge. However, they were distinguished from one another in terms of instructional approach, materials provided, intensity and cost. Both RSL and BTL are full-day comprehensive curricula; BELL is an add-on literacy program entailing only two 15-20 min sessions daily. The finding that both RSL and BTL had significant impacts on all literacy measures compared to the lack of impacts of BELL suggests that dosage of the intervention should account for part of the differences in impacts on children's outcomes. This is more persuasive considering that BELL had a much stronger focus on phonological awareness than the other two curricula, yet had no impact on children's phonological awareness while RSL and BTL did.

Child Background Characteristics

Findings of some studies also revealed that child background characteristics (such as family economic status, pretest performance, personality, and language ability) may moderate the impacts of interventions. The research demonstrated that the interventions were effective for all children, but were particularly effective for some children. For example, children who were more economically or academically disadvantaged were found to have gained more from interventions than their more advantaged peers (Assel et al, 2006; Ramey, Ramey, & Stokes, 2008 unpublished manuscript; Odom, Diamond, Hanson, et al., 2007). In a study that examined the contributions of child characteristics to the quality of teacher-child relationship, Rudasill, Rimm-Kaufman, Justice and Pence (2006), in their study of LFC, demonstrated that individual differences in child temperament and language skills affected teacher-child interactions, which ultimately contributed to intervention effect. This was especially true for early language and literacy curricula, in which teacher-child conversations are often key cornerstones of the implementation.

English Language Learner (ELL) status is also a very important factor when considering children's early language and literacy skills. Although most studies included ELLs in their study samples, results were not often reported by subgroup. This was perhaps due to power issues, since most studies were not powered to detect subgroup differences. An exception is the LA ExCELS (Los Angeles: Exploring Children's Early Learning Settings) study which explored ELL children' experience in early care and education settings and their school readiness outcomes (Fuligni, 2008). Preliminary results showed that low income bilingual Spanish children were behind monolingual peers in several language and school readiness domains during pre-kindergarten period. There were no differences in experiences of Spanish speaking and English monolingual children in early care and education programs at age 4. However, participation in early learning settings was particularly beneficial for Spanish speaking children. This is consistent with the aforementioned pattern that academically disadvantaged children benefited more from interventions or programs compared to their advantaged peers.

Child background in terms of family factors also includes family literacy environment and parent behavior. These issues were addressed by the Getting Ready Nebraska program. In this program, several studies examined the effects of home literacy and parents' belief or behavior on children's development. In a study investigating adolescent parents' participation in learning and their perceptions of professional support, Knoche, Woods, & Sheridan (2008) found that for children whose parents demonstrated low levels of parent learning behaviors, high levels of professional support were associated with higher scores in young children's language skills.

In sum, the federally-funded evaluation studies reviewed here provide support and replicate previous findings about factors that may be important as moderators of intervention effectiveness. However, many questions remain. For example, how much professional development is optimal? What amount of dosage of intervention is needed for children to progress? What interventions work best for which children? One way to address this in the future would be to conduct planned variation studies, in which hypotheses about "how much" and "for whom" can be tested. From this data, threshold levels for professional development and dosage, for example, could be more clearly understood and ultimately be used to inform intervention developers and policy makers.

Directions for Future Research

In addition to the specific suggestions for future research at the end of each section above, there are several comments regarding future research that apply in general based on the review of this corpus of research. From a substantive point of view, more focused attention should be paid to the needs of some subgroups of children, especially ELLs. As aforementioned, although most studies did a commendable job of including a diverse population of children in their studies, impacts on subgroups were seldom examined. In part, this may be an issue of research design, since effectiveness studies are designed to provide an overall mean, and are often not powered to be able to detect subgroup differences. However, future research should certainly focus on the specific needs of these children, who make up more and more of the population of children in early care and education settings.

Another substantive issue that should be addressed in future research is trying to determine the active ingredients in those interventions for which positive effects were found. Because most of the PCER interventions, for example, incorporated multiple components, when effects were found, it was not possible to identify which component had led to the positive effect. More fine-tuned research would be able to disentangle the effects of various components and move the field forward in terms of identifying the most critical ingredients of interventions. In addition, the NELP, a much more extensive review, should be able to provide further insights into this question.

From a methodological perspective, it was quite remarkable that there were more than a dozen randomized controlled trial (RCT) studies of early childhood interventions to review. On the one hand, the national push for more rigorous research in the field has certainly increased the number of RCTs that have been implemented and, in this way, has improved the rigor of the research available. On the other hand, effectiveness research studies utilizing RCT designs have their own set of limitations. For example, in terms of statistical power, it is clear that in order to detect the types of effects on children that we would expect across one school year, sample sizes must be fairly large. Although RCTs require fewer units of randomization than say, regression discontinuity designs, it is still the case that in order to detect small effects, sample sizes must typically be in the range of 60 units with nested designs (observations within children within teachers, for example). Since randomization often occurs at the center level to avoid contamination across teachers within the same center, this can be quite a challenge for most researchers. One way to decrease sample size requirements is to conduct random assignment at the child level. This alternative, however, is not always practically or pragmatically feasible.

In addition, there is a trade-off between internal and external validity. Although the strength of RCTs is their high internal validity, they can suffer from low external validity. Especially in early care and education settings, when researchers are often limited to creating their study samples based on those who agree to participate from their overall recruitment efforts, generalizability can still be quite limited and therefore less policy relevant.

Meaningful detectable effect is another methodological issue that arises after reading these studies. In general, effect sizes were reported in terms of Cohen's d, and Cohen's guidelines for what is considered small (.20), medium (.50), and large (.80) are used. However, unless the author reports what the range of the assessment is and what the expected growth across a school year is, it is difficult to make a judgment about the substantive meaning of a .20 versus a .50 versus a .80 effect size. What does this mean in real world terms? What is a meaningful effect size? How does that vary by assessment or domain? Without diminishing the advances made in the field in the reporting of effect sizes, it would be helpful to also report a translation of Cohen's d into assessment-relevant terms, such as months of growth.

Finally, the PCER and ISRC initiatives have certainly made huge strides in terms of providing examples of conducting evaluations of programs and practices in real world settings. Lessons learned from these initiatives will make an important contribution to the field, both substantive and methodological. Lessons learned could address the wealth of knowledge of the implementers after having done these studies; suggest possible hypotheses for effects, or lack of effects, on child outcomes; and provide direction for future rigorous studies, of which there are certain to be more.

From a policy perspective, the issue of cost was not addressed in any of the studies that were reviewed. In line with the suggestion above regarding cost-benefit analysis in terms of achieving positive child outcomes, research on the cost of implementing the interventions would be useful for policy makers and educators.

Summary

This preliminary review of the published and unpublished papers on these federally-funded intervention evaluations suggests that there is evidence for positive effects of some of the selected interventions on some of the important early childhood language and literacy outcomes. However, evidence from these studies is not sufficient to inform policy makers about ways in which to assemble the critical ingredients necessary for more widespread and consistent success in raising young children's literacy outcomes. Many inter-related factors influence the effectiveness of interventions, some of which are just beginning to be understood. In addition, the experimental studies supported by these federal initiatives have proven that it is possible to conduct rigorous studies in early childhood settings and have moved the field forward in terms of methodology. Ongoing improvements and attention to new issues arising from these more rigorous methodologies will, however, be necessary. Both in terms of substance and methodology, therefore, the studies examined here constitute an important contribution to the knowledge base that informs early language and literacy education. Research on the characteristics of high quality programs that are both developmentally appropriate and successful in bridging the achievement gap will be in demand from legislators and policy makers as they are called upon to make informed decisions about early learning systems.

References

Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press.

Assel, M. A., Landry, S. H., Swank, P. R., & Gunnewig, S. (2007). An evaluation of curriculum, setting, and mentoring on the performance of children enrolled in pre-kindergarten. Reading and Writing, 20, 463-494.

Bierman, K. L., Domitrovich, C. E., Nix, R. L., Gest, S. D., Welsh, J. A., Greenberg, M. T., Blair, C., Nelson, K. E. & Gill, D. (in press). Promoting Academic and Social-Emotional School Readiness: The Head Start REDI Program (Final Revision). Child Development.

Bierman, K. L., Nix, R. L., Greenberg, M. T., Blair, C. & Domitrovich, C. E. (2008). Executive functions and school readiness intervention: Impact, moderation, and mediation in the Head Start REDI Program. Development and Psychopathology, 20, 821-843.

Bryant, P., MacLean, M., & Bradley, L. (1990). Rhyme, language, and children's reading. Applied Psycholinguistics, 11, 237-252.

CEC, Liousville, K. Y. (2007, April 20). Children's School Success (CSS): Child Outcomes from Three Years of Research. Power point slides.

Chall, J. S., Jacobs, V., & Baldwin, L. (1990). The reading crisis: Why poor children fall behind. Cambridge, MA: Harvard University Press.

Clarke, L. K. (1988). Invented versus traditional spelling in first graders' writings: Effects on learning to spell and read. Research in the Teaching of English, 22, 281-309.

Clay, M. M. (1985). The early detection of reading difficulties (3rd ed.). Portsmouth, NH: Heinemann.

Clay, M.M. (1991). Becoming literate: The construction of inner control. Portsmouth, NH: Heinemann.

Cohen, J. (1988). Statistical power analysis for behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.

Cosgrove, M., Fountain, C., Wehry, S., Wood, J., and Kasten, K. (2006, April). Randomized Field Trial of an Early Literacy Curriculum and Instructional Support System. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, California.

Cunningham, A. E., & Stanovich, K. E. (1997). Early reading acquisition and its relation to reading experience and ability 10 years later. Developmental Psychology, 33, 934-945.

Dickinson, D. K. (2001a). Large-group and free-play times: Conversational settings supporting language and literacy development. In D. K. Dickinson & P. O. Tabors (Eds.), Beginning literacy with language: Young children learning at home and school (pp. 223-255). Baltimore, MD: Brookes Publishing.

Dickinson, D. K. (2001b). Putting the pieces together: The impact of preschool on children's language and literacy development in kindergarten. In D. K. Dickinson & P. O. Tabors (Eds.), Beginning literacy with language: Young children learning at home and school (pp. 257-287). Baltimore, MD: Brookes Publishing.

Dickinson, D. K., & Porche, M. (2008). The relationship between teacher-child conversations with low-income four-year olds and grade four language and literacy development. Manuscript submitted for publication.

Dickinson, D. K., & Smith, M. W. (1994). Long-term effects of preschool teachers' book readings on low-income children's vocabulary and story comprehension. Reading Research Quarterly, 29, 104-122.

Dickinson, D. K., McCabe, A. & Clark-Chiarelli, N. (2004). Preschool-based prevention of reading disability: Realities vs. possibilities. In C. A. Stone, E. R. Silliman, B. J. Ehren, & K. Apel (Eds.), Handbook of language and literacy: Development and disorders (pp. 209-227). New York: Guilford Press.

Dickinson, D., Darrow, C., Ngo, S., & D'Souza, L. A. (in press). Narrowing the gap between potential and possibility: The challenge of changing classroom conversations. Submitted for publication in: O.A. Barbarin, P. Frome, & D. Marie-Winn (Eds.), The Handbook of Developmental Science and Early Schooling: Translating Basic Research into Practice.

Fuligni, A. S. (2008, May). School readniness of English-speaking and English-learning children: Links with experiences in early learning settings. Power Point slides.

Gamse, B.C., Bloom, H.S., Kemple, J.J., Jacob, R.T., (2008). Reading First Impact Study: Interim Report (NCEE 2008-4016). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Good, R., III, Simmons, D. C., & Smith, S. (1998). Effective academic interventions in the United States: Evaluating and enhancing the acquisition of early reading skills. School Psychology Review, 27, 740-753.

Gunning, T. G. (2000) Phonological awareness and primary phonics. Boston: Allyn & Bacon.

Hart, B., & Risley, T. (1995). Meaningful differences in the everyday lives of American children. Baltimore, MD: Brookes Publishing.

Huttenlocher, J., Vasilyeva, M., Cymerman, E. & Levine, S. (in press). Language input and child syntax. Cognitive Psychology.

Iversen, S., & Tunmer, W. (1993). Phonological processing skills and the Reading Recovery program. Journal of Educational Psychology, 85, 112-126.

Juel, C. (1994). Learning to read and write in one elementary school. New York: Springer-Verlag.

Justice, L. M., Cottone, E. A., Mashburn, A., & Rimm-Kaufman, S. E. (2008). Relationships between Teachers and Preschoolers Who Are At Risk: Contribution of Children's Language Skills, Temperamentally-based Attributes, and Gender. Unpublished manuscript. University of Virginia.

Justice, L. M., Mashburn, A., Pence, K. L. & Wiggins, A. (2008). Experimental Evaluation of a Preschool Language Curriculum: Influence on Children's Expressive Language Skills. Journal of Speech, Language, and Hearing Research, Vol.51, 983-1001.

Justice, L. M., Pence, K., Bowles, R. B., & Wiggins, A. (2006). An investigation of four hypotheses concerning the order by which 4-year-old children learn the alphabet letters. Early Childhood Research Quarterly, 21, 374-389.

Knoche, L. L., Sheridan, S. M., Cline, K., Givens, J.A. & Fleissner, S. (2006, June). Moderating the effects of risk on children's school readiness: What are the roles of family literacy and parent sense of competence? Poster session presented at the annual National Research Conference of Head Start, Washington, DC.

Knoche, L. L., Woods, K. E., & Sheridan, S. M. (2008). Adolescent parents' participation in learning: Factors contributing to their children's development. Manuscript submitted for publication.

Layzer, J.L., Layzer, C.J., Goodson, B.D., & Price, C. (2007). Evaluation of child care subsidy strategies: Findings from Project Upgrade in Miami-Dade County. Final report prepared for U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research, and Evaluation, Washington, DC.

Leather, C. V., & Henry, L. A. (1994). Working memory span and phonological awareness tasks as predictors of early reading ability. Journal of Experimental Child Psychology, 58(1), 88-111.

Lieber, J., Goodman-Jansen, G., Horn, E., Palmer, S., Mansen, M., Czaja C., Butera, G., Daniels, J., & Odom, S. (2007, April). Factors that influence the implementation of a new curriculum: Results from two years of implementation. Poster presented at the Biennial Meeting of the Society for Research in Child Development. Boston, MA.

Lonigan, C. J. (2006, July). Impact of preschool literacy curricula: Results of a randomized evaluation in a public prekindergarten program. Paper presented at the 13th annual meeting of the Society for the Scientific Study of Reading, Vancouver, British Columbia, Canada.

Lonigan, C. J., Farver, J. M., Clancy-Menchetti, J. & Phillips, B. M. (2005, June). Promoting the development of preschool children's emergent literacy skills: A randomized evaluation of a literacy-focused curriculum and two professional development models. Paper presented at the 12th annual meeting of the Society for the Scientific Study of Reading, Toronto, Ontario, Canada.

Lynch, R.G. (2004). Exceptional returns: Economic, fiscal, and social benefits of investment in early childhood development. Washington, DC: Economic Policy Institute.

Massey, S. L., Pence, K. L., & Justice, L. M. (2008). Educators' Use of Cognitively Challenging Questions in Economically Disadvantaged Preschool Classroom Contexts. Early Education and Development, 19(2), 340-360.

McGill-Franzen, A., & Allington, R. L. (1991). The gridlock of low reading achievement: Perspectives on practice and policy. Remedial & Special Education, 12(3), 20-30.

NICHD Early Child Care Research Network (2005). Pathways to reading: The role of oral language in the transition to reading. Developmental Psychology, 41, 428 - 442.

Odom, S. L., Diamond, K., Hanson, M., Lieber, J., Butera, G., Horn, E., Palmer, S., Marquist, J., & Fleming, K. (April, 2007). Children's School Success: Treatment Dosage and Child Outcomes. Poster presented at the Biennial Meeting of the Society for Research in Child Development. Boston, MA.

Pence, K. L., Justice, L. M., & Wiggins, A. K. (in press). Preschool Teachers' Fidelity in Implementing a Comprehensive Language-Rich Curriculum. Language, Speech, and Hearing Services in Schools.

Pinnell, G. S. (1989). Reading recovery: Helping at risk children learn to read. Elementary School Journal, 90, 161-184.

Preschool Curriculum Evaluation Research Consortium (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington, DC: U.S. Government Print Office.

Pullen, P., Justice, L. (2003). Enhancing phonological awareness, print awareness, and oral language skills in preschool children. Intervention in School and Clinic. Austin: Vol. 39, Iss. 2; 87.

Ramey, C. T., Ramey, S. L., and Stokes, B. R. (2008). Effective pre-K programs: Research evidence about program dosage and student achievement. Unpublished manuscript. Washington, DC: Georgetown University.

Ramey, S. L., Ramey, C. T., Kleinman, B.E., Lee, L.M., Farnetti, C.C., Timraz, N.M., et al. (2008). The Effects of Curriculum and Coaching Supports on Classrooms and Literacy Skills of Prekindergarten/Head Start students in Montgomery County Public Schools. Unpublished manuscript. Washington, DC: Georgetown University.

Reynolds, A. J. (2005). Confirmatory program evaluation: Examples from early childhood programs. Teachers College Record, 107, 2401-2425.

Rudasill, K. M., Rimm-Kaufman, S. E., Justice, L. M., & Pence, K. (2006). Temperament and Language Skills as Predictors of Teacher-Child Relationship Quality in Preschool. Early Education and Development, 17(2), 271-291.

Shu, H., Anderson, R. C., & Wu, N. (2000). Phonetic Awareness: Knowledge of orthography-phonology relationships in the Character acquisition of Chinese children. Journal of Educational Psychology 92(1), 56-62.

Snow, C. E., Burns, S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children. Washington, D.C.: National Academy Press.

Spira, E. G., Bracken, S. S., & Fischel, J. E. (2005). Predicting improvement after first-grade reading difficulties: The effects of oral language, emergent literacy, and behavior skills. Developmental Psychology, 41(1), 225-234.

Stahl, S. A., & Murray, B. A. (1994). Defining phonological awareness and its relationship to early reading. Journal of Educational Psychology, 86(2), 221-234.

Stanovich, K. (1993). Does reading make you smarter? Literacy and the development of verbal intelligence. Advance in Child Development and Behavior, 25, 133-180.

Stanovich, K. E. (1986) Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360-407.

Stevenson, H. W., & Newman, R. S. (1986). Long-term prediction of achievement and attitudes in mathematics and reading. Child Development, 57, 646-659.

Storch, S. A., & Whitehurst, G. J. (2002). Oral language and code-related precursors to reading: Evidence from a longitudinal structural model. Developmental Psychology, 38, 934-947.

Tabors, P. O., Roach, K. A., & Snow, C. E. (2001). Home language and literacy environment: Final results. In D. K. Dickinson & P. O. Tabors (Eds.). Beginning literacy with language: Young children learning at home and school (pp. 111-138). Baltimore, MD: Brookes Publishing.

Tabors, P. O., Snow, C. E., & Dickinson, D. K. (2001). Homes and schools together: Supporting language and literacy development. In D. K. Dickinson & P. O. Tabors, Beginning literacy with language: Young children learning at home and school (pp. 313-334). Baltimore, MD: Brookes Publishing.

Torgesen, J. K., & Davis, C. (1996). Individual difference variables that predict response to training in phonological awareness. Journal of Experimental Child Psychology, 63, 1-21.

Torgesen, J. K, Wagner, R. K., & Rashotte, C. A. (1997). Prevention and remediation of severe reading disabilities: Keeping the end in mind. Scientific Studies of Reading, 1, 217-234.

U.S. Department of Education, National Center for Education Statistics. (2006). The Condition of Education 2006 (NCES 2006-071). Washington, DC: U.S. Government Printing Office.

U.S. Department of Education. (2007). National Evaluation of Early Reading First. (NCEE 2007-4007). Washington, DC: National Center for Education Evaluation and Regional Assistance. U.S. Government Print Office.

U.S. Department of Health & Human Services. (2005). Head Start Impact Study: First Year Findings. Administration for Children and Families. Washington, DC.

Vellutino, F. R., & Scanlon, D. M. (2001). Emergent literacy skills, early instruction, and individual differences as determinants of difficulties in learning to read: The case for early intervention. In S. B. Neuman & D. K. Dickinson (Eds.), Handbook of early literacy research (pp. 295-321). New York: Guilford Press.

Wagner, R .K., & Torgesen, J. K. (1987). The natural of phonological processing and its causal role in the acquisition of reading skills. Psychological Bulletin, 101, 192-212.

Walker, D., Greenwood, C., Hart, B., & Carta, J. (1994). Prediction of school outcomes based on early language production and socioeconomic factors. Child Development, 65, 606-621.

Wasik, B. A., & Slavin, R. E. (1993). Preventing early reading failure with one-to-one tutoring: A review of five programs. Reading Research Quarterly, 28, 178-200.

Wehry, S., Cosgrove, M., & Fountain, C. (2006). Preschool-to-Kindergarten: A Longitudinal Study of the Effectiveness of the Early Literacy and Learning Model (ELLM). Poster.

Whitehurst, G. J., & Lonigan, C. J. (2001). Emergent literacy: Development from prereaders to readers. In S. B. Neuman & D. K. Dickinson (Eds.), Handbook of early literacy research (pp. 11-29). New York: Guilford Press.

Endnotes

1. Potentially positive is a rating given by the What Works Clearinghouse (WWC) indicating that although the difference between the treatment and control groups was not statistically significant, the effect size was large enough to be considered substantively important according to WWC criteria (i.e., at least .25).

2. The authors note that program site was confounded with child ethnicity (i.e., more Hispanic children in Head Start and Title I versus Universal pre-K) so that controlling for site in their design essentially controls for child ethnicity.

School Readiness and Early Childhood Education: What Can We Learn from Federal Investments in Research on Mathematics Programs?

This paper is part of a series of working papers prepared for a meeting sponsored by the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE).В  Abt Associates Inc and the National Center for Children in Poverty (NCCP) were funded to convene the meeting.В  The views represented in this paper are those of the author(s) and do not necessarily reflect the opinions of the U.S. Department of Health and Human Services.

Introduction

Programs designed to promote young childrens school readiness have focused primarily on language and social emotional development.  While these remain important skills for young children to acquire, there is a growing awareness that readiness for mathematics is also critical. Promoting school readiness for mathematics is particularly important for low-income and/or ethnic minority children who are at greater risk for beginning kindergarten with markedly lower math skills (Lee & Burkham, 2002).  In fact, recent research shows that childrens mathematics ability at kindergarten-entry is a better predictor of future academic success than their reading achievement (Duncan et. al., 2007).  Preschool and prekindergarten programs can buffer children against school failure (Bogard & Takanishi, 2005) and prepare young children for success in primary school mathematics (Arnold & Doctoroff, 2003; Bogard & Takanishi, 2005; Goldbeck, 2001).  Considerations like these have led many states to develop early learning standards for mathematics.

In spite of the evidence that early childhood education is the most promising and cost-effective way to positively affect the development of children at-risk for later school failure (Reynolds, 2005), there has been widespread reluctance to teach mathematical concepts to young children.  This is because many mathematics educators were not convinced that young children could learn these concepts and because it was unclear how best to teach them (Perry & Dockett, 2002).  In fact, some early childhood educators continue to resist the use of any planned teaching or curricula given their long held beliefs that young children need to learn on their own in a child-centered holistic environment and that deliberate teaching is not developmentally appropriate (Golbeck, 2001).  In addition, many teachers own fear of math is an obstacle to their willingness to teach mathematics (Ginsburg, Lee, & Boyd, 2008).  The result has been that mathematics education has traditionally not figured prominently in early childhood education programs.  For example, two major early childhood programs that account for a large portion of the market, Creative Curriculum and High/Scope, have traditionally not emphasized a comprehensive mathematics curriculum.  However, both of these programs are in the process of expanding their mathematics offerings.

The historical reluctance to teach mathematics to young children stands in stark contrast to research showing that young children can understand mathematics in complex ways.  While it was once thought that young children were incapable of abstract or logical thought because they were in Piagets preoperational stage, recent research shows that young children can understand basic aspects of number and operations, geometric shapes, spatial relations, measurement, and patterns (Ginsburg, Lee, & Boyd, 2008; Perry & Dockett, 2002).  Childrens everyday mathematical skills can be cultivated and extended at this age level in ways that support a more advanced understanding of mathematical concepts (Ginsburg, Lee, & Boyd, 2008).

In response to the recent research findings demonstrating that young children are eager learners of everyday mathematics, leading mathematics and early childhood education professional organizations now stress the importance of deliberate early childhood mathematics education (National Association for the Education of Young Children and National Council of Teachers of Mathematics, 2002).  Their position is that curricula providing organized activities designed to promote students understanding of mathematical concepts can be used in a deliberate manner by teachers, while still allowing children the opportunity to play and explore the world flexibly (Ginsburg, Lee, & Boyd, 2008; Perry & Dockett, 2002).  This approach to early mathematics education fits into prevailing views of quality early childhood education: children should play and be taught, and both should occur in a warm, and nurturing environment.

The goal of this paper is to examine the effectiveness of new research based mathematics curricula that attempt to respond to the call for organized programs of mathematics learning for young children.  Given that relatively little rigorous research on preschool mathematics programs has been conducted  whether federally-funded or not  this paper will review research that has been supported by a number of different funding streams:  federally-funded studies that were part of the PCER and ISRC initiatives; federally-funded Head Start research; studies funded through other federal programs, including the Institute for Education Science (IES) and the National Science Foundation (NSF); as well as foundation-funded research based in the U.S. and international research.  All of the studies reviewed include pre-kindergarten or preschool-aged children (e.g., children who are approximately four years old) in their samples.  These children may be attending organized programs like Head Start, or may be in other preschool or child-care center settings. In addition, all of the studies focused on improving the math skills of children from low-income families as these children are most at risk for beginning formal schooling with a poorer understanding of mathematics than their non-poor peers.

The first section of the paper focuses on mathematics-specific curricula whose development and/or evaluations have been supported by the federal government, as well as two curricula that were developed or evaluated by other funding sources.  The second section will review federally-funded research on comprehensive curricula that include a mathematics component.  In each of these sections, we will identify the funding stream and, when applicable, the research initiative supporting the research. The paper concludes with a discussion of what the research does and does not tell us at this point, and recommends directions for future research that would better illuminate the processes of teaching and learning that support mathematics learning in early childhood settings, as well as research designed to determine which underlying components of curricula and implementation are beneficial under the varying preschool and childcare settings that serve children most at risk for starting school with academic skills that lag behind those of their peers.

What Can We Learn from Federally-Funded Research on Early Childhood Mathematics Curricula?

Although leading professional organizations call for research-based curricula, the meaning of research based is a bit problematic. A restrictive definition might be that the curriculum should derive directly studies that focus on how mathematics should be taught. By this criterion, almost no programs would qualify. The designs of early childhood mathematics curricula are based on research investigating the development of childrens mathematical thinking in the absence of instruction, not from teaching experiments. Thus, a more accurate definition of research-based curricula is one that is inspired by research on young children and attempts to translate the research into an organized program of teaching. The danger with this definition is that it can be over-inclusive. Publishers in particular may claim that their programs meet whatever standards are in place at the moment and, not surprisingly, will advertise that virtually any curriculum for young children is research-based (or developmentally appropriate or whatever the slogan of the day may be). Their goal is sales, not scientific rigor.

Our approach is not to take too seriously the claim of a basis in research. After all, the major question is not whether the program derived from research but whether it is effective. Sometimes, practical applications precede and indeed inspire scientific investigation (Stokes, 1997). A creative curriculum developer may have a hunch, possibly based on some informal exploration, that an activity might work, and indeed it might. The issue is not whether the program is research-based but whether it has been evaluated and is shown to be effective in improving learning outcomes. While to date there have been few rigorous studies examining the effectiveness of mathematics curricula for young children (National Research Council, 2004; D. Clements & Sarama, 2008), the studies that have been conducted indicate that young children from low-income families can indeed benefit from curricula designed specifically to address mathematics learning.

Federally-Funded Cluster Randomized Studies of Mathematics-Specific Curricula: PCER, IES, and NSF Research

Federal dollars have supported the rigorous evaluation of three mathematics-specific early childhood curricula, although the evaluation of each has been supported by a different funding stream. An intervention consisting of the Pre-K Mathematics Curriculum (PreK Math; Klein, Starkey, & Ramirez, 2002) supplemented with the DLM Early Childhood Express Math software (DLM; D. Clements & Sarama, 2003) was evaluated as part of the Institute for Education Sciences (IES) Preschool Curriculum Evaluation Research program (PCER; PCER Consortium, 2008). Development and evaluation of the Building Blocks curriculum (Sarama, 2004; D. Clements & Sarama, 2003, 2008) has been supported by the National Science Foundation. Building Blocks is a designed for use with children as young as three-years-old. The evaluation of the Big Math for Little Kids curriculum (BMLK; Greenes, Ginsburg, & Balfanz, 2004) was supported by a research grant from IES (M. Clements, Lewis, and Ginsburg, 2008). BMLK was developed for use by pre-kindergarten and kindergarten students.^[1]

The three curricula share a number of characteristics, including the types of professional development offered to teachers, the contexts in which the curriculum is designed to be taught, and the scope of the curricula. It is important to note that the similarities noted here do not represent precise similarities across the curricula, but rather broad characteristics that they share. The specific representation of each of these characteristics certainly varies across the three curricula, possible in meaningful ways that result in differences in their effectiveness.

Professional development activities were a component of the treatment condition in the rigorous cluster randomized studies used to evaluate each of the curricula. All three of the evaluations included at least one intensive workshop on the curriculum before the beginning of the school year. Each of the interventions was also supported throughout the course of the study with regularly scheduled, periodic professional development sessions for teachers. These ranged from bi-weekly, one-on-one sessions in a teachers classroom to bi-monthly refresher courses in which groups of teachers met to review particular aspects of the curriculum.

Another shared characteristic of the curricula is that all are designed to utilize multiple contexts for teaching mathematics. In terms of at school activities, the three curricula include whole class learning activities and small group activities. The curricula also incorporate information and activities designed to be sent home for parents and children to work on together at home.

A third characteristic shared by the these three curricula is that each was designed to be a comprehensive mathematics curriculum covering multiple important mathematics domains , such as numbers, counting, and operations; shapes (geometry); measurement; and pattern. Its important to note that here we are referring to very broad mathematical domains and that the specific content and emphasis of each curriculum may well vary. The major point is that each curriculum sets out to cover multiple important mathematical domains rather than just number and operations or just shapes. See Table 1 for a comparison of the domains covered (broadly defined) by curricula.

The PreK Math/DLM, Building Blocks, and BMLK curricula also differ in several possibly important ways. While it is true that the three curricula cover many of the same (broadly defined) mathematics domains, it is certain that the specific topics covered within each domain, the scope of coverage for each topic (in terms of depth and/or breadth), the types of activities and lessons developed to teach each topic, and the ways in which various topics and/or domains are integrated with each other varies across the curricula. Investigating the extent of this variation is beyond the scope of this paper. However, a review of published reports and descriptions of the curricula provide some information about these differences. For example, both PreK Math/DLM and Building Blocks incorporate regular use of computer software, while BMLK does not include a software component.

The findings of the rigorous evaluations of the developmentally appropriate mathematics-specific curricula stand in stark contrast to the findings from the Head Start Impact Study. The Head Start Impact study compared children randomly assigned to attend Head Start to a control group of children who, for the most part, attended some other type of center-based care on a number of cognitive domains. Among four-year-olds, the study found a statistically significant positive impact of Head Start attendance on four of eight language-related cognitive domains, but no difference in early math skills (U.S. Department of Health and Human Services, 2005). Given that two of the mathematics curricula reviewed above were evaluated in Head Start classrooms, it appears that the Head Start Impact Studys lack of significant findings regarding math is due to the dearth of effective early childhood mathematics curricula, not Head Start. In fact, the studys final report points to the need for effective early childhood mathematics curricula and teacher professional development in math education (US DHHS, 2005).

While all three of these curricula have been rigorously evaluated using cluster randomized trials, including variation in the length of the studies and the age of children in the study samples, mathematics outcome measures used in the evaluations, and the types of classrooms settings in which the mathematics curricula were evaluated. As well discuss below, these differences make it difficult to compare the relative effectiveness of the curricula, other than to conclude that all three demonstrate effectiveness in improving childrens understanding of mathematics. The PreK Math/DLM and Building Blocks evaluation studies examined the impact of each curriculum over the course of childrens pre-kindergarten year, and the research took place in a combination of Head Start classrooms and state-funded prekindergarten classrooms. The BMLK evaluation, on the other hand, examined the curriculums impact over the course of childrens pre-kindergarten and kindergarten years among children attending child care centers that are subsidized by the New York City Administration for Child Services and, thus, didnt include either Head Start classrooms or state-funded pre-kindergarten programs.

Another important difference across the studies is that each evaluation utilized a different mathematics assessment as the outcome variable. Both PreK Math/DLM and Building Blocks used assessments developed by the curriculums developer (and evaluators), neither of which is nationally normed. Both sets of authors clearly articulate that their assessment is not overly aligned with the curriculum; they are designed to evaluate childrens understanding of the concepts taught, but do not use the same activities and materials that are part of the curriculum. At the suggestion of IES, BMLK used the mathematics assessment developed for the Early Childhood Longitudinal Study, Birth Cohort (ECLS-B; National Center for Education Statistics) which is nationally normed. Each of the assessment procedures (using versus not using an assessment designed to evaluate a particular curriculum) has its strengths and weaknesses. On the one hand, using an assessment designed to evaluate a particular curriculum is likely to be able to provide a more nuanced understanding about what concepts the curriculum was more (or less) successful at teaching. On the other hand, an assessment that has not been nationally normed will not provide information about how children compare to their peers throughout the country both before and after being taught with the curriculum.

Results from the studies indicate that all three curricula were effective in promoting childrens mathematics learning. The effect sizes for the studies were .43 for BMLK  (compared to control classrooms; M. Clements et al., 2008), .55 for PreK Math/DLM (compared to control classrooms; Klein, Starkey, D. Clements, Sarama, & Iyer, in press), and for Building Blocks the effect sizes were 1.07 (for the comparison with control classrooms) and .47 (for the comparison with classroom using PreK Math/DLM^[2]; Clements & Sarama, 2008). See Table 2 for additional details on the evaluation studies. In all three studies, the control classrooms used a variety of curricula, including Creative Curriculum, High/Scope, Montessori, or other local curricula.

We should note that the difference between the BMLK  and control students did not become statistically significant until the second year of the study (the year following pre-kindergarten^[3]), while the Building Blocks and PreK Math/DLM studies found statistically significant differences at the end of the pre-kindergarten year. This could be due to several factors. One possibility is that our use of the ECLS-B math assessment (a standardized test that was designed as a general assessment of mathematics learning, and not developed to test a particular mathematics curriculum) resulted in a stricter test of the curriculums effectiveness and, as a result, additional months of exposure to the curriculum were necessary before differences in childrens learning could be detected by this assessment. A second possibility is that more than seven months of exposure to the curriculum were necessary before group differences emerged. In our opinion, the fact that the overall math achievement of children in the study was near the national median score throughout the course of the study rules out a third possibility: that the curriculum is too advanced for preschoolers and not appropriate until children reach kindergarten.

An advantage of using the ECLS-B math assessment in the BMLK evaluation is that we were able to determine the extent to which BMLK helped reduce the achievement gap between children from low-income families and the national average. Specifically, in the fall of pre-kindergarten the average student score on the ECLS-B was at the 48^th percentile, but increased to the 56th percentile by the end of the prekindergarten year, and was at the 55^th percentile at the end of the kindergarten year.

In summary, evaluations of all three curricula demonstrated that they are effective, with effect sizes ranging from moderate to large. Furthermore, the fact that their effectiveness was demonstrated across a variety of classroom contexts (Head Start, state-funded prekindergarten, and NYC ACS subsidized child care centers) suggests that these comprehensive mathematics curricula are likely to be effective in promoting mathematics-related school readiness among children from low-income families.

A Federally-Funded Cluster Randomized Study of a Comprehensive Curriculum: ISRC research

Among the studies that were funded as part of the Interagency School Readiness Consortium, only the Childrens School Success curriculum (CSS) included a component designed specifically to advance childrens mathematics knowledge. Odom and his colleagues (Leiber et al., 2007; Odom et al., 2007a and 2007b) refer to CSS as an early childhood education model designed to combine science, language, literacy, math, and social skills into a meaningful learning experience. The mathematics component of the curriculum is described as being adapted from D. Clements and Saramas Building Blocks curriculum, but details regarding the extent to which the curriculum was modified are not provided.

Based on research reports (consisting of slides and posters from conference presentations), it is difficult to discern whether or not CSS was effective in promoting more advanced mathematics knowledge among the children attending Head Start centers where it was implemented. Analyses for this study are still underway, and, to date, none of the presentations provide statistical results demonstrating the curriculums effectiveness. However, there are multiple presentations that examine the impact of treatment fidelity and childrens attendance rates on childrens math achievement scores. As would be expected, fidelity of implementation is associated with higher student math scores at the end of the school year.

Other Early Childhood Mathematics Research

In light of the fact that only three early childhood mathematics curricula have been subjected to federally-funded rigorous evaluations, this research review will briefly review additional curricula. They include curricula that were evaluated by non-federal funds and/or by study designs that were not as methodologically rigorous or extensive as those for Building Blocks, PreK Math/DLM, and BMLK.

The National Science foundation has funded recent research on the Round the Rug curriculum (Casey, 2004), which is a supplementary language arts-based curriculum designed to promote childrens understanding of key mathematical concepts including pattern, geometry (shape), measurement, and graphs. The curriculum consists of six books that teachers use to lead lessons that integrate oral story-telling with hands-on mathematics. The impact of one of the lessons (on geometry) has been evaluated in a smaller-scale random assignment study involving six kindergarten teachers (Casey, Erkut, Ceder & Young, 2008). This study found that a lesson taught using both the story-telling and hands-on components promoted greater mastery of the material than the hands-on lesson alone.

The Head Start-University Partnership, a program of the US DHHS Administration for Children and Families, has supported research on a preschool mathematics curriculum that Katherine Sophian developed for use with three- and four-year olds. The curriculum consists of weekly activities that parents and teachers are to complete with children. The emphasis of the program is on measurement with various units and exploring the relationships between shapes rather than identifying features of shapes (i.e., the number of sides or angles). The curriculum has been evaluated in a study of three Head Start centers with children ranging in age from 2.5 years old to 4.6 years. This study found that use of the curriculum had a small positive effect on the math scores of children at the end of the year.

Discussion

We consider several sets of questions concerning the effectiveness of the programs and what can be learned from the evaluations of them. We conclude with suggestions for a research agenda.

Questions Concerning the Current Programs

How successful are the programs? A basic finding is that math education, as exemplified by the programs described above, can work for young children. Studies of different curricula find relatively large effect sizes, as indicated above. They were at least fairly successful in accomplishing their various and sometimes diverse goals. There is little doubt that early education can promote early mathematics learning in different areas, including number, shape, space, and pattern. This is valuable information, and it gets the enterprise started: there should be no doubt that early childhood mathematics education can be effective, at least in the short term.

At the same time, there are many questions remaining to be addressed and much that still needs to be learned. One question refers to the differential effectiveness of the programs under consideration. Do some achieve better results than do others? The answer is probably yes, but it is hard to compare programs directly. As we showed, the research studies used a wide variety of outcome measures for evaluation. As a result, it is hard to examine the relative effectiveness of programs (even using effect size) when they are trying to accomplish different goals. One program may be effective in promoting spatial reasoning and another effective in teaching the reading of numerals. It is good that both are effective, but it is hard to compare programs when goals and subject matter differ.

Further, it is important to note that many evaluations use outcome measures developed in conjunction with the goals of the curriculum (e.g., PreK Math/DLM), whereas other programs (e.g., BMLK ) use measures that do not align with the curriculum itself. In a sense, the aligned outcome measures can be considered near transfer tasks and standardized measures, far transfer tasks. The use of an outcome measure that aligns with the curriculum increases the likelihood that the evaluation will find positive effects, but does not indicate whether the treatment group would perform better on mathematical topics not emphasized in the curriculum. The use of far transfer tasks can provide insight into general aspects of learning but provide little useful detail about the specifics. Each approach has strengths and limitations that need to be recognized.

We also need to be clear about the inevitable limitations of the various outcome measures. Although most have reasonably sound psychometric properties, it is fair to say that of necessity the measures generally focus on relatively easy to measure aspects of performance. The results of such an approach are valuable in establishing that some learning has occurred, but the approach often fails to illuminate that learning in any detail. It is conceivable, of course, that a curriculum works, in the sense of promoting high test scores on these kinds of evaluations, but that it does not promote thinking or enhance long-term motivation for learning mathematics. It is conceivable that teachers may teach to the evaluation and in the process fail to promote meaningful learning. High stakes assessment may have negative effects at the preK and kindergarten levels, just as it does at higher levels of education.

How successful are the programs at teaching various topics within the mathematics curriculum? Mathematics is a complex subject, even in preschool. It involves far more than teaching rote aspects of number. The discipline is both wide and deep (Ginsburg & Ertle, 2008), and includes topics ranging from the invariance of cardinal number across various transformations to the idea of mapping physical space. Following the advice of the NAEYC/NCTM, many of the curricula present mathematics as a broad array of topics, including number, measurement, space, shape and pattern. At the same time, the program evaluations generally present little information concerning childrens learning in each of these specific areas. Consequently, we need to know much more about program effectiveness in teaching the very different topics of mathematics, ranging from number to shape and pattern.

In particular, we need to learn much more about a very special topic, namely mathematical thinking and reasoning. Children need to learn to understand why a figure is a triangle, not a rectangle, and to reason about why one operation (like 2 + 3) yields the same result as another (like 3 + 2). Some of the programs seem to promote such mathematical thinking and reasoning, but in general, the evaluations do not attempt provide in depth information concerning thinking and reasoning processes, strategies employed, and understanding of important ideas. One reason is that random assignment studies involving large numbers of children need to employ tests that are easy to administer on a large scale and relatively short. Such tests, although useful for their purpose, are not optimal for measuring cognitive phenomena as subtle and complex as reasoning and understanding. Another reason is that the field lacks appropriate and practical measures of mathematical thinking and reasoning.

In brief, we need to know much more than that a program works. We need to know how it works in the different substantive areas of mathematics, and how it works in the key area of mathematical thinking and reasoning. This kind of information can be of great value for researchers, teachers, and curriculum developers alike.

What aspects of the programs pedagogical methods or materials are most powerful in promoting childrens mathematical learning?  The programs employ various methods and materials. Sometimes they use small groups, and sometimes the use large ones. Sometimes the approach is relatively didactic and sometimes more open-ended. Sometimes they use games, and sometimes stories. Sometimes they use computers, and sometime they do not. Sometimes they do mathematics as a stand-alone activity, and sometimes it is integrated into other activities.

There are many questions to ask about these practices. How effective are the various methods  games, manipulative, stories, and the like, under various circumstances? How should the various methods be used in presenting the material? These of course are the primary issues of interest to teachers who work every day on teaching mathematics.

A crucial set of questions revolves around teaching. Many of the studies attempt to ensure the fidelity of instruction, in the sense of determining whether teachers teach the material more or less as intended. But the studies pay very little, if any, attention to the ways in which teachers implement the activities, incorporate them into their own teaching styles, find some topics easier to teach than others, interpret the materials, adjust teaching to meet student needs, and understand (or misunderstand) the competence of their students. Teachers are at the heart of any program and curriculum, yet the present studies tell us little about their roles in the enterprise.

In general, because of their broad focus on student outcomes, the evaluations typically provide no information about the strengths and weaknesses of various aspects of the programs, or about intentional teaching. As a consequence, the questions about methods, materials and teaching  the questions of most interest to teachers (and creators of professional development programs)  remain unanswered.

What have we learned about group, individual, and developmental differences in childrens mathematics learning?  There are substantial differences between SES groups in mathematics achievement. As is well known, low SES children generally perform more poorly than their middle SES peers. It appears that preschool instruction can be effective for both groups, although it may not eliminate the initial gap between them. But it is important to know whether, how, and to what extent the groups differ in their reactions to and learning from various programs. How do the different groups of children interact with the teachers and activities and does that contribute to the outcomes?

There are also wide individual differences in preschool childrens psychological functioning, language and mathematical knowledge. Some children enter preschool knowing little English. Some have poor executive function. Some may be stronger than others in number (Dowker, 2005). It is conceivable that some children may benefit more than others from a particular pedagogical method or curriculum.

Similarly, there may be important developmental differences in learning mathematics. The old view that preschoolers in general are concrete thinkers, or preoperational and therefore cannot learn an abstract subject like mathematics has been discredited. Nevertheless, there may be important differences between typical 3-year-olds and 4-year-olds in their learning of mathematics. What is the nature of these differences?

In general, the evaluation studies, focused as they are on the measurement of broad outcomes, do not provide information useful for addressing issues of group, individual or developmental differences in learning mathematics.

What can we conclude about effectiveness?  The evaluation research has shown that the various programs are effective in varying degrees in achieving their varied goals. That is important to know, but the research tells us little more than that, perhaps in part because of the very nature and demands of large-scale random assignment research. The research has little to say about relative effectiveness of different programs, about their success in teaching specific topics, about the relative power of different pedagogical techniques and materials, about how teachers teach, and about group, individual and developmental differences in learning.

A Research Agenda

The current evaluation paradigm has taught us a great deal, and has taken a useful first step in the direction of sound early childhood mathematics education. Yet, as we have shown, the paradigm is limited in its ability to answer key questions. The productive solution is not simply more and bigger RCT studies. Instead, we need a new and wide research agenda dealing with several issues fundamental to early mathematics education.

What and how should we evaluate? One set of issues concerns further evaluation of mathematics programs. Now that we know that many of them work, it is important to conduct research targeted to more specific issues, like the relative effectiveness of different kinds of programs for teaching specific content. What are some effective ways for teaching 4-year-olds the analysis of geometric forms or 3-year-olds some fundamental properties of number? How effective are particular materials or pedagogical methods?

In conducting work of this type, the field can benefit from improved outcome measures that tap into essential aspects of learning across the various topics that comprise the content of early mathematics. We need to get beyond using measures because they are convenient or have sound internal or test-retest reliability. The fundamental question is whether they measure what is important to measure. Fortunately, NIH is now funding the development of new research based measures of mathematics knowledge and other topics relevant to early childhood.

And as we go forward, heres a topic that should not receive much research attention: the long-term effects of early mathematics curriculum. Childrens later mathematics outcomes must be influenced by the education children receive after preschool. We know that much of that education, particularly for poor children is lacking, with the likely result that children receiving good preschool math education may not do very well later in school. This outcome is entirely to be expected and does not reflect on the childrens abilities or what is possible to achieve. Hence not much effort need be put into studying it. A more effective approach is to work at improving and evaluating education at all levels.

What are the processes involved in mathematical teaching and learning? A second set of issues revolves around the processes of teaching and learning. Mathematics has seldom been taught at the early childhood level. Consequently we know little about how to teach it or how children learn it. Most of the cognitive developmental research that has provided a revolution in the way we conceptualize young childrens mathematical abilities does not focus at all on teaching or on how children learn from teaching and in an educational context. The various curricula are research-based mostly in the sense that they are inspired by research on childrens mathematical competence, and not in the sense that they derive from the research any particular guidance on how to present or teach any topic. Therefore, we need research, some of which needs to be exploratory, that focuses on teaching and on childrens learning from teaching in an organized setting. Because so little is known about these topics, this kind of research will ultimately be of great practical value to teachers. By contrast, current evaluation research does not speak to teachers about these issues, except to tell them that effective early math education is possible.

How can we effectively implement math curricula? Many early childhood teachers have no interest in early mathematics, fear it, and do not want to teach it, sometimes because of outmoded notions of developmental appropriateness. School districts, preschools, and childcare organizations typically give the teachers little help in their efforts to implement mathematics programs. Several questions then arise: What are the obstacles that stand in the way of successful implementation? How can they be overcome? How can one help teachers to cope with their fears of mathematics and learn effective teaching methods (assuming we learn what those are)? What kind of supports  especially professional development  do teachers need over the long term to implement early mathematics education? In general, the problem is first to set up and then examine the effectiveness of an infrastructure for promoting early mathematics education. In the end, everything boils down to helping and supporting teachers to do good work over the long term.

Additional Information on Mathematics Curricula, Reviewed by Ginsburg, Lewis, & Clements

Reviewed by Ginsburg, Lewis, & Clements

Big Math for Little Kids

The Big Math for Little Kids (BMLK) is a mathematics curriculum designed to facilitate mathematics learning for pre-kindergarten and kindergarten students (Greenes, Ginsburg, & Balfanz, 2004).  The program includes six units (number, shape, measurement, constructing and partitioning numbers, patterns and logic, and navigation and spatial concepts) containing a sequence of enjoyable activities designed to promote both mathematical understanding and language (Greenes, et al., 2004). The program is designed to be used in whole-class and small-group settings, as well as with individual students.  Early field-testing suggested that children taught using the curriculum achieved a high level of mathematical understanding, learned to count to high numbers, were able to take the perspective of others, and anticipated further events and predicted outcomes (Greenes et al., 2004).

The effectiveness of the curriculum has been examined using a two-year randomized controlled trial (RCT) that was funded by the US Department of Education Institute of Education Sciences. The study, which focused on low-income children attending subsidized child care centers in New York City for pre-kindergarten and kindergarten, compared the mathematics achievement of children whose teachers either used the BMLK  curriculum or continued to teach mathematics using the Creative Curriculum (Dodge, Colker, & Heroman, 2002) or a home grown early childhood curriculum. The treatment teachers attended monthly workshops to deepen their understanding of young childrens mathematical learning, as well as to demonstrate important components of the curriculum.

Student achievement was assessed using the mathematics assessment developed for the Early Childhood Longitudinal Study, Birth Cohort (ECLS-B; National Center for Education Statistics) at the beginning and end of their pre-kindergarten year and then again in kindergarten, yielding scores at four time points. The advantage of using the ECLS-B is that it is (1) nationally normed and standardized and (2) that the assessment is not directly aligned with the content of the intervention, providing a stricter test of impact. The norming was conducted using a large stratified random sample including 14,000 children born in 2001 and the measure has high internal reliability (Rock & Pollock, 2002). The test itself is adaptive, meaning that the accuracy of responses determine whether the test taker receives easier or more difficult items, and allows for precise estimation of ability with fewer administered questions.

Preliminary results, using latent growth modeling, are currently available for this study and suggest that children in the BMLK group demonstrate a larger increase in mathematics achievement compared to children in the control group. There were no significant differences between the two groups at the beginning of the study, but by the end of kindergarten these differences emerge with a medium effect size (Cohens d=.43). It should be noted that this study was conducted by the authors of this paper (Clements, Lewis, and Ginsburg, 2008).

Building Blocks

The Building Blocks (funded by the National Science Foundation) mathematics curriculum, designed for pre-kindergarten through 2^nd grade children, is designed specifically to develop competencies detailed in the National Council of Teachers of Mathematics Principals and Standards for School Mathematics (Sarama, 2004). To this end, the curriculum focuses on developing spatial and geometric competencies, as well as numeric and quantitative concepts (Sarama & Clements, 2004). Within these two areas, three mathematical themes are integrated including patterns, data, and sorting and sequencing (Sarama, 2004). In addition to classroom activities, the curriculum relies heavily on the use of computer software, designed as part of the curriculum, to meaningfully engage children as young as 3 years of age in mathematical concepts (Sarama, 2004; Clements & Sarama, 2003). As a result, teachers are required to provide guidance within and between formats.

A randomized controlled trial (RCT) was conducted, comparing three groups of teachers, namely a group using the Building Blocks Curriculum, a second group using the Pre-Kindergarten Mathematics Curriculum (PreK Math; Klein, Starkey, & Ramirez, 2002), and a third control group that experienced whatever teaching was involved in business as usual. These groups included equal numbers of classrooms, some serving low-income students only and other classrooms serving both low-income and middle-income students.

Researchers assessed the impact of the three mathematics curricula using a measure of mathematics ability that was constructed by Clements and Sarama and includes many of the same mathematics activities that are part of the Building Blocks curriculum. The Early Mathematics Assessment (EMA) is administered individually to children during two 10-20 minute interviews, which include detailed protocol, coding, and scoring for the interviewer to follow (Clements & Sarama, 2008). In this study, the interviews were videotaped and recoded to ensure reliability. EMA is a comprehensive assessment of mathematical knowledge, is not aligned with any particular curriculum, and is has high internal reliability (Clements & Sarama, 2008).

Results showed that both the Building Blocks and PreK Math curriculum groups performed significantly better on the EMA measure than the control group and the Building Blocks group performed significantly better than the PreK Math intervention group (Clements & Sarama, 2008). Building Blocks outperformed the control group with a large effect size of 1.07 and outperformed the PreK Math curriculum with a medium effect size of .47. The Pre-Kindergarten Mathematics Curriculum outperformed the control group with a medium effect size of .64. Overall, the program effects for Building Blocks were the same regardless of program type (i.e., Head Start or a state-funded program), classroom socioeconomic (SES) composition, and child-level SES. In other words, there was no evidence that the impact of Building Blocks varied for different groups of students.

In addition to the impact of the curricula on the composite scores, subscore analyses demonstrated that some skills benefited more from Building Blocks than PreK Math (count higher without committing errors, describing counting errors, and explaining how to correct counting errors), while for other skills the Building Blocks and PreK Math students performed equally as well (object counting, verbal counting, comparing numbers, sequencing, shape identification and representation, and identifying counting errors) (Clements & Sarama, 2008).

Pre-Kindergarten Mathematics Curriculum

The Pre-Kindergarten Mathematics Curriculum (PreK Math), originally designed as part of the Berkeley Math Readiness Project has been evaluated as part of the Preschool Curriculum Evaluation Research (PCER) Program. PreK Math, was developed for children in grades [XX through XX] (Klein & Starkey, 2004). The curriculum is organized around seven units: enumeration and number sense, arithmetic reasoning, spatial sense, geometric reasoning, pattern sense and unit construction, nonstandard measurement, and logical reasoning. The small-group activities included in the curriculum use concrete materials and are designed to improve mathematical knowledge, specifically numerical and spatial-geometric thinking (Klein & Starkey, 2004).

An effectiveness study compared children in equivalent numbers of low- and middle-income classrooms using PreK Math to a comparison group (Klein & Starkey, 2004). Both income levels were included in order to test the researchers hypothesis that because the curriculum provides experiences to low-income children that middle-income children were likely to receive at home, the impact of PreK Math would be more pronounced among low-income children (Starkey, Klein, & Wakely, 2004). In addition to classroom activities, the authors of PreK Math developed a home component of the curriculum, which includes parent classes three times per year designed to teach parents how to use the activities with their children (Starkey, Klein, & Wakely, 2004).

Researchers administered the Child Math Assessment (CMA; Klein & Starkey, 2004; Starkey, Klein, & Wakely, 2004) to both groups of students in the fall and spring of their PreK year. The CMA assesses a wide variety of mathematical concepts using 16 separate tasks, which are administered in two 20-30 minute individual testing sessions. For this study, the assessments were videotaped and coded for reliability (Starkey, Klein, & Wakely, 2004). Half of the children received the first section during the first testing session and the other half received the second section of the test during the first testing session.

The results demonstrated that that mathematics ability for middle-income children in both study groups was significantly higher than that of their low-income peers, and that their mathematics ability grew at a faster rate over the course of the study (Klein & Starkey, 2004). The results also indicated that there was a significant main effect for PreK Math, with the intervention group having significantly higher CMA scores.  The researchers conclude that while PreK Math was effective for both low- and middle-income children, it was particularly beneficial to the low-income students (Klein & Starkey, 2004; Starkey, Klein, & Wakely, 2004).

Researchers conducted a second study (also involving random assignment of classrooms) of PreK Math in two early childcare settings  Head Start and state-funded preschools  representing 40 pre-kindergarten classrooms (Klein, et al., in press).  Teachers in the treatment group implemented PreK Math and the DLM Early Childhood Express Math software (Clements & Sarama, 2003), part of the Building Blocks curriculum, while the control group continued their regular curriculum, which included Creative Curriculum, High Scope, Montessori, and other local curricula (Klein, et. al., in press). As in the study described above, the children were assessed using the Child Math Assessment (CMA) and coded from videotapes. As, expected, the math scores of the PreK Math/DLM and control groups did not differ between groups in the fall, but by spring the intervention group scored significantly higher than the comparison groups with a medium effect size of .55 (Klein, et. al., in press).  This study used a second mathematics outcome measure; this composite score consisted of the CMA, a Shape Compositions task, and the Woodcock Johnson Applied Problems score. Analyses using the composite score also demonstrated a significant difference between the treatment and control groups with an effect size of .62 (Klein, et. al., in press).

Childrens School Success (ISRC)

The Childrens School Success (CSS) Program is a comprehensive curriculum for preschool children implemented with at-risk children (low income families, students with disabilities, and/or ELL), which focuses on oral language and literacy, science, math, and social competence (Lieber, et. al., 2007). The program views young children as active, self-motivated learners and includes student choice, family involvement and individualization into the programs conceptual framework (Odom, et. al., 2007b). The curriculum utilizes linked learning, or activities that build upon the previous lessons content, integrates curricular domains across activities, includes a problem solving process, and capitalizes on childrens interests and experiences (Odom, et. al., 2007b). The mathematics aspect of the program was adapted from Douglas Clements Building Blocks curriculum and includes number and operations, geometry and spatial sense, measurement, pattern/algebraic thinking, and displaying and analyzing data (Odom, et. al., 2007b).

Three years of research was conducted with approximately 800 at-risk children in Head Start or state pre-k or private childcare centers, in which the majority of enrolled children were of Caucasian/Non-Hispanic dissent (Odom, et. al., 2007b). Student achievement was measured used the Woodcock Johnson Math Subtest. Although the authors did not provide information on the characteristics of the math subtest, a study conducted by the NICHD Early Child Care Research Network (2002) found that the Woodcock Johnson Applied Problems subtest has an internal consistency of .91. This assessment does not align directly with the curriculum itself and as such is less biased in favor of the curriculum.

Presentations on the research have not included analyses comparing the treatment and control groups. Instead, the focus of the presentations thus far has been on the impact of treatment fidelity on childrens assessment scores, as well as their initial ability levels. These presentations have presented analyses that show that treatment fidelity has a positive significant association with childrens post test scores (after controlling for their pretest scores) on many (but not all) of the outcome measures. The presentations have also shown that, not surprisingly, children with lower test scores at the beginning of the study learned more in high fidelity classrooms than initially low-achieving children in low-fidelity classrooms. The lack of research findings regarding the treatment and control group comparisons, combined with the focus on treatment fidelity in the majority of research conference presentations leads us to wonder whether the evaluation of the CSS curriculum model did not find a significant difference in the treatment and control children on study outcome measures.

Round the Rug Math: Adventures in Problem Solving

Round the Rug Math: Adventures in Problem Solving is a supplementary program for pre-K through 2^nd grade classrooms that uses stories to teach problem-solving (Casey, 2004; Casey, Kersh, & Young, 2004). This approach teaches mathematics concepts within a language rich medium that extends over the course of many lessons (Casey, 2004). The program specifically focuses on spatial and analytical skills, which can help address learning gaps, so it is not meant to be a comprehensive curriculum (Clements & Sarama, 2008). However, the focus on developing spatial skills is also intended to achieve equity between girls and boys, who consistently show better spatial and geometry skills (Casey, 2004). The program does two things simultaneously: (1) integrates mathematical content into the theme-based approach generally used throughout early childhood curricula, and (2) teaches mathematics content systematically with sequenced lessons (Casey, Kersh, & Young, 2004). Specifically, the Round the Rug Math curriculum teachers mathematical concepts in a systematic, hierarchical progression through the use of long epic stories, which allow characters to have multiple adventures the expose students to mathematical problems or concepts (Casey, Kersh, & Young, 2004). Students must solve the problem before going on to the next part of the story, which includes progressively more difficult concepts.

In the first evaluation of the effectiveness of one story on students geometric understanding was conducted with Kindergarten students, comparing the Round the Rug Math curriculum to a control group.  The initial results indicate that the students who learned the content with the storybook approach improved significantly more than students who learned the content without the storybook approach, although details what this control group received were not described (Casey, Kersh, & Young, 2004). However, no information on the outcome measure or any statistical information was provided on this study.

A second study comparing the effectiveness of the program by gender suggests that in Kindergarten girls benefit more than boys from learning the mathematical content in a storytelling format (Casey, Erkut, Cedar, & Young, 2008). In this experimental study, six kindergarten teachers were randomly assigned to either the treatment or control group, with 76 students in the treatment group and 79 students in the control group. Two measures were used for pre- and post-test, including Triangles subtest of the Kaufman Assessment Battery for Children (K-ABC) and the Tangram test (Casey, Erkut, Cedar, & Young, 2008). The overall reliability of the K-ABC using a split-half procedure is .86-.93, with the Triangles subtests factorial loading at .70 for boys and .76 for girls (Casey, Erkut, Cedar, & Young, 2008).

There were higher pretest scores for the intervention group on the Triangle test, but no differences on the Tangram test. For the Triangle test, a repeated measures ANOVA showed a significant improvement from pretest to posttest (p<.001), as well as a significant difference between the treatment and control group (p<.003), particularly for the girls (p<.001; partial eta² =.141). A comparison of the boys by condition did not yield a significant difference (Casey, Erkut, Cedar, & Young, 2008). For the Tangram test, a repeated measures ANOVA showed a significant improvement from pretest to posttest (p<.001), but  no other effects.

Mathematics Curriculum Developed by C. Sophian

Sophian (2004a) differentiates the developmentally appropriate curriculum as one that matches the cognitive abilities of the learner from what she coined the prospective developmental perspective, meaning that some mathematical skills are important for learning at a later developmental point. This is often the unspoken goal of early education: teaching students enough so that they are ready and able to learn effectively in later grades and for low-income children the hope is that this preparation closes the achievement gap (Sophian, 2004b). While the development of social competence has long been a goal of Head Start, recent trends in accountability have broadened the focus of early childhood educators generally, and Head Start specifically, to include reading and math skills needed for school success (Fantuzzo, et. al., 2007). In addition, research suggests that low-income children, such as those in Head Start, have less mathematical understanding compared to wealthier counterparts (Sophian, 2004b).

Sophian developed a mathematics curriculum specifically for 3- and 4- year old children attending Head Start centers, which focuses heavily on measurement, object properties, and geometry (Sophian, 2004b). The curriculum is meant to be integrated within the rest of the Head Start program rather than as a stand alone curriculum. The curriculum was organized into weekly project activities and parents and teachers were given specific activities to complete with the children.

There is great emphasis on combining shapes in new ways and measurement with various units (Sophian, 2004b). Specifically, Sophian (2004b) describes the program as exploring the relationships between shapes rather than identifying features of those shapes (i.e. number of sides or angles). Rather than including measurement as a separate unit within the curriculum, Sophian (2004b) used measurement throughout the curriculum with a specific focus on measuring the same objects with different units of measurement; something she claims is not present in other similar programs.

An evaluation of the program was conduced to determine whether this math program could improve the readiness of low-income young children. Specifically, three Head Start centers, two classrooms within each center, served in the treatment group. Then six Head Start centers were matched on center characteristics and served in the control group; three centers conducted a literacy intervention and three centers continued their regular curriculum (Sophian, 2004b). In this case, the treatment group was provided the mathematics curriculum while the comparison group received either a literacy curriculum or no intervention (i.e. business as usual group). Children were assessed in the fall and spring of their pre-K year using an assessment procedure intended to align closely with the curriculum:  the Developing Skills Checklist (DSC) and a supplemental measure developed for the study. The mathematics portion of the DSC assesses:

naming shapes, reproducing and extending patterns, counting, identifying numerals, matching sets and numerals, joining and separating sets, identifying original positions, and logically operations (classification, conservation of number, estimation, and seriation) (Sophian, 2004b, pp. 69).

Using the DSC score, the mathematics intervention group scored significantly higher than either the literacy intervention group or the no-intervention group and there was a significant difference between conditions, using pretest scores as covariates, with an effect size (partial n² = 0.092; Sophian, 2004b). The supplemental score also showed the mathematics group scored significantly higher and there was a significant effect for the mathematics intervention (partial n² = 0.083).

References

Arnold, D. H. & Doctoroff, G. L. (2003). The early education of socioeconomically disadvantaged children. Annual Review of Psychology, 54, 517-545.

Bogard, K. & Takanishi, R. (2005). PK-3: An aligned and coordinated approach to education for children 3 to 8 years old. Social Policy Report: Society for Research in Child Development, 19 (3), 3-23.

Casey, B. (2004). Mathematics problem-solving adventures: A language-arts-based supplementary series for early childhood that focuses on spatial sense. In Clements, Sarama, and DiBiase (Eds.), Engaging Young Children in Mathematics (377-389). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Casey, B., Erkut, S., Ceder, I., & Young, J. M. (2008). Use of a storytelling context to improve girls and boys geometry skills in kindergarten. Journal of Applied Developmental Psychology, 29 (1), 29-48.

Casey, B., Kersh, J. E., & Young, J. M. (2004). Storytelling sagas: An effective medium for teaching early childhood mathematics. Early Childhood Research Quarterly, 19, 167-172.

Clarke, D. (2001). Understanding, assessing, and developing young childrens mathematical thinking: Research as a powerful tool for professional growth. In J. Bobis, B. Perry, & M. Michelmore (Eds.) Numeracy and beyond (Proceedings of the 24^th Annual Conference of the Mathematics Education Research Group of Australia, vol. 1, pp. 9-26). Sydney: MERGA.

Clark, B. & Shinn, M. R. (2004). A preliminary investigation into the identification and development of early mathematics curriculum-based measurement. School Psychology Review, 33 (2), 234-248.

Clements, M., Lewis, A., and Ginsburg, H. (2008). Preliminary Findings from the Big Math for Little Kids Evaluation Study. Unpublished manuscript.

Clements, D. H. & Sarama, J. (2003). DLM Express math software. In DLM Express Math Resource Package. Columbus, OH: SRA/McGraw-Hill.

Clements, D. H. & Sarama, J. (2008). Experimental evaluation of the effects of a research-based preschool mathematics curriculum. American Education Research Journal, 45 (2), 443-494.

Dodge, D. T., Colker, L. J., & Heroman, C. (2002). The Creative Curriculum for Preschool (4^th Ed). Washington, DC: Teaching Strategies, Inc.

Dowker, A. (2005). Individual differences in arithmetic: Implications for psychology, neuroscience and education. Hove, UK: Psychology Press

Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P. Pagani, L. S., Feinstein, L., Duckworth, K., Brooks-Gunn, J., Sexton, H., & Japel, C. School readiness and later achievement. Developmental Psychology, 43 (6), 1428-1446.

Fantuzzo, J., Bulotsky-Shearer, R., McDermott, P. A., McWayne, C., Frye, D., & Perlman, S. (2007). Investigation of dimensions of socio-emotional classroom behavior and school readiness for low-income urban preschool children. School Psychology, 36 (1), 44-62.

Ginsburg, H. P., & Ertle, B. B. (2008). Knowing the mathematics in early childhood mathematics. In O. N. Seracho & B. Spodek (Eds.), Contemporary perspectives on mathematics in early childhood education (pp. 45-66). Charlotte, NC: Information Age Publishing, Inc.

Ginsburg, H. P., Lee, J. S., & Boyd, J. S. (2008). Mathematics education for young children: What it is and how to promote it. Social Policy Report: Society for Research in Child Development, 22 (1), 3-22.

Greenes, C., Herbert, H. P., & Balfanz, R. (2004). Big math for little kids. Early Childhood Research Quarterly, 19, 159-166.

Griffin, S. (2004a). Number worlds: A research-based mathematics program for young children. In Clements, Sarama, and DiBiase (Eds.), Engaging Young Children in Mathematics (325-342). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Griffin, S. (2004b). Building number sense with Number Worlds: A mathematics program for young children. Early Childhood Research Quarterly, 19, 173-180.

Goldeck, S. L. (2001). Instructional models for early childhood: In search of a child-regulated/teacher-guided pedagogy. In S. L. Goldbeck (Eds.), Psychological Perspectives on Early Childhood Education: Reframing Dilemmas in Research and Practice (3-34). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Klein, A. & Starkey, P. (2004). Fostering preschool childrens mathematical knowledge: Findings from the Berkeley math readiness project. In Clements, Sarama, and DiBiase (Eds.), Engaging Young Children in Mathematics (343-360). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Klein, A., Starkey, P., & Ramirez, A. B. (2002). Pre-K mathematics curriculum. Glenview, IL: Scott Foresman.

Klein, A., Starkey, P., Clements, D., Sarama, J., & Iyer, R. (in press). Effects of a pre-kindergarten mathematics intervention: A randomized experiment. Journal for Research on Educational Effectiveness.

Lieber, J., Goodman-Jansen, G., Horn, E., Palmer, S., Hanson, M., Czaja, C., Butera, G., Daniels, J., & Odom, S. (March, 2007). Factors that influence the implementation of new curriculum: Results from two years of implementation. Poster presented at the Annual Meeting of the Society for Research on Child Development, Boston, MA.

National Research Council. (2004). On evaluating curricular effectiveness: Judging the quality of K-12 mathematics education. Washington, DC: Mathematical Sciences Education Board, Center for Education, Division of Behavioral and Social Sciences and Education, National Academic Press.

NICHD Early Child Care Research Network (2002). Early child care and childrens development prior to school entry: Results from the NICHD study of early child care. American Education Research Journal, 39 (1), 133-164.

Odom, S., Diamond, K., Hanson, M., Lieber, J., Butera, G., Horn, E., Palmer, S., Marquist, J., & Fleming, K. (March, 2007a). Childrens school success: Treatment dosage and child outcomes. Poster presented at the Annual Meeting of the Society for Research on Child Development, Boston, MA.

Odom, S., Butera, G., Schneider, R., Lieber, J., Sarpatwari, S., Horn, E., Palmer, S., Goodman-Jenson, G., Diamond, K., Czaja, C., Hanson, M., & Ceja, M. (April 20, 2007b). Childrens school success (CSS): Child outcomes from three years of research. Paper presented at the Council for Exceptional Children, Louisville, KY.

Preschool Curriculum Evaluation Research Consortium (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington DC: U.S. Government Printing Office

Reynolds, A. J. (2005). Confirmatory program evaluation: Applications to early childhood interventions. Teachers College Record, 107 (10), 2401-2425.

Rock, D. A., & Pollock, J. M. (2002). Early childhood longitudinal study Kindergarten class of 1998-99 (ECLS-K), psychometric report for kindergarten through first grade (NCES Working Paper No. 2002-05). Washington, DC: U.S. Department of Education. Retrieved December 17, 2003, from http://nces.ed.gov/pubsearch/pubsinfor.aps?=200205.

Sarama, J. (2004). Technology in early childhood mathematics: Building Blocks as an innovative technology-based curriculum. In Clements, Sarama, and DiBiase (Eds.), Engaging Young Children in Mathematics (361-373). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Sarama, J. & Clements, D. H. (2004). Building blocks for early childhood mathematics. Early Childhood Research Quarterly, 19, 181-189.

Sophian, C. (2004a). A prospective developmental perspective on early mathematics instruction. In Clements, Sarama, and DiBiase (Eds.), Engaging Young Children in Mathematics (253-266). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Sophian, C. (2004b). Mathematics for the future: Developing a Head Start curriculum to support mathematics learning. Early Childhood Research Quarterly, 19, 59-81.

Starkey, P., Klein, A., & Wakeley, A. (2004). Enhancing young childrens mathematical knowledge through a pre-kindergarten mathematics intervention. Early Childhood Research Quarterly, 19, 99-120.

Thomas, G. & Ward, J. (2002). Exploring issues in mathematics education: An evaluation of the early numeracy project 2001. New Zealand Ministry of Education, New Zealand.

Young- Loveridge, J. M. (2004). Effects on early numeracy of a program using number books and games. Early Childhood Research Quarterly, 19, 82-98.

Endnotes

[1] It should be noted that the authors of this paper conducted the evaluation of BMLK (Clements, Lewis, and Ginsburg, 2008) and that Ginsburg is one of the curriculums developers.

[2] There are several reasons we do not draw any conclusions about the relative efficacy of Building Blocks and PreK Math even though results from the Building Blocks evaluation study found that children using it scored significantly higher than children using PreK Math. First, the evidence from the federally funded evaluations of the two curricula indicates that both PreK Math and Building Blocks are more effective than the control curricula used in the control conditions. Second, both evaluations were conducted using mathematics assessments designed by the curriculum developers/evaluators. As we said before, this is not to suggest that that either assessment tool is overly aligned with a specific curriculum or to question either assessments validity. Rather, we suggest that it is premature to draw conclusions regarding the relative effectiveness of the two curriculum based on two studies, each of which used a different mathematics assessment. In fact, both curricula have been evaluated by the What Works Clearinghouse and received its highest rating: strong evidence of a positive effect with no overriding contrary evidence.

[3] ACS child care centers offer a kindergarten year for their students, and many students choose to complete the kindergarten year at the child care center rather than transitioning into a public elementary school. Many kindergarten programs in New York City are half-day, while the ACS child care centers offer full-day care and are intended to meet the needs of working parents.

[4] All mathematics curricula reviewed except for Childrens School Success reported a positive statistically significant impact on childrens mathematics knowledge.

Promoting Children's Socioemotional Development in Contexts of Early Educational Intervention and Care: A Review of the Impact of Federally-Funded Research Initiatives on Young Children's School Readiness

This paper is part of a series of working papers prepared for a meeting sponsored by the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE).В  Abt Associates Inc and the National Center for Children in Poverty (NCCP) were funded to convene the meeting.В  The views represented in this paper are those of the author(s) and do not necessarily reflect the opinions of the U.S. Department of Health and Human Services.
 

Introduction

Three decades of research in the fields of developmental psychology and early childhood have suggested that childrens socioemotional development is clearly associated with their school readiness (see Blair, 2002; Zaslow et al., 2003).  Children have been argued to draw upon positive styles of self-regulation and social skill as key sources of support when navigating new contexts of school (Raver, 2002).  Conversely, children who are persistently emotionally dysregulated and behaviorally disruptive have been found to receive less instruction from teachers and to have fewer opportunities for learning from peers (see Arnold et al., 2006; McClelland & Morrison, 2003).  However, claims of the role of socioemotional competence for childrens later academic achievement have recently received greater scrutiny (Duncan et al., 2007).  In addition, recent analyses using the nationally representative Early Childhood Longitudinal Survey-Kindergarten (ECLS-K) data set suggest that preschool experience may pose both risks and benefits to childrens long-term chances of success in school (Magnuson, Ruhm, & Waldfogel, 2007).  It is against this backdrop that a new set of federally funded research initiatives funded by the U.S. Department of Health and Human Services and the U.S. Department of Education were to test innovative models of program improvement and support for childrens school readiness.  Findings from these sets of research initiatives are particularly timely from both the standpoints of science and social policy.

Tests of the role of childrens socioemotional development for their later chances of success in school become even more pressing in the context of income poverty.  Specifically, young children in poverty are more likely to be exposed to multiple ecological stressors such as higher levels of neighborhood and family violence, greater psychological distress among adult caregivers, and a range of other co-factors that appear to place childrens ability to regulate  their emotions and behavior in jeopardy (Brooks-Gunn, Duncan, & Aber, 1997; Li-Grining, 2007; Raver, 2004). Policy contexts (such as early childhood education) that provide direct services to children have been argued to be the most effective means of supporting low-income childrens optimal outcomes (Magnuson & Duncan, 2003).  This context underscores the significance of major federal investments in evaluations of the impact of interventions targeting low-income childrens school readiness (such as the interventions within the ISRC and PCER consortia).

This review provides the opportunity to briefly review emerging findings from this set of major federal research initiatives. After providing a brief definition for each relevant socioemotional construct, this review summarizes the rationale for targeting that domain. Models of program impact mediated through improvements in instructional support (such as changes in teachers use of emotionally and behaviorally supportive classroom practices) are also reviewed, with the recognition that children within this set of interventions were hypothesized to be affected primarily through improvement in the quality and quantity of teachers instruction. (It is important to note that interventions such as Head Start and Early Head Start have invested in more comprehensive approaches that include provision of family supports and services, but those more comprehensive approaches will not be discussed, here). This review also discusses some of the potential tradeoffs in implementing new curricula in early childhood settings. Specifically, this review examines whether there is any evidence for any unexpected benefits or of any unanticipated negative consequences for childrens socioemotional development or for emotionally supportive classroom practices from the implementation of a large number of interventions in preschool settings. Finally, new directions for applied developmental science in early childhood educational settings are briefly outlined.

Contrasting models of the role of socioemotional development for childrens school readiness

The empirical case for the importance of childrens socioemotional development in classroom contexts has emerged from several different traditions in developmental, clinical, and educational psychology. From developmental perspectives, converging lines of inquiry from social developmental and neurobehavioral literatures suggest that children enter schools with distinct profiles of emotional reactivity, regulation and executive functioning that appear to facilitate or hinder their engagement with other learners, teachers, and the process of learning (Blair, 2002; Fantuzzo et al., 2007; Howse, Calkins, Anastopoulos, Keane, & Shelton, 2003; Raver, 2002). Similarly, drawing from a tradition of attachment theory, developmental researchers have highlighted ways that some children establish and maintain relationships with teachers that are characterized by a high degree of mutual positive engagement while other children engage in relationships with teachers that are characterized by a high level of conflict (for review, see Pianta, Justice, Cottone, Mashburn, & Rimm-Kaufman, symposium presentation). Third, clinical and educational psychological studies have highlighted the extent to which childrens disruptive, aggressive, and withdrawn behaviors have serious implications for short-term opportunities as well as long-term opportunities for learning, both for children manifesting behavioral difficulty and for their peers (Campbell, Shaw, & Gilliom, 2000). A fourth tradition of observational research in classrooms has highlighted ways that teachers also bring their own regulatory and interpersonal profiles of strength and difficulty to classroom interactions and instruction with their students (LoCasale-Crouch et al., 2007). These four mechanisms are likely to be transactionally, bidirectionally related as children with varying self-regulatory profiles elicit differing patterns of responsiveness versus conflict with teachers. These variables are also likely to be highly confounded by omitted variables or unmeasured characteristics across children, teachers, and settings (Duncan, 2003). For these reasons, investigators across developmental, clinical, and educational fields have come to consensus that experimental and quasi-experimental approaches are integral to our ability to draw causal inferences on the roles and modifiability of these processes as predictors of childrens school readiness.

In each of the sections below, a brief literature review is provided for each of these four possible mechanisms supporting low-income childrens school readiness. Findings from federally funded research initiatives are then considered, with close attention to whether those interventions yielded clear evidence of significant impacts on childrens socioemotional development (see Table 1 for summary of interventions designs, samples, and findings).

Self-regulation: How children handle their emotions, attention, and behavior in classroom contexts

Preschool has long been viewed as an important social context where children learn to follow adults directions, to handle their own emotions, attention, and impulses with increasing independence from adult regulatory support. Imagine any one of a number of routine classroom scenarios, where children are expected to sit attentively through circle time, line up for trips to the playground or bathroom without pushing or shoving peers, and to follow teachers directions to gather materials for a writing activity, clean up, or share a favorite book even when children feel tired, bored, or frustrated. Investigators have identified individual differences and growth trajectories in childrens ability to handle these regulatory challenges, based on a research tradition focusing on reactivity and regulation (see McClelland et al., 2007; Graziano, Reavis, Keane, & Calkins, 2007 for recent reviews). More recently, childrens ability to handle classroom challenges has been examined through a second neurobehavioral lens with research on childrens executive functioning emphasizing the roles of childrens working memory, attention deployment, and ability to inhibit prepotent impulses in order to meet external demands (Diamond & Taylor, 1996; Greenberg, Riggs, & Blair, 2007). In applied developmental contexts, investigators have considered childrens modulation of positive affect, attention, and behavior in classroom contexts as important approaches to learning that are correlated with teacher reports and direct assessments of childrens academic skill (Fantuzzo et al., 2007; McDermott, Leigh, & Perry, 2002; Rimm-Kaufman, Fan, Chiu, & You, 2007).

Evidence from a small, extant literature on self-regulation and executive functioning among low-income children suggests that exposure to more poverty-related risks is associated with childrens greater difficulty in their executive functioning and self-regulation skills (Li-Grining, 2007; Lengua, 2002). Evidence from recent neurobehavioral research suggests that executive functioning skills are late-developing through early childhood, suggesting an important window of opportunity or sensitive period for the development of competent regulation of attention, impulses, and use of working memory in early childhood (Diamond & Taylor, 1996).  On the basis of this model of self-regulation and school readiness (see Greenberg, 2006), several federally funded interventions in the ISRC consortium posited that children in treatment group would show significant gains in this domain of school readiness as compared to their control group counterparts (Bierman, Nix, Greenberg, Blair, & Domitrovich, in press-b; Fantuzzo, in preparation, Raver et al., revised and resubmitted).

Was there evidence from these federally funded research initiatives of significant impact of interventions on childrens self-regulatory skills? Several studies within the ISRC have found that children would specifically gain in self-regulatory skills when in classrooms that provided greater regulatory support. These have included Project REDI (Bierman et al., in press-a, reporting effect size of d = .29 on direct assessments of task engagement) and the CSRP (unpublished findings). Across these two studies, children in the treatment group were found to demonstrate stronger levels of attention, engagement, or focused effort on a direct assessment of attention and impulsivity at post-test, compared to children in the treatment group. In contrast, no statistically significant differences were found on teacher reports of childrens attentiveness, persistence, and other learning-related skills, on the Preschool Learning Behavior Scale (McDermott, Green, Francis, & Stott, 1996; PCER final report, 2008). These null findings are interpreted with caution in this review. This caution is based on concerns for power and correspondingly, the relatively high values that effect sizes would have to achieve, in order to be minimally detectable (see cell sizes and MDEs listed in PCERS final report, pp. 31)

Findings from REDI and CSRP are in line with prior work by Greenberg and colleagues (e.g., Riggs, Greenberg, Kusché, & Pentz, 2006) with older children, suggesting significant program impact on childrens executive function, and by recent findings by Diamond, Barnett, Thomas, & Munro (2007) where children assigned to the treatment group receiving the Tools of the Mind curriculum demonstrated significant benefits on a directly-assessed executive function task (the flanker task) relative to their control group assigned counterparts. In short, these findings suggest substantial evidence for the modifiability of childrens self-regulatory skills across the preschool year.

What are the implications of these hypothesized and demonstrated short-term gains in childrens executive function or self-regulation skills? An optimistic hypothesis might be that children with improved self-regulatory skills may be placed on a more positive developmental trajectory, better able to capitalize on future opportunities for learning in kindergarten and early elementary years. A less optimistic hypothesis is that these behavioral gains will be sustained only as long as children continue to have access to the conditions and classroom practices that supported the development of executive function and adaptive self-regulation within the intervention year. Future research is needed to learn whether these early gains in childrens ability to regulate their engagement, attention, and behavior are sustained into early elementary school years.

Childrens social cognitions and prosocial skills in classroom contexts

A parallel area of research has focused on what children know about their emotions and the negotiation of interpersonal problems, emphasizing the social cognitive mechanisms revealed in childrens successes versus failures to get along with peers and adults (see classic work by Dodge, Pettit, & Bates, 1994; Conduct Problems Prevention Research Group, 2002). Additional research on childrens attachment relationships with teachers, with the development of relationships characterized by closeness versus conflict also informs several interventions funded by the ISRC and PCER initiatives (Hamre & Pianta, 2001). Childrens social skills and quality of relationship with teachers have been found to be correlated to their later social and academic competence in early elementary school (see Raver, Garner, & Smith-Donald, 2007 for review). Both of those research areas suggest that children develop relatively stable social cognitions or attributions regarding strategies of getting along with peers and adults in classroom contexts. These attributions appear to be built on a foundation of childrens knowledge of emotions, knowledge of prosocial behaviors (e.g., helping, sharing, and taking turns), and the ability to generate and use more effective social problem-solving skills (see Domitrovich, Cortes, & Greenberg, 2007).

Past correlational research has faced the persistent problems of omitted variables bias and reverse causality (or bidirectional influence). For example, children who are temperamentally prone to be more sociable have been found to elicit more positive responses from peers and teachers than do children who express more anger and distress in the classroom (see for example, Justice, Cottone, Mashburn, & Rimm-Kaufman, under review). In the context of those relationships, more well-liked children may have greater opportunities to talk about, process, and remember information about their own and others feelings, and about strategies for successfully navigating social relationships than might children who are less well-liked. Similarly, childrens placement in classrooms with more emotionally supportive teachers and their negotiation of academic as well as social challenges are likely to be at least partially influenced by time-invariant individual and contextual variables that are often omitted from models (see OConnor & McCartney, 2007 for exception and methodological solutions using longitudinal data).

It is within this framework that the federally funded research initiatives targeting childrens SEL skills are likely to be of major impact to the field. In this area, randomized trials represent a key opportunity to test causal claims of the role of Social Emotional Learning (SEL) curricula for childrens knowledge, attributions, and behaviors regarding prosocial versus aggressive behavior with peers. Interventions targeting teachers practices also offer the opportunity to test the modifiability of childrens relationships with adults in classroom contexts. Outcomes that are commonly tapped in interventions that target childrens social problem-solving with peers and positive relationships with teachers include direct assessments of childrens emotion understanding, of childrens selection of adaptive versus maladaptive strategies in hypothetical vignettes of conflict with peers. Outcome variables also include more general teacher reports of childrens social skills as well as teachers reports of the quality of their relationships with individual children.

With that brief review as an empirical backdrop, was there evidence from the federally funded research initiatives of significant impact of interventions on childrens social problem-solving skills and their ability to get along with peers? Evidence from Project REDI suggests that the intervention, comprised of cognitive and socioemotional curricula as well as teachers provision of emotion coaching and support was associated with moderate to medium-sized program impacts for childrens emotion understanding and interpersonal problem-solving (ds ranging from .15 to .39; Bierman et al., symposium presentation). These gains in childrens socioemotional skill acquisition were paralleled by substantial gains in treatment enrolled childrens generalized social competence, with effect sizes of d = -.28 for teacher rated aggression, d = .26 for observer-rated social competence (p<.08) (Bierman et al., in press-a ).  These findings are in keeping with prior randomized trial research by Bierman and colleagues (see Greenberg et al., 2007 and Domitrovich et al., 2007 for comparison) and by other senior leaders in the area of low-income childrens socioemotional development (see for example, Izard, Trentacosta, King, Morgan, & Diaz, 2007).

Was there evidence from the federally funded research initiatives of significant impact of interventions on childrens relationships with teachers? Building on their hallmark program of observational research across large samples in preschool and elementary school contexts, Pianta et al specifically targeted teacher-student relationships as a key socioemotional outcome for their web-based intervention, with evidence of improved teacher-student relationship using observational measures (see below). Similar findings of program impact on the teacher reports of the quality of teacher-student relationship have been informally discussed, but not yet submitted for publication from Project REDI and CSRP. These findings (should they be robust to sensitivity checks using alternative model specifications) would suggest that teacher-child relationships are modifiable. Additional analyses are also currently underway in both the REDI and CSRP labs to detect whether improvements in teachers relationships with children are bidirectionally related to childrens improvements in self-regulation (the teams are constrained from making causal claims regarding those linkages, however; see Raver et al., submitted, for further discussion).

Childrens behavior problems

While most of the studies in the ISRC consortium have highlighted childrens reductions in their risk for manifesting behavior problems, only two of the seven teams have submitted evidence of significant impact of intervention in this domain. These two studies include Project REDI, reporting reductions of childrens aggression by teachers (d = -.28) and by parents (d = -.13, at trend level of significance) (Bierman et al., in press-a).  These findings are similar to those yielded by the CSRP team, suggesting significant reductions in childrens externalizing and internalizing problems as reported by teachers, and trend-level reductions in childrens observed aggressive disruptive behavior in the classroom (Raver et al., revised and resubmitted).  Review of the PCER final report suggests that there were null impacts on childrens behavior problems in the pre-Kindergarten year, with point estimates of program impact using the SSRS Problem Behaviors Scale) reported to be small in magnitude and signed in inconsistent directions. Of concern is the finding that one intervention (Project Approach) appears to have yielded evidence of negative impact on childrens behavior problems in the Kindergarten year, with children in the treatment group showing significantly higher numbers of behavior problems than the control group. It is important to highlight however that that finding has not been replicated in any of the other 20 studies in the two consortia.

Mechanisms of improvement in childrens socioemotional development through improvement in the quality and quantity of instruction

How were these child-focused program impacts achieved? Consistent across all interventions reviewed was a clear emphasis on multi-day trainings for teachers, followed by extensive coaching support and attention to fidelity of implementation. Some studies (but not others) have also published findings of proximal improvement in classroom practices as a result of the implementation of the interventions planned. That smaller set of studies is reviewed below.

Findings from My Teaching Partner suggest that teachers who received web-based consultancy as well as web-based access to information on ways to improve instructional strategies made significant improvements in their classroom practices, as compared to teachers with access to web-based information, only (Pianta, Mashburn, Downer, Hamre, & Justice, submitted). Teachers in the treatment group were found to show significant gains in sensitivity, language modeling, and quality of instructional support to students, as compared to teachers in the control group. Effect size estimates are reported and therefore must be understood in terms of change over time: The investigators report unstandardized regression coefficients of B = .07 to .09 per unit of time (30 days). Briefly, this means that treatment group programs averaged .42 to .54 of a point gain (on the CLASS 7-point scale) relative to programs in the control group, in a six month period.  Importantly, gains were substantially larger for programs with very high proportions of poor children enrolled in their classrooms (see figures).

Similarly, Project REDI targeted teachers generalized classroom practices and induction strategies as well as their use of SEL curricular lessons to increase the level of emotional support and contingency to childrens emotional and social experiences (Bierman et al., in press-b; Bierman, personal communication, May 2008). Teachers use of emotion coaching and improvements in overall classroom management and behavioral support were significantly improved by the REDI intervention (Domitrovich et al., revised and resubmitted). Importantly, results from the REDI team suggest that these changes in classroom processes were powerful predictors (and likely mediators) of childrens language and socioemotional gains (Bierman et al., presentation). From a congruent theoretical framework, CSRP aimed to improve childrens self-regulation and opportunities for learning by increasing teachers use of emotionally supportive classroom practices where teachers maintained clear, firm yet warm patterns of limit-setting (see Raver et al., 2008). In contrast to project REDI, no specific child-focused curricula on emotional language or self-awareness were specifically targeted in CSRP. Findings from the CSRP intervention suggested that classroom climate was significantly benefited (d = .52 to d = .89). CSRP findings of intervention impact on positive classroom climate support the hypothesized mechanism of influence for intervention-enrolled childrens observed gains in self-regulation, relative to their control group enrolled counterparts.

Findings from some of the PCER studies provide sparse but congruent evidence of improved emotionally supportive classroom processes as a result of intervention. The University of Virginia team, for example, targeted both teachers increased use of language-rich classroom activities and the complexity of the language that teachers use when conversing with children (Pence, Justice, & Wiggins, in press). Analyses of the impact of this intervention suggest that teachers made changes in their activities most quickly, but were able to improve the quality of their conversations (described as a relational process) with the children in their classrooms, also (Pence et al., in press).  Ramey et al. (submitted) also primarily targeted teachers language and literacy instruction using two different levels of coaching (weekly and monthly) in the Building Language for Literacy intervention trial, but also collected independent observations of teachers time spent engaged in emotionally less supportive practices such as placing restrictions on children and negative/harsh treatment of children.  In the report included for this review, the investigators chose not to analyze whether difference between intervention conditions on these measures were statistically significant (see pp. 21), but inspection of the means on both measures suggests that point estimates of differences between the groups appear to favor treatment assigned classrooms.

Building relationships between teachers and intervention staff

All the intervention models reviewed above (e.g., MTP, REDI, CSRP) as well as most other models in the ISRC that are currently analyzing their data for evidence of treatment impact (led by Fantuzzo, Kupersmidt, Odom, Sheridan) have relied on significant investments in coaching of teachers in supporting gains in classroom climate. Similar levels of investment in training and coaching were found in all studies reviewed from the PCER consortium (e.g., Ramey et al., submitted; Assel, Landry, Swank, et al., in press; Cosgrove, Fountain, Wehry, Wood, & Kasten, submitted; Klein, Starkey, Clements, Sarama, & Iyer, in press).

Across all interventions using coaching or consultation approaches in the ISRC consortium, levels of coaching were commensurate with levels used in the language- and literacy interventions in the PCER group (e.g. N Florida ELLM used two days of intensive training followed by 1 hour weekly coaching sessions across the school year while training for Pre-K Mathematics included 2 4-day trainings and 15 on-site coaching sessions). Comparison of models across all ISRC and PCER studies that employed coaching suggests several commonalities, including emphasis on job embedded, collaborative models (including cycles of modeling, observation and feedback) between teachers and coaching staff (see Cosgrove et al., submitted; Raver et al., 2008). In short, intervention staff focused substantial levels of effort in building trusting, collaborative relationships with teachers (see Brown, Knoche, Edwards, & Sheridan, submitted for case study).

With variations on this coaching and training model, multiple teams demonstrated significant improvements in teachers classroom practices (see above). Building of positive, supportive coaching relationship may be particularly important given that interventions may be asking teachers to be reflective, self-critical, and willing to take the risk of trying new approaches in the ways that they run their classrooms. In one study, for example, teachers in the treatment group reported increasing levels of efficacy in implementing language stimulation techniques over the school year (Justice et al., under review). Importantly, teachers in the treatment group were also found to report lower, rather than higher levels of self-efficacy and comfort when compared to teachers ratings of self-efficacy in an untreated control group.  These findings, though drawn from a single intervention trial, are congruent with other studies that document the challenges that teachers face as well as the gains that they are capable of making in programs emphasizing professional development and quality improvement (see Li-Grining et al., submitted; Brown et al., submitted).  Extensive focus group and evaluation surveys conducted by Piantas team suggest that teachers generally reported feeling supported by consultancy services, even when they are web-based (Whitaker, Kinzie, Kraft-Sayre, Mashburn & Pianta, 2007).

An obvious next question is whether there is a threshold level to the amount of coaching needed to support improvements in the quality and quantity of instruction. Ramey, Ramey, and Stokes (in preparation) raise this by pointing to contrasting models of coaching in weekly versus monthly delivery schedules, with no clear evidence that more frequent coaching yields substantially greater benefit than less frequent coaching. This represents an important new direction for future research.

Checking to determine whether there were unanticipated benefits or drawbacks of early intervention for childrens socioemotional development

One fair question might be whether there are unanticipated spillover benefits from focusing on child language, literacy and math outcomes on childrens socioemotional outcomes. One hypothesis might be that children may gain increasingly strong regulatory skills through more cognitively demanding and engaging curricula, where the content of teachers lessons helps to entrain and strengthen childrens attentional and memory skills (see Doctoroff, Greer, & Arnold, 2006). A contrasting hypothesis might be that children might respond negatively to more cognitively demanding and firmly structured classroom practices and curricula, showing increased behavioral difficulty that might offset language, literacy, or math gains.

Several ISRC interventions used hybrid models combining foci on language/literacy as well as childrens socioemotional development and analyses of treatment impact will elucidate whether there were consistent benefits or costs to childrens behavioral development, across interventions (see interventions led by Pianta, Fantuzzo, Odom, Kupersmidt, and Bierman). Of the ISRC hybrid models tested, Project REDI provided data to support improvements, rather than decrements in childrens socioemotional development as well as in their language development (see above). Across 13 of the 14 interventions in the PCER evaluation, teachers in the intervention groups and teachers in the control group did not differ on the level of their students behavioral difficulty or social skills (using the SSRS; Gresham & Elliott, 1990). Again, these null findings should be interpreted with caution. The one exception was that children in the Learning Approaches treatment group were found to fare less well on socioemotional measures than were children in the control group, as rated by kindergarten teachers (see above). With that exception noted, there was no clear evidence of negative consequences for teacher-child interaction. Nor is there evidence for negative behavioral or emotional consequences for childrens socioemotional development, in almost all studies where teachers were extensively trained and monitored to implement significantly more cognitively demanding interventions.

Another way to explore this question is to consider whether teachers training, time, or curricular focus on academically focused outcomes might inadvertently lead classrooms to become too tightly structured, overly cognitively demanding, or somehow less emotionally or behaviorally supportive. Descriptive data from many of the non-experimental studies submitted for this review, however, suggest that the risk of preschool classrooms becoming overly cognitively demanding is relatively low. For example, descriptive work by the Howes & Fuligni team (Fuligni, revised and resubmitted) as well as work by Justice et al. (under review) on the preschool activity contexts and preschoolers exposure to language suggests that relatively low percentages of class time are spent engaged in instructional effort. Similarly, Massey, Pence, Justice and Bowles (2008) report that teachers use of more cognitively challenging questions is limited to approximately 11% of their utterances directed to the low-income children in their classrooms (pp. 12).  While speculative, it does not appear that those classrooms included in this broad range of studies were already too tightly paced or cognitively demanding, prior to implementation of the intervention.  Put another way, there may be significant regulatory benefits, and possibly fewer regulatory costs to raising the bar for teachers structure and pacing of cognitively demanding material in classrooms serving low-income children.

The PCER 14-study evaluation offers limited but important opportunity to examine this question: Data on the quality of teacher-child interaction were collected three times during the school year across all 14 studies (as rated by observers using Arnett scales) (Preschool Curriculum Evaluation Research Consortium, 2008). Overall, statistically significant evidence of beneficial spillover effects in improving the classroom climate were found for the Creative Curriculum intervention, where treatment-assigned teachers were observed to be less detached and more positive in spring than were teachers in control group classrooms. Though non-significant, evidence from seven of the exclusively literacy/language oriented curricula demonstrated point estimate differences between treatment and control groups that were in the right direction (e.g., with point estimates of effect sizes equal to .38 or higher) (see Preschool Curriculum Evaluation Research Consortium, 2008, pp. xliv). In sum, measured indicators of classroom quality across all studies but one suggest that placing higher demands on teachers instructional practices using either language/literacy or hybrid intervention models did not lead to measurably negative impacts and in one case (mentioned earlier), the implementation of these interventions led to clear benefits regarding the socioemotional climate of the classroom.

Directions for future research in promote childrens readiness for school

The role of child, family, classroom, and context characteristics as moderators

Increasingly, randomized trials have been analyzed with attention to moderating roles of person and place, where interventions may fit the needs of some children, in some contexts more than the intervention might for other children, in other contexts (Gorman-Smith & Tolan, 1998). The role of moderators was explored in some studies reviewed here, but not in others, and they represent a very promising direction for future research.

A small number of studies considered the role of child characteristics, such as child gender, race/ethnicity, English-language-learner status, and risks for self-regulatory or expressive language difficulty. For example, children at higher levels of behavioral and cognitive risk (e.g. those children who are more temperamentally or neurocognitively prone to high levels of shyness, impulsivity, or distractability) might be expected to benefit more greatly or less greatly from interventions (see Bierman et al., in press-a for review). Yet this review suggests that few of the socioemotionallyoriented. hybrid, or cognitively-oriented interventions (in ISRC and PCER) considered whether intervention impacts were greater or smaller for children with greater proneness to regulatory skill or difficulty.  One exception was the nonexperimental finding that children with greater proneness to shyness had significantly more difficult time establishing positive relationships with teachers in nonexperimental analyses of one PCER- funded intervention (Justice et al., under review). Importantly, child temperament moderated relations between childrens language skills and student-teacher relationship, where children who were temperamentally prone to anger and had low expressive language abilities were at particularly high risk of conflictual relationship with their preschool teachers (Justice et al., under review). Additional findings of moderation of intervention impact by child risk were found for Ravers team for observational measures of child behavioral problems (Raver et al., revised and resubmitted). In future, it will be important to carefully consider whether program impacts are larger or smaller for children with differing profiles of strength versus risk.

Family level risk may also be important and parsimonious way to consider fit of different intervention models for families with substantially differing economic and psychosocial resources. Findings by Piantas team of clear, larger benefit of the MTP program for serving very high-poverty classrooms as compared to programs serving proportionally fewer poor children highlights the importance of including family-level income poverty and related risks in models. A third important set of moderators are those of program type and program resources. For example, an intervention targeting the emotional climate of classrooms may be difficult to implement in settings that are chaotic or disorganized, or under-resourced (see Raver et al., 2008 for review). In contrast, programs that have mental health consultants on staff, on-site personnel to address teacher training, quality improvements, etc. may already be sufficiently resourced that they are likely to show little, if any benefit of additional services implemented through our intervention efforts. In short, it is important to include some observable indicators of level of program resources as covariates and as moderators, to detect whether programs with higher organizational capacity are able to benefit from intervention more so than others (see Assel, Landry, Swank, & Gunnewig, 2007 for examples of heterogeneity of child level program impacts across program type).

The importance of socioemotional measures in study analyses

Past reviews have highlighted the importance of including socioemotional measures as well as cognitively oriented measures when benchmarking intervention impact (e.g. Raver & Zigler, 1997). There are several key benefits (highlighted earlier) for including socioemotional measures at both child- and classroom levels, even when interventions are targeted toward childrens language and literacy. The inclusion of child social skills and behavior problem measures in the PCER evaluation and some individual PCER studies (e.g., Klein et al., in press) helps to rule out concern, for example, that there may be iatrogenic sequelae from the introduction of interventions targeting language and literacy. Similarly, the inclusion of childrens language and math skills in interventions that target only classroom socioemotional processes offers the opportunity to test whether there are costly tradeoffs (in terms of lower instructional time) or unanticipated benefits (in terms of childrens language gains) when focusing program improvement efforts on socioemotional processes. This cross-domain integration of measures at child- and classroom levels represents an important area of future collaboration and future research.

The importance of modeling cluster-randomized status in study analyses

From a methodological standpoint, the impact of a number of these interventions on childrens socioemotional development was difficult to interpret for this review because of variability in the ways that data were analyzed and reported.  A substantial number of studies provided careful, sophisticated analyses of program impact, using Intent-to-treat analyses, multi-level modeling (e.g., HLM), and clear description of model specification so that the role of cluster-randomized status to treatment versus control groups could be clearly identified. In contrast, a smaller number of studies limited their reports to analyses of program fidelity as a predictor of child-level or classroom-level outcomes, effectively reintroducing selection bias into designs that were initially randomized. Future research in this area would be substantially strengthened by a tiered reporting process, whereby intention-to-treat (ITT) analyses and treatment-on-treated/ dosage analyses could both be encouraged.

Summary

At this early stage of review, most research teams have only recently wrapped up final stages of data collection and completion of preliminary data analyses. Few research teams have completed the full set of ITT analyses that are needed to be able to determine the individual and collective impacts of preschool intervention on childrens socioemotional outcomes (A full set of ITT analyses would include tests of moderation and sensitivity checks regarding whether program impact estimates are sensitive to model specification). With that caveat in mind, preliminary review of the current set of published and unpublished papers suggests clear evidence for the benefits of several intervention approaches in supporting low-income childrens socioemotional development across their preschool year. Findings of improved classroom instructional processes and improved classroom emotional climate across both types of interventions suggest that interventions using teacher training and coaching models yielded substantial improvements in program quality. Children in treatment groups were found to show lower behavioral problems, increased self-regulatory skills, and greater prosocial skills with peers and with teachers, than their counterparts in control group classrooms, in a smaller number of interventions. As these trials are completed, they are likely to make a major contribution to our knowledge of the ways that scientists and policy makers can best support the school readiness of our nations low-income children.

References

Arnold, D. H., Brown, S. A., Meagher, S., Baker, C. N., Dobbs, J., & Doctoroff, G. L. (2006). Preschool-based programs for externalizing problems. Education & Treatment of Children, 29, 311339.

Assel, M. A., Landry, S. H., Swank, P. R., & Gunnewig, S. (2007). An evaluation of curriculum, setting, and mentoring on the performance of children enrolled in pre-kindergarten. Reading and Writing, 20, 463494.

Bierman, K. L., Domitrovich, C. E., Nix, R., Gest, S., Welsh, J. A., Greenberg, M. T., et al. (in press-a). Promoting academic and social-emotional school readiness: The Head Start REDI program. Child Development.

Bierman, K., Nix, R. L., Greenberg, M. T., Blair, C., & Domitrovich, C. (in press-b). Executive functions and school readiness intervention: Impact, moderation, and mediation in Head Start REDI program. Development and Psychopathology.

Blair, C. (2002). School readiness: Integrating cognition and emotion in a neurobiological conceptualization of children's functioning at school entry. American Psychologist, 57, 111127.

Brooks-Gunn, J., Duncan, G., & Aber, L. (Eds.). (1997). Neighborhood poverty: Context and consequences for children (Vol. 1). New York: Russell Sage Foundation.

Brown, J. R., Knoche, L. L., Edwards, C. P., & Sheridan, S. M. (submitted). Professional development: A case study of early childhood programs in the Getting Ready Project.

Campbell, S. B., Shaw, D. S., & Gilliom, M. (2000). Early externalizing behavior problems: Toddlers and preschoolers at risk for later maladjustment. Development and Psychopathology, 12, 467488.

Conduct Problems Prevention Research Group (2002). Using the Fast Track randomized prevention trial to test the early-starter model of the development of serious conduct problems. Development and Psychopathology, 14, 925943.

Cosgrove, M., Fountain, C., Wehry, S., Wood, J., & Kasten, K. (submitted). Randomized field trial of an early literacy curriculum and instructional support system.

Diamond, A., Barnett, S., Thomas, J. & Munro, S. (2007). Preschool program improves cognitive control. Science, 318, 13871388.

Diamond, A., & Taylor, C. (1996). Development of an aspect of executive control: Development of the abilities to remember what I said and to Do as I say, not as I do. Developmental Psychobiology, 29, 315334.

Doctoroff, G. L., Greer, J. A., & Arnold, D. H. (2006). The relationship between social behavior and emergent literacy among preschool boys and girls. Journal of Applied Developmental Psychology, 27, 113.

Dodge, K. A., Pettit, G. S., & Bates, J. E. (1994). Socialization mediators of the relation between socioeconomic status and child conduct problems. Child Development, 65, 649665.

Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young childrens social and emotional competence: A randomized trial of the preschool PATHS curriculum. Journal of Primary Prevention, 28, 6791.

Domitrovich, C. E., Gest, S. D., Gill, S., Bierman, K. L., Welsh, J. & Jones, D. (revised and resubmitted). Fostering high quality teaching with an enriched curriculum and professional development support: The Head Start REDI program.

Duncan, G. J. (2003). Modeling the impacts of child care quality on childrens preschool cognitive development. Child Development, 74, 14541475.

Duncan, G., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., et al. (2007). School readiness and later achievement. Developmental Psychology, 43, 14281446.

Fantuzzo, J., Bulotsky-Shearer, R., McDermott,P. A., McWayne, C., Frye, D., & Perlman, S. (2007). Investigation of dimensions of social-emotional classroom behavior and school readiness for low-income urban preschool children. School Psychology Review, 36, 4462.

Fuligni, A. (revised and resubmitted). Childrens experiences in early childhood programs for low-income children: Influence of program type and curriculum use. Early Education and Development.

Gorman-Smith, D., & Tolan, P. (1998). The role of exposure to community violence and developmental problems among inner-city youth. Development and Psychopathology, 10, 101116.

Graziano, P. A., Reavis, R. D., Keane, S. P., & Calkins, S. D. (2007). The role of emotion regulation in childrens early academic success. Journal of School Psychology, 45, 319.

Greenberg, M. T., Riggs, N. R., & Blair, C. (2007). The role of preventive interventions in enhancing neurocognitive functioning and promoting competence in adolescence. In D. Romer & E. F. Walker (Eds.), Adolescent psychopathology and the developing brain: Integrating brain and prevention science (pp. 441462). New York: Oxford University Press.

Gresham, F. M., & Elliot, S. N. (1990). The Social Skills Rating System. Circle Pines, MN: American Guidance Services.

Hamre, B. K., & Pianta, R. C. (2001). Early teacher-child relationships and the trajectory of childrens outcomes through eighth grade. Child Development, 72, 625638.

Howse, R. B., Calkins, S. D., Anastopoulos, A. D., Keane, S. P., & Shelton, T. L. (2003). Regulatory contributors to children's kindergarten achievement. Early Education and Development, 14, 101119.

Izard, C., Trentacosta, C., King, K., Morgan, J., & Diaz, M. (2007). In G. Matthews, M. Zeidner, & R. Roberts (Eds.), Emotions, emotionality, and intelligence in the development of adaptive behavior: Knowns and unknowns. Series in affective science (pp. 127150). New York: Oxford University Press.

Justice, L. M., Cottone, E. A., Mashburn, A., & Rimm-Kaufman, S. E. (under review). Relationships between teachers and preschoolers who are at risk: Contribution of childrens language skills, temperamentally-based attributes, and gender. Journal of Speech, Language, and Hearing Research.

Klein, A., Starkey, P., Clements, D., Sarama, J. & Iyer, R. (in press). Effects of a pre-K mathematics intervention: A randomized experiment. Journal of Research on Educational Effectiveness.

Lengua, L. J. (2002). The contribution of emotionality and self-regulation to the understanding of childrens response to multiple risk. Child Development, 73, 144161.

Li-Grining, C. P. (2007). Effortful control among low-income preschoolers in three cities: Stability, change, and individual differences. Developmental Psychology, 43, 208221.

Li-Grining, C. P., Raver, C. C., Smallwood, K. M., Sardin, L., Metzger, M. W., & Jones, S. M. (submitted). Classroom quality and teacher support in Head Start classrooms: The role of teachers psychosocial stressors. Early Education and Development.

LoCasale-Crouch, J., Konold, T., Pianta, R., Howes, C., Burchinal, M., Bryant, D., et al. (2007). Profiles of observed classroom quality in state-funded pre-kindergarten programs and associations with teacher, program, and classroom characteristics. Early Childhood Research Quarterly, 22, 317.

Magnuson, K., Ruhm, C., & Waldfogel, J. (2007). The persistence of preschool effects: Do subsequent classroom experiences matter? Early Childhood Research Quarterly, 22, 1838.

Magnuson, K., & Duncan, G. (2003). Parent- versus child-based intervention strategies for promoting children's well-being. In A. Kalil & T. DeLeire (2004), Family investments in children's potential: Resources and parenting behaviors that promote success. Monographs in parenting (pp. 209235). Mahwah, NJ: Lawrence Erlbaum Associates.

Massey, S. L., Pence, K. L., Justice, L. M., & Bowles, R. P. (2008). Educators use of cognitively challenging questions in economically disadvantaged preschool classroom contexts. Early Education and Development, 19, 340360.

McClelland, M. M., Cameron, C. E., Connor, C. M., Farris, C. L., Jewkes, A. M., & Morrison, F. J. (2007). Links between behavioral regulation and preschoolers literacy, vocabulary, and math skills. Developmental Psychology, 43, 947959.

McClelland, M. M., & Morrison, F. J. (2003).The emergence of learning-related social skills in preschool children. Early Childhood Research Review Quarterly, 18, 206224.

McDermott, P., Green, L. F., Francis, J. M., & Stott, D. H. (1996). Preschool Learning Behaviors Scale. Philadelphia: Edumetric and Clinical Science.

McDermott, P. A., Leigh, N. M., & Perry, M. A. (2002). Development and validation of the Preschool Learning Behaviors Scale. Psychology in the Schools, 39, 353365.

OConnor, E., & McCartney, K. (2007). Examining teacher-child relationships and achievement as part of an ecological model of development. American Educational Research Journal, 44, 340369.

Preschool Curriculum Evaluation Research Consortium (2008). Effects of preschool curriculum programs on school readiness (NCER 20082009). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.

Pence, K. L., Justice, L. M., & Wiggins, A. K. (in press). Preschool teachers fidelity in implementing a comprehensive language-rich curriculum.

Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (submitted). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms.

Ramey, C. T., Ramey, S. L., & Stokes, B. R. (in preparation). Effective pre-K programs: Research evidence about program dosage and student achievement.

Ramey, S. L., Ramey, C. T., Kleinman, B. E., Lee, L. M., Farneti, C. L., Timraz, N. M., et al. (submitted). The effects of curriculum and coaching supports on classrooms and literacy skills of prekindergarten/Head Start students in Montgomery County Public Schools.

Raver, C. C. (2002). Emotions matter: Making the case for the role of young childrens emotional development for early school readiness. Social Policy Report, 16, 36.

Raver, C. C. (2004). Placing emotional self-regulation in sociocultural and socioeconomic contexts. Child Development, 75, 346353.

Raver, C. C., Garner, P., & Smith-Donald, R. (2007). The roles of emotion regulation and emotion knowledge for childrens academic readiness: Are the links causal? In R. C. Pianta, M. J. Cox, & K. L. Snow (Eds.), School readiness and the transition to kindergarten in the era of accountability (pp. 121147). Baltimore: Brookes Publishing.

Raver, C. C., Jones, A. S., Li-Grining, C. P., Metzger, M., Smallwood, K., & Sardin, L. (2008). Improving preschool classroom processes: Preliminary findings from a randomized trial implemented in Head Start settings. Early Childhood Research Quarterly.

Raver, C. C., Jones, S. M., Li-Grining, C. P., Zhai, F., Bub, K., & Pressler, E. (submitted). CSRP's impact on low-income preschoolers' pre-academic skills: Self-regulation and teacher-student relationships as two mediating mechanisms. Child Development.

Raver, C. C., Jones, S. T., Li-Grining, C., Zhai, F., Metzger, M., & Solomon, B. (revised and resubmitted). Multi-component intervention targeting childrens behavior problems in Head Start-funded preschool classrooms: A cluster-randomized controlled trial. Journal of Consulting and Clinical Psychology.

Raver, C. C., & Zigler, E. F. (1997). Focus section: New perspectives on Head Start. Social competence: An untapped dimension in evaluating Head Starts success. Early Childhood Research Quarterly, 12, 363385.

Riggs, N. R., Greenberg, M. T., Kusché, C. A., & Pentz, M. A. (2006). The mediational role of neurocognition in the behavioral outcomes of a social-emotional prevention program in elementary school students: Effects of the PATHS curriculum. Prevention Science, 7, 91102.

Rimm-Kaufman, S. E., Fan, X., Chiu, Y.-J., & You, W. (2007). The contribution of the Responsive Classroom Approach on childrens academic achievement: Results from a three year longitudinal study. Journal of School Psychology, 45, 401421.

Whitaker, S., Kinzie, M., Kraft-Sayre, M. E., Mashburn, A., & Pianta, R. C. (2007). Use and evaluation of web-based professional development services across participant levels of support. Early Childhood Education Journal, 34, 379386.

Zaslow, M., Reidy, M., Moorehouse, M., Halle, T., Calkins, J., & Margie, N. G. (2003). Progress and prospects on the development of indicators of school readiness. In Child and youth indicators: Accomplishments and future directions (in press). Bethesda, MD: National Institutes of Health.

A Synthesis of Federally-Funded Studies on School Readiness: What Are We Learning About Professional Development?

This paper is part of a series of working papers prepared for a meeting sponsored by the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE).Т  Abt Associates Inc and the National Center for Children in Poverty (NCCP) were funded to convene the meeting.Т  The views represented in this paper are those of the author(s) and do not necessarily reflect the opinions of the U.S. Department of Health and Human Services.

Introduction

Effective teachers have long been considered essential to a high-quality early childhood education that prepares children to succeed in school.  Recent expansions of publicly-funded preschool programs have increased the need for effective early childhood teachers who have the skills necessary to interact with children in ways that promote their social-emotional development, learning, and overall readiness to succeed in school.  But, what are the best ways to improve the effectiveness of existing early care and education teachers and staff?  This paper draws lessons from recent federally-sponsored research to explore that question.

The Scope of Professional Development and the Focus of This Paper

In the early childhood field, professional development (PD) is often the encompassing term used to refer to pre-service and in-service training and education, whether offered through institutions of higher education or through community-based training programs. Individuals entering the early childhood workforce undertake PD to gain an initial credential (e.g., Child Development Associate credential) or degree (typically either AA or BA). Existing members of the workforce undertake PD to achieve degrees and/or to increase their teaching skills, knowledge of a particular subject area, or to learn to implement a particular curriculum. It is that latter type of professional development (in-service training to help existing staff improve their teaching skills or to implement a new curriculum) that is the focus of most of the studies reviewed for this paper.

A recent, extensive review of the early childhood PD literature revealed many gaps in the research, with few studies systematically varying PD to explore its effects on teacher practices or childrens learning outcomes, or to investigate necessary threshold dosage levels, optimal content, or the possible mediating effects of teacher or program characteristics (Zaslow and Martinez-Beck, 2006). Ramey & Ramey (2008) summarize the state of the field: Content, amount, and format of professional development varies but has not been linked to specific classroom instructional practices that have proven effective in promoting childrens developmental outcomes (p. 45).

Instead of focusing on these types of issues, much of the early research in the field focused on the relationship between a teachers educational background, especially whether or not the teacher had a B.A. degree, and early child care and education (ECE) quality or child outcomes.  Some studies of center- and home-based ECE programs found that the quality of care and instruction (as measured by scales such as ECERS, FDCRS, and ITERS) was likely to be better when teachers possessed BA degrees than when they did not (e.g., Burchinal, Cryer, Clifford, & Howes, 2002). Teachers with more education and training in child development specifically were likely to have more sensitive and less harsh interactions with children (Howes, 1997). And, children in both centers and family child care homes were more likely to show better outcomes when their teachers had higher levels of education (Clarke-Stewart, Vandell, Burchinal, OBrien, & McCartney, 2002; Howes, Whitebook, & Phillips, 1992; Weaver, 2002).

Based on these and similar studies, many reviewers concluded that the best quality ECE programs were those in which teachers possessed BA degrees, especially in child development or similar fields (Barnett, 2004; Bowman, Donovan, & Burns, 2001; Whitebook, 2003).  Indeed, the National Academy of Sciences Committee on Early Childhood Pedagogy recommended that Each group of children in an early childhood education and care program should be assigned a teacher who has a bachelors degree with specialized education related to early childhood (Bowman, Donovan, & Burns, 2001, p. 13). The research findings and these and other similar recommendations helped spur changes in policy and practice such that many publicly-funded state preschool and Head Start programs now require lead teachers to have BA degrees.

But, recent studies have led some to re-examine the emphasis on teacher education levels.  Studies from the Family and Child Experiences Survey (FACES) in Head Start showed statistically significant but small-in-magnitude associations between teacher qualifications and classroom quality (ACYF, 2001) and between teacher credentials and childrens early writing skills (ACF, 2003). The National Center for Early Learning and Developments (NCEDL) Multi-State Study of Pre-Kindergarten suggested that students of teachers who had a BA degree or higher demonstrated greater learning gains in math skills but not in other academic areas than those whose teachers had less than a BA (Early, Bryant, Pianta, Clifford, Burchinal, Ritchie et al, 2006). Subsequent analyses of seven major studies of the relationship between classroom quality and childrens educational outcomes and the educational attainment and majors of their teachers yielded null or contradictory findings (Early, Maxwell, Burchinal, Alva, Bender, Ebanks, et al, 2007).

Based on these new findings, research is beginning to explore more nuanced questions about PD such as the threshold and the amount of education that make a difference for quality and outcomes, the characteristics of teachers undergraduate programs (because not all B.A. programs are of the same quality or have the same course content), outcomes other than childrens academic achievement such as social interactions and behavioral management, the impact of teacher and program characteristics on classroom quality and child outcomes, and the supports that can help teachers gain the most from their PD experiences (Bryant, Barbarin, Clifford, Early, & Pianta, 2004; Hyson, Tomlinson, & Morris, 2008).

These are precisely the types of studies that are needed to fill the gaps in the research identified by Zaslow and Martinez-Beck (2006).  They build on findings that suggest that training is related to improved quality of ECE programs (Burchinal, Howes, & Kontos, 2002; Burchinal, Cryer, Clifford, & Howes, 2002) and more sensitive interactions with children (Clarke-Stewart et al, 2002).  But, what training strategies are most likely to be effective?  While there is limited research in the early childhood field to answer this question, Tout, Zaslow, and Berry (2006) suggest that more intensive and longer duration training is likely to be better than brief training, and a recent review of in-service training for K-12 teachers concluded that one-day programs, in most cases, are not worthwhile (Loeb, Rouse, & Shorris, 2007, p. 8).

Other research suggests that ECE quality and teacher practices can be influenced by characteristics of the workplace or the teachers involved. For example, workplace characteristics such as the levels of education and training of teachers within an ECE program can affect individual teacher performance (Whitebook et al, 2001). Workplace characteristics such as teacher compensation and teacher turnover levels, program type (e.g., location in a school), teacher-child ratios, full- or part-day, and levels of poverty of children in the classroom have all been associated with classroom or program quality (Bryant et al, 2004; Kontos, Howes, Shinn, & Galinsky, 1994; Whitebook at al, 1990, Whitebook et al, 1993).

Similarly, teacher characteristics such as teachers views about teaching have been associated with their classroom teaching behavior and ability to incorporate new instructional practices (Bowman et al, 2001). Teacher attitudes and knowledge were also identified as mediators of the effect of teacher qualifications on classroom quality in a study of FACES, such that teacher qualifications were associated with significant positive changes only when teacher attitudes and knowledge were also taken into account (ACF, 2003).  Teachers knowledge of childrens cultural and family backgrounds, and teachers who serve as role models have been linked to improved teacher-child relationship (Saft & Pianta, 2001).

Emerging Federal Research as a Source for New Evidence Concerning PD

In the early 2000s, several federal agencies funded research aimed at promoting childrens school readiness. The four sets of projects (Preschool Curriculum Evaluation Research Grants program (PCER), the Interagency School Readiness Consortium (ISRC), Evaluation of Child Care Subsidy Strategies, and the Quality Intervention for Early Care and Education Program (QUINCE)) have different aims and approaches, but all share a commitment to rigorous research. The findings from these projects are still emerging, but they provide some of the most recent direct and indirect evidence concerning PD and its relationship to quality of early childhood services and outcomes for children.  This paper reviews studies from three of the four sets of projects to distill lessons learned concerning PD.[1]

An Underlying Logic Model

The federally-funded projects have been designed to test curricula, improve the quality of instruction, and promote one or more aspects of school readiness (e.g., early language/literacy, mathematics, or science skills, social-emotional development, general school readiness, and parent involvement; see Appendix C.2). But, most of these studies share a common assumption: PD (in the form of in-service teacher training) affects teacher practices in the classroom, and those practices in turn result in benefits for children.

The logic model below (Figure 1) represents the common assumptions underlying these projects and illustrates that workplace and teacher characteristics can affect the results. We use this logic model as a framework for this paper. Our review focuses more on changes in teacher behavior and practices and less on changes in children because other papers will address child outcomes, but we do highlight those studies that connect changes in instructional practices or specific PD strategies with changes in children.

As illustrated in Figure 1, most of the projects reviewed in this paper included training of teachers (an exception was LA EXCeLS) to help the teachers improve their teaching practices and the overall quality of their classrooms and/or to help teachers implement a specific curriculum. Programs typically employed trainers to work with the teachers initially, and, sometimes those or other individuals also served as ongoing coaches or mentors to help the teachers implement the skills they had been taught. In some projects, the training for the coaches/trainers was described. Presumably, such training would improve the coaching delivered to teachers. Changes in teacher behavior/instructional practices, therefore, were either a direct result of the training that the teachers received or resulted from the better training of the teachers delivered by the coaches. The effects of training could be moderated by workplace or teacher characteristics. Changes in instructional practices are hypothesized to result in better outcomes for children.

Figure 1.
Logic Model

The logic model in Figure 1 illustrates relationships among professional development, teacher characteristics, workplace characteristics, coaching practices, changes in teacher behavior and instructional practices, and enhanced child skills, achievement, and development. In this model, professional development is divided into three types of training — training to improve teaching practices, training to implement a curriculum, and training of coaches that provide individualized support for teachers. According to the model, professional development training is hypothesized to affect changes in teacher behavior and instructional practices. In the case of training for coaches, the training is hypothesized to contribute to improvement in coaching practices, which in turn are believed to result in changes in teacher behavior and instructional practice. For all three types of professional development training, teacher characteristics and workplace characteristics are assumed to affect the relationship between training and changes in teaching. According to the last phase of the model, the resulting changes in teacher behavior and instructional practices are hypothesized to contribute to enhanced child development, skills, and achievement.

This paper begins by describing the projects included in the review and what insights they have to offer about PD.  Many of the projects were not developed as tests of PD strategies, to draw lessons about PD, so there are limits to the conclusions we can draw based on this body of work.  Next, project results are summarized in two ways:  (1) with a focus on the sub-set of the projects that directly tested different PD approaches; and then (2) across all projects, with findings reported in relation to the logic model.  We conclude by offering suggestions for future research.

Approach to This Review

Principal investigators submitted papers, presentations, and posters representing 15 projects.  Twelve projects (all but LA ExCELS, and the articles by Fantuzzo, Bulotsky-Shearer, McDermott, McWayne, & Perlman (2007) and Powell, Burchinal, File, & Kontos (2008)) were reports of the effects of specific training interventions that also included details concerning professional development activities.  While we draw lessons from all the projects, we focus most on these 12 projects.  Further, four projects submitted studies in which PD strategies (e.g., mentoring versus non-mentoring; mentoring of different intensity) were tested explicitly.  Table 1 lists the 15 projects reviewed, the 12 with relevant data regarding PD, and the 4 that explicitly compared PD strategies (Lets Begin with the Letter People/Doors to Discovery; Literacy Express; MyTeachingPartner; and Building Language for Literacy).  This review highlights findings from these four projects, although results from all the studies were examined for patterns related to PD or workplace and teacher characteristics associated with the effects of PD on teacher practices, program quality, or child outcomes.

Appendices 1-12 provide detailed information regarding the projects and submitted studies, including methods, measures, the PD strategies employed, and results. Briefly, the highlights of these appendices and the main implications for this review are as follows:

• Study design (see Appendix C.1):  While 11 of 12 projects employed randomized trial designs, only the four mentioned above were designed to hold curriculum constant, making it possible in those four studies to assess the effects of PD strategies without confounding them with the effects of the curriculum.
• Sample sizes (see Appendix C.1) ranged from 6 to 55 per group for analyses at the classroom level, and from 6 to 89 at the teacher level, with much larger groups for analyses at the child level.  For some of the teacher/classroom-level analyses, therefore, small sample sizes may limit finding significant effects or the generalizability of results.
• PD as a package of strategies:  Most projects typically employed one or more of several PD strategies as their in-service training approach (see Table 2). For most projects, therefore, it is not possible to determine which of the specific strategies in their PD package might be exerting more, less, or any effect on teacher practices or child outcomes. The most commonly employed strategies were workshops and coaches/mentors who worked with teachers to help them implement what they learned via workshops.
• Important factors not described. Since most of these studies were not designed as tests of PD approaches, they did not describe factors that might influence the effectiveness of the PD. For example, the studies did not always describe the qualifications of coaches/mentors/trainers, the PD they received, whether teachers and assistant teachers were both trained as part of the projects (see Appendix C.6), or the PD provided to control groups (see Appendix C.1). This makes it difficult to draw definitive conclusions about these factors.
• Mapping results onto the logic model, and measuring changes in implementation and teacher behavior. The logic model in Figure 1 suggests that training can be delivered for three purposes (to improve teacher practices, to help teachers implement a curriculum, and to train coaches). Most of the projects report results both for implementation of a curriculum and for improving teacher practices (see Appendix C.9). In some studies, the measures used to assess the implementation of the curriculum are conceptually similar to measures used to assess changes in classroom practices (e.g., a measure of the implementation of a curriculum designed to promote early language might be the extent to which teachers used open-ended questions to promote conversation and vocabulary, but that might also be considered a measure of change in teacher behavior). In this review, we report the effects of PD on implementation and teacher behavior separately, but we note that there is some conceptual overlap.
• Workplace characteristics. All 12 projects took place primarily (though not solely) in settings that serve low-income children, including Head Start, publicly-funded preschool, and/or community child care programs (see Appendix C.4). However, it is not clear if projects used terms consistently (e.g., a setting might be described as a Title I program in one study but a school-based preschool program in another), which limits the conclusions that can be drawn regarding the effects of auspices. Other workplace characteristics such as incentives to encourage participation in training (see Appendix C.5) and teacher turnover were described less frequently.
• Teacher characteristics. Projects differed in the extent to which they described the characteristics of participating teachers (see Appendix C.7). Most of the teachers described were either Caucasian or African-American and English-speaking. In the half of the projects that reported on teachers educational level or experience, most teachers had BAs or degrees higher than BAs, which may limit the extent to which these results generalize to the broader early childhood workforce, especially those in non-school-based settings.
• Emerging research. The studies submitted for this review represent only some of the federally-funded research that is relevant. More studies are forthcoming, both for the projects included in the review and for others (e.g., QUINCE), and many more details regarding PD and its effects will undoubtedly become available.

Results: What We Know

In this section, we report, first, from the four projects that explicitly tested different approaches to professional development and therefore provide the most direct evidence about PD (see Appendices 1-12 for additional details).  Then, second, we report findings across all projects, organized according to the logic model presented in Figure 1.

Projects that Tested Effectiveness of Specific PD Strategies

Four projects conducted randomized trials that held constant the effects of curriculum, isolating the effects of particular PD strategies. These four examined the use of mentors/coaches and the effects of differing levels of coaching intensity.

Lets Begin with the Letter People/Doors to Discovery

Assel, Landry, Swank, & Gunnewig (2006) report on the first-year results of a randomized trial designed to test two curricula (Lets Begin with the Letter People and Doors to Discovery) and two PD strategies (mentoring and non-mentoring). The authors hypothesized that (1) mentoring would help teachers implement a curriculum; (2) children would show greater academic gains when their teachers have been mentored than when they have not; and (3) the impact of mentoring would be greatest in classrooms with teachers who have lower levels of education.

School sites that had Head Start, Title I pre-K, and universal pre-K programs were randomly assigned to, first, a curriculum condition (Lets Begin with the Letter People or Doors to Discovery) and then to a PD condition (mentoring or non-mentoring), resulting in assignment to one of five conditions: (1) Lets Begin with the Letter People, mentored; (2) Lets Begin with the Letter People, non-mentored; (3) Doors to Discovery, mentored; (4) Doors to Discovery, non-mentored; and (5) control group. Seventy-six classrooms, 76 teachers, and 603 children participated in the study.

Teachers in experimental groups were trained in their curriculum in a four-day summer workshop that was teacher-centered, employed small groups, and focused on curricula-specific content for promoting language, literacy, and social-emotional skills. During the year, curriculum mentors supported teachers as they implemented their assigned curriculum. Mentors worked with teachers twice per month in classroom coaching sessions that focused on lesson planning, room arrangement, schedules, behavioral issues, curriculum fidelity, and demonstration of curriculum components.

Mentors completed fidelity checklists designed for each curriculum three times during the year, for both mentoring and non-mentoring classrooms. Fidelity of implementation improved over time for both curricula: Just 29.8% of Lets Begin teachers scored at high levels at the first evaluation, but that figure had increased to 71.5% by mid-year (comparable figures for Doors to Discovery were 28.6% and 59.6%, respectively). The authors speculate that implementation of Lets Begin might have been better because it has a single user-friendly teacher guide, as compared with the multiple guides teachers had to consult for Doors to Discovery.

Study findings revealed few main effects for mentoring across all settings or curricula or for all outcome measures.  For example, on phonological awareness, children in Title I and universal pre-K classrooms with mentoring had significantly greater gains than children in non-mentored classrooms regardless of curricula, but children in Head Start classrooms did better in non-mentored classrooms. Children in Title I classrooms using Doors to Discovery showed greater growth in vocabulary whether or not there was mentoring, while children in the Lets Begin classrooms did better when their teachers had been mentored. The authors summarize these and other findings by saying, When mentoring showed a positive impact, it was only in the Title I or universal pre-K classrooms, and, further, that benefits were more likely to be within the public school system utilizing Lets Begin and within the literacy rather than the language domain.

The authors suggest that, had more intense mentoring been offered, the results for mentoring might have been stronger.  They also note that all teachers received feedback about implementation of curricula, which may have lessened the impact of the mentoring overall, as that feedback could have served as a kind of intervention.

MyTeachingPartner (MTP)

Two studies of MTP, a web-based PD model designed to improve teachers instructional practice and interactions with children to promote language/literacy and social skills, were submitted for this review (Pianta, Mashburn, Downer, Hamre, & Justice, in press; Whitaker, Kinzie, Kraft-Sayre, Mashburn, & Pianta, 2007).

Whitaker et al (2007; also described in Kinzie, Whitaker, Neesen, Kelley, Matera, & Pianta, 2006) explored the relationship between levels of teacher support and teacher participation in MTP training activities. A total of 235 teachers were assigned to one of three levels of service support: (1) a Web-Only group that received a laptop computer and access to the MTP website; (2) a Materials group that received the same plus printed versions of MTP curricula and their corresponding materials; and (3) a Consultancy group that received all of the above plus a video camera to tape their classroom practice for bi-weekly on-line discussions with a teacher consultant who reviewed the video clips and provided feedback and recommendations.

Teachers in the Consultancy group logged on to the website more often than teachers in the Web-Only and Materials groups, but the Materials group spent more time on-line than did members of the other groups. The Consultancy and Web-Only groups agreed in their views of MTPs usefulness, but the Materials group responded significantly less positively than the other two groups. The authors conclude that teachers will voluntarily participate in PD if they find it useful and if they receive the level of supports they feel they need.

In a second study reporting on a randomized controlled trial of MTP, Pianta et al (in press) compared the effects of Consultancy versus Web-Only supports on the quality of observed teacher-child interaction in pre-K classrooms. Teachers in the Web-Only condition received materials and access to the MTP web-site, which included video clips of high-quality teaching exemplars. Teachers in the Consultancy group videotaped their own classroom teaching and sent the videotapes to a consultant (mentor) who provided feedback to teachers in on-line video chats twice each month over the course of the school year.

In the first year of the two-year study, teachers in the Consultancy group had more sensitive interactions with students, were better at engaging students in instruction, and improved their ability to stimulate childrens language more than teachers in the Web-Only group. The effects of Consultancy on teacher behaviors were moderated by the level of poverty of children in the classroom. Specifically, when 50% of the children in classrooms were classified as poor, there were no differences in the rates of change between teachers in the Consultation and Web-Only conditions. But, when 100% of the children in the classrooms were poor, then the teachers in the Consultation group had greater increases in the quality of teacher-child interactions than teachers in the Web-Only condition. This latter finding suggests that PD interventions in classrooms with a high density of children from low-income households may need to look different with respect to intensity and/or supportiveness for teachers than PD interventions in other classrooms.

All classrooms were in publicly-funded pre-K programs, and the teachers in these studies were highly experienced (averaging 15 years teaching; one-third with advanced degrees), so it is unclear if these findings would apply to other settings or to teachers with other backgrounds.

Literacy Express

Literacy Express is a comprehensive preschool curriculum for three- to five-year-olds with units on oral language, emergent literacy, basic math, science, general knowledge, and social-emotional development. The study submitted for this project included a brief description of a randomized trial that compared three groups: (1) training via workshops; (2) training via workshops plus mentoring; and (3) a business as usual control group.  As described in greater detail in Lonigan, Farver, Clancy-Menchetti, & Phillips (2005),[2] a total of 48 preschools (mostly Head Start centers) in Florida and California were randomly assigned to one of three PD conditions: Literacy Express workshops only (15 schools), Literacy Express workshops plus mentoring (15 schools), or a business as usual comparison group (18 schools).

In the workshop group, teachers and aides participated in a 2-day Literacy Express initial workshop plus three ½-day workshops during the school year. Teachers and aides in the mentoring group participated in the same workshops and received classroom visits by a trained project mentor. Preschools in the business-as-usual comparison group used the preschools standard curricula (most often High/Scope or Creative Curriculum) (Lonigan et al, 2005; Preschool Curriculum Research Consortium, 2008).

Results revealed statistically significant increases in print knowledge for children in the workshop-plus-mentoring group, but no differences were found between children in the two groups in phonological processing, oral language, or math.

Building Language for Literacy

The purposes of this randomized trial (Ramey, Ramey, Kleinman, Lee, Farnett, Timraz, et al, no date) were to understand the factors and instructional practices that promote childrens language and literacy in the context of Scholastics Building Language for Literacy (BLL) curriculum and two levels of coaching (monthly versus weekly).  The project emerged from a ten-year partnership between the Montgomery County Public Schools (MCPS) in Maryland and Georgetown Universitys Center on Health and Education. Three hypotheses addressed PD: (1) BLL coaching will lead to significant benefits in classroom instructional environment; (2) children with teachers who have BLL coaches will have significantly higher literacy levels; and (3) weekly coaching will lead to greater benefits than monthly coaching.

All classrooms used the BLL language and literacy curriculum. Twenty-four classrooms were randomized into the intervention (coaching) or control conditions, and the classrooms in the intervention condition were further randomized to either weekly (30 sessions) or monthly (8 sessions) coaching.

In the two intervention conditions, PD consisted of a three-day summer institute to introduce the curriculum (2 days for paraeducators), coaching, plus optional monthly evening group sessions with coaches and peers for more discussions. The teachers time for the evening sessions was covered by district stipends, and teachers could earn up to 16 units of PD credits. Coaches had Masters degrees in reading with additional training on the BLL curriculum, and they received ongoing supervision during the course of the study.

PD in the comparison condition consisted of the PD offered to all MCPS pre-K/Head Start classrooms. Certified teachers could participate in a voluntary summer training institute; aides could participate in a half-day of instruction. Teachers also had access to additional PD days and supervisors who were content specialists throughout the year. Teachers in all classrooms were certified teachers with a specialty in early childhood education.

Results indicated that BLL coaching classrooms had significantly higher levels of curriculum implementation than comparison classrooms. Contrary to the hypothesis on intensity, teachers who received monthly coaching had better implementation scores than teachers who received weekly coaching.

Classrooms in both coaching conditions had higher scores on the Early Language and Literacy Classroom Observation (ELLCO) than control group classrooms, but there were no differences in ELLCO scores between the weekly and monthly coaching groups.  Although no statistical tests were conducted, the authors report that teachers in the coaching condition may have displayed better performance on the Rameys Observation of Learning Essentials (an observational measure of teacher behavior).

Three measures of child outcomes, all focused on childrens early language and literacy skills (Test of Early Reading Ability (TERA); Get It, Got It, Go!; and a school district pre-K reading measure) were used to assess differences in the gains children made from fall to spring during the study year. Children in coaching conditions showed greater gains than children in the control group in TERA scores (total scores, and two of three subscales), but there were no significant differences in gains between children in weekly and monthly coaching conditions. Children who were English Speakers of Other Languages achieved greater fall-to-spring gains in TERA scores if they were enrolled in coaching rather than non-coaching conditions. There were no differences among any groups on the other two measures of child outcomes.

In summary, BLL coaching resulted in higher levels of literacy-rich classroom environments and instructional practices and higher early literacy skills on one standardized measure of childrens reading ability compared to typical classrooms. There was no benefit of the more intense coaching. This may perhaps be due to the small sample size, as there were only six classrooms in each of the coaching conditions. The small sample sizes, school-based settings, and high educational levels of teaching staff may also limit generalizability.

Examining the Results Across All Projects

The results of the studies above are direct tests of approaches to PD. This section examines those findings as well as those of the studies submitted by other projects, reporting all of them according to the logic model in Figure 1.

Effects of PD on Implementation of Curricula

Eleven of the 12 projects employed teacher training to help teachers implement a particular curriculum (see Appendix C.1). Ten of 12 reported on changes in teacher behavior as evidence of implementation of the curriculum or of the intervention on which teachers were trained. Projects employed implementation checks at frequencies ranging from three times per year to ongoing, primarily using curriculum-specific checklists or measures of implementation (see Appendices 9-10). Study not in the set submitted but used to supplement information.

Four projects (Childrens School Success, Project Upgrade, Language-Focused Curriculum, Lets Begin with the Letter People/Doors to Discovery) suggest that teachers gradually achieved better implementation and stronger fidelity over time, presumably as they had more practice. But, more frequent visits by a coach were not always associated with better implementation (e.g., Building Language for Literacy).

It makes intuitive sense that dosage of PD (or curriculum) is associated with both implementation and outcomes. The Childrens School Success project proposed a useful definition of dosage that combines fidelity, implementation, and child attendance. The authors conclude that, for an intervention to benefit children, teachers must deliver the curriculum as intended, the whole curriculum must be delivered, and children must attend class to receive the intervention. Their results suggest that fidelity measures were significantly associated with childrens post-test performance, but those associations were sometimes moderated by childrens attendance (and their pre-test performance). Similarly, the Language-Focused Curriculum project found that children in intervention classrooms who had better attendance benefited more than children with weaker attendance. These two projects suggest that future studies of PD should monitor fidelity, implementation, as well as child attendance.

Effects of PD on Changes in Teacher Practices

Appendix C.12 describes changes in teacher practices observed in the 12 studies. In addition to the MTP study that directly assessed and reported positive changes in teacher practices as a result of mentoring/coaching, four other projects that employed coaching/mentoring reported positive changes in teaching behaviors. For example, the Chicago School Readiness Project found the emotional climate of the classroom improved in intervention (5-6 Saturday workshops plus mental health consultants/coaches) classrooms, and teachers were more enthusiastic and responsive in their interactions with students and displayed fewer emotionally negative practices. In Project Upgrade, teachers in the intervention (workshop/coaching/curricula) groups out-performed members of a control group on behaviors related to promoting literacy (e.g., support for oral language, print knowledge, print motivation, support for phonological awareness, literacy resources, and literacy activities). Although these studies were not designed to isolate the effects of mentoring from the effects of other PD strategies employed in the projects or from the curriculum the PD was designed to help teachers implement, they may provide some support for the value of mentors/coaches in changing teacher practices.

Some teacher behaviors appear harder to change than others. For example, in the Language-Focused Curriculum project, researchers recorded the extent to which teachers made changes in classroom activity contexts (e.g., setting up a dramatic play corner for the week) and in instructional processes (e.g., asking open-ended questions to promote early literacy skills).  Activity contexts were more likely to be implemented soon after training, while changes in instructional processes took longer to achieve.  In Project Approach, an observational study of childrens engagement in public preschool classrooms, the researchers report that teachers were reluctant to work with students in small rather than large groups, even after training. These results suggest that future PD research might seek to establish the types of teacher behaviors that are harder to change and the specific PD strategies that might be more effective with such hard-to-change behaviors.

Effects of PD for Coaches/ Mentors/Consultants

Eleven projects employed individuals described as coaches, mentors, or mental health consultants. Their responsibilities included training teaching staff on the curriculum, visiting classrooms to observe the teaching staff in action and to model appropriate implementation of the curriculum, providing feedback to the staff, facilitating group meetings with teachers to reflect on practices, barriers, and successes, and, in the Chicago School Readiness project, providing stress reduction services to teachers and direct one-to-one mental health services to a few children in each classroom. But, despite the central role played by the coaches, most studies contained limited information about them, the training or supervision they received, or the effects of that training on their coaching skills or performance (see Appendix C.8). Future PD research specifically designed to identify the best approaches to PD for coaches would be useful.

The Effects of PD on Children

As described above, two of the four studies that tested specific PD approaches (BLL and MTP) suggest that coaching/mentoring produced better outcomes for children compared to PD that did not include mentoring, but the four studies also suggest that outcomes for children can vary depending on curricula, auspices, and outcome being assessed (see Appendix C.12).

Other projects that included coaching/mentoring also produced benefits in child outcomes (e.g., Project Upgrade, Head Start REDI, Pre-K Mathematics), but it is not possible to say what elements of the PD/curriculum interventions in those projects were associated with the benefits. However, across all 12 projects, the lone effort that produced no significant effects on either classroom/instructional quality or child outcomes was also the only project that did not include a coaching component along with its workshops (Language-Focused Curriculum).

Workplace Characteristics and Outcomes

The submitted studies described workplace characteristics such as program auspices/settings, incentives provided to PD recipients, poverty of children enrolled in the participating programs, and teacher turnover.

Auspices/Setting. All projects took place in at least some Head Start, publicly-funded preschool, and/or community-based child care programs (see Appendix C.4).  As described above, in the Lets Begin with the Letter People/Doors to Discovery project, the authors concluded that, when mentoring made a difference in child outcomes, it was mostly in Title I/ UPK classrooms, rather than in Head Start. In contrast, the Pre-K Mathematics project found no differences in effects on children across the participating Head Start and state preschool programs. Because of the mixed findings and the fact that few studies examined this issue directly; no firm conclusions about auspices can be drawn, except that future PD research should consider the effects of different preschool settings.

Childrens Poverty Level. Most projects operated in settings with a high percentage of low-income children. The MTP project, which reported the effects of PD by childrens poverty level, showed effects of consultancy when 100% of children were in poverty but no effects when 50% of children were in poverty. Because these subgroup analyses were not based on original randomization of the study, it is possible that the results are due to selection bias or some other factor. Nevertheless, the findings suggest that the impact of high levels of poverty should be assessed in future PD research.

Incentives.  Half of the 12 projects mentioned that teachers or programs were provided with incentives to encourage participation (see Appendix C.5). The incentives primarily included financial compensation for the time of the teachers and free sets of curricular materials. Only one project (Building Language for Literacy) allowed teachers to earn PD units for participation, which is a strategy that many ECE programs nationally are using. There were no direct tests of the effectiveness of these incentives in any project. Nevertheless, because of their policy importance, incentives may be an area for future PD research to explore.

Teacher Turnover. In Project Upgrade, turnover ranged from 28% to 44% in intervention classrooms to 49% in control group classrooms over the two-year period of the project.  The coaches/mentors in the project identified high turnover as a barrier to effective implementation. Turnover was either not reported or was lower in most other submitted studies.

Teacher Characteristics and Outcomes

Studies described teacher characteristics such as years of education, educational degrees, demographics, and teacher motivation (see Appendix C.7). Results suggest some teacher behavior or child outcomes can be affected by particular teacher characteristics, but, in other cases, training/PD/curricula interventions exert their effects without moderation by teacher characteristics.

Teacher Education Levels and Years of Experience

In the Childrens School Success and Pre-K Mathematics projects, teachers without BAs or with fewer years of experience implemented the curriculum or changed their classroom (Pre-K Mathematics) practices about as much as did teachers with BAs or with many years of experience. In the Early Literacy and Learning Model, changes in child outcomes were not affected by teachers educational level, leading the authors to conclude, This suggests that ELLM is successful in addressing the preparation deficiencies of early childhood and child care educators, though the issue merits further study (Cosgrove et al, 2006, p. 25).

But, in the LA ExCELS observational study, better classroom emotional climate was observed in settings taught by educators with BA degrees in a child development major, across all settings.

Teacher Language. Project Upgrade found PD interventions benefited teachers differently depending on their initial education levels, their dominant languages, and the outcomes being observed. While the curricular/PD intervention had strong effects on teacher behavior overall, the impacts were stronger for teachers whose primary language was Spanish than for their English-speaking peers. Further, for the two curricula that produced benefits for children, the benefits were larger for children in classrooms with Spanish-speaking teachers.

Teacher Attitudes and Motivation. In the Chicago School Readiness project, teachers who demonstrated a high level of dedication to their own PD (63% attended three or more of the Saturday trainings) were more likely to implement the curriculum as fully intended. In Project Upgrade, mentors reported that the best implementers of the curricula had, among other things, a positive attitude toward instructional change, while resistance to instructional change was a barrier to implementation.

In sum, teacher education, language, and motivation may influence the impact of a PD intervention and further research is warranted.

What We Dont Know and Recommendations for Future Research

These projects do much to illuminate the process and importance of professional development. Overall, findings suggest that teacher training and ongoing supports can help improve the implementation of curricula, and that such training and support is often associated with improvements in teacher behavior and instructional practice, and enhanced child outcomes. However, benefits are influenced by characteristics of the workplace and teachers, the type of professional development activities, and the intensity of supports. Because most of the studies were not designed to specifically test PD approaches, these findings are suggestive, not conclusive.  Nevertheless, the projects provide clues about areas where future research and exploration would be helpful.

How Can Research Provide a More Complete Picture of Professional Development and Its Effects?

Most of these studies were not designed and did not attempt to trace all the links across the full logic model.

Recommendation for Future Research:  Support analyses in existing and any new studies that trace the links from different types of PD to both shorter-term changes in teacher or instructional practice and longer-term changes in childrens academic achievement or social-emotional skills, while also examining how those effects are heightened or moderated by workplace and teacher characteristics.

What are the Threshold Levels of Implementation, Fidelity, and Dosage, and How Can They Be Measured?

These and previous studies of PD suggest that implementation, fidelity, and dosage are multi-dimensional concepts important for achieving good outcomes, but there is no definitive information about how best to measure these constructs, and how much of any given professional development activity is needed to achieve desired results.

Recommendation for Future Research:  Support the development of measures of implementation that identify the most critical elements of effective program delivery. Include regular reporting on child attendance as part of discussions of dosage and implementation. Conduct additional research to compare the effects of different types and amounts of professional development on these constructs.

How Does Coaching/Mentoring Produce its Effects, and How Should Coaches/Mentors Be Prepared for Their Roles?

Most projects were not designed to study the effectiveness of coaches/mentors, so descriptions of their backgrounds or of the PD they received were limited. While coaching/mentoring appears to be effective, it is not possible to draw conclusions from these studies about how it is producing its effects, or what pre-service or in-service training, professional development, or work experiences an effective coach should possess.

Recommendation for Future Research: Directly assess coaching and mentoring by conducting studies to: (1) determine what specific activities occur during coaching/mentoring that result in the most positive changes in teaching and instructional practice; (2) explore how much coaching and mentoring are needed to produce desired results for different types of teachers (e.g., new, or less educated or experienced teachers); or (3) test the effectiveness of coaches/mentors with varying backgrounds or experiences.

What Professional Development Strategy or Combination of Strategies Produces the Greatest Impact?

Most of the projects delivered multiple professional development strategies (e.g., workshops, coaching, individual/group reflections, etc.) as integral parts of a single training package, making it impossible to isolate the effects of a specific PD strategy.

Recommendation for Future Research:  Compare the effects of individual training strategies to determine the impact each has on teacher behavior/instructional practices and child outcomes.

How Do Teacher and Workplace Characteristics Influence Professional Development and Outcomes?

Generally, these projects produced improvements in teaching practices and outcomes for children, though sometimes those main effects were moderated or heightened by teacher and workplace characteristics (language teachers speak, teacher motivation, poverty status in the classroom, and program auspice).

Recommendation for Future Research:  Conduct studies that explore how teacher and workplace characteristics influence the effects of PD. Develop and test new professional development approaches to better meet the needs of teachers and programs for whom existing approaches may not work as well.

What are the Best Ways to Support All Educators, Not Just Lead Teachers?

A few of the reviewed projects urged future professional development activities be delivered to the whole teaching staff (not just the lead teacher) and involve program administrators. It is not clear from the results of the reviewed projects if these approaches yield greater benefits.

Recommendation for Future Research:  Assess professional development approaches that target not only lead teachers, but assistant teachers and aides as a team, as well as program administrators or school principals who provide supports and set the tone for what educators do in the classroom.

What Professional Development Strategies Will Best Benefit New Teachers and Teachers with Less Education and Experience?

The projects included in this review focus mostly on PD for existing staff, rather than on preparing new entrants for work in the early childhood field. Many participating teachers had BAs and more years of experience than may be reflective of the general ECE workforce. The projects therefore did not address questions related to PD for new entrants into the field, for existing staff with limited experience or degrees, or existing staff who undertake PD to achieve higher levels of education.

Recommendation for Future Research:  Explore the use and test the effectiveness of various PD practices with different populations: (1) new entrants into the ECE field; (2) existing staff with limited professional experience, and (3) existing staff striving for higher levels of education.

What is the Best Way to Alter Hard-to-Change Practices?

In several studies, it appears that some teaching practices are harder to change than others (e.g., working in small rather than large groups; changing instructional processes rather than classroom activity contexts), but there is not much information on how PD can be delivered or targeted to change those practices.

Recommendation for Future Research:  Support analyses of existing data to determine which practices are hardest to change. Conduct new research to test PD approaches that can help teachers alter those practices.

Conclusions

In sum, these projects demonstrate that PD can produce benefits in teacher behavior and instructional practices and in child outcomes ranging from academic achievement to social-emotional development. Generally, teachers trained on a curriculum demonstrated improvements in their classroom instructional practices, and children showed benefits in outcomes. This set of projects provides four rigorous assessments of PD, particularly as it relates to coaching and mentoring. When it comes to the benefits of particular PD strategies, the conclusions from this set of emerging findings are more limited. There is also new information about the relationship of PD and implementation of curricula with fidelity. However, results vary across curricula and outcomes, and are affected by workplace and teacher characteristics variations that have not yet been consistently considered or reproduced across projects. Exploring these factors in future research can help provide even more information about PD so that effective teaching supports can be put in place that ensure children are prepared to enter school ready to succeed.

References

Administration for Children, Youth, and Families (ACYF).  (2001).  Head Start FACES:  Longitudinal findings on program performance.  Third progress report.  Washington, D.C.: U.S. Department of Health and Human Services.

Administration for Children and Families (ACF). (2003).  Head Start FACES:  A whole-child perspective on program performance. Fourth progress report.  Washington, D.C.: U.S. Department of Health and Human Services.

Barnett, W. S. (2004). Better teachers, better preschools: Student achievement linked to teacher qualifications. Preschool Policy Matters, 2. New Brunswick, NJ: NIEER.

Bryant, D., Barbarin, O., Clifford, R.M., Early, D., & Pianta, R.  (June 2004).  The National Center for Early Childhood Development and Learning: Multi-state study of pre-kindergartenPresentation at the National Association of the Education of Young Childrens 13^th National Institute for Early Childhood Professional Development, Baltimore, MD.

Bowman, B.T., Donovan, M.S., & Burns, M.S. (Eds.). (2001).  Eager to learn: Educating our preschoolers.  Washington, DC.:  National Academy Press.

Burchinal, M. R., Howes, C., & Kontos, S. (2002). Structural predictors of child care quality in child care homes. Early Childhood Research Quarterly, 17, 87-105.

Burchinal, M. R., Cryer, D., Clifford, R. M., & Howes, C. (2002). Caregiver training and classroom quality in child care centers. Applied Developmental Science, 6(1), 2-11.

Clarke-Stewart, K. A., Vandell, D. L., Burchinal, M. R., O'Brien, M., & McCartney, K. (2002). Do features of child care homes affect children's development? Early Childhood Research Quarterly, 17, 52-86.

Early, D., Barbarin, O., Bryant, D., Burchinal, M., Chang, F., Clifford, R., Crawford, G., Weaver, W., Howes, C., Ritchie, S., Kraft-Sayre, M., Pianta, R., & Barnett, W.S.  (2005).  Pre-Kindergarten in Eleven States:  NCEDLs Multi-State Study of Pre-Kindergarten & Study of State-Wide Early Education Programs (SWEEP).  Preliminary Descriptive Report.  University of North Carolina-Chapel Hill.

Early, D., Bryant, D., Pianta, R., Clifford, R., Burchinal, M., Ritchie, S., et al.  (2006).  Are teachers education, major, and credentials related to classroom quality and childrens academic gains in pre-kindergarten?  Early Childhood Research Quality, 21, 174-195.

Early, D., Maxwell, K., Burchinal, M., Alva, S., Bender, R., et al.  (2007).  Teachers education, classroom quality, and young childrens academic skills:  Results from seven studies of preschool programs.  Child Development, 781(2), 558-580.

Howes, C. (1997). Childrens experiences in center-based child care as a function of teacher background and adult-child ratio. Merrill-Palmer Quarterly, 43(3), 404-425.

Howes, C., Whitebook, M., & Phillips, D. (1992). Teacher characteristics and effective teaching in child care: Findings from the National Child Care Staffing Study. Child & Youth Care Forum, 21(6), 399-414.

Hyson, M., Tomlinson, H.B., & Morris, C.  (2008). Quality improvement in early childhood teacher education:  Faculty perspectives and recommendations for the future.  Manuscript under review.

Kontos, S., Howes, C., Shinn, M., & Galinsky, E. (1994). Quality in family child care and relative care. NY: Teachers College Press.

Loeb, S., Rouse, C., & Shorris, A. (2007). Introducing the issue. Excellence in the classroom. The Future of Children, 17(1), 3-14.

Lonigan, C. J., Farver, J. M., Clancy-Menchetti, J., & Phillips, B. M. (2005, April). Promoting the development of preschool childrens emergent literacy skills: A randomizedevaluation of a literacy-focused curriculum and two professional development models. Paper presented at the biennial meeting of the Society for Research in Child Development, Atlanta, GA.

Preschool Curriculum Evaluation Consortium (2008). Effects of preschool curriculum programs on school readiness. (NCER 2008-2009). Washington, D.C.: Naitonal Center for Education Research, Institute of Education Sciences, U.S. Department of Education, Washington, D.C.: U.S. Government Printing Office.

Ramey, S.L. & Ramey, C.T. (2008). The effects of curriculum and coaching supports on classrooms and literacy skills of prekindergarten/Head Start students in Montgomery County Public Schools. Unpublished Manuscript. Georgetown University Center on Health and Education. Wahsington, DC.

Saft, E.W. & Pianta, R.C. (2001). Teachers perceptions of their relationships with students: Effects of child age, gender, and ethnicity of teachers and children. School Psychology Quarterly.16, 125141.

Tout, K., Zaslow, M., & Berry, D. (2006). Quality and qualifications: Links between professional development and quality in early care and education settings. In (M. Zaslow & I. Martinez-Beck, Eds.), Critical issues in early childhood professional development. Baltimore: Brookes Publishing.

Weaver, R.H.  (2002). Predictors of quality and commitment in family child care:  Provider education, personal resources, and support. Early Education and Development, 13(3), 265-282.

Whitebook, M., Howes, C., & Phillips, D. (1990). Who cares? Child care teachers and the quality of care in America. Oakland, CA: Child Care Employee Project.

Whitebook, M., Phillips, D., & Howes, C. (1993). National Child Care Staffing Study revisited: Four years in the life of center-based child care. Oakland, CA: Child Care Employee Project.

Whitebook, M., Sakai, L., Gerber, E., & Howes, C. (2001).  Then and now:  Changes in child care staffing 1994-2000, Technical report.  Washington, DC:  Center for the Childcare Workforce.

Whitebook, M.  (2003). Early education quality:  Higher teacher qualifications for better learning environments-A review of the literature.  Berkeley, CA:  Institute of Industrial Relations, University of California, Berkeley.

Zaslow, M. & Martinez-Beck, I. (2006). Critical issues in early childhood professional development. Baltimore: Brookes Publishing.

List of Appendices

C.1       Methods of the Studies
C.2       Content Focus of the Interventions
C.3       Training Activities Provided in the Interventions
C.4       Workplace Characteristics: Auspices/Settings
C.5       Workplace Characteristics: Incentives
C.6       Recipients of Professional Development Activities
C.7       Teacher Characteristics
C.8       Characteristics of Coaches/Mentors
C.9       Constructs Measured
C.10     Implementation Measures and Frequency of Implementation Checks
C.11     Measures Used to Assess Changes in Teacher Behavior or Instructional Practices
C.12     Professional Development, Implementation, and Changes in Classrooms/ Instructional Practices and Children

*SOURCES: All information from submitted articles except items marked with an asterisk. Those items are drawn from 2008 report on PCERS studies, available at: http://ies.ed.gov/ncer/pubs/20082009/pdf/20082009.pdf

Note:  Programs participating in Project Upgrade were described as child care centers that had to serve primarily low-income children, including some whose care was subsidized; and have at least one four-year-old classroom with at least five children. (p. 8)  No additional descriptions of the programs were provided.

Note: This table reports incentives, as they were described by project authors.

Endnotes

[1]       At the time publications and reports on these four initiatives were requested from the principal investigators, analyses had not yet been completed for all projects. QUINCE sent no papers to review, and several other projects indicated that additional studies would be forthcoming.

[2]       Study not in the set submitted but used to supplement information.

Closing the Gap in the School Readiness of Low-Income Children

This paper is part of a series of working papers prepared for a meeting sponsored by the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE). Abt Associates Inc and the National Center for Children in Poverty (NCCP) were funded to convene the meeting. The views represented in this paper are those of the author(s) and do not necessarily reflect the opinions of the U.S. Department of Health and Human Services.

Introduction

This paper is intended to stimulate discussion of several important questions related to the problems faced by children from low-income families as they enter school: first, what is the size of the school readiness gap and what does it indicate about the school readiness of children from different socioeconomic backgrounds?  Second, is it necessary to close the gap completely before children enter school in order to improve childrens chances of school success; and finally, what are the implications of what we have learned about the school readiness gap for future program and research efforts?

This paper adds to the body of knowledge about the school readiness gap through secondary analyses of several data sets aimed at describing the size and nature of the school readiness gap as well as the developmental trajectories of childrens school readiness skills. The paper begins by summarizing what we know about the gap in the school readiness of children from low-income families compared with their more advantaged peers, and its implications for later school achievement. As part of this discussion, we suggest one way in which the gap in achievement might be explained in a way that is easier to understand and that could be applied across a variety of different measures. The paper goes on to examine the question of whether the gap needs to be completely closed to reduce the likelihood of school failure. As part of this discussion, we look at children beginning school at varied levels of school readiness for differences in the rate of growth throughout the school years.

What is the nature of the gap in school readiness experienced by children from low-income families and how can it best be quantified?

The concept of school readiness is multi-faceted, encompassing the physical health, social-emotional, cognitive and linguistic status of children. There is abundant evidence that poor children lag behind their more advantaged counterparts on most if not all aspects of readiness. The effects of poverty manifest themselves early in a childs life. Children in low-income families have higher rates of asthma and dental disease, are more vulnerable to measles and other preventable illnesses, are more likely to have behavioral or developmental problems and less likely to see a pediatrician on a regular basis or live in a safe home environment that nurtures their development (Garbarino, 1990; Dryfoos, 1987; Rosenbaum, 1992; Gelles, 1992). At the end of preschool, they score up to one standard deviation below the norms on measures of language and early math (Lee & Burkham, 2002; U.S. DHHS, 2003). These problems are not readily overcome when children enter school; low-income children are more likely to perform poorly once in school (Dryfoos, 1987), to repeat grades and have frequent absences (Ravitch & Finn, 1987), fail to complete high school and lack basic literacy and numeracy skills (Gardner, 1990).

Ideally then, any consideration of the size of the gap in school readiness, its implications and the results of efforts to close it would consider all facets of the problem. In reality, there are difficulties in attempting this for several reasons: we have not specified and quantified the extent of the gap in several of these domains, in part because we lack good measures; we do not fully understand the implications of some early problems for school success; and there have been limited efforts to address problems in some domains. This paper confines discussion to the gap in cognitive (i.e., language) achievement for several reasons: early cognitive skills are foundational for later school performance and success (Whitehurst & Lonigan, 2001; Duncan et al., 2007) making the discussion substantively important; characteristically (though not always) they are measured using standardized tests with norms that allow us to examine and describe the data in different ways; and, to date, most evaluations of early childhood interventions have measured effects in these areas.

Using several large national data sets, researchers have both identified the gap in school readiness in terms of pre-literacy skills (as well as math and cognitive skills) and quantified it. Lee and Burkham, in their analysis of data from the Early Childhood Longitudinal Study  Kindergarten cohort (ECLS-K), found that low-SES children scored almost half a standard deviation (.47) lower on the reading skills test at the start of kindergarten than their middle-class counterparts, and 1.17 standard deviations below high-SES children (Lee & Burkham, 2002). Analysis of data from the Family and Child Experiences Survey (FACES), administered when a child enters Head Start, found a gap of a standard deviation (1.0) in the pre-reading and math skills of Head Start children compared with national norms (U.S. DHHS, 2003).^[1]

Why should we be concerned about this gap?  As the earlier discussion showed, a number of researchers have demonstrated the implications of the gap for future school achievement. Recently, Duncan and his colleagues, analyzing data from more than 35,000 preschoolers in Canada and the United States, found that language and literacy skills at the start of school along with early math skills were the most important predictors of later school success (Duncan et al., 2007).^[2] Children who enter school with many fewer of the necessary skills than their more advantaged counterparts do not catch up.

For this paper we used data from the ECLS-K to graphically illustrate the different trajectories traveled by the general population of children and those whose families lived in poverty at the time their children entered kindergarten and were in poverty at one or more measurement points before the end of fifth grade.  Exhibit 1 shows, from the start of kindergarten through the spring of grade five, children demonstrate steady growth in their reading abilities.  Children experiencing repeated poverty (i.e., children whose families are below the federal poverty line at baseline and at least one subsequent measurement point), however, follow a growth trajectory that is somewhat below the average trajectory for all children.  That is, the difference in the reading scores increases over time, with children living in continued poverty falling farther and farther behind.  These analyses suggest, consistent with previous studies, children living in continued poverty have a slightly slower rate of achievement in reading scores than the average of all children.

Exhibit 1:
Growth Trajectories of Children in ECLS-K Full Sample and Children in Continuing Poverty*
Mean at Each Measurement Point for Full Sample (top black) and
Children in Repeat Poverty (bottom orange)

* Children whose families are in repeated poverty
(poverty at the time of the Fall K test and poverty at one or more subsequent measurement points).

Children in continuing poverty are defined as children whose families are in poverty at the time of the fall kindergarten test and in poverty at one or more subsequent measurement points.

Exhibit 1 shows the growth in children's average reading IRT scores from the fall of kindergarten through the spring of grade 5. As shown in the X-axis, reading IRT scores were measured in the fall of kindergarten (baseline), spring of kindergarten (9 months later), spring of grade 1, (21 months after baseline), spring of grade 3 (45 months after baseline), and spring of grade 5 (69 months after baseline). The range of possible reading IRT scores shown on the Y-axis is 0 points to 150 points. From baseline through the spring of grade 5, average reading IRT scores for the full sample begin at approximately 25 points at baseline and increase to approximately 40 points in the spring of kindergarten, to approximately 75 points in the spring of grade 1, to approximately 120 points in the spring of grade 3, and to approximately 140 points in the spring of grade 5. The average trajectory for children in continuing poverty follows a similar pattern, but is somewhat below the average trajectory for the full sample and has a slightly slower rate of growth. Specifically, the average reading IRT scores for children in continuing poverty begin at approximately 20 points at baseline and increase to approximately 28 points in the spring of kindergarten, to approximately 60 points in the spring of grade 1, to approximately 100 points in the spring of grade 3, and to approximately 120 points in the spring of grade 5.

Ways to quantify the gap in readiness

Typically, as we have seen, achievement differences within a population and across populations are described in terms of the portion of a standard deviation (a measure of the variation in scores within a group) that the difference represents. Increasingly, the impacts of interventions designed to reduce those differences are expressed in effect sizes. Effect sizes are standardized measures of the magnitude (size) of treatment effects. For each measure, the effect size is equal to the estimated impact of the treatment, divided by the control group standard deviation. The standardization makes possible a comparison of the size of treatment effects across studies and, within limits, across outcome measures.^[3] For example, if the effect sizes of a treatment on outcome measures A and B are 0.50, and 0.25, respectively, then the size of the treatment impact on A is considered to be twice the size of the impact on B.^[4] This strategy also allows us to estimate how much of a gap in achievement has been narrowed by an intervention. For example, if the initial gap is 0.5 of a standard deviation, then an intervention that has an impact of 0.25 (to use the example given above) has succeeded in halving the gap. This is an important aid to our understanding of the effectiveness of an intervention.

However, for policymakers and other members of the interested public, standard deviations and effect sizes are quite abstract measures and offer no guidance about the educational or developmental meaning of the gap (or of the impact of an intervention). To try to address this problem, we developed a different metric for the school readiness gap, one that might help policymakers understand both the magnitude of the gap and the effectiveness of efforts to close it.

To do this, we identified some large data sets that included low-income preschoolers assessed with nationally-normed language/pre-reading tests. The first data set came from two evaluations of the Comprehensive Child Development Program (CCDP), in which almost 8,000 low-income children below the age of five were assessed repeatedly using the Peabody Picture Vocabulary Test (PPVT-R) and its Spanish equivalent (TVIP). The second data set came from the first evaluation of Even Start, in which more than 6,000 low-income preschool children were assessed using the PPVT-R and TVIP at several points in time.^[5] Since both evaluations showed that the programs had no effects on any of the outcomes measured, we used scores from both treatment and control group children. The readily available data are in the form of standard score equivalents. These were converted to raw scores using Table 1 of the PPVT-R Manual. We then converted the raw scores into age-equivalent scores for both the study samples and the norming samples, and assessed the difference in months of development between the low-income samples and the norming sample (which, of course, included some low-income children). Exhibit 2 presents the findings of those assessments.

For the CCDP sample of children, whose family incomes were substantially below the Federal Poverty Level (FPL) when they entered the study and moved only slightly upward over time, three-year-olds in the study sample lagged behind the norms by almost nine months. The gap continued to widen with age: the gap at four years of age was almost 14 months and before six years of age, the gap was almost 15 months. Findings from the analysis of data from the Even Start evaluation showed a similar gap in achievement  8 months for three-year-olds, increasing to 12 months for four-year-olds. The income criteria for participation in Even Start were not as uniform as those used in CCDP, so the income range is probably slightly wider. However, when we looked at scores for children in the lowest income group (less than 50% of the FPL), these showed an even wider gap in achievement as the children prepared to enter kindergarten  almost 18 months.^[6]

While both of these were large samples of low-income children drawn from many different parts of the country, neither is representative of children from low-income families nationally. The same analysis could be performed with data from the ECLS-K, for a nationally-representative sample. However, because the ECLS-K chose to use a compendium of subtests drawn from several existing measures rather than a single standardized test, standardization of the measure and creation of age-based norms would be a time-consuming undertaking that is beyond the scope of this project.

These analyses and descriptions assume that the gap to be closed is between the achievement of children from low-income families and the average child as they enter kindergarten. Although it may not have been quantified in these terms before, the gap in school readiness associated with socioeconomic status has certainly been recognized by policymakers at all levels of government for many decades. Considering it here in terms of childrens months of development gives new perspective to the weight and scale of the problem, the need for effective solutions, and the tremendous challenge these disparities pose for schools and social programs serving low-income families.

Do we have to close the gap all the way?

As the previous discussion made clear, progress in narrowing the school readiness gap is generally measured in terms of how close children are to national norms. Closing the school readiness gap refers to bringing children to national norms on a specific outcome measure, such as a standardized literacy or math assessment. Using this yardstick, numerous studies have demonstrated that school readiness interventions make progress toward narrowing the gap but generally do not close the gap entirely.^[7] Some researchers have asked whether it is necessary to close the gap all the way before children begin school. Can disadvantaged children be kept in the running to participate in school even if they do not catch up to national norms?  It is a tempting thought because, as we saw earlier, the size of the gap as children enter school is large and may be hard to close in one year before school. The question here is: Can school readiness research help identify a point that is sufficient to enable children to benefit fully from kindergarten and not continue to lose ground?

To investigate this question, we used longitudinal data from the ECLS-K, namely the scores on the reading test used from the fall of the kindergarten year through fifth grade (35,468 observations) to construct growth trajectories for children beginning kindergarten at varied literacy levels. If it were true that children entering school with scores at some point below the average score managed to catch up, we hypothesized that we would expect to see a different (i.e., slightly accelerated) growth trajectory for those children. In other words, we investigated the question of whether, at the start of kindergarten, children with lower than average literacy levels had faster rates of growth than children at higher literacy levels, moving their scores up to the average score. This would manifest itself as a steeper growth trajectory than the ones for both higher- and lower-performing peers. If this were the case, then helping to move children at least to this specified level before school might mean they could make somewhat greater gains than their peers in the early grades, and that the school experience itself would be instrumental in closing the gap.

Fall kindergarten Item Response Theory (IRT) scaled scores were used to sort children into deciles and then IRT scaled scores at four subsequent time-points were used to construct linear growth models for each decile. Exhibit 3 shows the mean growth trajectories for children from each of the ten deciles. For example, the curve at the bottom of the exhibit shows the mean Reading IRT scores over several time points, for the children whose scores taken in the fall of kindergarten were in the lowest 10 percent of the score distribution. The mean scores of these children were below the means of children from the higher deciles, as they progressed from kindergarten to first grade, on to grades 3 and 5. On average, the deciles move along nearly parallel tracks  there is no group that breaks away and moves closer to another. In addition to the plots shown in Exhibit 3, we fitted linear growth curve models to the data. Consistent with the results shown in the plots, these models indicated that childrens scores from the fall kindergarten are highly predictive (p = <.0001) of their scores on the test in fifth grade. Furthermore, analysis of the ECLS-K data did not indicate that childrens rate of growth in literacy over the elementary school years was related to their literacy level at the start of kindergarten. Childrens average rate of growth in literacy through elementary school was effectively the same regardless of their fall kindergarten score. Children with high, moderate, and low literacy scores at the start of kindergarten all grew at the same rate, on average. At no level along the continuum of kindergarten starting points did the gap narrow over time between any deciles.

Of course, one possible reason why children in the lowest deciles do not catch up to others as a result of the school experience is that these predominantly low-income children may enter schools that are of lower quality than the schools their more advantaged peers enter. If it were true that low-income children could depend on having a high-quality kindergarten experience that supported the gains made as a result of an effective preschool intervention, the results might be different.

Looking at the ECLS-K data from a different perspective offers some additional insights into the reasons why children do not catch up.  The reading test used for the ECLS-K actually was a composite of nine subtests drawn from existing tests. An examination of the decile growth curves for each of the subtests provides interesting insights and helps explain the overall growth pattern.  Exhibits 4-11 show the decile growth curves for each of the subtests. As some have suggested, there are foundational skills (letter recognition, beginning sounds, ending sounds, sight words) that most children (except those in the lowest decile) have acquired by third grade, even if they lagged behind badly on entry to kindergarten.  As the skill tested becomes more complex (understanding words in context), the time taken to catch up increases, and some of the deciles have not caught up by the end of fifth grade.  Exhibits 7-10 show that, for the skills that are needed to interpret and understand what is read, which for most children do not manifest themselves at all until the end of the kindergarten year, the gaps between the deciles actually widen over time  we are no longer looking at the parallel tracks that the overall reading score produces.  Although children in the lower deciles do catch up to others on some skills, the delay may cost them the opportunity to develop adequately the skills that are important for school success.

These analyses demonstrate the relationship between early and later achievement scores and suggest ways in which early skills may be foundational for later skills.  Findings also raise additional questions about whether it is adequate to narrow the school readiness gap or if the gap needs to be closed completely in order for children to benefit from later schooling.  That is, it is clear that early delays in literacy skills can result in delays in the acquisition of more complex skills and leaves underperforming children at risk for school failure.  The next step in exploring how to put children in the running for school success may be to identify the specific skills children need to benefit from learning opportunities in kindergarten (and later schooling) and the early childhood experiences that get them there.

Discussion

When we describe the gap that exists in terms that policymakers and the general public can easily comprehend, we get a sense of the size and scale of the school readiness gap in terms of the number of months of development that low-income children lag behind their more affluent peers.  Is it realistic to expect a single year of early childhood intervention before the start of kindergarten to make up for nine months to a year of development (i.e., to greatly accelerate the rate at which disadvantaged children develop and learn)?  The good news from the most recent reviews of early childhood research (e.g., Caswell & He, 2008; Ginsburg & Clements, 2008; Goodson, 2008; Raver, 2008) is that some interventions have been successful in moving children closer to this goal; the bad news is that none succeeded in accomplishing it completely (i.e., closing the gap).  As Hart and Risley (1995) pointed out, longer and more focused interventions are needed to help children acquire the language and vocabulary skills that will be essential to develop the more complex skills of comprehension and interpretation. Beginning a year earlier or continuing the intervention into kindergarten might magnify the impact of effective interventions sufficiently to accomplish that goal.  At the same time, it is important that we continue to work on improving the quality of interventions and curricula for preschool-aged children.

Many other questions about effective strategies for preparing low-income children for school and reducing the school readiness gap remain to be answered.  Although everyone recognizes that parents are essential agents in supporting childrens development, we have been almost completely ineffective in marshalling this resource.  Are we to give up on parents or are there ways they could be enlisted to help magnify the effect of direct services to children?  Another question has to do with the utility of continued experimental testing in single sites of different curricular or teacher training approaches, in the absence of a comprehensive theoretical framework for the research.  Is Head Start, with its commitment to continuous quality improvement, and its ability to move a very large number of programs in a desired direction, a better laboratory for the many aspects of early childhood education that need to be tested?  What if any evidence about the effectiveness of different strategies and approaches will be gained as states invest more resources in universal prekindergarten?  These and other questions are topics for later discussion.

Exhibit 3
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 3 shows average reading IRT scores from the fall of kindergarten through the spring of grade 5 for children in each of the deciles of the distribution of baseline reading IRT scale score. As shown on the X-axis, reading IRT scores were measured in the fall of kindergarten (baseline), spring of kindergarten, spring of grade 1, spring of grade 3, and spring of grade 5. The range of possible reading IRT scores shown on the Y-axis is 0 points to 200 points. For children in all deciles, the average growth trajectory follows a common pattern, similar to the average trajectory for the full sample. The vertical displacement between the trajectories for each of the deciles remains fairly constant, with slight widening over time. No trajectory ever crosses another; the order remains unchanged. The gap between the trajectory for children in the highest of the deciles is consistently larger than the gap between any of the other deciles.

Exhibit 4
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 4 shows average letter recognition scores from fall of kindergarten through spring of grade 5 for children in each of the deciles for baseline reading. The X-axis shows the measurement time points: fall of kindergarten, spring of kindergarten, and spring of grades 1, 3, and 5. The Y-axis shows the range of possible letter recognition scores from 0.0 to 1.0. In the fall of kindergarten, children vary widely in their letter recognition scores, with children in the lowest decile scoring near 0.0, on average and children in the highest decile scoring 1.0, on average. By the spring of kindergarten, the variation has diminished substantially, with children in the lowest decile scoring an average of 0.7 on the letter recognition measure. By the spring of grade 1, the average letter recognition score is 1.0 for students in all deciles.

Exhibit 5
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 5 shows average sounds on a measure of beginning sounds from fall of kindergarten through spring of grade 5 for children in each of the deciles for baseline reading. The plot shows a pattern similar to Exhibit 4; however, for students in most of the deciles, other than the two highest, the average beginning sounds score tends to be lower than the average letter recognition score in the fall of kindergarten. In addition, the trajectories take longer to converge at 1.0 - by the spring of grade 3 rather than by the spring of grade 1.

Exhibit 6
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 6 shows average scores on a measure of ending sounds from the fall of kindergarten through spring of grade 5 for children in each of the deciles of baseline reading scores. Again, the plot shows a pattern similar to Exhibit 5, although average ending sounds scores tend to be lower in the fall of kindergarten than average beginning sounds scores. By the end of grade 3, students in all deciles have average ending sounds scores of 1.0; however, the rate of change in average scores for ending sounds tends to be slower than for beginning sounds.

Exhibit 7
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 7 shows average scores on a measure of sight word recognition from fall of kindergarten through spring of grade 5 for children in each of the deciles of baseline reading scores. This plot continues the trend from earlier plots of lower average scores in the fall of kindergarten and slower rates of growth. Students in all but the highest of the deciles have an average sight words score of 0.0 in the fall of kindergarten. There is still a large amount of variation among the deciles in the average sight words scores in the spring of grade 1, with an average score below 0.5 for students in the lowest decile and nearly reaching 1.0 for students in the highest decile.

Exhibit 8
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 8 shows average scores on a measure of reading words in context from the fall of kindergarten through spring of grade 5 for children in each of the deciles of baseline reading score. This plot extends the trend from earlier plots - with each building literacy skill, initial scores are lower than for the previous skill and the rate of growth over elementary school is slower. Students in all but the highest of the deciles have an average word in context score of 0.0 in the fall of kindergarten and only slightly higher scores in the spring of kindergarten. Variation in average word in context scores is wide in the spring of first grade, ranging from 0.15 for the lowest decile to 0.9 for the highest decile. Over time, scores cluster at increasingly higher levels; however, even by the spring of grade 5, students in the lowest deciles have average scores that are still slightly below 1.0.

Exhibit 9
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 9 shows average scores for children's ability to make literal inferences when reading from the fall of kindergarten through the spring of grade 5, for children in each of the deciles of baseline reading score. In this plot, average literal inference scores remain low in the earliest grades for students in most deciles, with average scores of 0.0 in fall and spring of kindergarten, and average scores below 0.2 in spring of first grade for children in most deciles. Variation in average literal inference scores is wide in spring of grade 3, ranging from 0.4 for children in the lowest decile to nearly 1.0 for children in the highest. By the end of grade 5, children in the highest two deciles have average scores approaching 1.0; however, on average, most children have not mastered this skill, with average scores ranging down to 0.7.

Exhibit 10
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 10 shows average scores for children's ability extrapolate meaning from written text, beginning in the fall of kindergarten through the spring of grade 5, for children in each of the deciles of baseline reading score. Children in all but the highest decile have not yet begun to demonstrate this skill by the spring of grade 1, with average scores for all but the highest decile at 0.0. Average scores vary widely in spring of grade 3 (0.1 - 0.9) and spring of grade 5 (0.4 - 1.0). On average, most children have not mastered this skill by the end of grade 5.

Exhibit 11
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 11 shows average scores for children's ability to evaluate fictional text from the fall of kindergarten through the spring of grade 5, for children in each of the deciles of baseline reading score. This plot continues the previous trend, showing lower average scores and slower rates of growth with each increasingly complex reading skill. Average scores are near 0.0 for children in all but the highest decile through the spring of grade 1. Average evaluation scores vary at the lower range of the scale at the spring of grade 3 (0.1 - 0.5) and the spring of grade 5 (0.2 - 0.75) than average extrapolation scores (Exhibit 10). On average, children have mid-range scores on this skill by the end of grade 5.

Exhibit 12
Mean at Each Measurement Point
for Deciles Determined by Fall Kindergarten Score

Exhibit 12 shows average scores for children's ability to evaluate non-fiction text from the fall of kindergarten through the spring of grade 5, for children in each of the deciles of baseline reading score. On average, children are only beginning to demonstrate this skill by the spring of grade 5. Average scores for all but the highest decile are 0.0 through the spring of grade 3. In the spring of grade 5, children in all but the two highest deciles have average scores ranging between 0.0 and 0.1. Average scores are 0.15 for children in the second highest decile and are nearly 0.3 for children in the highest decile.

References

Caswell, L., & He, Y. (2008). Approaches to promoting childrens school readiness: A review of federally-funded research initiatives aimed at improving young childrens language and literacy skills in early education and care settings. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the U.S. Department of Health and Human Services, Washington, DC.

Dryfoos, J. G., (1987). Youth at risk: One in four in jeopardy. New York, NY: Carnegie Foundation

Duncan, G.J., Dowsett, C.J., Claessens, A., Magnusen, K., Huston, A.C., Klebanov, P., Pagani, L., Feinstein, L., Engel, M., Brookes-Gunn, J., Sexton, H., Duckworth, K. & Japel, C. School readiness and later achievement. (2007). Developmental Psychology, 43(6), 1428-1446.

Garbarino, J. (1990). The human ecology of early risk. In Miesels, S.M. & Shonkoff, J.P. (Eds.), Handbook of Early Childhood Intervention. New York, NY: Cambridge University Press.

Gardner, H. (1990). The difficulties of school: Probable cause, probable cures. Daedalus, 19, 85-113.

Gelles, R.J. (1992). Poverty and violence toward children. American Behavioral Scientist, 35, 258-274.

Ginsberg, H., Lewis, A., & Clements, M. (2008). School readiness and early childhood education: What can we learn from federal investments in research on mathematics programs?  Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the U.S. Department of Health and Human Services, Washington, DC.

Goodson, B. (2008). What it means and what it takes to prepare children for school: A synthesis of evidence for the impacts of federally-funded research initiatives in early childhood education. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the U.S. Department of Health and Human Services, Washington, DC.

Hart, B. & Risley, T.R. (1995). Meaningful differences in the everyday experiences of young American children. Baltimore: Paul H. Brookes Publishing Co.

Lee, V.E. & Burkham, D.T. (2002). Inequality at the Starting Gate. Washington, DC: Economic Policy Institute.

Raver, C.C. (2008). Promoting childrens socioemotional development in context of early educational intervention and care: A review of the impact of federally-funded research initiatives on young childrens school readiness. Working paper prepared for A Working Meeting on Recent School Readiness Research: Guiding the Synthesis of Early Childhood Research, sponsored by the U.S. Department of Health and Human Services, Washington, DC.

Rosenbaum, S. (1992). Child health and poor children. American Behavioral Scientist, 35, 275-289.

St. Pierre, R., Goodson, B., Layzer, J. & Bernstein, L. (1994). National Evaluation of the Comprehensive Child Development Program: Interim Report. Cambridge, MA: Abt Associates, Inc.

St. Pierre, R., Swartz, J., Gamse, B., Murray, S., Deck, D. & Nickel, P. (1995). Nation Evaluation of the Even Start Family Literacy Program: Final Report. Cambridge, MA: Abt Associates, Inc.

U.S. Department of Health and Human Services, Administration for Children and Families. (2003). Head Start FACES 2000. Washington, DC.

Whitehurst, G.J., & Lonigan, C.J. (2001). Emergent literacy: Development from prereaders to readers. In Neuman & Dickinson (Eds.), Handbook of Early Literacy Research (pp. 11-29). New York: Guilford Press.

Endnotes

1. The ECLS-K, a longitudinal study of a nationally-representative sample of children beginning in kindergarten, used a set of measures developed for the study that borrowed subtests and items from existing tests to create a battery that could be applied from kindergarten through fifth grade. The FACES study used an extensive battery that included several subtests from standardized measures of language and preliteracy skills, as well as some measures developed for the study.

2. Childrens attention-related skills and self-regulation were also important predictors; other social and emotional behaviors, such as aggression, disruptive behavior were not predictive for either gender or any social class.

Meeting Agenda

Meeting Participants

Gina Adams

The Urban Institute
2100 M Street, N.W.
Washington, DC 20037
Phone: 202-261-5709
Email: gadams@urban.org

Sarah Avellar
Mathematica Policy Research, Inc.
Email: savellar@mathematica-mpr.com

W. Steven Barnett
Director, NIEER
Rutgers, The State University of New Jersey
120 Albany Street, Suite 500
New Brunswick, New Jersey 08901
Phone: 732-932-4350 Ext. 228
Email: sbarnett@nieer.org

Jeanne Brooks-Gunn
Virginia and Leonard Marx Professor of Child Development
Teachers College and the College of Physicians and Surgeons
Columbia University
525 West 120th Street, Box 39
New York, NY 10027
Phone: 212-678-3338
Email: brooks-gunn@columbia.edu

Peg Burchinal
FPG Child Development Institute
521 S. Greensboro St
Carrboro NC 27510
Phone: 919-966-5059
Email: burchinal@unc.edu

Natasha Cabrera
Assistant Professor
Human Development
University of Maryland
College Park, Maryland, 20742
Phone: 301-405-2827
Email: ncabrera@umd.edu

Ajay Chaurdry
Director, Center on Labor, Human Services, & Population
The Urban Institute
2100 M Street, NW
Washington, DC 20037
Phone: 202-833-7200
Email: achaudry@urban.org

Peggy Clements
Education Development Center, Inc.
96 Morton Street, 7th Floor
New York, NY 10014
Phone: 212-807-4268
Email: pclements@edc.org

Ronna Cook
Westat
5912 Rossmore Drive
Bethesda, MD 20814
Phone: 301-564-9228
Email: ronnacook@comcast.net; ronnacook@westat.com

Sarah Daily
Senior Policy Analyst
National Governors Association
Center for Best Practices
Education Division
444 North Capitol, Suite 267
Washington, DC 20001
Phone: 202-624-7857
Fax: 202-624-7825
Email: sdaily@nga.org

Rachel Demma
Senior Policy Analyst, Education Division
National Governor's Association
444 N. Capitol St., Suite 267
Washington, D.C. 20001-1512
Phone: 202-624-5300
Email: rdemma@nga.org

Susanne A. Denham
Department of Psychology MS 3F5
George Mason University
4400 University Drive
Fairfax VA 22030-4444
Phone: 703-993-1378
Email: sdenham@gmu.edu

Celene E. Domitrovich
Pennsylvania State University
108 South Henderson Building
University Park, PA 16802
Phone: 814-865-2616
Email: cxd130@psu.edu

Jason Downer
Senior Scientist
Center for the Advanced Study of Teaching and Learning
University of Virginia
350 Old Ivy Way, Suite 100
Charlottesville, VA 22903
Phone: 434-924-0792
Email: jdowner@virginia.edu

Eilene A. Edejer
Chicago Public Schools
Office of Early Childhood Education
125 South Clark, 9th Floor
Chicago, IL 60603
Phone: 773-553-2333
Email: eaedejer@cps.k12.il.us

Janet Fischel
Stony Brook University
University Medical Center L5,
Nicolls Road,
Stony Brook, NY 11794-7564
Phone: 631-444-8211
Email: Janet.Fischel@stonybrook.edu

Cheryl Fountain
Florida Institute of Education
University of North Florida
University Center
12000 Alumni Drive
Jacksonville, 32224
Phone: 904-620-2496
Email: fountain@unf.edu

Ellen Frede
Co-Director
National Institute for Early Education Research
Rutgers, The State University of New Jersey
120 Albany Street, Suite 500
New Brunswick, NJ 08901
Phone: 732-932-4350
Fax: 732-932-4360
Email: efrede@nieer.org

Allison S. Fuligni
Associate Researcher
UCLA GSE&IS
BOX 951521, 8142A Math Sciences
Los Angeles, CA 90095-1521
Phone: 310-206-2417
Email: fuligni@gseis.ucla.edu

Walter S. Gilliam
Yale Child Study Center
230 South Frontage Road
PO Box 207900
New Haven, CT 06520-7900
Phone: 203-785-3384
Email: walter.gilliam@yale.edu

Herbert P. Ginsburg
Jacob H. Schiff Foundation Professor of Psychology and Education
Department of Human Development
542 Grace Dodge Hall
Teachers College, Columbia University
525 W. 120th Street
New York, NY 10027
Phone: 212-678-3443
Fax: 212-678-3837
Email: hpg4@columbia.edu

Barbara D. Goodson
Principal Scientist
Abt Associates Inc.
55 Wheeler St.
Cambridge, MA 02138
Phone: 617-349-2811
Fax: 617-349-2665
Email: barbara_goodson@abtassoc.com

David Grissmer
Principal Scientist
Center for the Advanced Study of Teaching and Learning
University of Virginia
350 Old Ivy Way, Suite 100
Charlottesville, VA 22902
Phone: 434-243-8200
Email: grissmer@virginia.edu

Brenda Jones Harden
Associate Professor
Institute for Child Study
University of Maryland
3304 Benjamin Building
College Park, MD 20742
Phone: 301-405-2580
Fax: 301-405-2891
Email: bjharden@umd.edu

Renate M. Houts
Senior Research Statistician / Psychologist
Statistics & Epidemiology
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: 919-316-3345
Fax: 919-541-6722

Marilou Hyson
NAEYC
1313 L St., NW, Suite 500
Washington, DC 20036
Phone: 202-365-4791
Email: mhyson@naeyc.org

Julia Isaacs
Brookings Institution
1775 Mass Ave, NW
Washington, DC 20036
Phone: 202-797-6466
Email: jisaacs@brookings.edu

Graciela Italiano-Thomas
Consultant
510 N Street, SW
Apt. N 623
Washington DC 20024
Phone: 202-550-4434
Email: Gift@gracielaitaliano.com

Stephanie M. Jones
Assistant Professor
Risk and Prevention Program
Harvard Graduate School of Education
Larsen 603, 14 Appian Way
Cambridge, MA 02138
Phone: 617-496-2223
Fax: 617-384-8177
Email: stephanie_m_jones@gse.harvard.edu

Sharon Lynn Kagan
Professor of Early Childhood and Family Policy
Teachers College, Columbia University
525 W. 125th Street, Box 226
New York, NY 10027-6696
Phone: 212-678-8255
Email: Sharon.kagan@columbia.edu

Elisa Klein
Department of Human Development
Institute for Child Study
3304 Benjamin Building (#143)
University of Maryland
College Park, MD 20742
Department Phone: 301-405-2827
Department Fax: 301-405-2891
Email: elklein@umd.edu

Lisa Knoche
Research Assistant Professor and Project Director, Getting Ready Project
Nebraska Center for Research on Children, Youth, Families and Schools
University of Nebraska-Lincoln
238 Teachers College Hall
Lincoln, NE 68588-0345
Phone: 402-472-4821
Fax: 402-472-8777
Email: lknoche2@unl.edu

Janis Kupersmidt
Innovation Research and Training
1415 W. NC Highway 54
Suite 121
Durham, NC 27707
Phone: 919-493-7700
Email: jkupersmidt@irtinc.us

Carolyn Layzer
Abt Associates Inc.
55 Wheeler Street
Cambridge, MA 02138
Phone: 617-520-3597
Fax: 617-386-8500
Email: carolyn_layzer@abtassoc.com

Jean I. Layzer
Belmont Research Associates
42 Fairmont St.
Belmont, MA 02478
Phone: 617-484-8189
Email: jeanlayzer1@mac.com

Deborah J. Leong
The Metropolitan State College of Denver
26093 Thea Gulch
Golden, CO 80403
Phone: 303-279-5589
Email: leongd@mscd.edu

Joan Lombardi
Georgetown University
37th and O streets, NW
Washington, DC 20057
Phone: 202-687-0100
Email: lombardij@aol.com

Michael L. LУГpez
Executive Director
National Center for Latino Child & Family Research
22610 Woodfield Road
Laytonsville, MD 20882
Phone: 301-537-6552
Email: milopez@earthlink.net

John M. Love
Mathematica Policy Research
1016 Canyon Park Drive
Ashland, OR 97520
Phone: 541-488-6987
Fax: 609-799-0005
Email: jlove@mathematica-mpr.com

Kelly L. Maxwell
FPG Child Development Institute
UNC-CH, CB # 8180
Chapel Hill, NC 27599-8180
Phone: 919-966-9865
Email: maxwell@unc.edu

Pamela Morris
Director, Family Well-Being and Child Development Policy Area
MDRC
19th Floor
16 East 34 Street
New York, NY 10016-4326
Phone: 212-532-3200
Fax: 212-684-0832

Susan B. Neuman
Professor, University of Michigan School of Education
3119 School of Education
610 E. University Ave.
Ann Arbor, MI 48109-1259
Phone: 734-615-4655
Email: sbneuman@umich.edu

Samuel L. Odom
Director, FPG Child Development Institute
University of North Carolina
CB-8180, 105 Smith Level Road
Chapel Hill, NC 27599-8180
Phone: 919-966-4250
Fax: 919-966-7532
Email: slodom@unc.edu

Dan Pederson
President
Buffett Early Childhood Fund
3555 Farnam
Omaha, NE 68131
Email: dp@buffettearly.org

Robert C. Pianta
Dean, Curry School of Education
University of Virginia
405 Emmet St. South
Charlottesville, VA 22903
Phone: 434-924-3332
Email: pianta@virginia.edu

Helen Holz Raikes
Professor
Child, Youth and Family Studies
257 Mabel Lee Hall
Lincoln, NE
68588-0236
Phone: 402-472-9147
Email: hraikes2@unl.edu

Craig T. Ramey
Georgetown University Distinguished Professor of Health Studies
Director, Georgetown University Center on Health and Education
Box 571107, 3700 Reservoir Road, NW
Washington, DC 20057-1107
Phone: 202-687-8818 or 202-687-2874
Fax: 202-784-3129
Email: ctr5@georgetown.edu

Sharon Landesman Ramey
Georgetown University Center on Health and Education
3700 Reservoir Rd, NW
Washington, DC 20057
Phone: 202-687-2874 or 301-656-6612
Email: sr222@georgetown.edu

Melissa Raspa
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: 1-866-860-5229
Email: mraspa@rti.org

C. Cybele Raver
Director, Institute of Human Development and Social Change
Associate Professor, Department of Applied Psychology
Steinhardt School of Culture, Education and Human Development
New York University
239 Greene Street, Suite 400
New York, NY 10003
Phone: 212-998-5519
Email: cybele.raver@nyu.edu

Christine Ross
Senior Researcher
Mathematica Policy Research, Inc.
600 Alexander Park
Princeton, NJ 08540
Phone: 609-750-3184
Email: cross@mathematica-mpr.com

Beth Rous
University of Kentucky
126 Mineral Industries Bldg.
Lexington, KY 40506-0051
Phone: 859-257-9115
Email: brous@uky.edu

Thomas Schultz
Director of Early Childhood
Council of Chief State School Officers
One Massachusetts Avenue, NW Suite 700
Washington, DC 20001-1431
Phone: 202-312-6432
Email: thomass@ccsso.org

Larry Schweinhart
High Scope Educational Research Foundation
600 North River Street
Ypsilanti, MI 48198-2898
Phone: 734-485-2000 ext. 256
Email: lschweinhart@highscope.org

Susan M. Sheridan
Nebraska Center for Research on CYFS
216 Mabel Lee Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0235
Phone: 402-472-6941
Email: ssheridan2@unl.edu

Emily Snell
MDRC
16 East 34th Street, 19th Floor
New York, NY 10016
Email: Emily.snell@mdrc.org

Kyle Snow
RTI, International
6110 Executive Boulevard, Suite 902
Rockville, Maryland 20852-3907
Phone: 301-816-4605
Fax: 301-230-4647
Email: ksnow@rti.org

Helene Stebbins
HMS Policy Research
4617 S. 8th Road
Arlington, VA 22204
Phone: 703-769-2772
Email: helene.stebbins@verizon.net

Jennifer Stedron
National Conference of State Legislatures
7700 East First Place
Denver, CO 80230
Phone: 303-856-1427
Email: Jennifer.stedron@ncls.org

Mark Steinmeyer
Senior Program Officer
Smith Richardson Foundation
60 Jesup Road
Westport, CT 06880
Phone: 203-222-6222
Email: Msteinmeyer@srf.org

Louisa B. Tarullo
Associate Director of Research and Area Leader, Early Childhood
Mathematica Policy Research, Inc.
600 Maryland Ave. SW, Suite 550
Washington, D.C. 20024-2512
Phone: 202-264-3479
Fax: 202-863-1763
Email: ltarullo@mathematica-mpr.com

Kathy R. Thornburg
Director, Center for Family Policy & Research
1400 Rock Quarry Rd.
Columbia, MO 65211
Phone: 573-882-9998
Fax: 573-884-0598
Email: thornburgk@missouri.edu

Kathryn Tout
Child Trends
615 First Ave N.E., Suite 225
Minneapolis, MN 55413
Phone: 612-692-5518
Email: ktout@childtrends.org

Elaine R. Weiss
Senior Associate
Partnership for America's Economic Success
The Pew Charitable Trusts
1025 F St., NW, Suite 900
Washington, DC 20004-1409
Phone: 202-552-2052
Fax: 202-552-2299
Email: eweiss@pewtrusts.org

Jerry West
Senior Fellow
Mathematica Policy Research, Inc.
600 Maryland Avenue, SW, Suite 550
Washington, DC 20024-2512
Phone: 202-484-4516
Fax: 202-863-1763
Email: jwest@mathematica-mpr.com

Martha Zaslow
Child Trends
4301 Connecticut Avenue NW, Suite 350
Washington, DC 20008
Phone: 202-572-6032
Email: mzaslow@childtrends.org

Edward Zigler
Sterling Professor of Psychology, Emeritus
Director, Emeritus, The Edward Zigler Center in Child Development and Social Policy
310 Prospect Street, Suite 215
New Haven, CT 06511-2187
Phone: 203-432-4576
Fax: 203-432-7147
Email: edward.zigler@yale.edu

Federal Staff

Anne F. Bergan
Social Science Research Analyst
Office of Planning, Research and Evaluation
Administration for Children and Families
370 L'Enfant Promenade, SW
Washington, DC 20447
Phone: 202-260-8515
Fax: 202-205-3598
Email: abergan@acf.hhs.gov

Barbara Broman
Deputy to the Deputy Assistant Secretary for Human Services Policy
Office of the Assistant Secretary for Planning and Evaluation
U.S. Department of Health & Human Services
200 Independence Avenue, SW
Washington, DC 20201
Email: barbara.broman@hhs.gov

Beth Caron
Education Program Specialist
US Department of Education
Office of Special Education and Rehabilitative Services
400 Maryland Ave, SW
Washington, DC 20202-2550
Phone: 202.245.7293
Fax: 202.245.7617
Email: beth.caron@ed.gov

Reeba Daniel
U.S. Department of Health and Human Services
370 L'Enfant Promenade, SW
Washington, DC 20447
Phone: 202-205-8666
Email: reeba.daniel@ed.gov

Kathleen Dwyer
Administration for Children and Families
Office of Planning, Research and Evaluation
370 L'Enfant Promenade, SW
7th Floor West
Washington, DC 20447
Phone: 202-401-5600
Fax: 202-205-3598
Email: kathleen.dwyer@acf.hhs.gov

Caroline Ebanks
National Center for Education Research
Institute of Education Sciences
U.S. Department of Education
555 New Jersey Avenue, NW
Room 610D
Washington, DC 20208
Phone: 202-219-1410
Email: Caroline.Ebanks@ed.gov

James A. Griffin
Deputy Chief, Child Development & Behavior Branch
Center for Research for Mothers and Children
Eunice Kennedy Shriver National Institute of Child Health and Human Development
6100 Executive Blvd., Suite 4B05
Rockville, MD 20852-7510
Phone: 301-435-2307
Fax: 301-480-0230
Email: James.Griffin@nih.gov

Laura R. Hoard
Social Science Research Analyst
Division of Child and Family Development
Office of Planning, Research, and Evaluation
Administration for Children and Families
370 L'Enfant Promenade, SW
Washington, DC 20447
Phone: 202-401-4561
Fax: 202-205-3598
Email: laura.hoard@acf.hhs.gov

Jamie Holcomb
Office of the Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Avenue, SW
Washington, DC 20201
Phone: 202-690-8505
Email: jamie.holcomb@hhs.gov

Amy Madigan
Office of the Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Avenue, SW
Washington, DC 20201
Phone: 202-690-6652
Email: amy.madigan@hhs.gov

Nancy Geyelin Margie
Office of Planning, Research, and Evaluation
Administration on Children and Families
U.S. Department of Health and Human Services
370 L'Enfant Promenade, S.W.
Washington, DC 20447
Phone: 202-401-5522
Email: Nancy.Margie@acf.hhs.gov

Ivelisse Martinez-Beck
Child Care Research Coordinator
Office of Planning, Research and Evaluation
Administration for Children and Families
370 L'Enfant Plaza Promenade, SW
7th Floor West
Washington, D.C. 20447
Phone: 202-690-7885
Fax: 202-205-3598
Email: ivelisse.martinezbeck@acf.hhs.gov

Martha Moorehouse
Office of the Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Avenue, SW
Washington, DC 20201
Phone: 202-690-6939
Email: martha.moorehouse@hhs.gov

Melissa Pardue
Deputy Assistant Secretary for Human Services Policy
Office of the Assistant Secretary for Planning and Evaluation
200 Independence Avenue, SW
Washington, DC 20201

Ann C. Rivera
Society for Research in Child Development/AAAS Fellow (2008-2009)
Office of Planning, Research and Evaluation
Administration for Children and Families
Aerospace Building 7W
370 L'Enfant Promenade
Washington, DC 20447
Phone: 202-401-5506
Email: Ann.Rivera@acf.hhs.gov

Lauren Supplee
Office of Planning, Research and Evaluation
Administration for Children and Families
370 L'Enfant Promenade SW, 7th Fl West
Washington, DC 20447
Phone: 202-401-5434
Email: lauren.supplee@acf.hhs.gov

Mary Bruce Webb
Director, Division of Child and Family Development
Office of Planning, Research and Evaluation
Administration for Children and Families
370 L'Enfant Promenade SW
Washington, DC 20447
Phone: 202-205-8628
Fax: 202-205-3598
Email: mbwebb@acf.hhs.gov

T'Pring Westbrook
Research Fellow
Administration for Children and FAmilies
Office of Planing, Research and Evaluation
370 L'Enfant Promenade, SW
7th Floor, West
Washington, DC 20447

Maria Woolverton
Administration for Children and Families
Office of Planning, Research and Evaluation
370 L'Enfant Promenade, SW
7th Floor, West
Washington, DC 20447
Phone: 202-205-4039
Email: maria.woolverton@acf.hhs.gov

Joseph Zogby
Room 18A-39 Parklawn
5600 Fishers Lane
Rockville, MD 20857
Phone: 301-443-4393
Email: jzogby@hrsa.gov

Abt Associates

Alyssa Rulf Fountain
Social and Economic Policy Division
Abt Associates Inc.
55 Wheeler Street
Cambridge, MA 02138
Phone: 617-520-2657
Fax: 617-386-7608
Email: alyssa_rulf_fountain@abtassoc.com

Anne Wolf
Social and Economic Policy Division
Abt Associates Inc.
4550 Montgomery Avenue
Suite 800 North
Bethesda, MD 20814
Phone: 301-634-1738
Phone: 301-828-9683
Email: anne_wolf@abtassoc.com

Hestia Advising

Lisa Klein
Hestia Advising
PO Box 6756
Leawood, KS 66206
Phone: 913-642-3490
Email: lklein@hestiaadvising.com

National Center for Children in Poverty

Jane Knitzer
The National Center for Children in Poverty
Mailman Schools of Public Health
Columbia University
215 W. 125th St., 3rd Floor
New York, NY 10027
Phone: 646-284-9615
Email: knitzer@nccp.org

Lee Kreader
The National Center for Children in Poverty
Mailman Schools of Public Health
Columbia University
215 W. 125th St., 3rd Floor
New York, NY 10027
Phone: 646-284-9625
Email: kreader@nccp.org

Jessica Vick
The National Center for Children in Poverty
Mailman Schools of Public Health
Columbia University
215 W. 125th St., 3rd Floor
New York, NY 10027
Phone: 646-284-9602
Email: vick@nccp.org