Proceedings from a Working Meeting on School Readiness Research: Guiding the Synthesis of Early Childhood Research. Examining the Results Across All Projects

12/15/2009

The results of the studies above are direct tests of approaches to PD. This section examines those findings as well as those of the studies submitted by other projects, reporting all of them according to the logic model in Figure 1.

Effects of PD on Implementation of Curricula

Eleven of the 12 projects employed teacher training to help teachers implement a particular curriculum (see Appendix C.1). Ten of 12 reported on changes in teacher behavior as evidence of implementation of the curriculum or of the intervention on which teachers were trained. Projects employed implementation checks at frequencies ranging from three times per year to ongoing, primarily using curriculum-specific checklists or measures of implementation (see Appendices 9-10). Study not in the set submitted but used to supplement information.

Four projects (Childrens School Success, Project Upgrade, Language-Focused Curriculum, Lets Begin with the Letter People/Doors to Discovery) suggest that teachers gradually achieved better implementation and stronger fidelity over time, presumably as they had more practice. But, more frequent visits by a coach were not always associated with better implementation (e.g., Building Language for Literacy).

It makes intuitive sense that dosage of PD (or curriculum) is associated with both implementation and outcomes. The Childrens School Success project proposed a useful definition of dosage that combines fidelity, implementation, and child attendance. The authors conclude that, for an intervention to benefit children, teachers must deliver the curriculum as intended, the whole curriculum must be delivered, and children must attend class to receive the intervention. Their results suggest that fidelity measures were significantly associated with childrens post-test performance, but those associations were sometimes moderated by childrens attendance (and their pre-test performance). Similarly, the Language-Focused Curriculum project found that children in intervention classrooms who had better attendance benefited more than children with weaker attendance. These two projects suggest that future studies of PD should monitor fidelity, implementation, as well as child attendance.

Effects of PD on Changes in Teacher Practices

Appendix C.12 describes changes in teacher practices observed in the 12 studies. In addition to the MTP study that directly assessed and reported positive changes in teacher practices as a result of mentoring/coaching, four other projects that employed coaching/mentoring reported positive changes in teaching behaviors. For example, the Chicago School Readiness Project found the emotional climate of the classroom improved in intervention (5-6 Saturday workshops plus mental health consultants/coaches) classrooms, and teachers were more enthusiastic and responsive in their interactions with students and displayed fewer emotionally negative practices. In Project Upgrade, teachers in the intervention (workshop/coaching/curricula) groups out-performed members of a control group on behaviors related to promoting literacy (e.g., support for oral language, print knowledge, print motivation, support for phonological awareness, literacy resources, and literacy activities). Although these studies were not designed to isolate the effects of mentoring from the effects of other PD strategies employed in the projects or from the curriculum the PD was designed to help teachers implement, they may provide some support for the value of mentors/coaches in changing teacher practices.

Some teacher behaviors appear harder to change than others. For example, in the Language-Focused Curriculum project, researchers recorded the extent to which teachers made changes in classroom activity contexts (e.g., setting up a dramatic play corner for the week) and in instructional processes (e.g., asking open-ended questions to promote early literacy skills).  Activity contexts were more likely to be implemented soon after training, while changes in instructional processes took longer to achieve.  In Project Approach, an observational study of childrens engagement in public preschool classrooms, the researchers report that teachers were reluctant to work with students in small rather than large groups, even after training. These results suggest that future PD research might seek to establish the types of teacher behaviors that are harder to change and the specific PD strategies that might be more effective with such hard-to-change behaviors.

Effects of PD for Coaches/ Mentors/Consultants

Eleven projects employed individuals described as coaches, mentors, or mental health consultants. Their responsibilities included training teaching staff on the curriculum, visiting classrooms to observe the teaching staff in action and to model appropriate implementation of the curriculum, providing feedback to the staff, facilitating group meetings with teachers to reflect on practices, barriers, and successes, and, in the Chicago School Readiness project, providing stress reduction services to teachers and direct one-to-one mental health services to a few children in each classroom. But, despite the central role played by the coaches, most studies contained limited information about them, the training or supervision they received, or the effects of that training on their coaching skills or performance (see Appendix C.8). Future PD research specifically designed to identify the best approaches to PD for coaches would be useful.

The Effects of PD on Children

As described above, two of the four studies that tested specific PD approaches (BLL and MTP) suggest that coaching/mentoring produced better outcomes for children compared to PD that did not include mentoring, but the four studies also suggest that outcomes for children can vary depending on curricula, auspices, and outcome being assessed (see Appendix C.12).

Other projects that included coaching/mentoring also produced benefits in child outcomes (e.g., Project Upgrade, Head Start REDI, Pre-K Mathematics), but it is not possible to say what elements of the PD/curriculum interventions in those projects were associated with the benefits. However, across all 12 projects, the lone effort that produced no significant effects on either classroom/instructional quality or child outcomes was also the only project that did not include a coaching component along with its workshops (Language-Focused Curriculum).

Workplace Characteristics and Outcomes

The submitted studies described workplace characteristics such as program auspices/settings, incentives provided to PD recipients, poverty of children enrolled in the participating programs, and teacher turnover.

Auspices/Setting. All projects took place in at least some Head Start, publicly-funded preschool, and/or community-based child care programs (see Appendix C.4).  As described above, in the Lets Begin with the Letter People/Doors to Discovery project, the authors concluded that, when mentoring made a difference in child outcomes, it was mostly in Title I/ UPK classrooms, rather than in Head Start. In contrast, the Pre-K Mathematics project found no differences in effects on children across the participating Head Start and state preschool programs. Because of the mixed findings and the fact that few studies examined this issue directly; no firm conclusions about auspices can be drawn, except that future PD research should consider the effects of different preschool settings.

Childrens Poverty Level. Most projects operated in settings with a high percentage of low-income children. The MTP project, which reported the effects of PD by childrens poverty level, showed effects of consultancy when 100% of children were in poverty but no effects when 50% of children were in poverty. Because these subgroup analyses were not based on original randomization of the study, it is possible that the results are due to selection bias or some other factor. Nevertheless, the findings suggest that the impact of high levels of poverty should be assessed in future PD research.

Incentives.  Half of the 12 projects mentioned that teachers or programs were provided with incentives to encourage participation (see Appendix C.5). The incentives primarily included financial compensation for the time of the teachers and free sets of curricular materials. Only one project (Building Language for Literacy) allowed teachers to earn PD units for participation, which is a strategy that many ECE programs nationally are using. There were no direct tests of the effectiveness of these incentives in any project. Nevertheless, because of their policy importance, incentives may be an area for future PD research to explore.

Teacher Turnover. In Project Upgrade, turnover ranged from 28% to 44% in intervention classrooms to 49% in control group classrooms over the two-year period of the project.  The coaches/mentors in the project identified high turnover as a barrier to effective implementation. Turnover was either not reported or was lower in most other submitted studies.

Teacher Characteristics and Outcomes

Studies described teacher characteristics such as years of education, educational degrees, demographics, and teacher motivation (see Appendix C.7). Results suggest some teacher behavior or child outcomes can be affected by particular teacher characteristics, but, in other cases, training/PD/curricula interventions exert their effects without moderation by teacher characteristics.

Teacher Education Levels and Years of Experience

In the Childrens School Success and Pre-K Mathematics projects, teachers without BAs or with fewer years of experience implemented the curriculum or changed their classroom (Pre-K Mathematics) practices about as much as did teachers with BAs or with many years of experience. In the Early Literacy and Learning Model, changes in child outcomes were not affected by teachers educational level, leading the authors to conclude, This suggests that ELLM is successful in addressing the preparation deficiencies of early childhood and child care educators, though the issue merits further study (Cosgrove et al, 2006, p. 25).

But, in the LA ExCELS observational study, better classroom emotional climate was observed in settings taught by educators with BA degrees in a child development major, across all settings.

Teacher Language. Project Upgrade found PD interventions benefited teachers differently depending on their initial education levels, their dominant languages, and the outcomes being observed. While the curricular/PD intervention had strong effects on teacher behavior overall, the impacts were stronger for teachers whose primary language was Spanish than for their English-speaking peers. Further, for the two curricula that produced benefits for children, the benefits were larger for children in classrooms with Spanish-speaking teachers.

Teacher Attitudes and Motivation. In the Chicago School Readiness project, teachers who demonstrated a high level of dedication to their own PD (63% attended three or more of the Saturday trainings) were more likely to implement the curriculum as fully intended. In Project Upgrade, mentors reported that the best implementers of the curricula had, among other things, a positive attitude toward instructional change, while resistance to instructional change was a barrier to implementation.

In sum, teacher education, language, and motivation may influence the impact of a PD intervention and further research is warranted.

View full report

Preview
Download

"index.pdf" (pdf, 307.32Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apa.pdf" (pdf, 78.45Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apb.pdf" (pdf, 401.85Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apc.pdf" (pdf, 288.78Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apd.pdf" (pdf, 90.04Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"ape.pdf" (pdf, 190.7Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apf.pdf" (pdf, 68.04Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®