Four projects conducted randomized trials that held constant the effects of curriculum, isolating the effects of particular PD strategies. These four examined the use of mentors/coaches and the effects of differing levels of coaching intensity.
Lets Begin with the Letter People/Doors to Discovery
Assel, Landry, Swank, & Gunnewig (2006) report on the first-year results of a randomized trial designed to test two curricula (Lets Begin with the Letter People and Doors to Discovery) and two PD strategies (mentoring and non-mentoring). The authors hypothesized that (1) mentoring would help teachers implement a curriculum; (2) children would show greater academic gains when their teachers have been mentored than when they have not; and (3) the impact of mentoring would be greatest in classrooms with teachers who have lower levels of education.
School sites that had Head Start, Title I pre-K, and universal pre-K programs were randomly assigned to, first, a curriculum condition (Lets Begin with the Letter People or Doors to Discovery) and then to a PD condition (mentoring or non-mentoring), resulting in assignment to one of five conditions: (1) Lets Begin with the Letter People, mentored; (2) Lets Begin with the Letter People, non-mentored; (3) Doors to Discovery, mentored; (4) Doors to Discovery, non-mentored; and (5) control group. Seventy-six classrooms, 76 teachers, and 603 children participated in the study.
Teachers in experimental groups were trained in their curriculum in a four-day summer workshop that was teacher-centered, employed small groups, and focused on curricula-specific content for promoting language, literacy, and social-emotional skills. During the year, curriculum mentors supported teachers as they implemented their assigned curriculum. Mentors worked with teachers twice per month in classroom coaching sessions that focused on lesson planning, room arrangement, schedules, behavioral issues, curriculum fidelity, and demonstration of curriculum components.
Mentors completed fidelity checklists designed for each curriculum three times during the year, for both mentoring and non-mentoring classrooms. Fidelity of implementation improved over time for both curricula: Just 29.8% of Lets Begin teachers scored at high levels at the first evaluation, but that figure had increased to 71.5% by mid-year (comparable figures for Doors to Discovery were 28.6% and 59.6%, respectively). The authors speculate that implementation of Lets Begin might have been better because it has a single user-friendly teacher guide, as compared with the multiple guides teachers had to consult for Doors to Discovery.
Study findings revealed few main effects for mentoring across all settings or curricula or for all outcome measures. For example, on phonological awareness, children in Title I and universal pre-K classrooms with mentoring had significantly greater gains than children in non-mentored classrooms regardless of curricula, but children in Head Start classrooms did better in non-mentored classrooms. Children in Title I classrooms using Doors to Discovery showed greater growth in vocabulary whether or not there was mentoring, while children in the Lets Begin classrooms did better when their teachers had been mentored. The authors summarize these and other findings by saying, When mentoring showed a positive impact, it was only in the Title I or universal pre-K classrooms, and, further, that benefits were more likely to be within the public school system utilizing Lets Begin and within the literacy rather than the language domain.
The authors suggest that, had more intense mentoring been offered, the results for mentoring might have been stronger. They also note that all teachers received feedback about implementation of curricula, which may have lessened the impact of the mentoring overall, as that feedback could have served as a kind of intervention.
Two studies of MTP, a web-based PD model designed to improve teachers instructional practice and interactions with children to promote language/literacy and social skills, were submitted for this review (Pianta, Mashburn, Downer, Hamre, & Justice, in press; Whitaker, Kinzie, Kraft-Sayre, Mashburn, & Pianta, 2007).
Whitaker et al (2007; also described in Kinzie, Whitaker, Neesen, Kelley, Matera, & Pianta, 2006) explored the relationship between levels of teacher support and teacher participation in MTP training activities. A total of 235 teachers were assigned to one of three levels of service support: (1) a Web-Only group that received a laptop computer and access to the MTP website; (2) a Materials group that received the same plus printed versions of MTP curricula and their corresponding materials; and (3) a Consultancy group that received all of the above plus a video camera to tape their classroom practice for bi-weekly on-line discussions with a teacher consultant who reviewed the video clips and provided feedback and recommendations.
Teachers in the Consultancy group logged on to the website more often than teachers in the Web-Only and Materials groups, but the Materials group spent more time on-line than did members of the other groups. The Consultancy and Web-Only groups agreed in their views of MTPs usefulness, but the Materials group responded significantly less positively than the other two groups. The authors conclude that teachers will voluntarily participate in PD if they find it useful and if they receive the level of supports they feel they need.
In a second study reporting on a randomized controlled trial of MTP, Pianta et al (in press) compared the effects of Consultancy versus Web-Only supports on the quality of observed teacher-child interaction in pre-K classrooms. Teachers in the Web-Only condition received materials and access to the MTP web-site, which included video clips of high-quality teaching exemplars. Teachers in the Consultancy group videotaped their own classroom teaching and sent the videotapes to a consultant (mentor) who provided feedback to teachers in on-line video chats twice each month over the course of the school year.
In the first year of the two-year study, teachers in the Consultancy group had more sensitive interactions with students, were better at engaging students in instruction, and improved their ability to stimulate childrens language more than teachers in the Web-Only group. The effects of Consultancy on teacher behaviors were moderated by the level of poverty of children in the classroom. Specifically, when 50% of the children in classrooms were classified as poor, there were no differences in the rates of change between teachers in the Consultation and Web-Only conditions. But, when 100% of the children in the classrooms were poor, then the teachers in the Consultation group had greater increases in the quality of teacher-child interactions than teachers in the Web-Only condition. This latter finding suggests that PD interventions in classrooms with a high density of children from low-income households may need to look different with respect to intensity and/or supportiveness for teachers than PD interventions in other classrooms.
All classrooms were in publicly-funded pre-K programs, and the teachers in these studies were highly experienced (averaging 15 years teaching; one-third with advanced degrees), so it is unclear if these findings would apply to other settings or to teachers with other backgrounds.
Literacy Express is a comprehensive preschool curriculum for three- to five-year-olds with units on oral language, emergent literacy, basic math, science, general knowledge, and social-emotional development. The study submitted for this project included a brief description of a randomized trial that compared three groups: (1) training via workshops; (2) training via workshops plus mentoring; and (3) a business as usual control group. As described in greater detail in Lonigan, Farver, Clancy-Menchetti, & Phillips (2005), a total of 48 preschools (mostly Head Start centers) in Florida and California were randomly assigned to one of three PD conditions: Literacy Express workshops only (15 schools), Literacy Express workshops plus mentoring (15 schools), or a business as usual comparison group (18 schools).
In the workshop group, teachers and aides participated in a 2-day Literacy Express initial workshop plus three ½-day workshops during the school year. Teachers and aides in the mentoring group participated in the same workshops and received classroom visits by a trained project mentor. Preschools in the business-as-usual comparison group used the preschools standard curricula (most often High/Scope or Creative Curriculum) (Lonigan et al, 2005; Preschool Curriculum Research Consortium, 2008).
Results revealed statistically significant increases in print knowledge for children in the workshop-plus-mentoring group, but no differences were found between children in the two groups in phonological processing, oral language, or math.
Building Language for Literacy
The purposes of this randomized trial (Ramey, Ramey, Kleinman, Lee, Farnett, Timraz, et al, no date) were to understand the factors and instructional practices that promote childrens language and literacy in the context of Scholastics Building Language for Literacy (BLL) curriculum and two levels of coaching (monthly versus weekly). The project emerged from a ten-year partnership between the Montgomery County Public Schools (MCPS) in Maryland and Georgetown Universitys Center on Health and Education. Three hypotheses addressed PD: (1) BLL coaching will lead to significant benefits in classroom instructional environment; (2) children with teachers who have BLL coaches will have significantly higher literacy levels; and (3) weekly coaching will lead to greater benefits than monthly coaching.
All classrooms used the BLL language and literacy curriculum. Twenty-four classrooms were randomized into the intervention (coaching) or control conditions, and the classrooms in the intervention condition were further randomized to either weekly (30 sessions) or monthly (8 sessions) coaching.
In the two intervention conditions, PD consisted of a three-day summer institute to introduce the curriculum (2 days for paraeducators), coaching, plus optional monthly evening group sessions with coaches and peers for more discussions. The teachers time for the evening sessions was covered by district stipends, and teachers could earn up to 16 units of PD credits. Coaches had Masters degrees in reading with additional training on the BLL curriculum, and they received ongoing supervision during the course of the study.
PD in the comparison condition consisted of the PD offered to all MCPS pre-K/Head Start classrooms. Certified teachers could participate in a voluntary summer training institute; aides could participate in a half-day of instruction. Teachers also had access to additional PD days and supervisors who were content specialists throughout the year. Teachers in all classrooms were certified teachers with a specialty in early childhood education.
Results indicated that BLL coaching classrooms had significantly higher levels of curriculum implementation than comparison classrooms. Contrary to the hypothesis on intensity, teachers who received monthly coaching had better implementation scores than teachers who received weekly coaching.
Classrooms in both coaching conditions had higher scores on the Early Language and Literacy Classroom Observation (ELLCO) than control group classrooms, but there were no differences in ELLCO scores between the weekly and monthly coaching groups. Although no statistical tests were conducted, the authors report that teachers in the coaching condition may have displayed better performance on the Rameys Observation of Learning Essentials (an observational measure of teacher behavior).
Three measures of child outcomes, all focused on childrens early language and literacy skills (Test of Early Reading Ability (TERA); Get It, Got It, Go!; and a school district pre-K reading measure) were used to assess differences in the gains children made from fall to spring during the study year. Children in coaching conditions showed greater gains than children in the control group in TERA scores (total scores, and two of three subscales), but there were no significant differences in gains between children in weekly and monthly coaching conditions. Children who were English Speakers of Other Languages achieved greater fall-to-spring gains in TERA scores if they were enrolled in coaching rather than non-coaching conditions. There were no differences among any groups on the other two measures of child outcomes.
In summary, BLL coaching resulted in higher levels of literacy-rich classroom environments and instructional practices and higher early literacy skills on one standardized measure of childrens reading ability compared to typical classrooms. There was no benefit of the more intense coaching. This may perhaps be due to the small sample size, as there were only six classrooms in each of the coaching conditions. The small sample sizes, school-based settings, and high educational levels of teaching staff may also limit generalizability.