Proceedings from a Working Meeting on School Readiness Research: Guiding the Synthesis of Early Childhood Research. Discussion

12/15/2009

We consider several sets of questions concerning the effectiveness of the programs and what can be learned from the evaluations of them. We conclude with suggestions for a research agenda.

Questions Concerning the Current Programs

How successful are the programs? A basic finding is that math education, as exemplified by the programs described above, can work for young children. Studies of different curricula find relatively large effect sizes, as indicated above. They were at least fairly successful in accomplishing their various and sometimes diverse goals. There is little doubt that early education can promote early mathematics learning in different areas, including number, shape, space, and pattern. This is valuable information, and it gets the enterprise started: there should be no doubt that early childhood mathematics education can be effective, at least in the short term.

At the same time, there are many questions remaining to be addressed and much that still needs to be learned. One question refers to the differential effectiveness of the programs under consideration. Do some achieve better results than do others? The answer is probably yes, but it is hard to compare programs directly. As we showed, the research studies used a wide variety of outcome measures for evaluation. As a result, it is hard to examine the relative effectiveness of programs (even using effect size) when they are trying to accomplish different goals. One program may be effective in promoting spatial reasoning and another effective in teaching the reading of numerals. It is good that both are effective, but it is hard to compare programs when goals and subject matter differ.

Further, it is important to note that many evaluations use outcome measures developed in conjunction with the goals of the curriculum (e.g., PreK Math/DLM), whereas other programs (e.g., BMLK ) use measures that do not align with the curriculum itself. In a sense, the aligned outcome measures can be considered near transfer tasks and standardized measures, far transfer tasks. The use of an outcome measure that aligns with the curriculum increases the likelihood that the evaluation will find positive effects, but does not indicate whether the treatment group would perform better on mathematical topics not emphasized in the curriculum. The use of far transfer tasks can provide insight into general aspects of learning but provide little useful detail about the specifics. Each approach has strengths and limitations that need to be recognized.

We also need to be clear about the inevitable limitations of the various outcome measures. Although most have reasonably sound psychometric properties, it is fair to say that of necessity the measures generally focus on relatively easy to measure aspects of performance. The results of such an approach are valuable in establishing that some learning has occurred, but the approach often fails to illuminate that learning in any detail. It is conceivable, of course, that a curriculum works, in the sense of promoting high test scores on these kinds of evaluations, but that it does not promote thinking or enhance long-term motivation for learning mathematics. It is conceivable that teachers may teach to the evaluation and in the process fail to promote meaningful learning. High stakes assessment may have negative effects at the preK and kindergarten levels, just as it does at higher levels of education.

How successful are the programs at teaching various topics within the mathematics curriculum? Mathematics is a complex subject, even in preschool. It involves far more than teaching rote aspects of number. The discipline is both wide and deep (Ginsburg & Ertle, 2008), and includes topics ranging from the invariance of cardinal number across various transformations to the idea of mapping physical space. Following the advice of the NAEYC/NCTM, many of the curricula present mathematics as a broad array of topics, including number, measurement, space, shape and pattern. At the same time, the program evaluations generally present little information concerning childrens learning in each of these specific areas. Consequently, we need to know much more about program effectiveness in teaching the very different topics of mathematics, ranging from number to shape and pattern.

In particular, we need to learn much more about a very special topic, namely mathematical thinking and reasoning. Children need to learn to understand why a figure is a triangle, not a rectangle, and to reason about why one operation (like 2 + 3) yields the same result as another (like 3 + 2). Some of the programs seem to promote such mathematical thinking and reasoning, but in general, the evaluations do not attempt provide in depth information concerning thinking and reasoning processes, strategies employed, and understanding of important ideas. One reason is that random assignment studies involving large numbers of children need to employ tests that are easy to administer on a large scale and relatively short. Such tests, although useful for their purpose, are not optimal for measuring cognitive phenomena as subtle and complex as reasoning and understanding. Another reason is that the field lacks appropriate and practical measures of mathematical thinking and reasoning.

In brief, we need to know much more than that a program works. We need to know how it works in the different substantive areas of mathematics, and how it works in the key area of mathematical thinking and reasoning. This kind of information can be of great value for researchers, teachers, and curriculum developers alike.

What aspects of the programs pedagogical methods or materials are most powerful in promoting childrens mathematical learning?  The programs employ various methods and materials. Sometimes they use small groups, and sometimes the use large ones. Sometimes the approach is relatively didactic and sometimes more open-ended. Sometimes they use games, and sometimes stories. Sometimes they use computers, and sometime they do not. Sometimes they do mathematics as a stand-alone activity, and sometimes it is integrated into other activities.

There are many questions to ask about these practices. How effective are the various methods  games, manipulative, stories, and the like, under various circumstances? How should the various methods be used in presenting the material? These of course are the primary issues of interest to teachers who work every day on teaching mathematics.

A crucial set of questions revolves around teaching. Many of the studies attempt to ensure the fidelity of instruction, in the sense of determining whether teachers teach the material more or less as intended. But the studies pay very little, if any, attention to the ways in which teachers implement the activities, incorporate them into their own teaching styles, find some topics easier to teach than others, interpret the materials, adjust teaching to meet student needs, and understand (or misunderstand) the competence of their students. Teachers are at the heart of any program and curriculum, yet the present studies tell us little about their roles in the enterprise.

In general, because of their broad focus on student outcomes, the evaluations typically provide no information about the strengths and weaknesses of various aspects of the programs, or about intentional teaching. As a consequence, the questions about methods, materials and teaching  the questions of most interest to teachers (and creators of professional development programs)  remain unanswered.

What have we learned about group, individual, and developmental differences in childrens mathematics learning?  There are substantial differences between SES groups in mathematics achievement. As is well known, low SES children generally perform more poorly than their middle SES peers. It appears that preschool instruction can be effective for both groups, although it may not eliminate the initial gap between them. But it is important to know whether, how, and to what extent the groups differ in their reactions to and learning from various programs. How do the different groups of children interact with the teachers and activities and does that contribute to the outcomes?

There are also wide individual differences in preschool childrens psychological functioning, language and mathematical knowledge. Some children enter preschool knowing little English. Some have poor executive function. Some may be stronger than others in number (Dowker, 2005). It is conceivable that some children may benefit more than others from a particular pedagogical method or curriculum.

Similarly, there may be important developmental differences in learning mathematics. The old view that preschoolers in general are concrete thinkers, or preoperational and therefore cannot learn an abstract subject like mathematics has been discredited. Nevertheless, there may be important differences between typical 3-year-olds and 4-year-olds in their learning of mathematics. What is the nature of these differences?

In general, the evaluation studies, focused as they are on the measurement of broad outcomes, do not provide information useful for addressing issues of group, individual or developmental differences in learning mathematics.

What can we conclude about effectiveness?  The evaluation research has shown that the various programs are effective in varying degrees in achieving their varied goals. That is important to know, but the research tells us little more than that, perhaps in part because of the very nature and demands of large-scale random assignment research. The research has little to say about relative effectiveness of different programs, about their success in teaching specific topics, about the relative power of different pedagogical techniques and materials, about how teachers teach, and about group, individual and developmental differences in learning.

A Research Agenda

The current evaluation paradigm has taught us a great deal, and has taken a useful first step in the direction of sound early childhood mathematics education. Yet, as we have shown, the paradigm is limited in its ability to answer key questions. The productive solution is not simply more and bigger RCT studies. Instead, we need a new and wide research agenda dealing with several issues fundamental to early mathematics education.

What and how should we evaluate? One set of issues concerns further evaluation of mathematics programs. Now that we know that many of them work, it is important to conduct research targeted to more specific issues, like the relative effectiveness of different kinds of programs for teaching specific content. What are some effective ways for teaching 4-year-olds the analysis of geometric forms or 3-year-olds some fundamental properties of number? How effective are particular materials or pedagogical methods?

In conducting work of this type, the field can benefit from improved outcome measures that tap into essential aspects of learning across the various topics that comprise the content of early mathematics. We need to get beyond using measures because they are convenient or have sound internal or test-retest reliability. The fundamental question is whether they measure what is important to measure. Fortunately, NIH is now funding the development of new research based measures of mathematics knowledge and other topics relevant to early childhood.

And as we go forward, heres a topic that should not receive much research attention: the long-term effects of early mathematics curriculum. Childrens later mathematics outcomes must be influenced by the education children receive after preschool. We know that much of that education, particularly for poor children is lacking, with the likely result that children receiving good preschool math education may not do very well later in school. This outcome is entirely to be expected and does not reflect on the childrens abilities or what is possible to achieve. Hence not much effort need be put into studying it. A more effective approach is to work at improving and evaluating education at all levels.

What are the processes involved in mathematical teaching and learning? A second set of issues revolves around the processes of teaching and learning. Mathematics has seldom been taught at the early childhood level. Consequently we know little about how to teach it or how children learn it. Most of the cognitive developmental research that has provided a revolution in the way we conceptualize young childrens mathematical abilities does not focus at all on teaching or on how children learn from teaching and in an educational context. The various curricula are research-based mostly in the sense that they are inspired by research on childrens mathematical competence, and not in the sense that they derive from the research any particular guidance on how to present or teach any topic. Therefore, we need research, some of which needs to be exploratory, that focuses on teaching and on childrens learning from teaching in an organized setting. Because so little is known about these topics, this kind of research will ultimately be of great practical value to teachers. By contrast, current evaluation research does not speak to teachers about these issues, except to tell them that effective early math education is possible.

How can we effectively implement math curricula? Many early childhood teachers have no interest in early mathematics, fear it, and do not want to teach it, sometimes because of outmoded notions of developmental appropriateness. School districts, preschools, and childcare organizations typically give the teachers little help in their efforts to implement mathematics programs. Several questions then arise: What are the obstacles that stand in the way of successful implementation? How can they be overcome? How can one help teachers to cope with their fears of mathematics and learn effective teaching methods (assuming we learn what those are)? What kind of supports  especially professional development  do teachers need over the long term to implement early mathematics education? In general, the problem is first to set up and then examine the effectiveness of an infrastructure for promoting early mathematics education. In the end, everything boils down to helping and supporting teachers to do good work over the long term.

View full report

Preview
Download

"index.pdf" (pdf, 307.32Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apa.pdf" (pdf, 78.45Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apb.pdf" (pdf, 401.85Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apc.pdf" (pdf, 288.78Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apd.pdf" (pdf, 90.04Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"ape.pdf" (pdf, 190.7Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"apf.pdf" (pdf, 68.04Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®