By
Martha J. Zaslow, Sharon M. McGroder, and Kristin A. Moore
Overview
As we seek to understand the effects on families of the 1996 welfare reform law, we can build on the foundation of a rigorous evaluation study focusing on the effects on families of welfare-to-work programs implemented under the previous welfare law, the Family Support Act of 1988. The National Evaluation of Welfare-to-Work Strategies (the NEWWS, funded by the U.S. Department of Health and Human Services and the U.S. Department of Education) is evaluating the impact of a set of welfare-to-work programs operated under "JOBS" (the Job Opportunities and Basic Skills Training Program). A pioneering feature of this national evaluation is that it simultaneously considers program impacts on adult economic outcomes and on the development and well-being of the children in the families.
This summary report presents a summary of findings from one of a set of three complementary reports from the National Evaluation of Welfare-to-Work Strategies' two-year follow-up (with results from a further follow-up, completed five years after families enrolled, to be presented in the future). We focus in the present summary report on the findings related to impacts on children, reporting results from a special component of the evaluation, the Child Outcomes Study (see McGroder, Zaslow, Moore and LeMenestrel, 2000, for a detailed presentation of findings). This component of the evaluation focuses in depth on children's development and well-being for a sample of families with young (preschool-age) children at the start of the evaluation, drawn from three of the evaluation's seven research sites. A second report in this series focuses primarily on economic impacts in all seven of the evaluation's research sites, with a more limited examination of impacts on children of all ages (Freedman, Friedlander, Hamilton, Rock, Mitchell, Nudelman, Schweder, and Storto, 2000). A third report draws together findings on children from the in-depth look at young children in the Child Outcomes Study, and brief markers of well-being collected regarding children of all ages in families in the full evaluation sample (Hamilton, with Freedman and McGroder, 2000).
The Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies examines the impacts on both the parental and child generations of two distinct approaches to welfare reform implemented as part of the federal JOBS Program: a labor force attachment approach (emphasizing a rapid transition to employment), and a human capital development approach (emphasizing a longer-term strategy of education and training in order to obtain a better job). These strategies are precursors of the welfare reform programs now being implemented under the 1996 welfare law, the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA). The labor force attachment approach under JOBS is especially germane because of its emphasis on moving clients quickly into employment, the clear priority of new policies. However, the human capital development approach may provide an informative model for states as caseloads drop, and those families remaining on welfare face more barriers to employment (such as low literacy or limited education).
Although welfare policies were initiated many years ago with the aim of protecting children in poor families, most of the evaluation research concerning these policies has focused on adult economic outcomes. This is perhaps not surprising, given that the most clearly targeted outcomes of these programs have been economic. The Family Support Act explicitly stated as its goal the reduction of long-term welfare dependency. Further, this law did not call for services aimed directly at enhancing the development of children (such as early childhood educational intervention, or developmental screening); rather authorized services focused on increasing adult employment.
Nevertheless, a mother's assignment to a welfare-to-work program has the potential to affect the development of children, for example, by affecting the material resources available within the family, and by affecting children's experiences of care both within and outside of the home. The Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies examines whether children can be affected by their mothers' assignment to a welfare-to-work program, how their development and well-being are affected (favorably or unfavorably), if at all, and in what ways any impacts on children come about.
The Child Outcomes Study uses a rigorous experimental design. Two years after mothers were randomly assigned to one of the two JOBS welfare-to-work strategies or to a control group, outcomes for children (at that point between about 5 and 7 years of age) were examined. The children's cognitive development and academic achievement were measured through a combination of direct assessment (an assessment of the children's cognitive school readiness) and maternal report (for example, mothers' reports of academic problems). The children's behavioral and emotional adjustment were measured through maternal report (for example, using measures of the child's behavior problems and positive social behaviors). The children's health and safety were also measured through maternal report (for example, using an interview measure indicating whether the child has had an accident, injury or poisoning requiring emergency medical attention; an interview measure widely used in national surveys in which the mother indicates whether she sees the child's overall health as excellent, very good, good, fair, or poor). Validation work indicates that the global health rating reflects primarily physical health problems (Krause and Jay, 1994).In general, all measures selected for use in the Child Outcomes Study have demonstrated sufficient validity and reliability (Bracken, 1984; Polit, 1996; Zill, 1985; Peterson and ZILL, 1986)
In addition to examining mean scores on measures of cognitive school readiness, problem and positive behavior, and overall health, we also examined program impacts on the proportion of children with extreme scores on these measures in the interest of ascertaining whether JOBS welfare-to-work programs changed the distribution of children's outcomes for example, reducing the proportion at the "unfavorable" end and/or increasing the proportion at the "favorable" end which is possible even if the programs had no impacts on mean scores. Thus, in some cases a single response to a survey question can give rise to two or more impacts (e.g., one relating to the mean and one relating to the distribution).
The Child Outcomes Study was conducted in three sites: Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. Families enrolled in the evaluation between September 1991 and January 1994. Data collection for the two-year follow-up was completed in January 1996, prior to the implementation of the new welfare law. Findings from this study must be seen in light of the fact that mothers were exempt from participation in JOBS welfare-to-work activities if they were needed at home to care for an ill or incapacitated family member, including a child. As a result, children with a health condition requiring such care were not included in the evaluation. The 1996 welfare law no longer provides an explicit exemption for a mother with an ill or incapacitated child.
In this experimental evaluation, mothers randomly assigned to a control group in each of the study sites were eligible for all welfare benefits. However, they did not receive the special messages and case management of a JOBS program, they were not mandated to participate in JOBS program activities, and they did not have access to the particular work preparation activities through JOBS. Control group members were eligible for child care assistance, similar to that offered to program group members, if they were participating in work preparation activities in which they had enrolled on their own.
The summary report that follows this overview and the full report of the two-year follow-up of the Child Outcomes Study provide extensive information on the impacts of the six JOBS programs examined here (i.e., the human capital development and labor force attachment approaches in each of the three study sites) on economic outcomes in these families with young children, as well as on such further measures of family functioning as marital status, father involvement as reported by mothers, maternal psychological well-being and parenting. In this overview, we focus specifically on the findings regarding impacts on children.
Overall, the six JOBS programs examined had relatively few statistically significant impacts on children on average, that is, across all families assigned to a given program. Nevertheless, there were more impacts than one would expect due to chance. In addition, findings differed according to the aspect of the children's development examined, with impacts in the area of cognitive development favorable, in the area of health unfavorable, and in the area of behavior mixed (including both favorable and unfavorable impacts).
Child impact findings at the aggregate level tended to differ according to site. Whereas impacts on child outcomes were relatively few, they tended to be favorable in Atlanta, unfavorable in Riverside, and mixed in Grand Rapids. Where impacts were found, they were generally small. Only a single impact(1) was of sufficient magnitude to be called "policy relevant." Policy relevant findings in the present study are those that meet the criterion, set at the start of the study, of having an effect size of at least a third of a standard deviation.(2) The strongest evidence on which to base conclusions about impacts on children is a consistent patterning of impact results, particularly when impacts meet or exceed the criterion for policy relevance.
In addition to examining impacts for the sample as a whole in each site, analyses also addressed the question of whether findings differed for children from families with different background characteristics. There was an initial concern that children in families at higher risk (in terms of such background characteristics as the mother's education, her work history, number and spacing of children, and maternal psychological well-being) might show a pattern of unfavorable program impacts. Mothers in higher-risk families might be less able to fulfill the requirement to participate in a welfare-to-work program, or might experience substantial stress in doing so. Children in higher-risk families might be more vulnerable to changes in family routines and their own care situations. Yet on the other hand, JOBS programs might provide exactly the kind of support the higher-risk families need in order make progress toward economic self-sufficiency. The combination of messages, enhanced case management, program requirements, and services might be particularly effective for such families. If this is indeed the case, then children in higher-risk families might be particularly likely to show favorable impacts of one or both JOBS program approaches.
As for the findings for children across all families, statistically significant impacts for subgroups of families were relatively few, though exceeding what might have been expected on the basis of chance. Findings for children in higher-risk families, when they emerged, tended to be favorable, though small, when the mother had been assigned to a human capital development program or to Atlanta's labor force attachment program(3). By contrast, program impacts, some of which reached the criterion for policy relevance, tended to be unfavorable for children from higher-risk families in the other two labor force attachment programs.
Contrary to the initial hypothesis, it was among lower-risk families that there were indications of some concentrations of unfavorable impacts for children, and these did not appear to vary by program approach. Rather, a pattern of unfavorable impacts occurred for children in three particular programs: Grand Rapids' labor force attachment program, and both of Riverside's programs. In addition, a number of the program impacts for children in lower-risk families met the criterion for policy relevance, and nearly all of these were unfavorable impacts. Further, many (one-third) of the unfavorable and policy relevant impacts found for children in lower-risk families substantially exceeded the threshold for policy relevance, in that effect sizes were .50 or larger (which is considered "moderate" to "large" in magnitude; Cohen, 1988).
Further analyses were conducted to begin to explicate the processes underlying program impacts on children. While the analyses of program impacts on children summarized above are experimental in nature, the analyses used regarding the processes underlying program impacts on children, mediational analyses, are non-experimental. Whereas experimental analyses provide strong evidence regarding the existence of program impacts on children, they cannot address the question of how these impacts came about. Because mediational analyses are non-experimental, they do not allow firm causal inferences to be made regarding the pathways through which these impacts come about. Nevertheless, if one hopes to gain insight into the processes through which a given program had impacts, then non-experimental statistical analyses are necessary.
Mediational analyses examined five selected program impacts on children in greater depth. The particular child impact findings focused upon in this way were selected in keeping with the overall patterning of findings of favorable impacts for children in the area of cognitive development, unfavorable health impacts, and mixed behavioral impacts. These analyses rely on information about processes and child outcomes measured contemporaneously and used a relatively modest statistical approach in modeling pathways; as such, results should be considered preliminary. Additional waves of data from the five-year follow-up, as well as statistical methods that more effectively partition effects among multiple mediators, more completely control for selection effects, and allow alternative models to be tested explicitly (e.g., structural equation modeling), are needed to provide more definitive answers regarding the multiple pathways through which program impacts on children came about.
These analyses suggest that the mechanisms through which children can be affected by a given welfare-to-work program include both outcomes that were directly targeted (e.g., employment), as well as outcomes not directly targeted by the programs (e.g., parenting, maternal psychological well-being). In particular, though, the findings highlight the role played by mechanisms more proximal to the child, especially maternal psychological well-being and parenting. As yet, the roles of child care, health insurance, and income in mediating the five child impacts selected for pathways analyses have not been identified, although it should not be concluded that these factors do not in general affect outcomes for children. Findings also indicate that welfare-to-work programs sometimes have effects on aspects of family life that are important to children that go in opposing directions (favorable and unfavorable). For example, Atlanta's labor force attachment program increased mothers' sense of time stress (which was associated with less favorable behavioral outcomes in children), but simultaneously led to improvements in parenting (which was associated with more favorable behavioral outcomes in children). Impacts on children reflect the net effect of such influences. Opposing influences of this kind on key aspects of family life may help explain the small number and size of significant impacts on child outcomes. Understanding the nature and direction of multiple influences on family life is central to strengthening pathways that yield favorable impacts on children and weakening the pathways that have detrimental implications for children.
In sum, the results indicate that the welfare-to-work programs implemented as part of the JOBS Program did have significant impacts on children's developmental outcomes, but these impacts were not widespread and were generally small. When impacts did occur, they were favorable in the area of the children's cognitive development and academic achievement, unfavorable in the area of the children's health and safety, and mixed in the area of behavioral and emotional adjustment. Looking at subgroups of families, there was a pattern of favorable impacts for children from higher-risk families assigned to human capital development programs or to the Atlanta labor force attachment program. Yet, at the same time, there was a concentration of unfavorable, and policy relevant, impacts for children from lower-risk families in three of the programs studied.
Given these findings, we must consider it a possibility that the welfare-to-work programs implemented under the 1996 law can have impacts both favorable and unfavorable on children's development. It is indeed possible that the intensified obligations and opportunities of the new policy are resulting in stronger or more pervasive impacts on children than were found for the JOBS Program. Furthermore, the variations in impacts suggest the value of research that would examine other policy strategies, for example, child care, health care, and other work supports, to determine how to promote favorable, and avoid unfavorable, child impacts. Thus, it is important to continue to measure the influences of welfare-to-work programs and other policy strategies on both the parent and child generations.
Bracken, B. A. (1984). Bracken Basic Concept Scale: Examiner's Manual. The Psychological Corporation, Harcourt Brace Jovanovich, Inc.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition). Hillsdale, NJ: Erlbaum and Associates.
Freedman, S., Friedlander, D., Hamilton, G., Rock, J., Mitchell, M., Nudelman, J., Schweder, A., and Storto, L. (2000). Evaluating alternative welfare-to-work approaches: Two-year impacts for eleven programs. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.
Hamilton, G., with Freedman, S., and McGroder, S. (2000). Do mandatory welfare-to-work programs affect the well-being of children? A synthesis of child research conducted as part of the National Evaluation of Welfare-to-Work Strategies. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.
Krause, N. M. and Jay, G.M. (1994). What do global self-rated health items measure? Medical Care, 32 (9), 930-942.
McGroder, S.M., Zaslow, M.J., Moore, K.A., and LeMenestrel, S.M. (2000). The National Evaluation of Welfare-to-Work Strategies: Impacts on young children and their families two years after enrollment: Findings from the Child Outcomes Study. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.
Peterson, J., and Zill, N. (1986). Marital disruption and behavior problems in children. Journal of Marriage and Family, 48, 295-307.
Polit, D. F. (1996). Self administered teacher questionnaire in the New Chance 42-month survey. New York: Manpower Demonstration Research Corporation.
Zill, N. (1985). Behavior Problem Index. Washington, D.C.: Child Trends, Inc.
Goals of this Report
This summary report highlights key findings from a study focusing on young children in the context of welfare-to-work programs: the two-year follow-up report of the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies (McGroder, Zaslow, Moore and LeMenestrel, 2000). Specifically, this report asks whether young children's development and well-being were affected when their mothers were assigned to participate in welfare-to-work programs implemented as part of the Job Opportunities and Basic Skills Training (JOBS) Program, the set of programs authorized by the Family Support Act of 1988.
The JOBS Program required mothers with children as young as age three (or age one, at state option) to participate in welfare-to-work activities. This was the first time that a federal program implemented nationally required participation by mothers with preschool-age children. Young children are particularly likely to experience changes in their daily lives when their mothers' activities change. In addition, there is evidence that young children's development is particularly sensitive to family poverty status (Duncan and Brooks-Gunn, 1997). Thus, it is important to consider the implications of welfare-to-work programs on children who are preschool-age when their mothers enroll in these programs.
In this report we ask whether different aspects of young children's development (specifically their cognitive development and academic achievement, behavioral and emotional adjustment, and health and safety) were affected at all, and, if so, for better or for worse, two years after their mothers were assigned to participate in a JOBS welfare-to-work program. We also ask whether aspects of family life that are important to children's development (for example, the stimulation and support available to the child in the home environment, participation in child care, and poverty status) changed when mothers were assigned to participate in a JOBS welfare-to-work program. Finally we explore the linkages between changes in family life and program impacts on children's developmental outcomes.
Why focus on outcomes for children in evaluating a set of adult-oriented welfare-to-work programs?
Much of what is known about the effects of welfare-to-work programs focuses on economic outcomes for adults, for example, the extent to which programs increase employment, earnings and household income, and reduce welfare receipt. Yet a majority of those receiving public assistance are children. Further, a primary purpose of welfare programs has historically been to protect the well-being of children growing up in poor, and especially single-parent households. Even those programs that target adult economic outcomes, and have few or no components explicitly focused on enhancing young children's well-being, have the potential to alter important aspects of children's lives. For example, programs that target family economic self-sufficiency can alter the time children spend in the home and in child care settings, and the material resources available for their needs. Thus, even though JOBS programs did not aim directly to change child well-being (for example, through an early childhood intervention, or through developmental screening and follow-up services), children's development could have been affected indirectly, because of changes within the family and in children's care situations.
Accordingly, this study asks: (1) whether JOBS programs affected the well-being and development of children; (2) what the direction of effects on children was, that is, whether children's development and well-being were influenced favorably or unfavorably; (3) which children were affected (all children, on average, or primarily children in certain types of families); and (4) how children were affected by their mothers' assignment to a JOBS program.
This study uses a rigorous design to examine program impacts on children. The sample for the Child Outcomes Study is a subset of the families that were participating in the larger evaluation of economic impacts of JOBS called the National Evaluation of Welfare-to-Work Strategies. The subset of families in the Child Outcomes Study includes families in three of the full evaluation's seven research sites, and specifically families that had a preschool-age child at the time of enrollment in the evaluation.
The larger evaluation, and the Child Outcomes Study embedded within it, followed an experimental design. Families were randomly assigned to a control group, or to one of two experimental groups. The control group families were eligible for all benefits associated with Aid to Families with Dependent Children; however, they had no requirement to participate in a welfare-to-work program, nor did they receive the special messages, case management, or services of a JOBS program. The two experimental groups were also eligible for all benefits, but were required to participate in a JOBS welfare-to-work program. The two experimental groups differed in terms of which kind of program they were randomly assigned to, with one approach (labor force attachment) emphasizing a rapid transition to employment, and the other (human capital development) emphasizing education and training as a means to obtaining a better job. Families in the two experimental groups had the encouragement and support of enhanced case management. At the same time, they faced the possibility of sanctions (reductions in welfare benefits) if they did not participate in welfare-to-work activities.
In an experimental study of this kind, we are confident that families did not differ systematically before they were randomly assigned to a control or experimental group (and this was confirmed statistically). Accordingly, differences between those in an experimental group as opposed to the control group can be attributed to the influence of having been assigned to a JOBS program. We use the term child outcome to refer to the measures of child well-being and development that we are focusing on. We reserve the term child impact specifically to describe differences on a child outcome measure between families in an experimental and control group.
In addition to assessing impacts, we also look at the child outcome scores for children in the control group in order to describe the developmental status of children in the absence of a JOBS welfare-to-work program. In this way, we can examine the extent to which these children were at risk in their development apart from exposure to a welfare-to-work program. This provides an important context for understanding program impacts.
We make this distinction in terminology for measures of family well-being as well, with the term "outcomes" referring to the measures of family functioning in economic and other areas, but the term "impacts" referring to experimental and control group differences on these outcome measures.
Did we anticipate effects on children to go in a particular direction?
Very little is known about how welfare-to-work programs affect children, and the Child Outcomes Study is one of the first to focus on this issue. Given the very limited basis for making predictions, we did not begin the study anticipating that JOBS programs would affect children in a particular direction, with either improvement or deterioration in the children's development. Instead, the study was initiated in order to explore four diverging possibilities:
- That children's development would be affected favorably when their mothers were assigned to a JOBS program. Mothers assigned to a JOBS program could potentially experience gains in educational attainment, basic skills, and job skills. Over time, participation could lead to employment, improvements in earnings and income. Through such changes, mothers could also experience increased psychological well-being, and the home environment could improve. Increased child care participation, given that the care was of good quality, could provide children with increased stimulation and support. Such changes could affect children's development positively.
- That children's development would be affected unfavorably when their mothers were assigned to a JOBS program. This direction of impacts on children is possible if mothers experienced substantial stress in making a change in their daily activities and in arranging family life so that they could participate in welfare-to-work activities. Such stress could affect the quality of mothers' interactions with their children and the degree of organization (for example of routines, the physical setting) of the home environment. Unfavorable impacts on children could occur if the young children were exposed to unstable (frequently changing) child care arrangements or child care of poor quality. Unfavorable impacts on children could also occur if mothers made the transition to employment, but did so without stable earnings, or with total family income falling below what they were receiving while on welfare.
- That only children in certain subgroups of families would be affected. Families with particular characteristics might respond in different ways to the mothers' assignment to JOBS programs. For example, mothers with limited education, literacy, and/or previous work experience, or mothers with symptoms of depression, might find it particularly difficult to fulfill welfare-to-work program requirements. Such mothers might fail to fulfill the requirements and face sanctioning. Alternatively, they might fulfill the requirements, but do so with substantial stress. Children in these families, but not others, might show unfavorable program impacts. On the other hand, there might be families especially likely to benefit from JOBS programs. For example, mothers with limited education and literacy, mothers with limited work experience, or mothers experiencing psychological distress, might benefit especially from the combination of supports (enhanced case management), program requirements (a participation mandate), and program messages and services (labor force attachment and human capital development welfare-to-work activities) that comprised JOBS programs. If the mothers in such higher-risk families are particularly responsive to the JOBS program components, and make changes in educational attainment, job-related skills, employment and income, such changes could affect children favorably in these particular families.
- That there would be no effects on children. Changes may occur within the family, but not in aspects of family life that affect children immediately. For example, it may not be enough (from the point of view of impacts on children) to change mothers' employment status or the sources of family income if these do not result in changes in the child's immediate environments of home or child care. Perhaps changes in the home environment or child care do occur but are too small to result in impacts on child development. There could also be influences on aspects of family life going in opposing directions, with these influences resulting in no net change in children's development. For example, maternal earnings might increase, but the home environment might simultaneously deteriorate with increased maternal stress. These different possibilities point to the importance of examining not only whether JOBS affected children, but also what kinds of changes occurred within families, and how these changes jointly help to shape impacts on children.
The study was designed with sufficient power to reasonably assess whether there were harmful impacts on children, an important hypothesis to examine when the new provisions of the Family Support Act were being implemented. The design also has sufficient power to assess whether favorable impacts occurred or whether different impacts occurred for different subgroups.
At what point should policymakers take findings on children into account in making policy decisions?
In the present study, we report all program impacts on children that are statistically significant. These program impacts are reliable: they are very unlikely to have occurred just on the basis of chance. As such, these program impacts warrant continued monitoring. In the present study, we will want especially to monitor whether the kinds of outcomes on which statistically significant impacts were found at the two-year point continue to show differences at the final follow-up (five years after the families enrolled in the evaluation), and if such differences grow in magnitude.
We also report on whether a statistically significant result meets a further criterion: that of "policy relevance." At the start of the study and as the study proceeded, researchers and policy makers met to grapple with the question of the point at which child impact findings should be taken into account in considerations about policy. A decision was made that statistically significant findings that were of a particular magnitude should be considered relevant to policy discussions. Specifically, for this study, a policy relevant impact was defined as one in which the effect size was at least one-third of a standard deviation on a given measure.
This threshold sets aside impact findings that are so small that, while they are reliable statistically and warrant continued monitoring over time, may at this point in time have limited importance in terms of children's development. At the same time, the threshold for policy relevance does not require that an impact be "large" in magnitude(4) in order to meet the criterion. By setting the threshold in this way, we can be reasonably confident that we are being inclusive in identifying instances of possible harm as well as of possible beneficial effects on children, without focusing on effects that are so small as to be of limited importance for children's development.
In presenting results, we discuss not only the significance and policy relevance of impacts but also the patterning of findings. We also identify those impacts for which effect sizes substantially exceeded the threshold for policy relevance, that is, were .50 or larger (which is considered "moderate" to "large" in magnitude; Cohen, 1988). The strongest evidence on which to base conclusions about impacts on children is a consistent patterning of impact results, particularly when impacts meet or exceed the criterion for policy relevance. A patterning of results, for example, might show consistently favorable impacts for families in a particular site, or a particular program approach. A patterning of results might also pertain to a type of child outcome, with findings in one aspect of development (such as health) consistently affected favorably (or unfavorably) across programs.
What is the relevance of these findings in the new policy context?
Data collection for the two-year follow-up in the present evaluation was completed in January of 1996, prior to implementation of the new welfare law. Are findings from this evaluation still relevant in the broader sense of having implications in the new policy context? With the passage of the new welfare law, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA), AFDC and the JOBS programs were supplanted by the Temporary Assistance to Needy Families (TANF) program. Nevertheless, the experiences of families participating in JOBS offers guidance to states implementing their own welfare-to-work programs. In particular, findings from the evaluation of JOBS may suggest which programmatic approaches affect child outcomes, and in what ways. Apart from program impact findings, the Child Outcomes Study also provides much information about the developmental status of control group children, and about the family contexts of these children; information indicating how these children and families would be faring in the absence of the JOBS Program. This information, too, can help to guide efforts to shape programs that include the needs of both children and adults in families receiving public assistance (see Knitzer, Cauthen and Kisker, 1999).
The present welfare policy environment differs substantially from that under AFDC in several important respects (see Zaslow, Tout, Smith and Moore, 1998). For example, welfare benefits are no longer considered an entitlement under the 1996 law, and states have substantial latitude in implementing their particular programs. Under TANF almost all states emphasize a quick transition to employment. The new federal law requires that recipients of public assistance be working within two years of the time they start to receive assistance (or sooner at state option), places a lifetime limit of 60 months on the receipt of federal TANF funds, and requires states to have a certain proportion of their caseloads meeting work requirements. But there are also greater supports for work, with almost all states now treating earnings and assets more generously, and significantly more federal funds are available for child care. Another difference is that under AFDC, JOBS also encompassed program approaches focusing on remediation of basic education skills to improve job prospects. Whereas the Family Support Act required mothers of children three and over to participate in work-related activities (with a state option to lower the age to 12 months), the new law lowered the age for all states to 12 months (or younger, if a state chooses). Finally, while under the Family Support Act, mothers were exempt from participation in welfare-to-work activities if they were needed in the home to care for a sick or incapacitated family member (including a child), there is no such explicit exemption under PRWORA.
Over the years, public assistance has become more and more predicated on custodial parents' involvement in work or mandatory welfare-to-work program activities, as policymakers have sought to encourage adult self-sufficiency while also meeting the goal of fostering poor children's well-being. Under TANF and AFDC, recipients of public assistance were required to take steps toward economic self-sufficiency or face a reduction in benefits (sanctioning). In many ways, programs implemented under PRWORA may be seen as an intensification of both the obligation and the opportunity dimensions of those implemented earlier, with respect to this common priority.
The present study is one of only very few that examines impacts on the development of children in the context of welfare-to-work programs. Findings on what happened to children in the context of JOBS programs will help build toward an understanding of how the new policy context is affecting children. We will need to be cautious about extrapolating from the specific findings of the present study to outcomes for children under new policies. Yet addressing the general questions of whether children were affected even in a program directed primarily at economic outcomes in adults, and how children were affected, will provide us with information that will continue to be of vital importance. In addition, documenting the effects of labor force attachment and human capital development approaches will yield important evidence regarding the implications for children of very different welfare-to-work strategies.
The results from this study are an important piece of the emerging picture on the impacts of welfare on children and families. It will be important to put the findings from this study in the context of findings from other studies, such as the Project on State Level Child Outcomes (Child Trends, 1999), which examines the impacts on children of five state approaches to experimenting with welfare policies through obtaining waivers from various AFDC provisions.
Study Approach
How is the development of children being studied in the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies?
The present report focuses on a sample of about 3,000 families with young children whose development is being followed over a period of five years in the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies. The Child Outcomes Study focuses in depth on children's development and family life in a subset of the approximately 55,000 families participating in the full seven-site evaluation of economic impacts of JOBS programs as part of the National Evaluation of Welfare-to-Work Strategies.
The children whose development is being tracked over time were all preschool-age when their mothers enrolled in the evaluation (enrollment occurred between September 1991 and January 1994). The families in the study come from three of the seven sites of the National Evaluation of Welfare-to-Work Strategies: Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. These are study sites in which two different JOBS program approaches were tested: a labor force attachment approach, which encouraged welfare recipients to make a rapid transition to employment, and a human capital development approach, which encouraged recipients to engage first in education and training as a longer-term strategy to improve employment prospects.
What are the characteristics of the families at enrollment in the Child Outcomes Study?
Looking at the characteristics of the families in the sample when they entered the evaluation, there are numerous indications of disadvantage. For example, for the sample as a whole, a majority of mothers had never been married, though there was substantial variation across the sites (71 percent of mothers in Atlanta, 59 percent in Grand Rapids, and 43 percent in Riverside had never been married). In addition, in each of the sites, a majority of the mothers had received welfare for two years or more (about 75 percent in Atlanta; 72 percent in Grand Rapids; and 65 percent in Riverside). Between a third and half of the mothers in the three sites had limited literacy according to an assessment completed at baseline (about 48 percent in Atlanta, 33 percent in Grand Rapids, and 35 percent in Riverside). More than a third of the mothers in the sample reported a moderate to high number of depressive symptoms (about 36 percent in Atlanta; 43 percent in Grand Rapids; and 35 percent in Riverside), a prevalence of symptoms substantially greater than in community samples.
Yet in some respects the characteristics of the families contradict commonly held assumptions about families receiving welfare. For example, while it is common to think of mothers receiving public assistance as having low levels of education, many of the mothers in this sample had completed high school or received a General Educational Development (GED) degree. Indeed, in two of the study sites (Atlanta and Grand Rapids), a majority of mothers had a high school diploma or GED. Many think of families receiving public assistance as having large numbers of children, yet most of the families in the sample had only one or two children. Some view welfare as an intergenerational pattern. Yet in the present sample, most of the mothers did not report receiving welfare when they were themselves children (between one-fifth in Riverside, and one-third in Grand Rapids). Finally, most of the mothers in the sample agreed with the statement that "It's wrong to stay on welfare if you can get a job, even a job you don't like," suggesting positive attitudes about employment.
Even though all of the mothers in the sample had applied for or were receiving welfare at the start of the evaluation, there was substantial variation in terms of key background characteristics. This variation underscores the importance of asking whether children were affected differently in families of different backgrounds. For example, there were sizeable subgroups within this sample who had and had not completed high school or a GED; with higher and lower reading and math literacy scores; with few and many symptoms of depression.
How were the data for this report collected?
This is the second report from the Child Outcomes Study. The first report described the family context and the children's development in one study site at the start of the evaluation.(5) In the current report, for the first time, the possibility of program impacts on young children is being examined. The results come from a follow-up interview carried out with the families in all three sites approximately two years after enrollment in the evaluation, when the children were about five to seven years old. The interviews were conducted in the families' homes. During the course of the interview, a direct assessment of the child's cognitive development was carried out, and mothers completed further measures regarding the children's development. The interview also collected information about aspects of family life that may have been affected by the JOBS programs, and thus, may help to explain program impacts on the children (for example, measures of family economic circumstances, maternal psychological well-being, the home environment, and the children's child care participation).
The measures we report on here from the two-year follow-up, particularly those pertaining to the children's behavioral and emotional adjustment and health and safety, while often well validated in other research, must be seen as having the limitation of relying exclusively on mothers' reports. For example, in the area of health and safety, we do not have doctors' assessments of overall health, nor do we have hospital records data concerning emergency room visits. Mothers' reports carry with them all of the influences that can affect maternal perceptions. For example, a mother who is concerned about being able to make it to work (particularly in a job without benefits such as sick leave) may find child health issues more salient and, thus, any indications of the child's compromised health (e.g., the common cold) may lead these mothers to rate the focal child's overall health less favorably than mothers without similar employment concerns.
In the future, results will be presented from a final follow-up with the Child Outcomes Study families and children. This follow-up was conducted about five years after the families enrolled in the evaluation, when sufficient time had passed for longer-term changes in the families and children in response to JOBS to have manifested themselves. This report will include, in addition to information on assessments of the children's cognitive development and maternal report measures of the children's health and safety, and behavioral and emotional adjustment, data from teacher reports of the children's progress and adjustment in school, and children's reports of their own behavior and engagement in school.(6) The direct assessments of academic achievement are also more extensive at the five-year follow-up point.
The larger, seven site evaluation of primarily economic impacts in the National Evaluation of Welfare-to-Work Strategies includes a more limited set of questions addressed to mothers about the well-being of all of the children in the family. Impact results for these questions in the larger evaluation sample are included in the two-year follow-up report focusing on economic impacts in the full sample (Freedman, Friedlander, Hamilton, Rock, Mitchell, Nudelman, Schweder, Storto, 2000).
In addition, a brief Synthesis Report (Hamilton, with Freedman, and McGroder, 2000) is also being made available, drawing together and integrating the two-year follow-up results from the Child Outcomes Study, and the set of brief measures focusing on all children in the family in the larger evaluation.
JOBS Welfare-to-Work Programs
What were the key components present in JOBS welfare-to-work programs?
The programs implemented under JOBS had four key components: (A) mandated participation in education, training, and/or employment activities; (B) messages about the importance of such activities; (C) enhanced case management services to direct and monitor clients' progress; and (D) services to facilitate employment.
All families, whether they had been randomly assigned to one of the JOBS program groups or to a control group, received AFDC benefits. Families assigned to the program as well as control group members were also eligible for child care subsidies and for Medicaid while receiving AFDC and for a year following a transition off of AFDC and into employment ("transitional benefits"). Control group members were eligible for child care assistance, similar to that offered to program group members, if they were participating in work preparation activities in which they had enrolled on their own.
However, only the families assigned to one of the JOBS program groups were exposed to the four key program components:
A. Mandated Participation
Participation in approved activities to enhance economic self-sufficiency was mandatory. Failure to participate in approved activities could result in sanctioning (i.e., a reduction in welfare benefits). The activities emphasized, and the sequencing of activities, differed in the labor force attachment and human capital development programs.
Clients in labor force attachment programs were first assigned to "job club," which emphasized learning how to find a job through classroom instruction and experiential activities such as phoning employers to set up interviews. After three to five weeks, clients who had not secured employment either continued their job search, were assigned to basic education, or received short-term training.
Depending on their educational level and work history, clients in human capital development programs participated either in a basic education program, for example, a high school completion program, or a General Educational Development (GED) program, or in vocational training or college. If they did not find employment upon completion of their first activity, it was intended that they enroll in job club, individual job searches, on-the-job training or other work experiences, or go on to obtain additional education or vocational training.
B. Messages
Messages provided to clients in the labor force attachment programs (at orientation and through case management) emphasized finding a job quickly. Messages provided to clients in the human capital development programs emphasized building skills in order to find "better" jobs. In addition, while both control group and program group mothers were provided information about child care (which, as described below, in specific sites, included suggestions about the type of child care to use), program mothers may well have had greater exposure to these messages to the extent that enhanced case management put them in greater contact with their case managers.
C. Enhanced Case Management Services
Clients worked with a case manager to develop a plan for progressing toward economic self-sufficiency. Client and case manager then together monitored progress in light of this plan. And while both control group and program group mothers were entitled to child care services, program mothers by virtue of receiving enhanced case management and, thus, greater opportunities for more frequent interactions with their case manager may have a received more sustained assistance in securing child care.
D. Services to Facilitate Employment
The messages and sequence of activities in the labor force attachment and human capital development programs differed, but both emphasized finding employment and both included services, such as job club and job search, to facilitate a transition to employment.
Did JOBS programs differ across sites?
States had latitude in the implementation of their JOBS programs within the framework of these key components. Accordingly, the sites in this study varied in their orientation and in how they implemented their JOBS programs. These differences are documented in detail in a discussion of program practices and characteristics in the three sites (Hamilton, Brock, Farrell, Friedlander, and Harknett, 1997). What follows is a brief summary of key differences.
- In Atlanta, case managers had relatively small caseloads, and they tended to see their role as supportive, actively seeking out services for clients. Even though the labor force attachment and human capital development programs in Atlanta were clearly distinct, case managers in Atlanta more than the other sites tended to give messages to participants in both programs emphasizing the importance of skills and education (human capital development-oriented messages), perhaps due to their experience running human capital development-type programs. They also tended to be somewhat ambivalent about imposing financial sanctions. Nevertheless, sanction rates in Atlanta were fairly high. In Atlanta, families were free to choose among child care options (unlicenced provider in a home setting, licensed provider in a home setting, or child care center), but were reimbursed only for licensed care, and were encouraged to use center-based care, on the grounds that such care was of better quality and more reliable. Assistance in securing and paying for child care was also promoted as a benefit of participating in JOBS.
- Grand Rapids' human capital development program was unique in its up-front, in-depth assessment of clients' needs. However, caseloads were relatively high, making it difficult to give personal attention and encouragement in either program. A greater emphasis was placed on monitoring client participation and imposing financial sanctions for noncompliance. Clients in Grand Rapids were not encouraged to use one form of child care over another and case workers did not actively promote assistance with child care as a benefit of participating in JOBS.
- In Riverside, the welfare agency had extensive experience and success in running labor force attachment-type programs. Thus, while the labor force attachment and human capital development programs in Riverside were distinctly different, in general, case managers tended to emphasize to clients in both programs the importance of a quick transition to the labor force. Despite moderate to large caseloads, Riverside case managers were held responsible for the educational and employment outcomes of their clients. Case managers closely monitored clients' progress, and they encouraged clients to succeed in their assigned activities. In addition, while staff did not hesitate to enforce participation requirements, clients were afforded many opportunities to come into compliance before financial sanctions took effect. Clients in Riverside were encouraged to use informal child care arrangements, in part because such care involved lower reimbursement rates, but also because it was felt that many clients preferred to leave children with friends or family members. In the Riverside site, though not the other sites, the human capital development program was only available for recipients considered in need of basic education (with no high school diploma or GED, non-English speakers, or those with low scores on baseline literacy or math test scores).(7)
In light of these program variations, we use the term "program" in reference to one of the six specific programs implemented in the three study sites, reserving the phrase "the JOBS Program" only when discussing the federally mandated welfare-to-work program authorized in the Family Support Act. We use the term "program approach" to refer to the human capital development and the labor force attachment welfare-to-work strategies. In order to take the local variation in programming into account, we report on the impacts of each program approach (labor force attachment and human capital development) for each of the research sites separately. Comparing and contrasting the results from each of the three sites gives us a picture of the range of impacts on children of their mothers' assignment to human capital development and labor force attachment program approaches as implemented in different locations, and with somewhat different emphases. It also directs us to consider different participant characteristics in each site and differences in the local job markets when interpreting program impacts.
What was the extent of mothers' exposure to JOBS programs?
Assignment to a labor force attachment or human capital development program under JOBS did significantly increase participation in appropriate activities for mothers in the Child Outcomes Study sample. Thus, mandatory participation requirements (combined with the JOBS programs' messages, case management, and services), affected participation even among mothers with preschool-age children.
Yet the duration of program exposure was generally not long. This is actually in keeping with program goals. If participation in such activities as job search or basic education had occurred for all or even most of the months of the two-year follow-up period, this would indicate that in this time frame, mothers were not progressing off welfare and to employment the actual goals of the programs. In the Atlanta, Grand Rapids, and Riverside sites, in the full evaluation sample, the average duration of participation (among those who participated for at least a day following baseline), ranged from about 3 to 6 months of the two-year follow-up period in the labor force attachment programs, and about 6 to 9 months in the human capital development programs (Hamilton et al., 1997).
In considering the occurrence and magnitude of program impacts of JOBS on children, it is important to keep in mind that exposure to JOBS programs, first, was generally much shorter than the periods of exposure often studied in evaluations of early childhood care and education programs (see Barnett, 1995); and, second, for the most part involved the direct exposure to the programs of mothers, not children. There were no program components like early childhood intervention addressed directly to the children. Thus, any impacts on children should be seen as transmitted through changes in family life brought about by mothers' assignment and exposure to JOBS programs. These would include increases in the use of child care arrangements.
Figure SR-1 reflects this understanding of how JOBS, a set of programs directed to mothers, could have affected children. The conceptual framework that is illustrated notes how:
- assignment to a JOBS program (A), could affect:
- mothers' exposure to, and experiences in, the JOBS program as the program was actually implemented (B);
- and, through program exposure, could affect (C): outcomes directly targeted by the JOBS program (such as maternal educational attainment, employment, and economic resources), and outcomes derivative of these targeted outcomes (such as use of child care and access to health insurance);
- as well as (D): further outcomes not directly targeted by JOBS programs (such as maternal psychological well-being, and parenting);
- and through changes in these aspects of family life and children's child care situations, affect child outcomes (E).
We turn now to a summary of the findings. The findings are presented in three sections. Section V summarizes results for the children in the sample: first, how children in the control groups were doing in the absence of JOBS program influences, and second, whether the JOBS programs altered this course of development. Section VI follows the same sequence regarding families, first asking how control group families were progressing in the absence of program influences, and then whether the JOBS programs had affected family life. Section VII reports findings regarding the linkages between program impacts on families and children, asking which aspects of family life help to explain selected findings regarding program impacts on children. A final section, Section VIII, summarizes the findings and discuss their implications.
Findings for Children
Development and Well-Being of Control Group Children,
and Program Impacts on Children
We first describe findings at the two-year point for children in the control groups of the Child Outcomes Study, as a background for understanding child impact results that follow.
A. Children in the Control Groups
How were children in the control groups of the three study sites faring at the time of the two-year follow-up?
In the absence of JOBS program influences, the children in the sample were at risk of poor developmental outcomes, especially in the area of cognitive development.
For example, as can be seen in Figure SR-2, children in the control groups were significantly less cognitively ready for school than a national sample of children of the same age, as measured by standardized scores on an assessment of cognitive school readiness, the School Readiness Composite of the Bracken Basic Concept Scale (Bracken, 1984).
Children in the control groups also had higher average scores on a measure of behavior problems than age-mates in a national sample (Figure SR-3). On the Behavior Problems Index (Peterson and Zill, 1986; Zill, 1985), they tended to have more frequent behavior problems overall, and more frequent externalizing (aggressive/acting out) behavior problems in particular, than five- to seven-year-olds in the National Longitudinal Survey of Youth-Child Supplement. Differences here are not as marked, however, as the differences in cognitive school readiness, though they are statistically significant.
At the same time, as can be seen in Figure SR-4, mothers rated the overall health of focal children in the control groups as favorably, or more favorably, than mothers of children aged five- to-seventeen in a national health survey.(8) However, as we have noted, the Family Support Act exempted mothers who were needed in the home to care for a sick or incapacitated family member, including a child, from mandated participation in JOBS. Thus, it is not surprising that mothers of children in the control groups of the Child Outcomes Study sample rated their children's overall health in this way.
B. Program Impacts on Children
Were there program impacts on measures of children's development and well-being at the time of the two-year follow-up?
Did mothers' assignment to a JOBS program exacerbate this developmental risk in children? Alternately, did it diminish risk? Was it the case that a program directed primarily to mothers had little or no impact on children? Or were there impacts that varied in direction (favorable and unfavorable) according to the type of child outcome, the research site, program approach, or the characteristics of the families? We summarize impact findings first "overall" or in the "aggregate." That is, within each study site, we ask whether the measures of child well-being and development differed for the human capital development group as a whole as opposed to the control group as a whole, and for the labor force attachment group as a whole as opposed to the control group as a whole. In subsequent sections we turn to the question of whether children in families who entered the evaluation at higher and lower risk showed different patterns of program impacts. Impact analyses control for key background characteristics, including family race/ethnicity and child gender. Detailed reporting on child impacts by race/ethnicity and child gender are planned in the future.
In analyses at the aggregate level there were relatively few statistically significant program impacts on children. Further, those impacts that did occur tended to be small in magnitude. In fact, only one of the impacts reached the criterion of policy relevance. The findings indicate that the welfare-to-work strategies implemented under JOBS did indeed have impacts on selected child outcomes, though it is important to underscore that these aggregate impacts were relatively few and small in magnitude.
What were the program impacts in different aspects of children's development?
Table SR-1 provides a brief summary of the number and direction ("+" indicating favorable, and "-" indicating unfavorable) of statistically significant program impacts for each of the aspects of children's development studied (cognitive development and academic achievement, behavioral and emotional adjustment, and health and safety) and for each of the six programs studied (the human capital development and labor force attachment program approaches in each of the three sites). This table provides an overview, summarizing impact findings across discrete measures for each aspect of development. Some of the specific measures summarized in this table pertain only to the focal child (for example, the assessment of the focal child's cognitive school readiness, the measure of behavior problems, and the rating of overall health described above). However, some of the specific measures summarized in the table pertain to all of the children in the family. For example, mothers were asked whether any of the children in the family had been suspended or expelled from school, and whether any of the children in the family had had an accident, injury, or poisoning requiring emergency medical attention. Thus, the table includes both "focal child" measures and "any child" measures.(9)
| DEVELOPMENTAL DOMAIN: | ATLANTA | GRAND RAPIDS | RIVERSIDE | TOTAL IMPACTS | TOTAL IMPACTS POSSIBLE | ||||
|---|---|---|---|---|---|---|---|---|---|
| HCD | LFA | HCD | LFA | HCD | LFA | FAVORABLE | UNFAVORABLE | ||
| COGNITIVE/ ACADEMIC | 2+ | 3+ | 1+ | 0 | 1+ | 0 | 7+ | 0 | 30 |
| BEHAVIORAL/ EMOTIONAL | 0 | 2+/1- | 0 | 2- | 1+ | 1- | 3+ | 4- | 78 |
| PHYSICAL HEALTH/SAFETY | 0 | 0 | 0 | 0 | 2- | 2- (1) | 0 | 4- (1) | 24 |
| TOTAL NUMBER OF IMPACTS FOUND | 2+ | 5+/1- | 1+ | 2- | 2+/2- | 3- (1) | 10+ | 8- (1) | 132 |
| NOTES: Numbers represent statistically significant impacts (at p < .10); numbers in parentheses represent the number of statistically significant impacts that meet our criterion for policy relevance (i.e., > .33 SD). "+" indicates a favorable aggregate impact; | |||||||||
The table notes the number of possible impacts (that is, the total number of measures of each aspect of development considered, across all six programs), as well as the number of statistically significant impacts actually found. The contrast of actual and potential number of impacts underscores our conclusion that there were relatively few impacts overall, though more than one would expect to occur on the basis of chance. Overall, there were 18 statistically significant treatment-control group differences in measures of child development and behavior, out of 132 impact analyses conducted across all child outcomes and programs. While the number of significant impacts is relatively small, nevertheless, it is a greater number of findings than we would expect by chance alone(10), indicating that JOBS welfare-to-work programs did indeed have impacts on selected child outcomes.
As can be seen, child impact findings at the aggregate level differed in direction according to aspect ("domain") of development. All of the impacts in the domain of children's cognitive development and academic achievement were favorable. That is, when there were statistically significant differences, it was always the case that children whose mothers had been assigned to a JOBS program scored better on measures of this aspect of development than children whose mothers had been assigned to a control group. Moreover, at least one favorable program impact on a measure of cognitive development and academic achievement occurred in four of the six programs examined here.
By contrast, all of the statistically significant impacts in the area of child health and safety were unfavorable. That is, when there were statistically significant impacts, they always occurred in a direction indicating that children of mothers assigned to a JOBS program had less favorable scores than children of mothers assigned to a control group.
All of the unfavorable health impacts occurred in the Riverside site, and unfavorable health impacts occurred in this site in both programs (labor force attachment and human capital development). Mean scores indicated that program group mothers in the Riverside site rated the focal children as less healthy overall. In addition, a smaller proportion of mothers in the program groups than in the control groups rated the focal children as in excellent or very good health (as opposed to good, fair or poor) on this overall health rating. The unfavorable impact of Riverside's labor force attachment program on the proportion of focal children rated as in very good or excellent health was large enough to be considered "policy relevant."
These unfavorable impacts indicate that a larger proportion of program than control group mothers in the two Riverside programs rated focal children in the range of good, fair, or poor health (rather than very good or excellent). We examined the origin of this impact more closely, asking especially whether the higher proportion of experimental group mothers rating their children in this range was largely attributable to the "good" rating, and thus did not indicate a shift toward fair or poor health. However, the findings in fact reflect a higher proportion of mothers rating their children as in poor or fair health in the two programs.
No statistically significant differences were found at the aggregate level in any of the research sites on the proportion of mothers reporting that an accident or injury had occurred to any of their children that was severe enough to require emergency medical attention. Thus, differences in the area of health and safety occurred in terms of mothers' ratings of focal children's overall health, but not on outcomes related to emergencies for any of the children in the family.
The overall health rating reflects the mothers' perceptions of the focal child's health. We do not have doctors' assessments of the child's overall health or the incidence and/or severity of concrete illnesses, such as ear infections. Maternal perceptions may, in part, reflect the salience of this issue. For example, employed mothers may be concerned about the possibility of missing work because of a child's illness, and, thus, any indications of the child's compromised health (e.g., the common cold) may lead employed mothers more so than non-employed mothers to rate the focal child's overall health less favorably. At the same time, lower health ratings could, in fact, reflect poorer child health, which could result, for example, from exposure in child care settings to other children with health symptoms, from diminished attention to such family routines as bed time, or from mothers' having less time to bring children to health care providers for diagnosis and treatment. In either case, it is possible that lower health ratings by program mothers are, to some extent, derivative of maternal employment. (See Section VII below for results suggesting the degree to which maternal employment and/or child care did, in fact, help to explain unfavorable impacts on this rating of focal children's health.)
Impacts on measures of the children's behavioral development and emotional adjustment, in contrast with measures of cognitive development and health, included both favorable and unfavorable impacts. Unfavorable behavioral impacts occurred in each site's labor force attachment program, whereas there does not appear to be a pattern (by site and/or program approach) to the occurrence of favorable behavioral impacts.
Should we have expected child impact findings to go in the same direction across all aspects of development?
Is it problematic that findings went in different directions for different aspects of development, with impacts on cognitive development and academic achievement favorable; health and safety unfavorable; and behavioral and emotional adjustment including both favorable and unfavorable impacts? In fact, there are precedents in other bodies of research on children's development for such complex patterns. Child care research provides an example. Findings in this literature show that children who have participated in formal child care settings tend to have better scores on measures of cognitive development (e.g., Zaslow, Oldham, Magenheim and Moore, 1998), mixed outcomes on measures of behavioral development (e.g., Barnett, 1995; Yoshikawa, 1995), and some unfavorable outcomes in terms of health (such as more ear and intestinal infections, e.g., Hirsh-Pasek, 1998). In giving this example, our intent is not to suggest that impacts of JOBS programs on children are transmitted primarily through child care influences. Rather, our intent is to point out that different aspects of development can be affected in differing directions by a developmental context (like child care, or mothers' assignment to a JOBS program).
What patterns were observed with respect to the three sites and six programs?
Child impact findings at the aggregate level tended to differ according to site, as shown in Table SR-1.
Cognitive development: Favorable impacts were found in Atlanta's labor force attachment program and in all three human capital development programs. Favorable program impacts on children's cognitive development and academic achievement occurred especially in Atlanta's labor force attachment program. In this program, children had significantly higher mean cognitive school readiness scores, as well as a more favorable distribution of scores on the assessment of cognitive school readiness (i.e., the proportion of children scoring at the high end of the distribution was greater, and the proportion of children scoring at the low end was smaller).
In addition, at least one favorable impact in the domain of cognitive development occurred in each site's human capital development program. Specifically, in Atlanta and Grand Rapids, children in the human capital development program had scores indicating a more favorable distribution on the Bracken assessment (i.e., the proportion of children scoring at the high end of the distribution was greater). In the Atlanta and Riverside sites, a smaller proportion of mothers in the human capital development program than in the control group reported that any of the children in the family had had an academic problem since the start of the evaluation.
It is possible that programs that improve mothers' educational outcomes provide the mothers themselves with a source of cognitive stimulation. This in turn may influence how much and how mothers interact with their children, with implications for the children's cognitive development. Children may also be influenced by observing their mothers pursuing educational goals. Mothers who themselves participate in educational activities may place more emphasis on school work, and/or may feel more effective in assisting their children with homework.
The concentration of favorable cognitive impacts on children in each site's human capital development program and in Atlanta's labor force attachment program is consistent with this hypothesis. These four programs increased mothers' educational attainment (e.g., the receipt of a high school diploma or GED, the receipt of a trade degree).
Health: Unfavorable impacts were found only in Riverside. All of the unfavorable child health impacts at the aggregate level occurred in the Riverside site. Unfavorable impacts occurred in both programs in this site, and all findings pertained to the focal child's health, as rated by the mother. One of these impacts was large enough to be considered policy relevant. No unfavorable health impacts were statistically significant at the aggregate level for either of the programs in the other two study sites. As noted above, no program impacts were found at the aggregate level on the more extreme measure of accidents, injuries or poisonings requiring a visit to an emergency room or clinic. Further, as noted above, the maternal report measure of overall health likely reflects not only the actual occurrence of particular health symptoms in the focal child, but also the salience of the focal child's general health status (which may vary as a function of maternal employment status and history).
Behavior: Unfavorable impacts were found only in labor force attachment programs, though favorable impacts were also found in Atlanta's labor force attachment program. Each of the labor force attachment programs studied had at least one unfavorable program impact in the area of behavior and emotional adjustment as reported by the mother. However, the picture was mixed for the labor force attachment program in Atlanta. For this program there were also two favorable impacts on measures of behavior and emotional adjustment. (A single favorable behavioral impact also occurred in Riverside's human capital development program.)
Pattern of findings in Atlanta and Riverside. On balance, impacts in Atlanta were favorable for each program approach. Of eight statistically significant child impact findings in Atlanta, seven were favorable (with the single unfavorable impact in this site occurring on a behavioral outcome for children whose mothers were in the labor force attachment program).
By contrast, on balance, in the Riverside site, significant child impact findings were unfavorable. Of seven statistically significant child impacts in the Riverside site, five were unfavorable. The unfavorable findings in Riverside were concentrated in the health area, but one also occurred on a behavioral outcome for children whose mothers were in the labor force attachment program.
We started out with an interest in exploring possible favorable as well as unfavorable program impacts on children. What can we conclude about favorable impacts?
In the context of previous research on welfare-to-work programs, it is important to note that the aggregate impact findings in the present study include evidence of favorable impacts on children, particularly in the cognitive domain of development, and particularly in Atlanta.
Findings at the aggregate level indicate slightly more favorable than unfavorable significant impacts on child outcome measures. All of these favorable impacts may be considered small in magnitude, with none reaching the threshold for policy relevance. They are concentrated in the domain of cognitive development. Each site's human capital development program had at least one favorable impact in the cognitive domain, but the greatest concentration of favorable cognitive impacts occurred in Atlanta's labor force attachment program.
To date, only three evaluations of welfare-to-work programs with rigorous experimental designs have had an explicit focus on child impacts: the present study, the Teenage Parent Demonstration (Kisker, Rangarajan, and Boller, 1998), and the New Chance Evaluation (Quint, Bos and Polit, 1997). The Child Outcomes Study is the first noting a pattern of favorable child impacts at the aggregate level for cognitive outcomes, and a greater number of favorable than unfavorable impacts, on balance, at the aggregate level.
Favorable impacts on children, and especially boys, were recently reported in a program evaluation of a different kind, however: the New Hope Evaluation (Bos, Huston, Granger, Duncan, Brock, and McLoyd, 1999). New Hope is a demonstration program in which low-income adults, already working or willing to work 30 or more hours a week, are assured of an income above the poverty level, along with benefits such as health insurance and child care subsidies. Participants in this program, in contrast with those in the three welfare-to-work demonstrations described above, were not all receiving welfare, were not all women, and they did not all have children. Also, unlike current and recent welfare-to-work programs, participation in New Hope was voluntary. Children, and especially boys, were more likely to have been in organized after-school activities and were reported by their teachers to be doing better academically and to be showing more positive social behavior. Parents in the program group reported more positive social behavior in their sons, and the boys themselves had higher aspirations and expectations in terms of future occupations and advanced education.
Perhaps New Hope and the JOBS programs examined here, to a greater extent than New Chance or the Teenage Parent Demonstration, affected family economic activities and resources, child care use, or the stimulation and support available in the home environment. Such impacts on families might help to explain the presence of favorable child impact results. We will return to these possibilities for the Child Outcomes Study in examining the evidence on the processes through which the child impacts arose (Section VII).
How were children from higher- and lower-risk families affected?
As noted above, the Child Outcomes Study was initiated in order to examine four diverging possibilities: that there would be favorable program impacts on children, unfavorable program impacts on children, no net impacts overall, or impacts especially for particular subgroups. We turn now to the fourth possibility, examining child impacts occurring for families with particular characteristics.
Previous research examines whether welfare-to-work programs and other kinds of interventions have different effects on children according to whether or not families have particular risk factors and also according to the total number of risk factors they have. For example, the New Chance Demonstration (Quint et al., 1997) found unfavorable impacts on measures of child behavior problems as perceived by the mother. Unfavorable impacts occurred especially for families in which the mother had entered the evaluation at high risk of depression, in which the mother had been out of school two years or more, and in families with more risk factors overall. The Infant Health and Development Program, which aimed at improving the cognitive development of children born at low-weight, provided center-based early intervention, home visits, and parent support services. This program had positive effects for children from poor families when these families had no or few risk factors, but not when poor families had multiple risk factors (Brooks-Gunn, 1997). Thus there is evidence that children in low income families respond differently to interventions according to the presence of particular risk factors, and the total number of risk factors, in their families.
Following in this tradition, in the present study we looked at child impacts in light of the presence of particular risk factors in families and also the total number of risk factors in families. We defined risk in terms of characteristics found in previous research on welfare families to be associated both with greater difficulty for the mother in making a transition to employment and also with a less positive course of development in children, apart from any intervention.
Research on factors that impede employment among families receiving welfare ("barriers to employment"), point to the importance of demographic characteristics of the family (such as having more children in the family), human capital factors (such as limited educational attainment and low basic skills; lack of work experience and history of longer welfare receipt), and psychosocial factors (such as maternal depressive symptoms) (evidence reviewed in Zaslow, Hair, Dion, Ahluwalia, and Sargent, 1999). These same factors have been found to be associated with the well-being and development of young children in families receiving welfare (Moore et al., 1995). Thus, there is evidence that family configuration, educational background and skills, work and welfare history, and maternal psychological well-being, are associated both with employment and with children's development in samples of families receiving welfare, apart from welfare-to-work programs.
It is not clear from the existing research, however, which type or types of risk may be important to the pattern of program impacts for children in light of mothers' assignment to a JOBS welfare-to-work program. Accordingly, we examined program impacts on children in light of a number of different types of risk. Each family within the study was categorized as at higher or lower risk in four different ways:
- sibling constellation risk (with higher risk involving more children and/or more closely spaced children);
- educational risk (with higher risk involving limited maternal educational attainment or low maternal math or reading literacy scores);
- work risk (with higher risk involving limited maternal employment history, multiple barriers to employment, or a history of welfare receipt of five or more years);
- maternal psychological well-being risk (with higher risk involving mothers with more indications of psychological distress);
Families could be categorized as at higher risk on more than one category. For example, a family might fall into the higher-risk category for all four types of risk, or might be categorized as at higher risk in terms of sibling constellation and maternal psychological well-being risk (but not in terms of educational and work risk). By studying each of these types of risk, we could ask whether program impacts on children were especially linked with risk on one (or more) dimension.
In keeping with the previous research, we also created a summary variable looking at the number of these risk factors for which each family was in the higher-risk category:
- cumulative risk, (with higher risk involving a larger cumulative number of the preceding four categories for which the family fell into the higher-risk category).
This made it possible to ask whether program impacts on children differed according to whether the family had a greater or lesser total number of risk factors.
Finally, we also explored a measure of risk that we thought might be informative, but for which there was little precedent in the research on families receiving welfare. The research on maternal employment documents differences in outcomes for children according to whether the mother's work role is concordant or discrepant with her preferred role (Zaslow, Rabinovich, and Suwalsky, 1991). Extending this to the present study, we asked whether impacts of JOBS on children would differ according to whether, at baseline, mothers felt more or less positively about working. On a more exploratory basis, then, we examined reservations about working (with higher risk involving mothers who tended to endorse statements indicating a preference for taking care of family over finding employment).
The question that the subgroup impact analyses address is that of whether risk factors that are associated with less progress toward employment in mothers, and less positive development in children (in the absence of any programs or interventions), are also associated with different patterns of impacts for children in the context of JOBS welfare-to-work programs. Given the priority in the Child Outcomes Study of providing a rigorous examination of the possibility of unfavorable impacts for children, careful scrutiny must be given to the possibility that unfavorable effects occurred for children in certain subgroups. An important possibility is that unfavorable impacts might occur especially for families at higher risk, both in terms of specific risk factors and/or on the overall number of risk factors.
While this prediction must be examined carefully, it is not the only pattern that can be anticipated. Indeed, JOBS programs were intended to address the difficulties faced by families in making a transition to employment, through a combination of supports and requirements. The possibility exists that especially for families with risk factors, one or both JOBS programs might be especially helpful, with children benefitting accordingly.
The findings for subgroups did not indicate any one approach to defining risk as more important than the others. Indeed, the pattern of findings generally held across each of the different approaches to defining risk.(11) Accordingly, we discuss below the pattern of findings for higher-risk families overall, and for lower-risk families overall.
As for the aggregate impact findings, the impact findings for subgroups were not pervasive; that is, there were relatively few statistically significant program impacts for particular higher- or lower-risk subgroups. Yet when looking separately at findings for each particular type of risk (sibling constellation risk, educational risk, work risk, maternal psychological well-being risk, cumulative risk, or reservations about working), the number of statistically significant differences exceeded what would be expected to occur simply on the basis of chance.(12) We begin with the question of whether there was a pattern of unfavorable impacts for children in higher-risk subgroups. However, we also explore the possibility of a pattern of favorable impacts for children in higher-risk subgroups, and we summarize findings for lower risk subgroups as well.
Higher-risk subgroups. The findings for higher-risk subgroups do not support the simple and straightforward hypothesis that there would be a concentration of unfavorable impacts for children in higher-risk families. Rather, the findings are in accord with a more complex pattern. We underscore that as for the aggregate impacts, statistically significant findings were relatively few.
Findings for children in higher-risk families were generally favorable when the family had been assigned to a human capital development group or to Atlanta's labor force attachment program. The only exceptions relate to unfavorable health impacts of Riverside's human capital development program (though all other findings for higher-risk families in this program were favorable), a single unfavorable health impact of Atlanta's labor force attachment program, and a single unfavorable behavioral impact of each of Atlanta's JOBS programs.By contrast, impacts for children in higher-risk families assigned to either Grand Rapids' or Riverside's labor force attachment program tended to be unfavorable.
Impacts for children in higher-risk subgroups tended to be small in magnitude. Yet several of the findings, all of these unfavorable and occurring in labor force attachment programs, were of sufficient magnitude to be called policy relevant.
That largely favorable impacts were found for children from higher-risk families when mothers had been assigned to human capital development programs or to Atlanta's labor force attachment program may be of particular interest to policy makers. In the new policy context, as caseloads drop, there are concerns that perhaps families remaining on welfare, like the higher-risk families studied here, face barriers to making the transition from welfare-to-work. Programs that have favorable impacts on mothers' education or training may help address barriers, and in doing so, also benefit children.
Lower-risk subgroups. For children in lower-risk subgroups, there was a concentration of unfavorable impacts in three programs: both of Riverside's programs, and the labor force attachment program in Grand Rapids. Across these three programs, a number of the statistically significant impacts were large enough to meet the criterion for policy relevance. In all but one instance, these policy relevant impacts were unfavorable. Further, many (one-third) of the unfavorable and policy relevant impacts found for children in lower-risk families substantially exceeded the threshold for policy relevance, in that effect sizes were .50 or larger (which is considered "moderate" to "large" in magnitude; Cohen, 1988).
On balance, the significant impacts for lower-risk families in Atlanta's labor force attachment program were favorable. None of these impacts was of a magnitude to be called policy relevant. Lower-risk families in human capital development programs outside of Riverside showed either no impacts or only isolated impacts, mixed as to direction.
In all three programs in which there were unfavorable and policy relevant impacts for lower-risk families, such impacts occurred both in the domain of child health and also in the domain of child behavior. In these analyses, there is also evidence of unfavorable health impacts of sufficient magnitude to be called policy relevant occurring beyond Riverside. In general, the subgroup impact analyses, in contrast with the aggregate impact analyses, provide evidence of more widespread health impacts. Significant, although not always policy relevant, impacts in an unfavorable direction went beyond Riverside in these analyses. Indeed they occurred for lower-risk subgroups in all programs and sites with the exception of Grand Rapids' human capital development program. All health impacts for children in lower-risk families were unfavorable.
In the Riverside site, children in lower-risk families assigned to the labor force attachment program also showed unfavorable (and in some instances policy relevant) program impacts in the cognitive domain. This was the only set of unfavorable impacts in the cognitive domain across all of the subgroup impact (and aggregate impact) analyses.
In sum, the findings for lower-risk subgroups are not pervasive (that is, the proportion of tested impacts to reach statistical significance was not high) nor do they appear specifically in one program approach or another (human capital development or labor force attachment). Rather, the larger unfavorable impacts are concentrated in three particular programs (Grand Rapids' labor force attachment program, and both of Riverside's programs).
What are the implications of the child impact findings for the concern about possible unfavorable impacts of JOBS welfare-to-work programs on children?
On the one hand, the dearth of policy relevant impacts at the aggregate level diminishes concern about unfavorable effects for children of mothers' assignment to a JOBS welfare-to-work program. Yet the concentration of unfavorable and policy relevant impacts, occurring especially among lower-risk subgroups in three programs suggests a need for continued vigilance.
It was not the case that the subgroup impact findings were all large in magnitude. Yet some subgroup impacts were moderate to large in magnitude (i.e.,were .50 or larger). Moreover, all of these moderate to large impacts were unfavorable, and all occurred for lower-risk subgroups. The fact that the unfavorable policy relevant impacts occurred in a concentrated manner (for lower-risk families in three specific programs), and that some of these were moderate to large in magnitude, suffices to indicate that we cannot entirely dismiss the possibility of unfavorable effects on children.
We need to continue to follow the development of children in particular lower-risk subgroups over time. In analyses of the five-year follow-up data in the future, it will be critical for understanding the impacts on children of these programs to ask whether the impacts for these subgroups are sustained, grow in magnitude, diminish, or disappear and to examine the context of such findings. For example, if unfavorable impacts are sustained, it will be important to explain whether the impacts are occurring in the context of ongoing economic struggle for families trying to sustain employment at low wages. If, on the other hand, the unfavorable impacts for these subgroups disappear over time, it will be important to ask if an initial period of economic struggle gave way to one of increased and stable economic well-being.
The findings suggest the need for continuing vigilance regarding the possibility of unfavorable impacts on children, though especially in particular programs and at particular risk levels. At the same time, the occurrence of favorable program impacts, albeit rarely large enough to call policy relevant, is encouraging and important to monitor over time. Favorable impacts were present especially for cognitive outcomes, and in Atlanta's programs. In addition, there was a tendency for favorable impacts to occur for children from higher-risk families whose mothers had been assigned to human capital development programs or to the Atlanta labor force attachment program.
Findings: Outcomes for Control Group Families and Program Impacts on Family Outcomes
Outcomes for Control Group Families and Program Impacts on Family Outcomes
What was the family context for these child impact findings? We turn now to an examination of outcomes for families. As we did for the child outcomes, we first consider how families were faring in the control groups, in the absence of JOBS programs, and then turn to an examination of program impacts.
A. Control Group Families
How were families faring economically in the absence of a JOBS program?
Analyses focusing on control group families indicate that, in the absence of welfare-to-work strategies under JOBS, sample families would have made some educational and economic progress over the two-year follow-up period, though many would still be considered disadvantaged.
Following random assignment, between 8 and 19 percent of control group mothers in the three sites received some sort of educational credential. Economically, whereas all sample families had applied for or were receiving AFDC at baseline, at the time of the two-year follow-up study, only 53 to 79 percent (with the percent varying by site) of control group mothers were receiving welfare. More mothers in the control groups (between 29 and 47 percent) were employed in the month preceding the survey than two years earlier. Employed mothers in the control groups had average earnings of approximately $6.10 an hour, but many (between 11 and 25 percent) were working at or below minimum wage.
Considering all sources of income(13), an estimated 69 (in Grand Rapids) to 77 percent (in Atlanta) of households in the control groups were living below the poverty line(14) at the two-year follow-up. Not surprisingly (given the population studied in the evaluation), these rates exceed the 17.4 percent of families with related children in the U.S. as a whole who were living in poverty in 1994. In addition, 11 percent in Riverside, 19 percent in Atlanta, and 21 percent in Grand Rapids, reported total net adjusted household incomes reflective of "deep poverty" (i.e., less than 50 percent of the poverty line). This compares to 7.2 percent of families in deep poverty nationwide (see Figure SR-5).
In the absence of a JOBS program, how were families faring in terms of non-economic aspects of family life?
There were also indications of a range of risk on the non-economic aspects of family life among control group mothers.
Depressive symptomatology. More than a quarter of the mothers in each site (from 29 percent in Riverside to 37 percent in Atlanta and Grand Rapids) reported "many" depressive symptoms, suggestive of clinical levels of depression. The rates in Atlanta and Grand Rapids are comparable to those reported by mothers in Atlanta in the months immediately following random assignment, when 42 percent of these mothers reported high levels of depressive symptoms (Moore et al., 1995). However, rates of "many" depressive symptoms are lower in the Child Outcomes Study sample compared to the more disadvantaged mothers enrolled in the New Chance Demonstration (all of whom were teenage mothers who had dropped out of school at the start of that evaluation); 52 percent of these mothers had high levels of depressive symptoms (Quint et al., 1997). Nonetheless, rates of depressive symptomatology suggestive of clinical depression among control group mothers in the Child Outcomes Study are higher than those typically found in community samples, which are closer to 20 percent for women (see review in Devins and Orme, 1985) (Figure SR-6).
Parenting. Interviewers rated a nontrivial minority of control group mothers (between 8 and 19 percent) as behaving in a harsh manner toward the focal child during the interview. Interviewers also rated control group mothers as moderately warm, on average, in interacting with their children, but there was a wide range in ratings of warmth, and ratings varied by site. On average, control group mothers did not rate themselves as particularly aggravated in the parenting role. However, about 10 percent of mothers in each site had scores indicating substantial aggravation and stress in the parenting role.
Father involvement as reported by mothers. We first examined the proportion of focal children living with their biological fathers. For those not living with their fathers, we note the proportion who had seen their fathers in the past 12 months. We also note the proportion of nonresidential fathers who had purchased clothing, toys or presents; had provided groceries; and who had served as a babysitter for the child. Few control group mothers (between 4 and 12 percent) were living with the focal child's biological father at the time of the follow-up survey, though over half of focal children who were not living with their biological fathers had seen them in the previous 12 months. In terms of other forms of support (buying clothes, toys, presents; providing groceries; occasional babysitting), focal children's biological fathers provided, on average, less than one of the three kinds of support.
Marital status and fertility. Whereas only 2 percent or fewer of control group mothers had been married at baseline, between 4 and 15 percent of control group mothers were married at the two-year follow-up. In addition, between 12 percent and 19 percent reported having had a baby between baseline and the two-year follow-up.
B. Program Impacts on Families
Program impact findings for families in the full sample of the National Evaluation of Welfare-to-Work Strategies have been presented elsewhere (Hamilton et al., 1997; Freedman et al., 2000). Here we summarize findings specifically for the Child Outcomes Study sample. These findings help ascertain whether the JOBS programs brought about changes in economic and non-economic aspects of family life specifically in families that entered the evaluation with preschool-age children. It is important to determine if the presence of such young children impeded program participation or prevented families from making economic progress. It is also important to ascertain whether being assigned to a JOBS program when at least one child in the family was of preschool age resulted in changes in parenting behavior or the home environment, or in maternal psychological well-being.
Did assignment to a JOBS program result in increased participation in program-related activities for mothers with young children?
Regardless of program approach, the six JOBS programs studied resulted in greater participation in job search activities. The largest impacts on participation in job search activities were concentrated in labor force attachment programs. Mothers in each site's labor force attachment program were about ten times as likely as those in the control groups to have ever participated (for at least one day) in job search activities since random assignment. Large impacts also occurred for mothers assigned to Riverside's human capital development program: 29 percent of these program mothers had participated in job search activities, compared to less than 5 percent of mothers in Riverside's (human capital development) control group.
Participation in basic education was higher for mothers assigned to a human capital development program than for control group mothers. The largest impact on participation in basic education activities (regardless of whether a degree was ultimately obtained) occurred for mothers assigned to Riverside's human capital development program: over half of these program mothers had participated in basic education activities since random assignment, whereas only 16 percent of control group mothers had done so.
Corresponding to the participation mandate faced by program, but not control group, mothers, all six programs in the Child Outcomes Study resulted in sanctioning of program families who failed to comply with the participation mandate. Sanctioning rates were highest for mothers in the two Grand Rapids programs (27 percent for those in the human capital development program and 38 percent in the labor force attachment program). Mothers in Atlanta's human capital development program were the least likely of the three human capital development programs to be sanctioned (15 percent as opposed to 27 percent in Grand Rapids and 24 percent in Riverside). Similarly, sanctioning rates were lower for mothers in Atlanta's labor force attachment than in the other two labor force attachment programs (11 percent in Atlanta; 38 percent in Grand Rapids, and 14 percent in Riverside).
Did assignment to a JOBS program result in impacts on targeted outcomes and/or on outcomes derivative of targeted outcomes for families with young children?
Educational attainment. Mothers in the human capital development programs, but generally not those in the labor force attachment programs, progressed in terms of educational attainment. This is in keeping with program goals: labor force attachment programs generally did not aim to increase educational attainment, while human capital development programs did.
Overall, mothers in human capital development programs in each site were more likely than controls to have received any educational credential since random assignment. By contrast, each site's labor force attachment program tended not to improve mothers' educational attainment (except in Atlanta, where mothers in the labor force attachment program were more likely than controls to have obtained a trade degree). Indeed, mothers assigned to Grand Rapids' labor force attachment program were less likely than controls to have obtained a high school diploma or GED, trade certificate, or any degree since random assignment.
Economic circumstances. Two years after random assignment, improvements in families' economic circumstances occurred mainly for mothers assigned to labor force attachment programs and for mothers assigned to Riverside's human capital development program. Impacts were largest and most numerous for the two programs in Riverside.
Program mothers in Atlanta witnessed relatively few economic impacts. From among multiple economic outcome measures, impacts were found on only a few in this site. As intended, both programs led to a reduction in AFDC receipt at the two-year follow-up. In addition, those in Atlanta's human capital development program were less likely to report earning the minimum wage or less and had somewhat greater earnings from their current jobs, and those in Atlanta's labor force attachment program were more likely to be employed, and less likely to be in deep poverty in the month prior to the two-year follow-up. Neither of Atlanta's JOBS programs had a statistically significant impact on total net adjusted household income at the two-year point.
The only statistically significant unfavorable economic impacts of any of the six JOBS programs in this study occurred for mothers in Grand Rapids' JOBS programs. Both JOBS programs in Grand Rapids' reduced the proportion of families at or above the poverty line. Mothers assigned to Grand Rapids' human capital development program were less likely to be employed 40 or more hours per week, and results for this program show an increase in the proportion of mothers working for less than the minimum wage. The only favorable economic impact of Grand Rapids's labor force attachment program was on the likelihood that program mothers were employed at some point since random assignment. It is worth noting that those in Grand Rapids' labor force attachment program had greater employment rates in the month prior to the two-year follow-up than those in any of the other sites' JOBS programs, but because control group mothers in Grand Rapids also had relatively high rates of employment (compared to control group mothers in the other sites), there was no net impact of Grand Rapids' labor force attachment program on current employment. Neither of Grand Rapids' JOBS programs had a statistically significant impact on total net adjusted household income at the two-year point.
Economic impacts in Riverside were more widespread. Mothers assigned to either of Riverside's JOBS programs were more likely than controls to have been employed at some point since random assignment, were more likely to have been employed in the month prior to the two-year follow-up, had more hours of employment, higher wages, and greater earnings. (Riverside's JOBS programs were the only ones in this study to have favorable impacts on hourly wage and hours employed.) In addition, mothers enrolled in Riverside's labor force attachment program were less likely to be receiving AFDC, and mothers in Riverside's human capital development program were more likely than their control group counterparts to have incomes above the poverty line. These impacts also occurred for many lower-risk and higher-risk subgroups. In addition, despite the absence of impacts on total income at the aggregate level, Riverside's human capital development program resulted in higher total household income for one lower-risk and two higher-risk subgroups of mothers. However, for another lower-risk subgroup, this program resulted in lower earnings. Also, despite the absence of impacts on deep poverty at the aggregate level, Riverside's labor force attachment program increased deep poverty for some higher-risk families. Nevertheless, employment and economic impacts of Riverside's JOBS programs were generally favorable, which is consistent with this site's focus and experience implementing work-oriented programs.
Employment-related child care. Program mothers (except those in Grand Rapids' human capital development program) were more likely than controls to be using child care for any of their children while they were employed at their current or most recent job. The largest impact on the use of employment-related child care for any of their children occurred for Riverside's labor force attachment program, where program mothers experienced the largest employment impacts (and where control group mothers reported relatively infrequent employment-related child care). Relatively large impacts also occurred in Riverside's human capital development program and in Grand Rapids' labor force attachment program, where program mothers were more likely than controls to have been employed at some point since random assignment. The small impacts in Atlanta likely reflect, in part, the absence of large employment impacts in this site.
Impacts on the current use of employment-related child care for the focal child were more limited and occurred only in Riverside: Only Riverside's JOBS programs increased the overall likelihood that mothers were both employed and using any non-maternal care for the focal child in the month prior to the two-year survey. Regardless of program approach, program mothers in this site reported more employment-related hours of child care for the focal child in the prior month, were more likely than controls to be employed and using informal care for the focal child, and to be employed and using care for the focal childduring irregular hours or with a varying schedule. These findings are consistent with the focus in Riverside likely experienced more intensively by program than control group mothers on helping mothers arrange low cost, primarily informal, child care, which provides greater flexibility as to hours of employment.
Did assignment to a JOBS program affect further, non-targeted outcomes for families?
There was less evidence of impacts on aspects of family life not targeted by JOBS welfare-to-work programs, yet such impacts did occur. When these impacts occurred, however, there were indications (discussed below) that they could be quite important to child outcomes.
Maternal psychological well-being. Specific aspects of maternal psychological well-being were affected in different ways by JOBS welfare-to-work programs, and impacts did emerge, especially for subgroups of mothers.
Assignment to a JOBS welfare-to-work program often led to increased feelings of time stress. Perhaps more worrisome, enrollment in a JOBS program increased depressive symptomatology, in the aggregate, for mothers assigned to Grand Rapids' labor force attachment program, as well as for a few lower-risk subgroups exposed to Grand Rapids' or Riverside's human capital development program, and a lower-risk subgroup assigned to Atlanta's labor force attachment program. Interestingly, Atlanta's labor force attachment programincreased the number of depressive symptoms among mothers reporting relatively many psychological problems at baseline, and decreased the number of depressive symptoms among mothers reporting no or relatively few psychological problems at baseline. This pair of offsetting impacts indicate that this JOBS program had completely opposite impacts (which resulted in an absence of impacts in the aggregate) depending on the initial level of mothers' depressive symptoms.
At the same time, findings indicate that, despite an absence of impacts at the aggregate level, four of the six JOBS programs led to greater feelings of control over one's life for certain (especially lower-risk) subgroups of mothers. This was especially true for Riverside's labor force attachment program, in which a majority of lower-risk subgroups examined had increases on the measure of subjective sense of control.
Parenting. In the aggregate, Atlanta's human capital development program led to greater warmth (as reported by the mother) and to higher scores on a summary measure of "favorable" parenting. Additional impacts on parenting emerged for a few subgroups, with Atlanta's human capital development program having both favorable and unfavorable impacts in both lower-risk and higher-risk subgroups. In the aggregate, Atlanta's labor force attachment program led to less harsh discipline, greater maternal warmth (as reported by the interviewer), greater verbal interactions with the focal child, and higher scores on a summary measure of "favorable" parenting. The additional impacts that emerged in a few subgroups were uniformly favorable in lower-risk subgroups and were both favorable and unfavorable in higher-risk subgroups.
There was only one aggregate impact on parenting in Grand Rapids: Mothers assigned to Grand Rapids' labor force attachment program reported less warmth toward the focal child than did control group mothers. Additional parenting impacts emerged in subgroups, and most were unfavorable regardless of risk level.
In the aggregate, there were no impacts of either of Riverside's JOBS programs on any measure of parenting. Nevertheless, a few impacts emerged at the subgroup level, with both favorable and unfavorable impacts of each program in lower-risk subgroups, but only favorable impacts of each program in higher-risk subgroups.
Father involvement as reported by mothers. Mothers in each of Grand Rapids JOBS programs reported receiving more forms of support from the focal child's biological father, such as purchasing groceries, toys, clothes, and babysitting, compared to control group mothers. This difference, while statistically significant, was small. Further, such support was reported to be occurring at low levels: program mothers in Grand Rapids still reported less than one kind of support, on average. Riverside's labor force attachment program decreased the likelihood that the focal child's biological father lived with the focal child at the two-year follow-up. Additional impacts emerged when subgroups were considered; notably, each of Atlanta's JOBS programs decreased the likelihood that the focal child's father lived with him or her in four higher-risk subgroups. In Grand Rapids, all impacts on father involvement in subgroups were favorable. By contrast, in Riverside, all impacts on father involvement in subgroups were unfavorable.
Marriage and fertility. In the aggregate, none of the six welfare-to-work programs had an impact on mothers' fertility, or their marital and cohabitation status, by the two-year follow-up, though a few scattered impacts emerged for five of the six JOBS programs in both lower- and higher-risk subgroups.
Findings: Linking Impacts on Children to Impacts on Families
Linking Impacts on Children to Impacts on Families
Through what pathways do children appear to have been affected by their mothers' assignment to a JOBS welfare-to-work program? In this section, we summarize results from statistical analyses that identify which program impacts on targeted and non-targeted outcomes appear to underlie selected impacts on children. The analyses of program impacts on children summarized above are experimental in nature. Experimental analyses provide strong causal evidence regarding the existence of program impacts on children; however, they cannot address the question of how these impacts came about. Analyses seeking to identify the pathways through which a program had its impacts are necessarily non-experimental. Because mediational analyses are non-experimental, they do not allow firm causal inferences to be made regarding the pathways through which these impacts came about.(15) In addition, the specific mediational analyses conducted in this study rely on information about mediators and child outcomes measured contemporaneously and used a relatively modest statistical approach in modeling pathways; as such, results should be considered preliminary.
To what extent were program impacts on measures of focal children's cognitive school readiness, externalizing behavior problems, and general health explained by program impacts on mothers (i.e., on targeted and non-targeted outcomes)?
In a modest attempt to understand the pathways through which specific welfare-to-work programs may affect children's developmental outcomes, five of the aggregate impacts on focal children in the Child Outcomes Study sample were examined in more detail through mediational analyses. These five impacts were selected because they illustrate the general pattern of findings at the aggregate level of favorable cognitive, unfavorable health, and mixed behavioral impacts. In particular, we examine:
A favorable impact on a measure of cognitive development:
- the favorable impact of Atlanta's labor force attachment program on focal children's mean cognitive school readiness scores;
Two unfavorable impacts on measures of children's health:
- the unfavorable impact of Riverside's human capital development program on ratings of the focal child's overall health;
- the unfavorable impact of Riverside's labor force attachment program on ratings of focal children's overall health;
A favorable and an unfavorable impact on behavioral and emotional adjustment:
- the favorable impact of Atlanta's labor force attachment program on focal children's reported externalizing behavior problems; and
- the unfavorable impact of Grand Rapids' labor force attachment program on focal children's reported externalizing behavior problems.
Mediational analyses are statistical analyses that attempt to identify one or more variables that appear to explain, statistically, the relationship between two other variables (see Baron and Kenny, 1986). In the present study, targeted and non-targeted outcomes that were affected by a given JOBS program (i.e., there was an experimental impact on the outcome) were examined as possible "mediators" of the program impact on children. The mediational analyses ask whether the statistical significance of the association between the JOBS program and a child outcome (the significant impact) is diminished or eliminated when the mediating variable, or set of mediating variables, is taken into account. When this happens, there is evidence that the mediator is a conduit through which the child impact is coming about. In these analyses, we find that some of the child impacts are "fully mediated" (the statistical significance of the impact on a child outcome is eliminated when considering the role of the mediators), while in other instances there was only "partial mediation" (the statistical significance of the impact on the child outcome is diminished but not eliminated when considering the mediators).
It is also possible that a mediator operates to increase (instead of decrease) the statistical significance (and, thus, the magnitude of the experimental impact) of a given JOBS program on a child outcome. In this case, the mediator is not one that helps to explain the experimental impact on children. Instead, this mediator is operating in an opposing direction, indicating that the program impact on the given child outcome would have been even more pronounced if not for this variable's buffering, or offsetting, effect.
Mediational analyses first examined for which of the 60targeted and non-targeted adult and family outcomes studied in the COS there were statistically significant impacts of any of the six JOBS programs studied (see Chapter 9, Table 9.1). Next, for each particular analysis (i.e., focusing on one of the selected impacts on children noted above), the set of variables that was included in each specific model was then narrowed to those adult and family outcomes on which there were statistically significant impacts in the program we were focusing on. Of all the adult and family outcomes that emerged as mediators in any of these five analyses (see Chapter 10, Table 10.1), three sets of adult variables consistently emerged as important in explaining child impact findings: variables related to the mothers' employment, to the mothers' parenting, and to maternal psychological well-being.
Cognitive development. The favorable impact of Atlanta's labor force attachment program on focal children's mean cognitive school readiness scores appears to be related to program mothers' greater employment and improvements in parenting.Health. The unfavorable impact of Riverside's human capital development program on mothers' ratings of focal children's overall health appears to be related to program mothers' increased feeling of time stress. However, program impacts would have been even more unfavorable in the ratings of children's health if program mothers were not also more likely to have been sanctioned, to have participated in basic education, and obtained a high school diploma or GED. (Perhaps these factors being sanctioned, participating in basic education, and obtaining a high school diploma or GED go together, indicating a program in which there was both more encouragement and pressure from case workers to participate, and indeed resulting in more participation and educational progress).(16)
The unfavorable impact of Riverside's labor force attachment program on ratings of focal children's overall health appears to be related to decreases in AFDC receipt and increases in mothers' work hours, although the picture remains somewhat unclear.(17) One possibility that would require further examination is that the loss of cash welfare benefits led to a loss of Medicaid coverage which, in turn, if coverage was not replaced by employer-provided coverage or other types of coverage, led to a decline in children's overall health ratings.
Behavioral and emotional adjustment. The favorable impact of Atlanta's labor force attachment program on focal children's reported externalizing behavior problems appears to be related to the program's favorable impact on parenting, despite also increasing mothers' reports of time stress and perceptions of feeling "pushed" by the welfare office.
The unfavorable impact of Grand Rapids' labor force attachment program on focal children's reported externalizing behavior problems appears to be related to the program's unfavorable impacts on maternal depressive symptomatology and parenting.
What have we learned about possible pathways through which various welfare-to-work programs can affect children?
Several themes emerge from these results. First, the mechanisms through which children can be affected by a given welfare-to-work program include both targeted outcomes (e.g., employment, AFDC receipt) as well as non-targeted outcomes (maternal psychological well-being and parenting). In particular, this study highlights the role played by intervening mechanisms more proximal to the child in the home environment namely, maternal psychological well-being and parenting. Aspects of family life targeted by welfare-to-work programs emerged less consistently as mediators of the five child impacts selected for mediational analyses. It may be, however, that these targeted outcomes do play a role inasmuch as program impacts on these outcomes activate processes more proximal to the child. For example, changes in employment or earnings may not, in and of themselves, consistently lead to program impacts on children's behavior problems unless these lead to changes in parenting or depressive symptomatology. Testing such multi-step hypotheses with statistical methods that can model such hypotheses is warranted as a next step.
It is worth noting that, because none of the six JOBS programs studied here had aggregate impacts on total household income, this impact could not serve as a pathway through which any of the aggregate impacts on children (not just the five impacts selected for further study) came about.
In addition, it is also noteworthy that despite pervasive program impacts on child care and more circumscribed program impacts on health insurance coverage - these impacts did not help to explain either the favorable or the unfavorable child impacts examined here.However, this does not mean that child care and health insurance coverage are not important to children's outcomes - only that these variables were not found to explain the particular impacts examined here. The reader should keep in mind that we examined the relation between mediators and child outcomes in an experimental context, that is, only in cases where there was an experimental impact both on a mediator and on a child outcome. This addresses one specific type of relation between a mediating variable and a child outcome, namely, whether a given mediating variable such as child care plays a role in helping to explain a particular child impact finding. However, there are likely to be associations between child care and child outcomes in this sample even where there were no program impacts on child care and/or no impacts on children. Indeed, findings from the experimental literature indicate that quality preschool experiences can have lasting benefits for low-income children (Barnett, 1995), and findings from the non-experimental literature show that formal child care arrangements bode well for low-income children's developmental outcomes (Zaslow, McGroder, Moore, and LeMenestrel, 1999; Zaslow, Oldham, Magenheim, and Moore, 1998). For example, a recent set of analyses shows that among control group families in the Riverside site of the Child Outcomes Study, five- to seven-year-old children identified to be at risk in terms of cognitive school readiness (because they were two or more years behind in basic concepts they should have mastered by school entry) were less likely to be currently enrolled in a formal child care arrangement (Zaslow, McGroder, Moore, and LeMenestrel, 1999). In short, because we restricted our focus to instances in which there was an impact on both a child outcome measure and a mediator, our analyses do not address the broader question of whether these mediators are important to children's outcomes, in general.
A second theme relates to the size of the mediating effects. Mediational analyses show that even relatively "small"(18) impacts on targeted outcomes (e.g., the favorable impact of Atlanta's labor force attachment program on mothers' current employment) and non-targeted outcomes (e.g., the favorable impact of Atlanta's labor force attachment program on measures of parenting; the unfavorable impact of Grand Rapids' labor force attachment program on a measure of parenting) can help to explain relatively small program impacts on children. Thus, program impacts on mothers need not be "large" to translate into impacts on children.
A third theme to emerge from these findings is that some welfare-to-work programs have effects in opposing directions on aspects of family life that are important to children. For example, Atlanta's labor force attachment program increased mothers' feelings of time stress (which was associated with more frequent externalizing behavior problems) but, at the same time, this program also led to improvements in parenting (which was associated with less frequent externalizing behavior problems). The net effect of these opposing influences for focal children's externalizing behavior problems was favorable in this program.
These findings suggest that, in modeling pathways through which children can be affected by welfare-to-work programs, it is imperative to consider both positive and negative pathways, regardless of whether one is trying to explain a favorable child impact or an unfavorable child impact. Influences of mediating variables going in contrasting directions may help explain the small number and size of significant impacts on child outcomes: counteracting influences may, at times, result in no or little "net" influence on children. Understanding such influences is central to strengthening pathways that yield favorable impacts on children.
As noted above, because the present analyses rely on information about mediators and child outcomes measured contemporaneously, results should be considered preliminary. Additional waves of data from the five-year follow-up, as well as statistical methods that more effectively partition effects among multiple mediators, more completely control for selection effects, and allow alternative models to be tested explicitly (e.g., structural equation modeling), are needed to provide more definitive answers regarding the multiple pathways through which program impacts on children came about. Nevertheless, given the limited state of knowledge on the pathways through which welfare-to-work programs can affect children and families, it is important to begin to address this issue, statistically, even with contemporaneous measures of mediators and child outcomes and using a relatively modest statistical approach. We expect that the mediational results reported here begin to shed light on this important topic, and will serve as the basis for model-building and testing explicit hypotheses in the future, especially when five-year data become available.
Summary and Implications
Overall, the results described here indicate that the welfare-to-work programs implemented as part of the JOBS Program did have significant impacts on children's development, but these impacts were not widespread and were generally small. There were not strong indications of unfavorable impacts on children when their mothers were assigned to participate in a JOBS program, and indeed it is important to note that the significant impacts included favorable impacts on children.
In the aggregate, program impacts on children were not found on many outcomes, and those impacts that were found tended to be small. In addition, the direction of impacts differed for the three aspects of development examined: There were favorable impacts particularly in the area of cognitive development, and particularly in the Atlanta site, while there were unfavorable (albeit generally small) impacts in the area of health, specifically in the Riverside site. Findings in the area of children's behavioral and emotional adjustment encompassed both favorable and unfavorable impacts. Only a single finding at the aggregate level (an unfavorable impact on a rating of child health in Riverside's labor force attachment program ) was of sufficient magnitude to be called "policy relevant."
In terms of subgroup impacts, we acknowledged that children from families at higher risk could show a pattern of unfavorable or of favorable child impacts. However, we hypothesized that children from higher-risk families might experience disproportionately unfavorable impacts resulting from their mothers' assignment to a JOBS program.
As for impacts at the aggregate level, findings at the subgroup level were few and tended generally to be small. The subgroup impact findings identified a pattern of unfavorable impacts, with a number of these large enough to be considered policy relevant, for children from lower-risk families. In particular, such a pattern occurred for children from lower-risk families in three programs: Grand Rapids' labor force attachment program (in which unfavorable and policy relevant impacts were found on behavioral and health outcomes); Riverside's human capital development program (again with unfavorable and policy relevant impacts in the areas of child behavior and health); and in Riverside's labor force attachment program (with unfavorable and policy relevant impacts in all three aspects of development). It is also worth noting that some of these unfavorable and policy relevant impacts may be considered "moderate" to "large" in terms of effect sizes. The fact that the unfavorable and policy relevant impacts occurred in a concentrated manner (for lower-risk families in particular programs), and that some of these were "moderate" to "large" in magnitude, suffices to indicate that we cannot entirely dismiss the possibility of unfavorable impacts. It will be important to continue to track findings for children in lower-risk families assigned to these programs in the five-year follow-up of the Child Outcomes Study.
At the same time, there was a pattern of favorable program impacts for children in higher-risk families whose mothers had been assigned to human capital development programs or to Atlanta's labor force attachment program.(19) These favorable impacts were generally small, however, with none of sufficient magnitude to be called policy relevant. Again this pattern needs to be followed over time. Interestingly, all four of these programs also had favorable impacts on mothers' educational attainment (e.g., receipt of a high school diploma or GED; receipt of a trade degree). We need to determine if, in the longer-term, programs that improve mothers' educational outcomes continue to have beneficial implications for children from higher-risk families.
In sum, most children were not adversely affected and in some cases, children were helped by their mothers' participation in a JOBS program. Yet for delimited subgroups (particularly lower-risk families in both of Riverside's programs and in Grand Rapid's labor force attachment program) there is reason for vigilance. Further follow-up of the children (five years after enrollment) will be important.
Given these findings, we must consider it a possibility that the welfare-to-work programs put in place by the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 and implemented in the context of other policy changes, such as increased child care funding, increased earnings disregards, and Medicaid and SCHIP eligibility expansions, can have impacts on children's development. It is indeed possible that the different obligations and opportunities of the new welfare policy will result in stronger or more pervasive impacts on children than we have found here. Yet we should not begin with an assumption that the newer programs will have uniformly unfavorable or favorable impacts on children. Rather, the present findings suggest that outcomes for children are likely to differ by domain of development, by the specific features of local programs, and by characteristics of the families. The match between type of program and whether the family entered the program at higher or lower risk may continue to prove important. Regarding children, then, the present findings underscore the importance of continuing to monitor the magnitude and direction of program impacts on children's developmental outcomes in the new policy context.
We have learned that welfare-to-work programs implemented under JOBS had the potential to affect multiple aspects of family life, including aspects of family life not explicitly targeted by the programs. Non-experimental statistical analyses indicate that impacts on children sometimes reflected the simultaneous influence of both favorable and unfavorable program effects on families. Further, we have learned that child impacts reflect multiple program influences on families. Pathways through which child impacts appear to have come about in the present study included JOBS programs' impacts on outcomes explicitly targeted by the programs (such as employment and receipt of AFDC) and especially, impacts on further, non-targeted aspects of family life (such as parenting behavior and maternal psychological well-being).
Thus far it appears that the new welfare-to-work policies implemented in the states following the passage of PRWORA, combined with a robust economy and other policies, have contributed to recent changes in employment rates and on receipt of public assistance (Moffitt, 1999). The fact that impacts on children in the present study were linked to such targeted outcomes underscores the possibility that current policies may also be having effects on children. The present set of findings point to the importance of examining not only the effects of current policies on employment, receipt of public assistance, earnings and income, but also on such aspects of family life as parenting behavior and mothers' psychological well-being.
In sum, we have learned that welfare-to-work programs do indeed have the potential to affect children, both favorably and unfavorably. It is important to identify whether, to what extent, and how, impacts on children are occurring in the present policy context. Welfare policies originated in a concern for the well-being of children. Effects on children should continue to be monitored and considered in policy decisions.
References
Barnett, W. S. (1995). Long-term effects of early childhood programs on cognitive and school outcomes. Future of Children, 5(3), 25-50.
Baron, R.M., and Kenny, D.A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173-1182.
Bos, H., Huston, A., Granger, R., Duncan, G., Brock, T., and McLoyd, V. (1999). New hope for people with low incomes: Two-year results of a program to reduce poverty and reform welfare. New York: Manpower Demonstration Research Corporation.
Bracken, B.A. (1984). Bracken Basic Concept Scale: Examiner's manual. The Psychological Corporation, Harcourt Brace Jovanovich, Inc.
Bradley, R.H., Whiteside, L., Caldwell, B.M., Casey, P.H., Kelleher, K., Pope, S., Swanson, M., Barrett, K., and Cross, D. (1993). Maternal IQ, the home environment, and child IQ in low-birth weight, premature children. International Journal of Behavioral Development, 16, 61-74.
Brooks-Gunn, J. (1997, Summer/ Fall). Early intervention, cognition, and school achievement: Findings from two generations of model programs. Focus, 19(1), 12-17.
Brooks-Gunn, J.B., Berlin, L.J., and Fuligni, A.S. (1998). Early childhood intervention programs: What about the family? In J.P. Shonkoff and S.J. Meisels (Eds.), Handbook of early childhood intervention (second edition). New York: Cambridge University Press.
Burchinal, M.R., Campbell, F.A., Bryant, D.M., Wasik, B.H., and Ramey, C.T. (1997). Early intervention and mediating processes in cognitive performances of children in low-income African-American families. Child Development, 68, 935-954.
Child Trends. (1999). Children and welfare reform: A guide to evaluating the effects of state welfare policies on children. Washington, DC.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition). Hillsdale, NJ: Erlbaum and Associates.
Current Estimates from the National Health Interview Survey. (1993). Vital and Health Statistics, 10 (190), Table 70.
Current Population Survey Annual Demographic Survey, March Supplement. [On-line] available: http://www.census.gov
Devins, G. M. and Orme, C. M. (1985). Center for epidemiological studies depression scale. In D. J. Keyser and R. C. Sweetland (Eds.), Test critiques (pp. 144-160). Kansas City, MO: Test Corporation of America.
Duncan, G.J., and Brooks-Gunn, J. (Eds.). (1997). Consequences of growing up poor. New York: Russell Sage Foundation.
Freedman, S., Friedlander, D., Hamilton, G., Rock, J., Mitchell, M., Nudelman, J., Schweder, A., and Storto, L. (2000). Evaluating alternative welfare-to-work approaches: Two-year impacts for eleven programs. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.
Hamilton, G., with Freedman, S., and McGroder, S. (2000). Do mandatory welfare-to-work programs affect the well-being of children? A synthesis of child research conducted as part of the National Evaluation of Welfare-to-Work Strategies. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.
Hamilton, G., Brock, T., Farrell, M., Friedlander, D., and Harknett, K. (1997). National Evaluation of Welfare-to-Work Strategies: Evaluating two welfare-to-work program approaches: Two-year findings on the labor force attachment and human capital development programs in three sites. Washington DC: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation.
Hirsh-Pasek, K. (1998, May). In sickness and in health: Results from the NICHD Study of Early Child Care. Paper presented at the meeting of the Science of the Ecology of Early Development (SEED) Consortium of Federal Agencies, Bethesda, MD.
Kisker, E.E., Rangarajan, A., Boller, K. (1998). Moving into adulthood: Were the impacts of mandatory programs for welfare-dependent teenage parents sustained after the programs ended? Princeton: Mathematica Policy Research.
Knitzer, J., Cauthen, N.K., and Kisker, E. (1999). Enhancing the well-being of young children and families in the context of welfare reform: Lessons from early childhood, TANF, and family support programs. Washington, DC: U.S. Department of Health and Human Services.
McGroder, S. (2000). Parenting among low-income, African-American single mothers with preschool-age children: Patterns, predictors, and developmental correlates. Child Development, 71(3).
McGroder, S.M., Zaslow, M.J., Moore, K.A., and LeMenestrel, S.M. (2000). The National Evaluation of Welfare-to-Work Strategies: Impacts on young children and their families two years after enrollment: Findings from the Child Outcomes Study. Washington, DC: U.S. Department of Health and Human Services.
Moffitt, R. (1999). Presentation at the congressional briefing, "Is Welfare Reform Working? The Impact of Economic Growth and Policy Changes," sponsored by the Consortium of Social Science Associations.
Moore, K. A., Zaslow, M. J., Coiro, M. J., Miller, S. M., and Magenheim, E. B. (1995). How well are they faring? AFDC families with preschool-aged children in Atlanta at the outset of the JOBS evaluation. Washington, DC: U.S. Department of Health and Human Services.
Ogawa, J., Winfield, N., and Egeland, B. (1999, April). All in a day's work: Maternal employment and mother-child relationships in a low-income sample. Presented at the biennial meeting of the Society for Research in Child Development, Albuquerque, NM.
Peterson, J., and Zill, N. (1986). Marital disruption and behavior problems in children. Journal of Marriage and Family, 48, 295-307.
Quint, J., Bos, H., and Polit, D. (1997). New Chance: Final report on a comprehensive program for young mothers in poverty and their children. New York: Manpower Demonstration Research Corporation.
Yoshikawa, Hirokazu (1995). Long-term effects of early childhood programs on social outcomes and delinquency. The Future of Children, 5 (3), 51-75.
Zaslow, M.J., Dion, M.R., Sargent, J., Weinfield, N., Ogawa, J., Egeland, B., DeTemple, J., Tabors, P., Snow, C., Morrison, D.R., Hair, E., & Ahluwalia, S. (2000). Program impacts on parenting behavior during the first half-year of an educationally-focused welfare-to-work program. Manuscript submitted for publication.
Zaslow, M.J., Hair, E., Dion, M.R., Ahluwalia, S., and Sargent, J. (1999). Maternal depressive symptoms and low literacy as potential barriers to employment in a sample of families receiving welfare: Are there two-generational implications? Manuscript submitted for publication. Zaslow, M.J., McGroder, S., Cave, G., & Mariner, C. (1999). Maternal employment and measures of children's health and development among families with some history of welfare receipt. Research in the Sociology of Work, 7, 233-259.
Zaslow, M.J., McGroder, S., Moore, K.A., and LeMenestrel, S.M. (1999, April). Behavior problems and cognitive school readiness among children in families with a history of welfare receipt: Diverging patterns and their predictors. Paper presented at the meetings of the Society for Research in Child Development, Albuquerque, NM.
Zaslow, M.J., Oldham, E., Magenheim, E., and Moore, K.A. (1998). Welfare families' use of early childhood care and education programs, and implications for their children's development. Early Childhood Research Quarterly 13(4), 535-563.
Zaslow, M. J., Rabinovich, B. A., and Suwalsky, J. T. D. (1991). From maternal employment to child outcomes: Preexisting group differences and moderating variables. In J. V. Lerner, and N. L. Galambos (Eds.), Employed mothers and their children (pp. 237-282). New York, NY: Garland Publishing, Inc.
Zaslow, M.J., Tout, K., Botsko, C., and Moore, K.A. (1998). Welfare reform and children: Potential implications. New Federalism Issues and Options for States, Series A., No. A-23, 1-5. Washington, DC: The Urban Institute.
Zaslow, M. J., Tout, K., Smith, S., and Moore, K. A. (1998). Implications of the 1996 welfare legislation for children: A research perspective. Social Policy Report of the Society for Research in Child Development, XII(3).
Zill, N. (1985). Behavior Problems Index. Washington, DC: Child Trends, Inc.
Acknowledgments
We want to thank the distinguished scholars Lois Hoffman, Ph.D.; Aletha Huston, Ph.D.; and Sheila Smith, Ph.D.; and the veteran child and family policy analyst, William Prosser, for their detailed and insightful comments on an earlier draft of this report.
Footnotes
1. An unfavorable impact of Riverside's labor force attachment program on the percent of focal children rated by their mothers to be in very good or excellent health.
2. All statistically significant impacts on measures of children's development warrant continued monitoring. However, impacts that meet the criterion for policy relevance are relatively larger impacts and, thus, may have greater implications for children's development. Consequently, policy makers may want to take special note of these impacts in their policy decisions.
3. The only exceptions relate to unfavorable health impacts of Riverside's human capital development program, a single unfavorable health impact of Atlanta's labor force attachment program, and a single unfavorable behavioral impact of each of Atlanta's JOBS programs.
4. Researchers in the behavioral sciences often rely on Cohen's (1988) characterization of effect sizes (in standard deviation units) of .20 as "small," .50 as "medium," and .80 as "large."
5. The report of the Descriptive Study ("How Well Are They Faring? AFDC Families with Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation") asked whether, overall, this group of children was at risk for poor developmental outcomes, and which factors were most closely associated with the development and well-being of the children (Moore, Zaslow, Coiro, Miller, and Magenheim, 1995). This study identified risk factors for poor developmental outcomes in the children, but also protective factors associated with more positive development.
6. Findings will also be reported in the future from a special in-depth study of parenting behavior (the JOBS Observational Study; see Zaslow et al., 2000). This study, carried out within a subset of the Child Outcomes Study sample, involved direct observation of mother-child interaction soon after baseline and again four and a half years after baseline. Fine-grained observational measures of parenting behavior are used to ask whether mother-child interaction was affected by assignment to one of the JOBS programs (the Atlanta human capital development program). The JOBS Observational Study was funded by the Foundation for Child Development, the William T. Grant Foundation, the George Gund Foundation, and an anonymous funder, with additional funds for pretesting of middle childhood observational measures provided by the U.S. Department of Health and Human Services.
7. Individuals in need of basic education, and those not in need of basic education were then assigned to different random assignment processes in the Riverside site. Those who were considered to be in need of basic education were randomly assigned to any one of the three research groups. However, those considered not in need of basic education could be randomly assigned only to the labor force attachment or control groups (see Hamilton et al., 1997). As a result, when contrasts of research groups are carried out in the Riverside site, those in the human capital development group are compared to control group members who are likewise considered in need of basic education, whereas members of the labor force attachment group (who could be in need or not in need) are compared to all control group members. By contrast, Atlanta and Grand Rapids did not take mothers' basic education needs into account when randomly assigning them to a program or control group; thus, mothers in these sites' human capital development programs were not as disadvantaged (with respect to literacy and/or educational attainment) as the mothers assigned to Riverside's human capital development program. Accordingly, in considering the patterning of findings for human capital development programs across all three sites, the reader should keep in mind, not only how the sites differ in terms of population and economic conditions (see Chapter 3), but also differences in the experimental designs.
8. Mothers were asked to rate the focal child's overall health in the following question: "Would you say that (CHILD's) health in general is: excellent, very good, good, fair, or poor?"
9. Chapter 2 of the report provides details about the specific "focal child" and "any child" measures. Chapter 6 presents impact results for specific child outcome measures.
10. Some would argue that each statistically significant finding is valid irrespective of the number of analyses conducted. Others would argue that the greater the number of analyses conducted, the greater the likelihood of chance findings and, thus, one needs to take the number of analyses into account. In order to minimize the likelihood of reporting chance findings, we chose to calculate the proportion of statistically significant impacts across all child outcome measures and across all six programs. Given that the experiment-wise Type I error rate was set at .10, we might expect to find significant results 10 percent of the time due to chance alone, translating into just over 13 statistically significant impacts across the 22 child outcome measures and the six programs (22 x 6 = 132, 132 x .10 = 13.2). Some argue that a more stringent standard is needed, requiring that the number of significant impacts within each program must exceed chance levels, or that the number of significant impacts within each domain of child development must exceed chance levels. Because there is a lack of consensus on this issue among statisticians, and given that a goal of the Child Outcomes Study was to provide a thorough examination of program impacts, we did not adhere to this more stringent standard. Moreover, our examination of impacts at the subgroup level (see Chapter 7) and of pathways through which particular impacts on children appear to have come about (see Chapter10) provides further evidence that even rare impacts are not necessarily chance findings.
11. The full report discusses the more subtle ways in which subgroup impact findings differed across the approaches to defining risk (sibling constellation risk, educational risk, work risk, maternal psychological well-being risk, and cumulative risk; with an exploratory examination of reservations about working as well).
12. The proportion of findings to reach statistical significance relative to the number of impacts examined, exceeded what might be expected on the basis of chance for all six approaches to defining risk. The general pattern of findings that we discuss for higher- and lower-risk subgroups held across the different approaches to defining risk.
13. Includes all earnings, AFDC, food stamps, WIC, SSI, social security, any unemployment insurance or workers' compensation, refugees assistance, foster care payments, family or friends outside the household, estimated earned income tax credit, and is net of estimated child care expenses.
14. The 1994 poverty level for a single mother with two children was $11,940 and for a single mother with three children, $15,081. This poverty line for the appropriate household size was divided by 12.
15. Notably, moving from experimental to non-experimental analyses creates some potential difficulties with respect to selection bias. For instance, because JOBS did not randomly assign mothers to various types of child care, mothers' child care choices reflect, to some degree, their preferences, motivations, and other characteristics. As a hypothetical illustration, if mothers with problem behavior children are disproportionately more likely than mothers without problem behavior children to seek quality child care in hopes of curbing the problem behavior and assuming that quality child care truly leads, causally, to better behavioral outcomes in children then the observed statistical "effect" of child care on children's behavioral outcomes will underestimate the true positive effect of child care. The mediational analyses reported on here and in the full report (as well as the experimental impact analyses that preceded them) did not control for children's prior developmental and behavioral outcomes; however, we did control for numerous other variables representing prior characteristics of the child, mother, and family at baseline that may serve as selection factors. (For a complete list, see the third footnote of Chapter 10 of the full report.) The availability of this number and range of baseline variables is a great asset of this dataset, and previous research using these data indicate that many of these baseline variables do, to some extent, capture selection into employment (Zaslow, McGroder, Cave, and Mariner, 1999), selection into child care (Zaslow, Oldham, Magenheim, and Moore, 1998), and selection into parenting pattern (McGroder, 2000). Thus, we are likely to have controlled for many, though not all, possible selection effects.
16. Future work may attempt to examine in greater detail how such variables are linked.
17. The analyses for this particular child outcome indicate a statistically significant role for both AFDC receipt and mother's work hours in models considering these variables separately (along with covariates), but not in a combined model. This appears to be due to multicollinearity of the mediating variables. For all other results, the findings summarized held in a final combined model as well as in models considering each mediating variable separately.
18. As we have noted, researchers in the behavioral sciences often rely on Cohen's (1988) characterization of effect sizes (in standard deviation units) of .20 as "small," .50 as "medium," and .80 as "large." Cohen (1988) acknowledges that this characterization is somewhat arbitrary and "is recommended for use only when no better basis for estimating the effect size index is available" (p. 25). For example, when the accumulation of empirical research demonstrates that a given effect size is predictive of a "meaningful" difference in an outcome generally agreed to be "important," this effect size may well be considered "large" regardless of its absolute size.
19. The only exceptions relate to unfavorable health impacts of Riverside's human capital development program, a single unfavorable health impact of Atlanta's labor force attachment program, and a single unfavorable behavioral impact of each of Atlanta's JOBS programs.