This summary report highlights key findings from a study focusing on young children in the context of welfare-to-work programs: the two-year follow-up report of the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies (McGroder, Zaslow, Moore and LeMenestrel, 2000). Specifically, this report asks whether young children's development and well-being were affected when their mothers were assigned to participate in welfare-to-work programs implemented as part of the Job Opportunities and Basic Skills Training (JOBS) Program, the set of programs authorized by the Family Support Act of 1988.
The JOBS Program required mothers with children as young as age three (or age one, at state option) to participate in welfare-to-work activities. This was the first time that a federal program implemented nationally required participation by mothers with preschool-age children. Young children are particularly likely to experience changes in their daily lives when their mothers' activities change. In addition, there is evidence that young children's development is particularly sensitive to family poverty status (Duncan and Brooks-Gunn, 1997). Thus, it is important to consider the implications of welfare-to-work programs on children who are preschool-age when their mothers enroll in these programs.
In this report we ask whether different aspects of young children's development (specifically their cognitive development and academic achievement, behavioral and emotional adjustment, and health and safety) were affected at all, and, if so, for better or for worse, two years after their mothers were assigned to participate in a JOBS welfare-to-work program. We also ask whether aspects of family life that are important to children's development (for example, the stimulation and support available to the child in the home environment, participation in child care, and poverty status) changed when mothers were assigned to participate in a JOBS welfare-to-work program. Finally we explore the linkages between changes in family life and program impacts on children's developmental outcomes.
Why focus on outcomes for children in evaluating a set of adult-oriented welfare-to-work programs?
Much of what is known about the effects of welfare-to-work programs focuses on economic outcomes for adults, for example, the extent to which programs increase employment, earnings and household income, and reduce welfare receipt. Yet a majority of those receiving public assistance are children. Further, a primary purpose of welfare programs has historically been to protect the well-being of children growing up in poor, and especially single-parent households. Even those programs that target adult economic outcomes, and have few or no components explicitly focused on enhancing young children's well-being, have the potential to alter important aspects of children's lives. For example, programs that target family economic self-sufficiency can alter the time children spend in the home and in child care settings, and the material resources available for their needs. Thus, even though JOBS programs did not aim directly to change child well-being (for example, through an early childhood intervention, or through developmental screening and follow-up services), children's development could have been affected indirectly, because of changes within the family and in children's care situations.
Accordingly, this study asks: (1) whether JOBS programs affected the well-being and development of children; (2) what the direction of effects on children was, that is, whether children's development and well-being were influenced favorably or unfavorably; (3) which children were affected (all children, on average, or primarily children in certain types of families); and (4) how children were affected by their mothers' assignment to a JOBS program.
This study uses a rigorous design to examine program impacts on children. The sample for the Child Outcomes Study is a subset of the families that were participating in the larger evaluation of economic impacts of JOBS called the National Evaluation of Welfare-to-Work Strategies. The subset of families in the Child Outcomes Study includes families in three of the full evaluation's seven research sites, and specifically families that had a preschool-age child at the time of enrollment in the evaluation.
The larger evaluation, and the Child Outcomes Study embedded within it, followed an experimental design. Families were randomly assigned to a control group, or to one of two experimental groups. The control group families were eligible for all benefits associated with Aid to Families with Dependent Children; however, they had no requirement to participate in a welfare-to-work program, nor did they receive the special messages, case management, or services of a JOBS program. The two experimental groups were also eligible for all benefits, but were required to participate in a JOBS welfare-to-work program. The two experimental groups differed in terms of which kind of program they were randomly assigned to, with one approach (labor force attachment) emphasizing a rapid transition to employment, and the other (human capital development) emphasizing education and training as a means to obtaining a better job. Families in the two experimental groups had the encouragement and support of enhanced case management. At the same time, they faced the possibility of sanctions (reductions in welfare benefits) if they did not participate in welfare-to-work activities.
In an experimental study of this kind, we are confident that families did not differ systematically before they were randomly assigned to a control or experimental group (and this was confirmed statistically). Accordingly, differences between those in an experimental group as opposed to the control group can be attributed to the influence of having been assigned to a JOBS program. We use the term child outcome to refer to the measures of child well-being and development that we are focusing on. We reserve the term child impact specifically to describe differences on a child outcome measure between families in an experimental and control group.
In addition to assessing impacts, we also look at the child outcome scores for children in the control group in order to describe the developmental status of children in the absence of a JOBS welfare-to-work program. In this way, we can examine the extent to which these children were at risk in their development apart from exposure to a welfare-to-work program. This provides an important context for understanding program impacts.
We make this distinction in terminology for measures of family well-being as well, with the term "outcomes" referring to the measures of family functioning in economic and other areas, but the term "impacts" referring to experimental and control group differences on these outcome measures.
Did we anticipate effects on children to go in a particular direction?
Very little is known about how welfare-to-work programs affect children, and the Child Outcomes Study is one of the first to focus on this issue. Given the very limited basis for making predictions, we did not begin the study anticipating that JOBS programs would affect children in a particular direction, with either improvement or deterioration in the children's development. Instead, the study was initiated in order to explore four diverging possibilities:
- That children's development would be affected favorably when their mothers were assigned to a JOBS program. Mothers assigned to a JOBS program could potentially experience gains in educational attainment, basic skills, and job skills. Over time, participation could lead to employment, improvements in earnings and income. Through such changes, mothers could also experience increased psychological well-being, and the home environment could improve. Increased child care participation, given that the care was of good quality, could provide children with increased stimulation and support. Such changes could affect children's development positively.
- That children's development would be affected unfavorably when their mothers were assigned to a JOBS program. This direction of impacts on children is possible if mothers experienced substantial stress in making a change in their daily activities and in arranging family life so that they could participate in welfare-to-work activities. Such stress could affect the quality of mothers' interactions with their children and the degree of organization (for example of routines, the physical setting) of the home environment. Unfavorable impacts on children could occur if the young children were exposed to unstable (frequently changing) child care arrangements or child care of poor quality. Unfavorable impacts on children could also occur if mothers made the transition to employment, but did so without stable earnings, or with total family income falling below what they were receiving while on welfare.
- That only children in certain subgroups of families would be affected. Families with particular characteristics might respond in different ways to the mothers' assignment to JOBS programs. For example, mothers with limited education, literacy, and/or previous work experience, or mothers with symptoms of depression, might find it particularly difficult to fulfill welfare-to-work program requirements. Such mothers might fail to fulfill the requirements and face sanctioning. Alternatively, they might fulfill the requirements, but do so with substantial stress. Children in these families, but not others, might show unfavorable program impacts. On the other hand, there might be families especially likely to benefit from JOBS programs. For example, mothers with limited education and literacy, mothers with limited work experience, or mothers experiencing psychological distress, might benefit especially from the combination of supports (enhanced case management), program requirements (a participation mandate), and program messages and services (labor force attachment and human capital development welfare-to-work activities) that comprised JOBS programs. If the mothers in such higher-risk families are particularly responsive to the JOBS program components, and make changes in educational attainment, job-related skills, employment and income, such changes could affect children favorably in these particular families.
- That there would be no effects on children. Changes may occur within the family, but not in aspects of family life that affect children immediately. For example, it may not be enough (from the point of view of impacts on children) to change mothers' employment status or the sources of family income if these do not result in changes in the child's immediate environments of home or child care. Perhaps changes in the home environment or child care do occur but are too small to result in impacts on child development. There could also be influences on aspects of family life going in opposing directions, with these influences resulting in no net change in children's development. For example, maternal earnings might increase, but the home environment might simultaneously deteriorate with increased maternal stress. These different possibilities point to the importance of examining not only whether JOBS affected children, but also what kinds of changes occurred within families, and how these changes jointly help to shape impacts on children.
The study was designed with sufficient power to reasonably assess whether there were harmful impacts on children, an important hypothesis to examine when the new provisions of the Family Support Act were being implemented. The design also has sufficient power to assess whether favorable impacts occurred or whether different impacts occurred for different subgroups.
At what point should policymakers take findings on children into account in making policy decisions?
In the present study, we report all program impacts on children that are statistically significant. These program impacts are reliable: they are very unlikely to have occurred just on the basis of chance. As such, these program impacts warrant continued monitoring. In the present study, we will want especially to monitor whether the kinds of outcomes on which statistically significant impacts were found at the two-year point continue to show differences at the final follow-up (five years after the families enrolled in the evaluation), and if such differences grow in magnitude.
We also report on whether a statistically significant result meets a further criterion: that of "policy relevance." At the start of the study and as the study proceeded, researchers and policy makers met to grapple with the question of the point at which child impact findings should be taken into account in considerations about policy. A decision was made that statistically significant findings that were of a particular magnitude should be considered relevant to policy discussions. Specifically, for this study, a policy relevant impact was defined as one in which the effect size was at least one-third of a standard deviation on a given measure.
This threshold sets aside impact findings that are so small that, while they are reliable statistically and warrant continued monitoring over time, may at this point in time have limited importance in terms of children's development. At the same time, the threshold for policy relevance does not require that an impact be "large" in magnitude(4) in order to meet the criterion. By setting the threshold in this way, we can be reasonably confident that we are being inclusive in identifying instances of possible harm as well as of possible beneficial effects on children, without focusing on effects that are so small as to be of limited importance for children's development.
In presenting results, we discuss not only the significance and policy relevance of impacts but also the patterning of findings. We also identify those impacts for which effect sizes substantially exceeded the threshold for policy relevance, that is, were .50 or larger (which is considered "moderate" to "large" in magnitude; Cohen, 1988). The strongest evidence on which to base conclusions about impacts on children is a consistent patterning of impact results, particularly when impacts meet or exceed the criterion for policy relevance. A patterning of results, for example, might show consistently favorable impacts for families in a particular site, or a particular program approach. A patterning of results might also pertain to a type of child outcome, with findings in one aspect of development (such as health) consistently affected favorably (or unfavorably) across programs.
What is the relevance of these findings in the new policy context?
Data collection for the two-year follow-up in the present evaluation was completed in January of 1996, prior to implementation of the new welfare law. Are findings from this evaluation still relevant in the broader sense of having implications in the new policy context? With the passage of the new welfare law, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA), AFDC and the JOBS programs were supplanted by the Temporary Assistance to Needy Families (TANF) program. Nevertheless, the experiences of families participating in JOBS offers guidance to states implementing their own welfare-to-work programs. In particular, findings from the evaluation of JOBS may suggest which programmatic approaches affect child outcomes, and in what ways. Apart from program impact findings, the Child Outcomes Study also provides much information about the developmental status of control group children, and about the family contexts of these children; information indicating how these children and families would be faring in the absence of the JOBS Program. This information, too, can help to guide efforts to shape programs that include the needs of both children and adults in families receiving public assistance (see Knitzer, Cauthen and Kisker, 1999).
The present welfare policy environment differs substantially from that under AFDC in several important respects (see Zaslow, Tout, Smith and Moore, 1998). For example, welfare benefits are no longer considered an entitlement under the 1996 law, and states have substantial latitude in implementing their particular programs. Under TANF almost all states emphasize a quick transition to employment. The new federal law requires that recipients of public assistance be working within two years of the time they start to receive assistance (or sooner at state option), places a lifetime limit of 60 months on the receipt of federal TANF funds, and requires states to have a certain proportion of their caseloads meeting work requirements. But there are also greater supports for work, with almost all states now treating earnings and assets more generously, and significantly more federal funds are available for child care. Another difference is that under AFDC, JOBS also encompassed program approaches focusing on remediation of basic education skills to improve job prospects. Whereas the Family Support Act required mothers of children three and over to participate in work-related activities (with a state option to lower the age to 12 months), the new law lowered the age for all states to 12 months (or younger, if a state chooses). Finally, while under the Family Support Act, mothers were exempt from participation in welfare-to-work activities if they were needed in the home to care for a sick or incapacitated family member (including a child), there is no such explicit exemption under PRWORA.
Over the years, public assistance has become more and more predicated on custodial parents' involvement in work or mandatory welfare-to-work program activities, as policymakers have sought to encourage adult self-sufficiency while also meeting the goal of fostering poor children's well-being. Under TANF and AFDC, recipients of public assistance were required to take steps toward economic self-sufficiency or face a reduction in benefits (sanctioning). In many ways, programs implemented under PRWORA may be seen as an intensification of both the obligation and the opportunity dimensions of those implemented earlier, with respect to this common priority.
The present study is one of only very few that examines impacts on the development of children in the context of welfare-to-work programs. Findings on what happened to children in the context of JOBS programs will help build toward an understanding of how the new policy context is affecting children. We will need to be cautious about extrapolating from the specific findings of the present study to outcomes for children under new policies. Yet addressing the general questions of whether children were affected even in a program directed primarily at economic outcomes in adults, and how children were affected, will provide us with information that will continue to be of vital importance. In addition, documenting the effects of labor force attachment and human capital development approaches will yield important evidence regarding the implications for children of very different welfare-to-work strategies.
The results from this study are an important piece of the emerging picture on the impacts of welfare on children and families. It will be important to put the findings from this study in the context of findings from other studies, such as the Project on State Level Child Outcomes (Child Trends, 1999), which examines the impacts on children of five state approaches to experimenting with welfare policies through obtaining waivers from various AFDC provisions.