Impacts of a Mandatory Welfare-to-Work Program on Children at School Entry and Beyond: Findings from the NEWWS Child Outcomes Study

By
Sharon M. McGroder, Ph.D., Martha J. Zaslow, Ph.D., Kristin A. Moore, Ph.D., and Jennifer L. Brooks, Ph.D.
Child Trends

"

Overview

Policy makers and others have expressed an interest in how children may be affected by mandatory welfare-to-work programs. Though superceded in 1996 by the Personal Responsibility and Work Opportunity Reconciliation Act, the Job Opportunities and Basic Skills Training (or JOBS) Program shares the current goal of replacing welfare with work. It also contained many of the elements  such as work requirements and sanctions for non-compliance  still operating in welfare-to-work programs today. This brief presents findings from the Child Outcomes Study, a substudy of the National Evaluation of Welfare-to-Work Strategies (NEWWS),⁽¹⁾ which examined the impacts of 11 JOBS programs in seven sites across the country.⁽²⁾

In three of these sites, the Child Outcomes Study looked at the long-term impacts of two alternative pre-employment strategies  employment-focused and education-focused  on children ages 3 to 5 at the start of the study. It sought to determine whether one approach was more or less beneficial than the other for children's development. Because these programs did not provide services aimed at improving the development and well-being of children  as in early childhood education programs  any impacts on children would likely result from their mothers' exposure to the program (for example, self-sufficiency messages from case managers) and from program-induced changes in maternal education, employment, and/or family income. Three general areas of child development were studied  cognitive development and academic functioning, social skills and behavior, and health and safety.

Overall, there were few impacts of the six JOBS programs studied when children were of elementary school age. When found, impacts on cognitive outcomes were favorable early on but faded over time; impacts on behavioral outcomes were both favorable and unfavorable both early and later on, and impacts on health outcomes were unfavorable, both early and later on. Of particular interest was the finding that impacts on young children did not vary according to the type of welfare-to-work strategy used but, rather, tended to vary more according to the site in which the program was implemented. Researchers conclude that impacts on outcomes important to children  such as stable maternal employment, adequate family income, and supportive environments  were too few, occurred for too brief a period, or were of an insufficient magnitude to lead to large, widespread impacts on elementary school-age children. They also emphasize that even when favorably affected by these programs, young children still remained at risk for problem outcomes, especially pertaining to academic achievement and school progress.

The Family Support Act and the JOBS Program

The Family Support Act of 1988 created the Job Opportunities and Basic Skills Training (JOBS) Program, which required states to develop and operate mandatory welfare-to-work programs for welfare recipients. The explicitly-stated goal of JOBS was to reduce long-term welfare dependency. Research at the time had shown that welfare-to-work programs adopting a "job-search-first" strategy generally did not benefit the most disadvantaged welfare recipients. Consequently, the law placed an emphasis on education and training, in the hopes that investing up-front in clients' skills might have a longer-term payoff in terms of employment in better-paying jobs. This was especially anticipated for those without a high school degree, who are at greatest risk for long-term welfare receipt.

The JOBS Program had four main components:

• Services to enhance the employability of welfare recipients, such as adult basic education (including high school or GED preparation classes, remedial education, and English as a Second Language classes), vocational training, and job search;
• Mandated participation in these work preparation activities (and financial sanctions for non-compliance with this mandate);
• Messages about the importance of such activities for self-sufficiency; and
• Case management, to direct and monitor clients' progress through these activities.

The Family Support Act marked the first time in national policy that welfare recipients with children ages 3 to 5 (or as young as one, at state option) were required to enroll in welfare-to-work programs. It also required that at least 55 percent of funds be reserved for services for welfare recipients deemed to be at the greatest risk of long-term welfare dependency. Enrollees were required to participate in employment-preparation activities for as long as they were on welfare and remained eligible for services. Case managers were expected to monitor participation and use a variety of informal and formal responses when enrollees did not comply. Finally, welfare recipients were supposed to be assigned additional activities if they completed participation in employment-preparation activities without finding a job.

The National Evaluation of Welfare-to-Work Strategies (NEWWS)

The Family Support Act required a rigorous evaluation of the JOBS Program. The National Evaluation of Welfare-to-Work Strategies (NEWWS) examined the long-term effects of 11 JOBS programs on welfare recipients, particularly on their welfare use, employment, and earnings. A central question of the NEWWS was "What works best, for whom?"⁽³⁾  Using an experimental design, evaluators randomly assigned recipients to either a program group (with services) or to a control group (without services). Individuals in program groups were required to take part in program services or face a reduction in welfare benefits. Control group members were not required to participate in and received no services through the program but were free to seek out similar services in the community on their own.

The evaluation compared the impacts of two alternative pre-employment strategies for different groups of welfare recipients:

• Programs that emphasized short-term job search assistance and encouraged people to find employment quickly, referred to as Labor Force Attachment or employment-focused programs; and
• Programs that emphasized skill-building activities, primarily basic education, referred to as Human Capital Development or education-focused programs.

States had wide latitude in terms of the sequencing, content, and overall implementation of JOBS programs. The result was that some sites' programs were more education-focused, while others were more work-focused. The evaluation sought to determine how a variety of JOBS programs, operated in a diverse range of conditions, affected adults' economic outcomes. Consequently, seven sites were selected that varied geographically, economically, and in terms of welfare benefit levels. In four of these sites, multiple JOBS programs were implemented and evaluated.

Program impacts were estimated from a wealth of data, collected over an extended period of time. (Random assignment took place between late 1991 and early 1994; data from the five-year follow-up was thus obtained between 1996 and 1999.) These data include standard client characteristics at baseline (such as educational attainment and welfare history), administrative records (unemployment insurance, state and county welfare payments, and food stamp data) for 41,715 single-parent families across the seven study sites. For a subset of about 7,000 families in four of the seven sites, survey data were also obtained. Questions focused on economic outcomes, family context and, to a limited extent, behavioral problems and safety of children in the family. These core surveys were conducted two and five years after random assignment.

The Child Outcomes Study: A Substudy of the NEWWS

Under the Family Support Act, welfare recipients with preschool-age children were required to participate in work preparation activities. Policy makers were interested in determining whether requiring low-income mothers to secure employment would affect their young children and, in particular, whether employment- or education-focused programs were more beneficial or detrimental to children.

Why would one expect a program aimed at mothers  a program that did not provide services directly to children  to affect their children? Researchers have found, and many practitioners have witnessed, that interventions aimed at individual family members can have "spillover" effects on other family members and on family functioning as a whole. Mothers required to participate in JOBS may find this a welcome push and may benefit from the work preparation services. The increased family income from earnings, and any accompanying improvements in mothers' self-esteem, would enhance the child's home environment, thereby improving outcomes for young children. On the other hand, the mandate to participate in JOBS activities (with financial sanctions for non-compliance) may be difficult for some mothers already stressed by their responsibilities of providing for and raising their children. Increased stress  perhaps even leading to depression  may lead mothers to interact more harshly with their children, which can prove detrimental for children's outcomes.

Policy makers were also interested in learning whether different welfare-to-work approaches had different effects on children. Some believed that increases in mothers' educational attainment that resulted from participating in an education-focused program would bode well for their children (particularly for their cognitive and academic success at school), even if such participation did not immediately or ultimately lead to increased employment and earnings for the families. Others argued that the quicker the mothers secured employment (through the employment-focused approach), the sooner their children would reap the financial benefits.

Policy makers also sought basic, descriptive information on how these families were faring economically, as well as information about the well-being of these single mothers and the developmental status of their children. To address these issues, the Department of Health and Human Services, with additional funding from the Department of Education, launched the Child Outcomes Study.

The Child Outcomes Study collected descriptive information on child and family functioning and evaluated the impacts on young children and their families of the six JOBS programs in the three NEWWS sites operating both an education-focused and an employment-focused program: Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. Single mothers with a preschool-age child at random assignment who were applying for or receiving welfare in these sites were eligible for the Child Outcomes Study. Just over 3,000 families were included in the two-year follow-up (when children were between about 5 and 7 years old), and approximately 2,300 families were included in the five-year follow-up (when children were between about 8 and 10 years old).

In addition to administrative data and information from the "core" survey, detailed information on outcomes for young children were obtained from surveys given to mothers two years after random assignment, and to mothers, children, and teachers five years after random assignment. Standardized assessments were also conducted to tap young children's academic school readiness (at the two-year follow-up) and achievement in math and reading (at the five-year follow-up). Measures of maternal and family well-being were obtained from mothers' reports and through interviewer ratings at both the two-year and five-year follow-up.⁽⁴⁾

Who are These Families and Children?

Though all families in the Child Outcomes Study had applied for or were receiving welfare at study entry, these families were fairly diverse in ways that challenge common stereotypes. For example, though all study participants were single mothers at the time they were enrolled in the study, between 31 percent (in Atlanta) and 57 percent (in Riverside) had been married previously. Between two-thirds and three-quarters of mothers in each site had only one or two children when entering the study. Most mothers (between 51 and 60 percent) had a high school diploma or its equivalent, though between one-third and one-half scored low enough on a literacy test to indicate they would have difficulty reading a map or a bus schedule. Between one-quarter and one-third of these mothers reported receiving welfare for less than two years, and two-thirds agreed that "it's wrong to stay on welfare if you can get a job, even a job you don't like."

How were children in these families doing?  An examination of outcomes of young children in the control groups two and five years after study entry indicates how these children would have been faring had their mothers not been exposed to the requirements and services of JOBS.

Findings suggest that, when between 5 and 7 years old, children were developmentally disadvantaged especially with respect to academic school readiness, but also (to a lesser degree) with respect to behavior problems.⁽⁷⁾ For example, only about 18 percent of young children in the Child Outcomes Study scored as high on a measure of academic school readiness as half of the children in a national sample of 5- to 7-year-olds.⁽⁸⁾ This is not to say that all children the Child Outcomes Study showed delays in academic school readiness; some control group children answered all items correctly, consistent with the performance of an "average" 7- or 8-year-old. Moreover, an analysis of control group children in the Riverside site found one-quarter of the children performing at or above grade level. Nevertheless, half of these children were already at least one year behind in the number of basic concepts they should have mastered by school entry, indicating developmental risk for a substantial portion of this sample.⁽⁹⁾

Earlier delays in average school readiness appear to have manifested three years later as lower academic performance and more problem behaviors for some children. A disproportionate share of control group children in Atlanta and Riverside scored "below average" on reading achievement tests, compared to a national sample of 8- to 10-year-olds.⁽¹⁰⁾ In addition, between 35 and 45 percent of control group children across the sites were performing below grade-level in math, and 43 to 54 percent were performing below grade-level in reading, according to their teachers. Teachers also reported that between one-quarter and one-third of these children were in a remedial math group, and almost 40 percent were in a remedial reading group. School behavior was also a problem for many of these children: One-third of teachers reported taking disciplinary action at least weekly with control group children. About one-third of teachers reported that these children needed but did not receive special resources or services, such as remedial instruction, speech therapy, or resources for emotional, psychological, or behavioral problems.

Impacts on Young Children

Did this set of JOBS programs improve upon or worsen these levels of cognitive and behavioral functioning and health for these young children? Because random assignment ensures that families and their young children are identical (on average) in every way except for exposure to the JOBS program, we can confidently conclude whether a program had a causal effect (called an "impact") by comparing subsequent outcomes for children in each program group to outcomes for children in the corresponding control group.⁽¹¹⁾

• This set of mandatory welfare-to-work programs had few impacts on young children. Looking across the six programs and the numerous child outcomes examined, there were few impacts at either the two- and five-year follow-up. Nevertheless, there were more numerous impacts than one would expect due to chance.
• When young children were affected
  • it was sometimes for the better, sometimes for the worse. Earlier impacts found were slightly more positive than negative; later impacts found were slightly more negative than positive.
  • it was in each of the three child development areas examined. Young children's academic school readiness was favorably affected early on, though impacts on academic functioning three years later were more circumscribed and were concentrated on measures of behavioral adjustment to school (for example, school engagement and disciplinary problems) and attendance. Young children's positive behavior and problem behaviors were affected, positively and negatively, both early and later on. Finally, few impacts were found at either the two- or five-year point on maternal report measures relating to young children's health and safety; however, when found, impacts were negative.
  • it did not vary according to welfare-to-work strategy. In each site, there were very few differences in outcomes for young children whose mothers were in the employment-focused program compared to children whose mothers were in the education-focused program. In addition, the pattern of impacts on young children was similar for both programs in each site. When differences occurred, neither approach emerged as uniformly more or less beneficial to children.
  • it tended to vary according to the site in which the programs were implemented.
    • Favorable impacts occurred in Atlanta. The majority of impacts found in Atlanta, in both the short- and longer-term, regardless of program approach, were favorable. There were very few differences in the impacts on young children according to the welfare-to-work strategy employed in Atlanta. Earlier impacts on young children occurred largely on measures of academic school readiness, and they were all positive, though small. Three years later, impacts of Atlanta's programs in the academic area consisted improvements in reading and reductions in disciplinary problems, though each program also increased absenteeism or tardiness by a moderate amount. The majority of longer-term impacts found related to children's social skills and behavior  both at home and in the classroom  and these impacts were uniformly positive.⁽¹²⁾
    • Unfavorable impacts were found in Grand Rapids. The majority of impacts found in Grand Rapids, in both the short- and longer-term, regardless of program approach, were unfavorable. Grand Rapids employment-focused program decreased positive behaviors (such as cooperation), and Grand Rapids' education-focused program both decreased positive behaviors and increased problem behaviors (including disciplinary problems in school) when children were 8 to 10 years old  despite not having affected behavioral outcomes earlier on.⁽¹³⁾
    • Impacts of Riverside's programs varied by area of development, and differed according to mothers' initial levels education. Neither of Riverside's programs affected behavioral or academic outcomes when children were 5 to 7 years old. However, when Riverside's employment-focused program had impacts in the full sample, it tended to worsen young children's behavior, and both improve and worsen different aspects of academic functioning when they were 8 to 10 years old. Limiting this sample to the subgroup of children whose mothers lacked a high school diploma at study entry, when impacts were found, both programs in Riverside improved children's behavior at the five-year follow-up.⁽¹⁴⁾ In a couple of instances, impacts were moderate in size.⁽¹⁵⁾

Impacts on Key Economic-Related Outcomes

In addition to answering key evaluation questions pertaining to program impacts on economic outcomes, an examination of such impacts might shed light on how it came to be that children were affected. So how were these children's parents and families affected by JOBS? The results for mothers in the Child Outcomes Study indicate that these six JOBS programs generally affected key economic outcomes as intended, especially in the short-run.

• By the two-year point, all six programs had substantially increased mothers' participation in activities designed to promote employment. Both the education- and work-focused programs increased participation in job search activities, with employment-focused programs increasing these rates by nearly tenfold over control group levels of less than 5 percent. In addition, all three education-focused programs at least doubled participation in basic education activities, with Atlanta's education-focused program quadrupling this rate from 5.3 to 21.9, and Riverside's education-focused program tripling this rate from 15.7 to 51.8.
• The three employment-focused programs generally did not alter, or even decreased, the likelihood of obtaining a high school diploma, GED, or trade certificate. Riverside's employment-focused program did not affect degree receipt either in the short-run or in the long-run. Grand Rapids' employment-focused program decreased degree receipt, by 11 percentage points, both early and later on. Atlanta's employment-focused program was the only one of these six JOBS programs to increase the proportion of mothers receiving a trade degree (from 5.9 to 8.3 percent) by the two year point  though even this program had no longer-term impacts on the likelihood of obtaining an educational credential.
• Some, but not all, employment-focused programs increased employment over the five-year period, and one education-focused program did. Atlanta's employment-focused program only slightly increased employment in the month prior to the two-year survey but showed no longer-term employment impacts. By contrast, Grand Rapids' employment-focused program and each of Riverside's programs increased the proportion of mothers reporting any employment early on (in the first two years of follow-up), as well as averaged over the full five-year follow-up period. Employment impacts were sustained in these three programs; a greater percent of these program than control group mothers were employed in each quarter in the last year of follow-up. In Riverside, these findings also held for the subgroup of mothers who were in need of basic education at study entry.
• All three education-focused programs increased degree receipt by the two-year point, though only two of these programs had long-term impacts on mothers' educational attainment. In the short run, mothers in each site's education-focused programs were more likely than mothers in the respective control groups to obtain a degree. However, these impacts faded in Grand Rapids by the fifth year, where a sizable minority of control group mothers had received an educational credential during the follow-up period.
• Impacts on welfare receipt varied over time and across programs. Early on, only Atlanta's programs and the employment-focused program in Riverside reduced the likelihood of receiving welfare in the month prior to the two-year survey. However, these welfare impacts faded in Atlanta. Longer-term reductions in welfare were concentrated in three programs: Grand Rapids' employment-focused program, and both programs in Riverside, reduced time on welfare and total welfare payments averaged over the entire five-year follow-up period. Findings held for Riverside's "in-need" subgroup as well. Notably, these are the same three programs that increased employment averaged over the entire follow-up period.
• Only Riverside's programs increased total earnings, both early on and averaged over the entire follow-up period. This is perhaps because they were the only programs to increase the average hourly wage and average hours worked at the two-year point; they were also the only two programs to increase the proportion of mothers employed full-time at the five-year interview. Both programs in Riverside increased full-time employment and cumulative earnings for educationally disadvantaged mothers as well.
• Impacts on income and poverty were even more limited, and sometimes unfavorable. None of these six JOBS programs affected  neither increased nor decreased  total income, on average, measured in the month prior to the two-year survey.⁽¹⁶⁾ Similarly, none of these programs affected income accrued in years 1 to 5 to these families with young children. Nevertheless, four programs affected families' poverty status at the two-year point, with two programs decreasing poverty and two programs increasing poverty. For example, Riverside's education-focused program decreased the proportion in poverty measured the month prior to the two-year survey, from 79 to 73 percent. In addition, Atlanta's employment-focused program reduced the proportion in "deep poverty" (incomes at less than half the poverty line) in the prior month, from 19 to 15 percent. At the same time, about 70 percent of mothers in Grand Rapids' control group had income in the month prior to the two-year survey placing them below the poverty line in that month, and each of Grand Rapids' programs increased this likelihood by about 5 percent. Notably, even programs that reduced poverty still left a majority of families below the poverty line at the two-year point.

In sum, while some welfare recipients with young children were able to make educational gains and economic progress without the benefit of JOBS, each of these six JOBS programs was able to improve upon this progress in one way or another in the short run. Four programs increased educational attainment early on; three years later, only two education-focused programs continued to show positive impacts on education. Only three programs had sustained increases in employment over the five-year period; two of these programs had sustained increases in earnings. However, improvements in income, either early or later on, remained elusive.

The Role of Family Risk

The Child Outcomes Study also sought to determine whether impacts on children varied according to initial levels of family risk.⁽¹⁷⁾ The Family Support Act sought explicitly to reduce long-term welfare dependency by providing the services necessary to move long-term recipients into jobs. Many argued that the opportunities that JOBS provided would be more beneficial for the most disadvantaged participants, thereby improving outcomes for children. Others feared that higher-risk participants might not be able to meet JOBS requirements, would face sanctions, and they and their children would suffer.

As already noted, these programs had few impacts on young children, which was true for children from families at lower initial risk as well as for children from families at higher initial risk. When impacts on children were found at the two-year point, they tended to be favorable for children in higher-risk families (in four programs)⁽¹⁸⁾ and unfavorable for children in lower-risk families (in three programs).⁽¹⁹⁾ In fact, some of the largest impacts on young children to occur at the two-year point were among these unfavorable impacts for children in lower-risk families assigned to one of these three programs.

At the five-year point, only programs in Riverside showed a pattern of favorable impacts on children of more disadvantaged mothers (i.e., those who lacked a high school diploma at study entry), serving to improve behavior. At the same time, however, Riverside's employment-focused program continued to have unfavorable effects on children in lower-risk families: This program lowered the academic performance, lessened school engagement, and increased problem behaviors of children of the "least disadvantaged" mothers.⁽²⁰⁾ In addition, at the five-year point, there emerged a concentration of unfavorable impacts on children of the least disadvantaged mothers in the Atlanta LFA program.⁽²¹⁾ These impacts were not widespread, however, but they did include unfavorable impacts on academic progress and placement, a decline in positive behaviors, and an increased likelihood of needing but not receiving special services.

The Role of Site

Even though the Family Support Act provided clear guidance on the kinds of education-focused and employment-focused services that all JOBS programs had to provide, it also gave states wide latitude in designing and implementing their programs. As a result, states' JOBS programs varied widely. In addition, the sites in which these programs were implemented varied  geographically, economically, demographically, and in the policies, practices, and ethos of the local welfare office. And, interestingly, program impacts on children varied more by site than by program approach. In fact, the same program approach  the employment-focused approach  produced favorable impacts on school-age children in one site (Atlanta) and unfavorable impacts on school-age children in another site (Grand Rapids). It may be that the relatively large caseload per case worker in Grand Rapids made it more difficult to give personal attention to clients, whereas the smaller caseloads in Atlanta, along with this site's "customer-oriented" approach to case management, facilitated greater personal attention to clients. Such attention, coupled with JOBS program services, may have amounted to a more positive experience for program mothers in Atlanta than program mothers in Grand Rapids, spilling over positively to their children.

It is also possible that the different pattern of impacts on children in Atlanta and Grand Rapids is related to differences in these sites' caseload characteristics. The Child Outcomes Study was the first to note a pattern of unfavorable impacts on children in lower-risk families  a pattern that has since been found in other welfare evaluations.⁽²²⁾ Child Outcomes Study families in Grand Rapids may have been less disadvantaged at study entry, on average, than Child Outcomes Study families in Atlanta. For example, compared to their counterparts in the Atlanta site at study entry, mothers in the Grand Rapids site were less likely to have three or more children, less likely to be living in public or subsidized housing, more likely to have ever been married, more likely to have "higher" literacy, and were more likely to have had earnings in the past 12 months. Though the mechanisms are not yet clear, it would appear that the context in which Atlanta's and Grand Rapids' employment-focused programs were implemented helped to shape impacts on children.

Why Were Impacts on Children So Limited?

Impacts on young children were infrequent, suggesting that while impacts on young children can occur, they most often do not. It may be that impacts on outcomes important to children  such as stable maternal employment, adequate family income, and sufficiently supportive environments experienced in children's daily lives (home, child care, school)  were too few, occurred for too brief a period, or were of an insufficient magnitude to lead to large, widespread impacts on young children.

In addition, impacts of a given JOBS program on children's environments may have occurred in opposite directions. For example, Atlanta's employment-focused program increased time stress reported by mothers at the two-year point (which predicted greater problem behaviors in children), but this program also increased employment and improved mothers' parenting (which predicted fewer problem behaviors). The net effect for children was slightly positive, that is, a reduction in behavior problems. In general, when program impacts on children's environments occur in countervailing directions, overall impacts on children may be small or non-existent.

What Explains the Few Impacts that Did Occur?

Despite the absence of far-reaching impacts on children in either the short-run or longer-term, the evidence for young children suggests that mandatory welfare-to-work programs aimed at adults can have "spillover" effects on children. Understanding the mechanisms that carry program impacts to children can inform policy makers about how to bolster positive impacts on young children and buffer against potentially negative impacts. So how did these impacts on young children come about? Unfortunately, the answers are not straightforward. As suggested above, there does not appear to be a single pathway through which children were affected by each of these six JOBS programs.

There is some evidence that program impacts on mothers' psychological well-being and parenting may underlie impacts on children in some cases. For example, a statistical examination of factors explaining selected impacts on children at the two-year point revealed that improvements in parenting were related to improvements in young children's academic school readiness and behavior in Atlanta's employment-focused program. In addition, the program-induced increase in maternal depressive symptomatology and reduction in warm parenting in Grand Rapids' employment-focused program was related to the increase in young children's externalizing behaviors in this program.

There is some evidence that program impacts on maternal education may underlie impacts on children. When programs affected maternal education at either the two-year or five-year point, children's outcomes were often affected in the same direction. This pattern was especially strong for higher-risk families at the two-year point. Four programs increased (relative to the control group) the proportion of mothers receiving either a high school education or GED or obtaining a trade degree by the two-year follow-up, and these were the same four programs having predominantly favorable impacts on children at the two-year point.⁽²³⁾

It is important to note that impacts on young children did not likely come about as a result of program-induced changes in total income, since none of these six JOBS programs altered  at either the two-year of five-year point  total income, on average, in the Child Outcomes Study sample.⁽²⁴⁾ In addition, there is little evidence that impacts on children resulted from contemporaneous changes in families' poverty status, employment, earnings, or child care.⁽²⁵⁾ Nevertheless, the possibility exists that earlier program impacts on key adult and family outcomes led to later impacts on children. For example, it is worth noting that the pattern of impacts on poverty (in four programs) at the two-year point is consistent in direction with the pattern of impacts found for these program children at the five-year point. Specifically, both programs in Grand Rapids increased poverty in the month prior to the two-year survey and worsened child outcomes at both the two- and five-year follow-ups. The Riverside education-focused program decreased poverty, and the Atlanta employment-focused program decreased "deep" poverty, at the time of the two-year survey, and the pattern of impacts for children in these programs was largely favorable three years later. This pattern is consistent with research suggesting that childrens early poverty experiences may be more important for their developmental outcomes than later poverty experiences.⁽²⁶⁾

In sum, there is no single answer to the question of what led to impacts on children. It is likely the result of multiple factors  some of which, like education and employment, were targeted by these programs, others, like income and parenting, were not. In addition, the ways in which a given welfare program affected children may not be the same as the ways in which a different welfare program affected children. And despite the in-depth information collected on children's environments, it is possible that the evaluation failed to measure other key outcomes that served as the conduits through which young children were affected by their mothers' enrollment in one of these JOBS program. Findings from the Child Outcomes Study suggest that characteristics of the research site  including characteristics of the population served, local economic conditions, welfare benefit levels, and the ethos and practices of the local welfare office  may be important for understanding impacts on children and families.

Implications for Policy and Programs

For the past 30 years, federal and state policy makers have sought ways to decrease long-term welfare receipt and increase employment among welfare recipients. The work requirements of the current federal welfare law, the Personal Responsibility and Work Opportunity Reconciliation Act, had their origins in earlier welfare policies, including the Family Support Act's JOBS Program. The Family Support Act and the JOBS Program introduced key features that are still major elements in today's welfare policies and programs  most notably, the social contract that requires welfare recipients to prepare for and secure employment as a condition of receiving public assistance, or face financial repercussions. Consequently, findings pertaining to the short- and long-term impacts of these earlier mandatory welfare-to-work approaches as revealed by the National Evaluation of Welfare-to-Work Strategies can be informative in today's policy context.

This groundbreaking multi-site study of the long-term impacts of mandatory welfare-to-work programs found few effects on young children of single mothers assigned to one of these JOBS programs. In addition, neither the employment- or education-focused programs emerged as more beneficial or detrimental to children. Nevertheless, impacts were found, indicating that policies seeking to increase employment among low-income single mothers can, in fact, affect their young children.

Findings may also be informative given the increasing devolution of welfare policy to the state and local levels. Local welfare policies, local economic conditions, characteristics of the population served, and the practices and ethos of the welfare office, constitute the context in which these JOBS programs were implemented and, thus, likely shaped the impacts of these programs on both adults and children. These same factors are likely to play a role in shaping impacts of today's even more diverse TANF programs. When states have even greater latitude in designing their welfare-to-work programs, policy makers and program operators will increasingly want to know "What works for whom, under what circumstances?"

It is important to keep in mind that, even when they were favorably affected by these programs, young children still remained at risk for problem outcomes, especially pertaining to academic achievement and school progress. In sum, the results from the Child Outcomes Study provide important information on the well-being of low-income children and how they can be affected by welfare reform, which could be informative as welfare programs and polices are developed at the federal, state, and local levels.

References

Freedman, S., Friedlander, D., Hamilton, G., Rock, J., Mitchell, M., Nudelman, J., Scheweder, A., & Storto, L, (2000). National Evaluation of Welfare-to-Work Strategies: Evaluating Alternative Welfare-to-Work Approaches: Two-Year Impacts for Eleven Programs. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

Hamilton (2002). Lessons from the NEWWS Evaluation: Moving People from Welfare to Work. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

Hamilton, G., Freedman, S., Gennetian, L., Michalopoulos, C., Walter, J., Adams-Ciardullo, D., Gassman-Pines, A., McGroder, S., Zaslow, M., Brooks, J., & Ahluwalia, S. (2001). National Evaluation of Welfare-to-Work Strategies: How effective are different welfare-to-work approaches? Five-year adult and child impacts for eleven programs. Executive Summary. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

Hamilton, G., Freedman, S., Gennetian, L., Michalopoulos, C., Walter, J., Adams-Ciardullo, D., Gassman-Pines, A., McGroder, S., Zaslow, M., Brooks, J., & Ahluwalia, S. (2001). National Evaluation of Welfare-to-Work Strategies: How effective are different welfare-to-work approaches? Five-year adult and child impacts for eleven programs. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

McGroder, S.M., Zaslow, M.J., Moore, K.A., & LeMenestrel, S.M. (2000). The National Evaluation of Welfare-to-Work Strategies: Impacts on young children and their families two years after enrollment: Findings from the Child Outcomes Study. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

Moore, K., Zaslow, M., Coiro, M.J., Miller, S.M., & Magenheim, E.B. (1995). How Well Are They Faring? AFDC Families with Preschool-Age Children at the Outset of the JOBS Evaluation. [Link is to Executive Summary] Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

Zaslow, M.J., McGroder, S.M., & Moore, K.A. (2000). The National Evaluation of Welfare-to-Work Strategies: Impacts on young children and their families two years after enrollment: Findings from the Child Outcomes Study. Summary Report. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. Department of Education.

Endnotes

1.  The Manpower Demonstration Research Corporation (MDRC) conducted the NEWWS under a contract with the Department of Health and Human Services (HHS), funded by HHS under competitive award # HHS100-89-0030. HHS also received funding for the evaluation from the U.S. Department of Education. Child Trends, as a subcontractor to MDRC, conducted the Child Outcomes Study. Additional funding for the Child Outcomes Study came from the Foundation for Child Development and the William T. Grant Foundation.

2.  Employment-focused programs were operated in Atlanta, GA; Grand Rapids, MI; Riverside, CA; and Portland, OR. Atlanta, Grand Rapids, and Riverside also operated education-focused programs, as did Columbus, OH (which operated two programs varying in the structure of case management); Detroit, MI; and Oklahoma City, OK.

3.  See Hamilton et al. (2001) and Hamilton (2002) for greater detail.

4.  Data from the NEWWS, including the Child Outcomes Study, are available free as public use files from the following HHS website: http://aspe.hhs.gov/hsp/NEWWS/. Restricted access files, which contain some additional information, are available for a nominal fee and must be used on site at the National Center for Health Statistics.

5.  This average vocabulary score for this subsample of Child Outcomes Study children is two standard deviations below this average score for the national sample. In other words, whereas about half of the Child Outcomes Study subsample scored 70 or lower, only about 3 percent of the 3- to 5-year-olds in the national sample scored this low.

6.  "Limiting condition" was defined as a "handicap, illness, emotional problem, or mental condition that limits his/her ability to attend school, to exercise or participate in sports, or that requires special medical equipment". Parents with severely ill or disabled children were generally not mandated to participate in welfare-to-work programs in the early to mid-1990s; as a result, such families were not included in the NEWWS samples. Their exclusion, however, is unlikely to have affected very much the overall level of assessed health for the children in the control groups. Data available from the NEWWS suggest that fewer than 3 percent of the exemptions from participation were granted due to children's severe health problems.

7.  About three-quarters of mothers in the control groups rated the young child as being in "very good" or "excellent" health at both the two- and five-year follow-up.

8.  The average score for Child Outcomes Study control group children was one full standard deviation below that of 5- to 7-year-old children in the national sample used to standardize the Bracken Basic Concepts Scale/School Readiness Composite. See Bracken, B.A. (1984). Bracken Basic Concepts Scale: Examiner's Manual. The Psychological Corporation, Harcourt Brace Jovanovich, Inc.

9.  Zaslow, M.J., and McGroder, S.M. (April, 1999). Behavior Problems and Cognitive School Readiness Among Children in Families with a History of Welfare Receipt: Diverging Patterns and their Predictors. Presented at the biennial meeting of the Society for Research in Child Development, held in Albuquerque, NM, April 15-18, 1999.

10.  Children were administered the Woodcock-Johnson Revised Tests of Achievement (Woodcock & Johnson, 1989; 1990) at the five-year follow-up. Information on the achievement test scores of a national sample of children permits comparisons to other samples of children. Scores between 90 and 110, representing the 25^th and 75^th percentiles, respectively, are considered "average" or "normal" (Woodcock & Mather, 1989, 1990). Thirty-four and 37 percent of control group children in Riverside and Atlanta (respectively) scored in the "below-average" range on the Woodcock Johnson Broad Reading Score. Only about 17 and 20 percent (in Atlanta and Riverside, respectively) scored in the "above-average" range on this measure.

11.  Program-control group differences on an outcome that occurred with a probability of less than 10 percent were considered "statistically significant". That is, we posited that a difference of this magnitude did not occur by chance, and we concluded that this program did, in fact, have an impact on this outcome.

12.  Earlier cognitive impacts in Atlanta did not exceed one-fifth of a standard deviation in size. Later impacts on reading level, discipline problems, and social skills and behavior were between one-quarter and one-third of a standard deviation in size. Five-year impacts of Atlanta's programs on absenteeism and/or tardiness were larger, approaching one-half of a standard deviation in size.

13.  The unfavorable impacts of Grand Rapids' education-focused program on behavior at the five-year point were typically between one-quarter and one-third of a standard deviation.

14.  Both favorable and unfavorable behavioral and academic impacts of Riverside's programs at the five-year point generally ranged from one-quarter to one-third of a standard deviation in size.

15.  Among children whose mothers lacked a high school diploma at study entry, Riverside's employment-focused program increased math scores by almost one-half a standard deviation, and Riverside's education-focused program increased absenteeism by almost one-half a standard deviation.

16.  "Total net adjusted household income" adds up to 15 sources of income from all household members, an estimated amount of the Federal earned income tax credit, and subtracts out estimated child care costs. See Freedman et al., 2000, p. 105, for details.

17.  Though "family risk" was defined differently at the two-year and five- year follow-ups, each definition took into account the number and/or severity of such factors as work history, time on welfare, maternal educational attainment, maternal literacy, number and spacing of children it he family, and maternal depressive symptoms. Findings summarized here reflect the pattern of findings for children in "lower-risk" and children in "higher-risk" families, variously defined.

18.  These were each of the three education-focused programs and Atlanta's employment-focused program.

19.  These were the employment-focused program in Grand Rapids and both programs in Riverside.

20.  Mothers who lacked a high-school diploma at study entry, had not worked in the year prior to study entry, and who had been receiving welfare for at least two years before the study were defined as the "most disadvantaged". Mothers with one or two of these barriers were considered "moderately disadvantaged," and mothers with none of these barriers were considered "least disadvantaged".

21.  Though samples sizes for the least disadvantaged subgroups in these programs were relatively small, impacts for this subgroup are being reported cautiously because they were relatively numerous and they formed a pattern (i.e., they were consistently unfavorable).

22.  See Knox, V., Miller, C., & Gennetian, L.A. (2000). Reforming Welfare and Rewarding Work: Final Report on the Minnesota Family Investment Program. New York: Manpower Demonstration Research Corporation; Bloom, D., Kemple, J.J., Morris, P., Scrivener, S., Verma, N., & Hendra, R. (2001). The Family Transition Program: Final Report on Florida's Initial Time-Limited Welfare Program. New York: Manpower Demonstration Research Corporation. See review in Zaslow, M.J., Brooks, J.L., Moore, K.A., Morris, P., Tout, K., & Redd, Z. (2001). Impacts on children in experimental studies of welfare-to-work programs. Report to the Edna McConnell Clark and David and Lucille Packard Foundations; and Zaslow, M.J., Moore, K.A., Brooks, J.L., Morris, P., Tout, K., Redd,, Z., & Emig, C. (2002). Experimental studies of welfare reform and children. Future of Children, Vol 12 (1), 79-95.

23.  A recent study using two-year Child Outcomes Study data found that the number of months mothers participated in basic education activities predicted a greater readiness for school and fewer academic problems in their young children. Magnuson, K. & McGroder, S. M. The effect of increasing welfare mothers' education on their young children's academic problems and school readiness. Submitted to the Journal of Public Policy and Management.

24.  Emerging evidence suggests that young children appear to benefit from welfare-to-work programs only when both parental employment and family income are improved. Three recent syntheses of findings from evaluations of numerous welfare-to-work and anti-poverty programs found that total income can be affected by programs containing strong financial incentives for work (such as enhanced earned income disregards). Morris, P.A., Huston, A.C., Duncan, G.J., Crosby, D.A., & Bos, J.M. (2001). How welfare-to-work policies affect children: A synthesis of research. New York: Manpower Demonstration Research Corporation. Sherman, A. (2001). How children fare in welfare experiments appears to hinge on income. Washington, D.C.: Children's Defense Fund. Zaslow et al. (2001). Zaslow et al. (2002). See also Duncan, G.J., & Chase-Lansdale, P.L. (2001). Welfare reform and child well-being. In R.M. Blank & R. Haskins (Eds.), The New World of Welfare (pp. 391-412). Washington, D.C.: Brookings Institution.

25.  Failure of adult and child impacts to map at the aggregate level does not rule out the possibility that these adult impacts operate as pathways for certain subgroups of families.

2.  Duncan, G. J., & Brooks-Gunn, J., Ed. (1997). Consequences of growing up poor. New York, Russell Sage Foundation.

How to Obtain a Printed Copy

To obtain a printed copy of this report, send or fax the title and your mailing address to:

Human Services Policy, Room 404E
Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Av, SW
Washington, DC 20201
Fax: (202) 690-6562