How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Discussion and Conclusions

12/01/2001

The major goal of the Child Outcomes Study was to determine whether mandated participation in a welfare-to-work program could have implications for enrollees' children  particularly preschool-age children whose mothers were newly required by the Family Support Act of 1988 to engage in work preparation activities as a condition of receiving welfare. Some policymakers feared that requiring mothers of preschool-age children to secure employment would harm their children; others argued that the economic benefits of employment that would accrue to these families would benefit children. Findings from the COS indicate that there were relatively few long-term impacts of the JOBS programs evaluated in NEWWS on children's developmental outcomes. The lack of pervasive impacts on children is perhaps not surprising, given that the JOBS program was not aimed at children. It may be that impacts on outcomes important to children, such as mothers' employment, family income, and/or the children's immediate environments (home, school, child care), were too few, occurred for too brief a period, or were of an insufficient magnitude to lead to consistent, enduring impacts on young children. It is also possible that impacts at the five-year point are understated in Atlanta and Grand Rapids because some control group members in these sites became eligible for welfare-to-work program services in years 4 or 5 of the follow-up period.

However, the number of impacts found exceeds chance levels, and there was a discernible pattern of impacts on young children.(28) At the five-year follow-up, all six JOBS programs examined affected children's social skills and behavior. Both Atlanta programs increased positive behaviors, and decreased problem behaviors; both Grand Rapids programs decreased positive behaviors, and the HCD program increased problem behaviors; and the Riverside LFA program increased problem behaviors in the full sample.(29)(30) For young children whose mothers lacked a high school diploma or basic skills at study entry, both programs in Riverside increased positive behaviors. Changes in problem behaviors may be especially important; research has shown that a history of antisocial behavior in childhood  which is partially measured by the problem behaviors examined in this study  is the strongest risk factor for chronic delinquency among adolescents.(31)

Impacts on young children's academic achievement were also relatively rare, but findings relating to behavioral adjustment to school (school engagement, disciplinary problems) were in accord with the patterns described above, with uniformly favorable impacts in Atlanta and uniformly unfavorable impacts in Grand Rapids. Impacts relating to how children were performing academically in school were scarce; however, when impacts were found on these outcomes and others, they were at the lower end of the effect size range for early intervention programs aimed directly at children. These findings suggest that welfare-to-work programs can affect children's performance in school, albeit not as frequently nor as systematically as they appear to affect children's behavior.

Finally, there were also few impacts in the domain of health and safety; however, they were all unfavorable, occurred in four of the six programs,(32) and tended to occur in programs that also increased children's school absenteeism and/or tardiness. In many cases, these unfavorable health impacts were being driven by the unfavorable impacts for boys, with girls' health left largely unaffected by these programs.

A second goal of the COS was to determine whether program impacts on young children of enrollees differed according to the welfare-to-work strategy employed  specifically, a Labor Force Attachment or a Human Capital Development strategy. Some believed that increases in mothers' educational attainment resulting from participation in an HCD program would bode particularly well for children  particularly for their school success  even if such participation did not lead immediately or ultimately to increased employment and earnings. Others argued that the quicker that mothers secured employment (through the LFA approach), the quicker that financial benefits would accrue to children. However, contrary to these initial hypotheses, the welfare-to-work strategy employed did not consistently produce different impacts on children at the five-year follow-up.(33) In direct comparisons of LFA and HCD programs, neither approach emerged as uniformly better or worse for children.

Did five-year impacts on children vary according to initial levels of family risk? The Family Support Act sought explicitly to reduce long-term welfare dependency by providing the services necessary to move long-term recipients into jobs. Many argued that the opportunities that JOBS provided would be more beneficial for the most disadvantaged participants (five-year impact findings for adults discussed in Chapter 7 support this hypothesis), thereby improving outcomes for children. Others feared that higher-risk participants might not be able to mobilize to meet JOBS requirements and would face sanctions, and they and their children would subsequently suffer. Contrary to these expectations, findings from the five-year follow-up do not show a discernible pattern of favorable or unfavorable impacts  either across programs or within a single program  for children of the "most disadvantaged" mothers.(34) Thus, children aged 8 to 10 in these higher-risk families do not appear to have been consistently helped or hurt by their mothers' enrollment in the LFA and HCD programs. Rather  and somewhat unexpectedly  there emerged a concentration of unfavorable impacts on children of the least disadvantaged mothers in the Riverside LFA program.(35) Interestingly, the Riverside LFA program unfavorably affected the same problem behavior and discipline outcomes for girls, which begs the question of whether these unfavorable impacts were concentrated among the least disadvantaged girls.(36)

In the Riverside LFA program, the unfavorable impacts on the academic performance, school engagement, and problem behaviors of focal children with the least disadvantaged mothers found at the five-year point are consistent with those found at the two-year point: increased academic problems, emotional problems, and suspensions or expulsions of children in lower-risk families. Other than these impacts, the JOBS programs examined in COS did not appear to have a similar pattern of short- and long-term impacts on children from families varying in initial risk. The only other pattern of findings to emerge by family risk level at the two-year follow-up(37)(38)  a pattern of favorable (though relatively small) impacts on children in higher-risk families in all HCD programs and in the Atlanta LFA program  did not emerge at the five-year point for children with the most disadvantaged mothers.

What led to the program impacts on children at the five-year point? It is worthwhile to note what did not lead to these impacts. Recent reviews of experimental findings for children indicate that, when welfare-to-work programs increase both maternal employment and family income, children often benefit.(39) However, because none of the six JOBS programs examined here affected the income of COS sample families, income cannot be a pathway through which children were affected. Also, use of child care does not appear to underlie program impacts on children.

There is some evidence from a nonstatistical "mapping" of adult and child impacts at the five-year point that maternal education may have been an important pathway through which some children were affected by their mothers' enrollment in an LFA or HCD program. Three of these programs affected both maternal education and child outcomes in the same direction; in the other three programs, though maternal education was not affected, children were. Thus, changes in mothers' education may not necessarily lead to changes in child outcomes, but when mothers' education is affected, it may have implications for their children. This finding is consistent with results showing that increased participation in educational activities by mothers in these three COS sites predicted greater academic school readiness in their children.(40)

In addition to examining the pattern of impacts across programs, it is important to consider impacts specific to a particular program for identifying possible pathways through which children were affected by that program. For example, the Riverside LFA program reduced time spent in child care activities after school and decreased the use of formal child care as a regular arrangement. Given some emerging evidence on the potential benefits of more formal child care arrangements on children,(41) the reduction in the use of formal care may be related to the pattern of unfavorable impacts of this program on these young children. In addition, research has shown a negative association between family residential moves and children's behavior.(42) This may help to explain the pattern of unfavorable impacts in Grand Rapids, given that both the LFA and the HCD programs in this site increased the likelihood that families had moved since study entry and also had unfavorable impacts on social skills and behavior among children aged 8 to 10. In short, because program impacts on children represent the net effect of all impacts on outcomes important to their development and well-being, and because impacts on these "intervening" outcomes may differ in different programs, future research needs to examine pathways for each program separately and needs to consider, simultaneously, all the intervening outcomes that were affected by each program.(43)

What is clear is that the sites in which these six JOBS programs were implemented varied  geographically, economically, demographically (including the racial/ethnic composition of the caseloads), and in the policies, practices, and ethos of the local welfare offices.(44) Differences in program implementation may also help to explain the pattern of impacts. For example, though the Atlanta site successfully implemented distinct LFA and HCD models (in terms of the content and sequence of services), a large proportion of both HCD and LFA case managers in this site believed that an education-oriented welfare-to-work strategy was the best means of leaving welfare.(45)(46) In addition, case managers in the Atlanta site viewed their roles as very customer-oriented; they actively sought out necessary support services for their clients and were less strict about monitoring compliance. It may be that the policies, practices, and ethos of the Atlanta welfare office "fit" well with the needs of its female clients to balance both work and family responsibilities and that this context helps to explain the beneficial results for children in both the LFA and the HCD programs.

Although a national program, the JOBS program was implemented locally  in sites that differed in economic conditions, the population served, and in the ethos and practices of the welfare offices. These conditions, though not included in the statistical analyses reported on here, shape the way the JOBS programs were implemented in each site and, thus, can shape the impacts that these programs have on targeted and nontargeted adult outcomes and on child outcomes in as yet unknown ways. Future research on the effectiveness of welfare-to-work programs  for both generations  may need to focus increasingly on identifying what works for whom, and under what circumstances. Such research would be informative to state and local policymakers and program planners who increasingly bear the responsibility, in this age of devolution, to design and effectively target welfare-to-work programs.

Endnotes

1.  Although focal children were aged 8 and 10 at the five-year follow-up, they are characterized as "young" children because they were preschool-age at random assignment.

2.  COS families consisted solely of single mothers with preschool-age children at random assignment; the larger samples included married couples and families not necessarily with a preschool-age child but with children of any age. COS mothers were somewhat younger, less likely to have been married, more likely to have a high school diploma or GED, and they had a slightly higher number of children on average than parents in the full evaluation sample. In addition, in Atlanta and Riverside, COS mothers were less likely to have ever worked full time for six months or more for the same employer at baseline. See Hamilton, 2000, p. 11.

3.  A more detailed analysis of program impacts on boys and girls at both the two-year and the five-year follow-up is beyond the scope of this report and will be done as part of future analyses.

4.  Gresham and Elliot, 1990, p. 1.

5.  Woodcock and Johnson, 1989, 1990.

6.  As noted in Chapter 2, the sample for the analyses of achievement test scores and of the mother- and child-reported child outcomes comprises 2,332 mothers and their focal children, and the sample for the analyses of teacher-reported child outcomes includes responses from 1,472 teachers of focal children in the final mother and child survey sample.

7.  Achenbach, 1991.

8.  Admittedly, even objective assessments of childrens' academic skills may contain some cultural biases.

9.  Interestingly, the favorable impacts on behavior in the Atlanta programs were largely concentrated among girls, with girls showing fewer problem behaviors and more positive behaviors in both programs and, in the HCD program, fewer disciplinary problems as well.

10.  There was a single finding that did not fit with this overall pattern of favorable impacts on social skills in the Atlanta programs: The LFA program decreased child-reported cooperation, though this difference was just beyond the cutoff for statistical significance (p = .11).

11.  Although the difference for teacher-reported cooperation in the Grand Rapids HCD program was just beyond the cutoff for statistical significance, it was of a size comparable to similar impacts in other programs that were statistically significant.

12.  The impacts on behavior in the Riverside LFA program occurred especially (or, in many cases, only) for the least disadvantaged subgroup and were part of a larger picture of unfavorable impacts for this subgroup that extended past behavior into teachers' reports of academic progress and placement. Further, the increases in problem behavior in this program were concentrated largely among girls and were accompanied by an increase in disciplinary problems among girls.

13.  Though the latter two differences were above the cutoff for statistical significance, they were both of a size comparable to impacts in other programs that were statistically significant.

14.  Berrueta-Clement et al., 1984; Campbell and Ramey, 1994.

15.  Although relatively little is known about predictors and consequences of positive development in children and youth, there are some indications that positive development can lead to more successful future functioning. See Moore and Glei, 1995.

16.  Gresham and Elliot, 1990, p. 1. See, for example, Coie and Dodge, 1983; Cowen et al., 1973; Parker and Asher, 1987.

17.  Farrington, 1987, as cited in Yoshikawa, 1995.

18.  Scores on that fall between 90 and 110, representing the 25th to the 75th percentiles, respectively, are classified as "average" or "normal" (Woodcock and Mather, 1989, 1990).

19.  Nevertheless, there were impacts on mean math scores within gender subgroups in both Riverside programs. Both programs increased mean math scores for boys whose mothers lacked a high school diploma or GED at baseline. Additionally, the Riverside HCD program also increased boys' mean reading scores and, interestingly, decreased girls' mean math and mean reading scores.

20.  While the impacts of the Riverside LFA program may seem inconsistent (simultaneously decreasing the number of children repeating a grade and increasing the number in a remedial reading group), these impacts occurred on experiences that apply to only a minority of the sample and thus may not pertain to the same children. In other words, the children who were in a remedial reading group may not be the same individuals who repeated a grade. Alternatively, it may be that teachers' evaluations of children's reading and math skills are highly dependent on children's classroom behavior, in which case the impact on being in a remedial reading group may reflect the behavioral impacts of this program more so than its impacts on measures of academic progress or placement. This speculation is supported by a set of findings for the least disadvantaged subgroup, in which there is a general pattern of unfavorable impacts on teachers' reports of behavior. At the same time, however, there is a single finding that appears to diverge from this pattern. Program group children of the least disadvantaged mothers are more likely to score above average on the reading test than controls yet are less likely to be rated as above grade level in reading by their teachers. This suggests that teachers' evaluations of children's performance in reading incorporate more than the children's actual reading skills (which, in fact, increased for program group children). For example, teachers perceiving less school engagement and more disciplinary problems in children (which was more true for the Grand Rapids HCD program group than for the control group) may have difficulty accurately gauging a child's reading skills if the child is having difficulty remaining seated or following instructions during reading class. Likewise, even among children with identical capabilities in reading, a teacher might be less likely to move a child into a higher-level reading group (and/or more likely to move him or her into a lower-level group) if the child tends to be disruptive to other students in the class.

21.  The increase in absenteeism in the Atlanta LFA program occurred especially among the least disadvantaged subgroup, a finding that was part of a broader pattern of unfavorable impacts for this subgroup.

22.  Adams and Marano, 1995. It may be surprising that mothers in the COS sample reported higher health ratings for their children aged 8 to 10 than were reported for both all children aged 5 to 17 and those from low-income families in a national sample. However, mothers were exempt from participation in JOBS welfare-to-work activities if they were needed at home to care for an ill or incapacitated family member, including a child. Consequently, the COS sample of children is relatively healthy, whereas national samples of children would include some severely and chronically ill children.

23.  The pattern of unfavorable health impacts, including some new impacts in subgroups that were masked in the aggregate, was found to be largely concentrated among boys.

24.  The adult economic impacts discussed in this section pertain to adults in the COS. See Appendix I for detailed tables of these impacts. The smaller sample sizes of the COS at times led differences that were of a similar magnitude to significant impacts in the larger evaluation samples to be nonsignificant in the COS sample. Hence, this discussion reports on differences in the COS that were either statistically significant or of a similar or greater size than a significant difference in the larger sample of adults within a given program.

25.  See Duncan and Magnuson, 2001.

26.  Duncan and Brooks-Gunn, 1997.

27.  Duncan and Brooks-Gunn, 1997.

28.  Impacts on young children two years after enrollment (when focal children were aged approximately 5 to 7) tended to vary by developmental domain, with only favorable aggregate impacts in the cognitive domain and only unfavorable impacts in the health and safety domain. Impacts on young children's behavior and socioemotional functioning were both favorable and unfavorable. See McGroder et al., 2000; and Zaslow, McGroder, and Moore, 2000.

29.  These patterns of impacts for young children do not match findings discussed in Chapter 11, as the client survey sample respondents did not provide any information on social skills and behavior for all of their children. Only mothers in the COS provided this information for the focal children, as presented in this chapter, and it is precisely this domain of child outcomes most consistently affected by these programs.

30.  As stated previously, the favorable impacts of the Atlanta programs on young children's positive and problem behaviors were especially pronounced for girls. At the same time, the unfavorable impacts of the Riverside LFA program on young children's problem behaviors were also especially pronounced for girls.

31.  Farrington, 1987, as cited in Yoshikawa, 1995.

32.  Interestingly, as reported in Chapter 11, favorable impacts were found for preschool-age children on whether any child had a condition that impeded on the mother's ability to go to work or school in the Riverside LFA program (for both the full sample and for those in need). Approximately two-thirds of the children in this age group were included in the COS sample presented in this chapter. As shown in Table 12.4, though, there were no favorable health impacts on this measure for the Riverside LFA program in the COS sample. However, the difference in the means between the program and control groups did go in the same direction. Perhaps the larger size of the client survey sample (and, thus, smaller standard errors) accounts for the difference in the presence of impacts.

33.  Similarly, impacts on young children at the two-year follow-up did not vary consistently by welfare-to-work strategy employed. See Hamilton, 2000; McGroder et al., 2000; and Zaslow, McGroder, and Moore, 2000.

34.  The most disadvantaged mothers did not have a high school diploma as of random assignment, did not work in the year prior to random assignment, and had been on welfare for at least two years as of random assignment. The least disadvantaged mothers had none of these barriers, and the moderately disadvantaged mothers had one or two of these barriers. See Chapter 7.

35.  Although samples sizes for the least disadvantaged subgroup in these programs were relatively small, impacts for this subgroup are being reported because they were relatively numerous and were consistent in direction (they were mainly unfavorable). A similar pattern of unfavorable impacts was found for children in lower-risk families (defined by greater employment and low levels of welfare receipt) assigned to Florida's Family Transition Program. See Bloom et al., 2000a.

36.  Small sample sizes precluded testing this hypothesis empirically.

37.  See McGroder et al., 2000.

38.  These comparisons rely on different child outcomes measured at the two-year and five-year follow-ups (with no child or teacher reports in the two-year follow-up) and on different (though similar) characterizations of subgroups.

39.  Zaslow et al., forthcoming.

40.  Magnuson and McGroder, 2001.

41.  NICHD Early Child Care Research Network, 2000; see also U.S. Department of Education, 1995. As discussed in Chapter 10, the NICHD findings also suggest that extensive participation in formal child care arrangements may have the unfavorable result of increasing children's externalizing behavior problems.

42.  Zaslow and Eldred, 1998.

43.  Findings from the two-year COS suggest that young children may be affected through multiple pathways, that pathways can vary according to the child outcome considered, and that different programs may activate different pathways in affecting children. See McGroder et al., 2000.

44.  Hamilton and Brock, 1994.

45.  Hamilton et al., 1997.

46.  This may be one reason that only the LFA program in Atlanta improved mothers' educational attainment by the two-year follow-up. See McGroder et al., 2000; Zaslow et al., 2000.