Prepared for:
U.S. Department of Health and Human
Services
Administration for Children and
Families
Office of the Assistant Secretary for Planning
and Evaluation
U.S. Department of Education
Office of the Under
Secretary
Office of Vocational and Adult
Education
June 2000
Prepared by:
Gayle Hamilton
with
Stephen Freedman
Manpower Demonstration Research Corporation
and
Sharon M. McGroder
Child Trends
| This document was prepared as part of the
National Evaluation of
Welfare-to-Work Strategies (NEWWS). The Manpower Demonstration
Research Corporation (MDRC) is conducting the NEWWS Evaluation under a contract
with the U.S. Department of Health and Human Services (HHS), funded by HHS
under a competitive award, Contract No. HHS-100-89-0030. Child Trends,
as a subcontractor, is conducting the analyses of outcomes for young children
(the Child Outcomes Study). HHS is also receiving funding for the
evaluation from the U.S. Department of Education. The study of one
of the sites in the evaluation, Riverside County (California), is also conducted
under a contract from the California Department of Social Services (CDSS).
CDSS, in turn, is receiving funding from the California State Job Training
Coordinating Council, the California Department of Education, HHS, and the
Ford Foundation.
The findings and conclusions presented herein do not necessarily represent the official positions or policies of the funders. |
This report is available on the Internet at:
http://aspe.hhs.gov/hsp/NEWWS/child-synthesis/index.htm
How to Obtain a Printed Copy of This Report
Appendix A: Two-Year Child Outcome Measures
Appendix B: (Table) Impacts on Average Total Welfare Payments Received in Years 1 and 2
Appendix C: A Comparison of National Samples of Children and Control Group Children Two Years After Study Entry
Appendix D: (Table) For All Families in the Client Survey Sample: Control Group Child Outcomes and Impacts on Child Outcomes
B.1 Impacts on Average Total Welfare Payments Received in Years 1 and 2
This synthesis is based on the work of two teams of researchers, one at MDRC and one at Child Trends, and this document owes much to them. The productive collaboration among these research teams and their affiliate organizations was instrumental in bringing to fruition this study of the effects of mandatory welfare-to-work programs on children.
Gratitude is also due to the people in the NEWWS Evaluation research samples. These people shared detailed information about themselves and their children, thoughtfully completed batteries of tests and indices, and, in many cases, opened their homes to enable researchers to obtain particularly sensitive information and directly assess children's well-being.
Finally, the ongoing cooperation of welfare department staff and administrators in the evaluation's seven sites has been indispensable. Their willingness to subject their programs to rigorous evaluation requiring an elaborate research design and detailed data has been of crucial importance throughout the NEWWS Evaluation. Welfare policymakers will reap the benefits of their commitment to discovering the true effects of various welfare program approaches for many years to come.
[ Go to Contents ]
Since its inception the primary goal of the Aid to Families with Dependent Children (AFDC) program, as well as successor programs funded under Temporary Assistance for Needy Families (TANF), has been to provide government support for poor children. Over the years, this public assistance has become more and more predicated on custodial parents' involvement in work or mandatory welfare-to-work program activities, as policymakers have sought to balance the goal of fostering poor children's well-being with that of encouraging adults' self-sufficiency. Currently, there are strong incentives for states to run mandatory, work-focused welfare-to-work programs: States face financial penalties if they fail to meet TANF-defined participation standards, which require large proportions of welfare recipients to be working or in work-related activities, and states must require recipients to work after two years of assistance. In addition, federal funds now may not be used to support most families on welfare for longer than five years, and a number of states and localities have shorter welfare time limits.
This document examines the effects of welfare-to-work programs on the children of the adults (almost all single mothers) mandated to participate in such programs. Synthesizing the results from two recently completed reports from a large-scale evaluation — the National Evaluation of Welfare-to-Work Strategies (NEWWS Evaluation) — the two-year effects of 11 welfare-to-work programs that operated in seven sites in the early to mid 1990s are summarized.(1) The sites included in the evaluation are Atlanta, Georgia; Columbus, Ohio; Detroit and Grand Rapids, Michigan; Oklahoma City, Oklahoma; Portland, Oregon; and Riverside, California. While the programs operated under the federal Job Opportunities and Basic Skills Training (JOBS) program that preceded TANF, and thus did not invoke a time limit on eligibility for welfare, they shared TANF's primary goal of moving welfare recipients into paid work and off assistance, and they reflect a range of approaches, implementation features, and environments: Some were strongly employment-focused while others emphasized basic education; they varied in how broadly the program participation mandate was applied to the welfare caseload and how strictly it was enforced, in the amount of child care support provided for program participation or employment, and in methods of case management; and the programs served different welfare populations and operated in a variety of labor markets. Although the NEWWS evaluation was designed to address the effects on children of requiring parents to participate in welfare-to-work programs, there are many other policies — for example, child care and health insurance policies — that can affect children, and those policies can be examined only indirectly in this evaluation.
To determine program effects on children, the NEWWS Evaluation uses a very strong research design: a random assignment experiment. In each evaluation site, adults who were required to participate in the program were assigned, by chance, either to a program group that had access to employment and training services and whose members were required to participate in the program or risk a reduction in their monthly welfare grant or to a control group that received no services through the program but could seek out such services from the community(2). (Control group members were eligible for child care assistance, similar to that offered to program group members, if they were participating in nonprogram activities in which they had enrolled on their own.) Notably, in four of the sites, there were two program groups (plus a control group). In three of the sites, one program group was employment-focused while the other program group was education-focused; in the fourth site, the two program groups varied in their case management staffing structure. This random assignment design assures that, within each site, there were no systematic differences between the background characteristics of families in the program and control groups when they entered the study. Thus, any subsequent differences in outcomes between the groups — for adults, children, or families as a whole — can be attributed with confidence to the effects of the programs. These differences between outcomes are called impacts, and all those reported are statistically significant and hold for the whole sample unless otherwise noted.
[ Go to Contents ]
No aspects of the 11 welfare-to-work programs studied as part of the NEWWS Evaluation were targeted directly to children. Theoretically, however, the programs could affect children through their effects on the parents required to participate in them, as described in the conceptual model presented below. Program effects on parents' education levels, welfare status, employment, earnings, and income could potentially result in changes in family routines, parents' self-esteem or stress, or parents' supervision of children or use of child care arrangements. Thus, as a first step, it is necessary to examine how the programs were implemented and their impacts on adults.
The 11 programs differed in the messages that they sent to welfare recipients about how best to obtain and retain employment: Some stressed getting a job quickly and others stressed initial investments in basic education or training. Most of the programs imposed a mandatory participation requirement on all recipients, and several used financial sanctions (that is, welfare grant reductions) extensively to enforce this mandate. All programs substantially increased participation in activities designed to promote employment, beyond what would have happened in their absence.
Most of the programs had impacts on their targeted outcomes: Over the two-year follow-up period, four programs that had an education focus increased the probability that welfare recipients would obtain a high school diploma or General Educational Development (GED) certificate. All programs decreased some aspect of welfare dependency, and 8 of the 11 increased two-year employment levels. While most programs increased individuals' reliance on earnings, as opposed to welfare, net income for these individuals was largely unchanged.
Impacts on nontargeted outcomes were found as well: Some programs led to a reduction in health insurance coverage, and most programs increased the use of paid child care. Across the programs, there were few effects on fertility or family structure, housing status, or mothers' psychological functioning, stress, or parenting.
Impacts on children were based on data collected in three child development areas: behavioral and emotional adjustment; cognitive functioning and academic achievement; and health and safety. In-depth data are available for preschool-age children in three of the sites (as part of a special Child Outcomes Study); more limited data are available for children of all ages in all seven sites. An examination of findings from both sets of data suggests the following regarding all children:
An examination of impacts on subgroups of young children (as part of the Child Outcomes Study) indicates the following:
Further research and longer follow-up are needed to clearly determine the mechanisms through which some of the programs affected children. Nonexperimental methods — which lack the rigor of the experimental methods that produced the findings reported above but are needed to examine the processes through which programs might affect children — were used in an attempt to explain the few found child impacts. The results are thus suggestive, but not definitive. They suggest that, for families with all school-age children, programs that place little emphasis on helping welfare recipients obtain good child care or that result in decreases in family income may tend to have unfavorable impacts on children. (There is also some indication that increases in employment may be connected with unfavorable child effects, but this finding held true for one source of data on employment and not for the other.) Most likely, these program characteristics or effects interact with each other in particular (as yet unknown) ways to affect children. Other examined program features or effects — whether programs were employment- or education-focused, the extent to which a mandatory participation requirement was enforced, increases in parents' high school diploma or GED receipt, or decreases in health insurance coverage — do not appear, by themselves, to relate to impacts on children. Analyses of selected impacts on the younger children in the study also suggest that programs might affect children to the extent that they affect mothers' employment and/or affect children's home environment (for example, mothers' psychological well-being and parenting). These analyses of preschool-age children did not find that increases in the use of child care, decreases in health insurance coverage, or changes in family income played roles in explaining the selected child impacts examined.
Because the 11 programs operated under JOBS (Job Opportunities and Basic Skills Training program), which preceded TANF, they did not invoke the TANF time limit on eligibility for welfare, try to meet its participation goals, impose full-family financial sanctions, or put in place the generous financial work incentives of many current programs. They also did not have available to them the recent and substantial increases in federal funding for child care or expanded eligibility for health insurance through Medicaid and the State Children's Health Insurance Program. It is plausible that some of the current generation of programs will produce larger effects on adults than those reported here; as a result, it is possible that they may have larger effects on children. The new policies also may result in stronger and more divergent impacts on children with varying initial levels of being at risk for poor development(3).
The remainder of this synthesis expands on the above findings. Section II presents a conceptual model of how mandatory welfare-to-work programs might affect children. Section III describes aspects of child well-being examined in the NEWWS Evaluation. Section IV discusses the characteristics of the adults and children in the evaluation samples. Sections V, VI, and VII summarize program implementation and program effects on targeted and nontargeted outcomes, highlighting any situations where effects were different for mothers in a special Child Outcomes Study sample focusing on young children, as compared to mothers with children of all ages. Section VIII — the heart of the document — presents child impacts.
[ Go to Contents ]
At the outset of the NEWWS Evaluation, it was an open question as to whether, and how, welfare-to-work programs designed for adults would affect children. For 20 years prior to 1988, women receiving welfare who had children under age 6 generally were not required to participate in such programs. With the passage of the Family Support Act in 1988, women with children as young as age 3 (or as young as age 1, at state option) were newly designated as mandatory participants. Thus, in the early 1990s there was much interest in how welfare-to-work programs might affect children, particularly preschool-age children, who were seen as particularly vulnerable to changes in their family situation. Expectations regarding possible effects on children varied. On the one hand, given that all aspects of the programs were aimed at changing adult behavior, one might expect few effects on children, particularly if effects on adults were not dramatic. On the other hand, if effects on adults — for example, increased employment or participation in program activities such as job search or education, or increases or decreases in family income — were large or pervasive enough, one might expect effects on children, either positive or negative.
The most prevalent current theories about how mandatory welfare-to-work programs might affect children hypothesize that program effects on adults' employment, earnings, and income may, in turn, affect the resources available to children's development, either positively or negatively(4). The resources available to children shape the daily experiences that contribute to their health, safety, and development. These resources can be material (for example, housing) or social (for example, interactions between mothers and children)(5). As a positive example, welfare-to-work programs that raise income might allow families to afford better and safer housing. Additionally, employment may improve mothers' self-esteem, enhancing their ability to be a role model for their children. As a negative example, the reduction of working mothers' time at home may result in decreased overall supervision of their children. Additionally, the requirement to participate in a welfare-to-work program or the experience of holding a new job may result in increased stress for mothers, affecting parenting practices.
Figure 1 depicts the theoretical model described above(6). The pivotal box in this model, labeled "Targeted Outcomes" (box C), represents the adult outcomes targeted by welfare and employment policies and programs (box A) that can be affected through implementation of the policies and programs (box B). Changes in the targeted outcomes, which can affect the resources available to families as well as family socialization patterns, can produce effects on nontargeted outcomes (box D). These nontargeted outcomes represent other avenues through which child outcomes (box E) might be affected(7).
As evident in the model, it is important to establish that impacts on the targeted outcomes or nontargeted outcomes exist. If impacts on children are found, but no impacts are apparent for the targeted or nontargeted outcomes, it will be unclear what led to the child impacts. (While the rigorous NEWWS research design can provide solid evidence about the existence of impacts on children, it does not allow firm causal inferences to be made about the processes through which mandatory welfare-to-work programs may affect children's well-being.) Associations found among program features, adult impacts, environmental effects, and child impacts, however, can give clues about possible pathways of effects. These can then be investigated through further research and result in modifications to the conceptual model.
[ Go to Contents ]
In order to allow sufficient time for the full effects of the NEWWS Evaluation programs to manifest themselves, sample members in the evaluation are being followed for five years from the time they entered the study. Comprehensive data on economic outcomes, including information on quarterly unemployment insurance-reported earnings and monthly welfare and Food Stamp payments, are being collected over the five-year follow-up period for 44,569 single parents (the full sample) who have been randomly assigned to research groups across the seven evaluation sites. (See box A in Figure 2.) At this point in the evaluation, economic outcome data covering the first two years of follow-up have been analyzed and results have been made public.
In addition, a wealth of information was collected through interviews with a subsample of 9,675 individuals, randomly chosen from all research groups in all seven sites, two years after they entered the study. Interviewees in this client survey sample were asked about their experiences in the program, educational attainment, family composition, housing status, employment and wage progression, and total family income, as well as about their children's behavior, school progress, and health and safety(8). (See box B in Figure 2.)
For a subset of the surveyed individuals (3,018 surveyed sample members in three of the seven evaluation sites who had a child aged 3 to 5 at study entry), additional, more detailed data were collected at the time of the interview for the young child(9). (If a family had more than one child aged 3 to 5, one was randomly selected to be studied in depth.) (See box C in Figure 2.)
These "focal" children, who would have been between about ages 5 and 7 at the two-year interview, were given a direct assessment of cognitive development, and the mothers completed multiple and more detailed measures regarding the children's behavioral and emotional adjustment and physical health and safety. Mothers in the Child Outcomes Study (COS) sample were also asked additional questions concerning nontargeted aspects of the welfare-to-work programs: for example, mothers' psychological well-being, child care arrangements, fathers' involvement with children and the payment of child support, and the family's home and neighborhood environment.
Data relating to all children of the 9,675 mothers interviewed in seven sites and to the young focal children of the 3,018 mothers interviewed in three sites were collected for three child developmental domains or outcome areas: behavioral and emotional adjustment; cognitive functioning and academic achievement; and health and safety. Appendix A details the measures used at the two-year follow-up point to assess changes in each of these areas. Data collection on children (and adults) is ongoing in the NEWWS Evaluation(10), and Box 1 describes the future analyses planned.
[ Go to Contents ]
In most aspects, the samples in the seven NEWWS Evaluation sites are diverse. Across all sites, almost all adult sample members were female single parents and, on average, 30 years old with two children at the time of study entry. The samples in Grand Rapids, Detroit, and Oklahoma City included teen parents, who represented, at most, 10 percent of each site's full sample. In Grand Rapids, Detroit, Oklahoma City, and Portland, mothers had children as young as age 1 at study entry; in these four sites, about two-fifths of the sample members entered the program when their youngest child was under age 3. The remainder of the samples in these four sites, as well as the full samples in the other three sites, were about evenly split between parents whose youngest child, as of study entry, was aged 3 to 5 and those whose youngest child was aged 6 or over. Depending on the site, between one-third and one-half of sample members gave birth to their first child when they were teenagers.(11)
Future Analyses of Children in the NEWWS Evaluation Will Cover Five Years of Follow-Up and Include Elementary School Teacher Assessments While extensive and rich data on both adult and child development outcomes were collected in the NEWWS Evaluation at the two-year follow-up mark, two years is not enough time to fully assess program impacts on child (or adult) outcomes. Another round of data collection is in progress for study sample members at their five-year follow-up point. These data will indicate whether the impacts on children observed at the two-year point persist, grow, or decline by the end of five years. In addition, new program impacts on child outcomes may emerge. At the five-year follow-up point, a small set of questions will again be asked of parents with children of all ages. In addition, for the sample of preschool-age focal children in three sites (who will be 8 to 10 years old at this point), math and reading skills will be assessed, and elementary school teachers will report on scholastic performance (whether performing at grade level, whether skipped or repeated a grade, and how performing in comparison to classmates). Finally, parents and teachers — and the focal children themselves — will report on focal children's behavior, maturity, social competence, and engagement in school. |
The ethnic make-up of the samples varied across the sites, reflecting the ethnic composition of the localities from which the samples were drawn. In Atlanta and Detroit, almost all sample members were African-American. About half of the sample members in Grand Rapids, Riverside, Columbus, and Oklahoma City, and two-thirds of those in Portland, were white. Only Riverside had a substantial portion (one-third) of Hispanics.
Slightly more than one-half of sample members had a high school diploma or General Educational Development (GED) certificate when they entered the program, and in all seven sites at least some study enrollees had some college or post-secondary schooling. On average, however, sample members had completed just 11 years of school as of study entry. None of the programs served populations who, as a whole, had much work history; fewer than half the individuals in all sites but Oklahoma City had worked at some point during the year prior to study entry. In all sites except Oklahoma City, between a quarter and a half of sample members had received welfare cumulatively for at least five years.(12) Furthermore, up to a quarter of sample members in any site met a definition of "most disadvantaged"; that is, they did not have a high school diploma or GED, lacked any work history in the year prior to enrolling in the program, and already had received welfare cumulatively for two years or more before entering the study.
In contrast to the full samples of parents described above, adults in the Child Outcomes Study (COS) sample (a subset of parents in the Atlanta, Grand Rapids, and Riverside samples) consisted solely of mothers with preschool-age children. At study entry, the mothers' average age was 27 in the Grand Rapids sample and 29 in the Atlanta and Riverside samples, below the average age of parents in the full sample.(13) Nevertheless, COS mothers had, on average, a slightly higher number of children than parents in the full sample. In addition, these mothers were less likely than those in the full sample to have ever been married and more likely to have a high school diploma or GED. Finally, in two of the three sites (Atlanta and Riverside), COS mothers were less likely than those in the full sample to have ever worked full time for six months or more for one employer. Some of these sample differences — for example, the younger average age of COS mothers and their relative lack of work experience — are typical of mothers with young children. Other sample differences — for example, COS mothers being more likely to have a high school diploma or GED — reflect the fact that women on welfare with all older children are often those who have more barriers to finding a job and leaving welfare, while women with young children, particularly those not married, are perhaps more likely to have just recently started receiving welfare, and the most advantaged of this group will leave welfare within a few years.
The children of adult sample members across the seven NEWWS Evaluation sites ranged in age from 1 to 17. Approximately 49 percent of all families had all school-age children, 23 percent had no school-age children, and 28 percent had both school-age and preschool-age children. As noted above, about two-fifths of the families in four sites included a child as young as age 1 or 2. At the other end of the age spectrum, approximately one-third of the families in all seven sites included at least one high school-age child, that is, a child between ages 13 and 17. Other characteristic data for children are available only in Atlanta and only for a small number of young children who were between ages 3 and 5 at study entry. (See Box 2.)
[ Go to Contents ]
As shown in Figure 1, features of program implementation lead directly to the targeted outcomes of welfare and employment policies and programs and can also influence nontargeted outcomes. This section discusses the program services, mandates, and activities shown in box B of the figure. Results for the full sample are highlighted. Where evident, distinctions are drawn between program implementation as it was experienced by the seven-site client survey sample (for which impacts on primarily school-age children are available) and by the COS sample in three of the evaluation sites (for which impacts on preschool-age children are available).
In At Least One Aspect of Development, Preschool-Age Children in the Atlanta COS Sample at Study Entry Were More Disadvantaged Than Children in National Samples In the Atlanta NEWWS Evaluation site, a special descriptive study was conducted close to the start of the evaluation, in order to describe the lives and circumstances of a sample of welfare families with preschool-age children and to inform policymakers about the developmental status of young children receiving welfare. (See Moore et al., 1995.) Approximately three months after sample mothers entered the study, interviews were conducted in the home and direct assessments were made of their children's cognitive development in the areas of receptive vocabulary and school readiness. In addition, mothers reported on their children's socioemotional development and health status. A brief summary of how this sample of 790 children were faring at roughly the start of the NEWWS Evaluation follows. Cognitive development: The Peabody Picture Vocabulary Test-Revised was used to assess cognitive development. This measure is highly correlated with measures of both intelligence and school achievement and is a predictor of IQ. Mean scores of children in the Atlanta sample on this measure were lower (by .4 of a standard deviation) than the mean scores of African-American children from welfare families in a national sample. (Comparisons were made solely for African-American children because of the possibility of racial bias with this measure.) Children in the Atlanta sample scored approximately two-thirds of a standard deviation below nonpoor children in a national sample. Socioemotional development: Using the Personal Maturity Scale, mothers described their children as showing fairly high levels of maturity. The average score on this scale was 8 out of a possible 10, with 10 indicating the highest level of maturity. National results using a comparable scale were not available. |
For analysis purposes, distinctions are drawn in the evaluation between employment-focused programs and basic education-focused programs, as well as between programs with high and low levels of enforcement of the participation mandate. Taking into account these two dimensions of program characteristics, as well as the types of program activities to which welfare recipients were initially assigned, four categories of welfare-to-work program approaches emerge, shown in Table 1.
| National Evaluation of Welfare-to-Work Strategies Table 1 Categorizing NEWWS Programs, by Approach, First Activity, and Enforcement Level |
|||
|---|---|---|---|
| Employment-Focused Approach | Education-Focused Approach | ||
| Job Search First: High Enforcement |
Varied First Activity: High Enforcement |
Education or Training First: High Enforcement |
Education or Training First: Low Enforcement |
| Atlanta LFA Grand Rapids LFA Riverside LFA |
Portland | Atlanta HCD Grand Rapids HCD Riverside HCD Columbus Integrated ColumbusTraditional |
Detroit Oklahoma City |
The distinction between employment- and education-focused approaches is central to the NEWWS Evaluation. To promote ongoing work and self-sufficiency among welfare recipients, states have traditionally implemented one or the other of these two approaches. The employment-focused approach emphasizes placing people in jobs quickly, even at low wages, reflecting a view that the workplace is where welfare recipients can best build their work habits and skills. The education-focused approach, which emphasizes education and training as a precursor to employment, is based on the belief that the required skill levels for many jobs are rising and that an investment in the "human capital" of welfare recipients will allow them to obtain better and more secure jobs. The two approaches convey different messages to welfare recipients about the best route to self-sufficiency, and they emphasize different program components. One aim of the NEWWS Evaluation is to determine the relative effects of the two approaches on both adults and their children.
Four of the sites in the evaluation — Atlanta, Grand Rapids, Riverside, and Columbus — operated two different programs simultaneously, to enable rigorous side-by-side tests of the comparative effectiveness of various approaches. Each of the first three of these sites implemented a "labor force attachment" (LFA) program as well as a "human capital development" (HCD) program, versions of employment-focused and education-focused programs that magnified the differences between the two types of approaches. The fourth site, Columbus, implemented a program using a "traditional" case management model, in which welfare eligibility and employment program functions were performed by separate sets of staff, as well as a program using an "integrated" case management model, in which these two functions were performed by the same staff. In the remaining three sites in the evaluation — Oklahoma City, Detroit, and Portland — the sites' established programs were studied. In all, 11 programs were examined in the seven NEWWS Evaluation sites.
The four employment-focused programs (see Table 1) provided job search assistance to a large segment of their caseload and encouraged enrollees to find work as quickly as possible. The three LFA programs, however, differed from the Portland program in two important ways. First, the LFA programs routinely assigned individuals to job search as their first activity, whereas Portland offered GED preparation classes to people who were thought to have a good chance of attaining a GED certificate relatively quickly. Second, Portland case managers, more often than those in the LFA programs, encouraged enrollees to hold out for a job that paid well above the minimum wage and offered the best chance for long-lasting and stable employment. In contrast, LFA case managers, especially in Riverside, stressed the value of taking any job, even a low-paying one, and trying to advance.
In the education-focused programs, a large percentage of program enrollees were initially assigned to some type of skill-building activity. Their first assignments depended, in part, on their educational levels on entering the program. Those with low reading or math skills were assigned to adult basic skills classes; those with higher skills but lacking a high school diploma or GED were assigned to GED preparation classes; and non-English speakers could be assigned to English as a Second Language (ESL) classes. Finally, those with a high school diploma or GED could be assigned to vocational training or employment-oriented skills courses at local community colleges. All in all, however, assignments to GED preparation or basic education courses predominated, vocational training program assignments were less common, and enrollment in college was minimal. Riverside's HCD program was unique among this group in that it did not serve high school graduates and GED holders who, at program entry, scored above minimum levels in reading and math tests.
As specified in the research design, no control group members were subject to a participation requirement and, as a result, none of them experienced any sanctions. In contrast, a wide cross section of program group members were enrolled in most programs, and participation was monitored closely. Failure to participate could result in a sanction, that is, a reduction in a family's total welfare grant. Sanction rates were high in four programs (Grand Rapids LFA and HCD and Columbus Integrated and Traditional), where at least 26 percent of sample members were sanctioned at some point during the two-year follow-up period, and low in two programs (Detroit and Oklahoma City), where less than 5 percent of sample members were ever sanctioned. Sanction rates for the remaining programs fell between these two extremes and were considered to be moderate to somewhat high.
Many control group members took part in employment-related activities, such as basic education, skills training, post-secondary education, or formal job search, on their own initiative at some point during the two-year follow-up period. All programs, however, were able to increase participation levels in such activities above those achieved by the control groups. Of the nine programs with at least a moderate enforcement of the participation mandate, all but one (Grand Rapids LFA) produced large impacts on participation, ranging from 21 percentage points (Grand Rapids HCD and Columbus Traditional) to 40 percentage points (Riverside HCD).(14) As expected, all of the employment-focused programs produced large increases in participation in job search activities, but two also produced small increases in participation in education and one produced a small increase in training. Most of the education-focused programs raised participation levels in education or training. These programs also, to a lesser extent, increased participation in job search.(15)
[ Go to Contents ]
Figure 1 suggests that numerous features of mandatory welfare-to-work programs are hypothesized to directly impact the adults subject to such programs and that these targeted effects on adults can potentially, through effects on nontargeted outcomes, influence child well-being. This section briefly summarizes effects on targeted outcomes for the 11 programs in the seven evaluation sites. The effects of the programs are measured by comparing outcomes for program groups with those of control groups; the resulting differences, or impacts, can be confidently viewed as the effects of the programs.(16) As in the previous section, while most shown results are for the full sample, contrasts between impacts for the client survey sample and for the COS sample, where they exist, are highlighted.(17)
While most education-focused programs increased participation in basic education among those who entered the study without a high-school diploma or GED, only three of these programs (Grand Rapids and Riverside HCD and Columbus Traditional) increased GED certificate attainment. For all sample members, not just those who entered the study without these education credentials, impacts on GED receipt in the three programs ranged from 3 to 8 percentage points. (Table 2 shows these impacts for the several different samples for which child impacts are discussed later in this document.) Portland's employment-focused program, which used a varied first activity approach, achieved similar gains in GED receipt. (The other three employment-focused programs had no effect on GED attainment.)
For the full sample, seven programs decreased cumulative welfare expenditures by 10 percent or more relative to the control groups, a historically large effect. (See Appendix B.) The Portland and Grand Rapids LFA programs produced unusually large decreases of 17 and 19 percent, respectively. The Riverside LFA welfare expenditure impact was also large (14 percent). Among COS sample mothers, four of the six programs to which they were subject reduced welfare dependency. For this sample, the Grand Rapids and Riverside LFA programs decreased cumulative welfare expenditures by at least 15 percent, a large impact; the Atlanta LFA and Riverside HCD programs decreased expenditures by a smaller percentage. Finally, while welfare expenditure reductions were found for the full samples in the Atlanta and Grand Rapids HCD programs, they were not evident for the COS samples in these same programs.
Eight of the 11 programs increased two-year employment levels. Among COS sample members, five of the six programs to which they were subject increased employment, and impacts on employment rates were generally substantially higher for this sample than for the client survey samples in these five programs.
A majority of control group members worked for pay at some point during the two-year follow-up period. For the client survey sample, three of the employment-focused programs (all except Atlanta LFA) and four of the seven education-focused programs (Atlanta, Grand Rapids, and Riverside HCD; and Detroit) increased two-year employment levels. (See Table 2.) Among COS sample members, all programs except the Atlanta LFA program increased two-year employment levels. Employment impacts ranged from 6 percentage points in the Grand Rapids HCD program to 25 percentage points in the Riverside LFA program. These employment impacts were substantially larger than those for the client survey sample in all but one of the five programs. The biggest difference occurred in the Riverside LFA program, where the impact for the COS sample was nearly 7 percentage points higher than the impact for the client survey sample.
For the client survey sample, earnings gains for three of the programs (Riverside LFA and HCD and Portland) were moderate and statistically significant, while earnings gains for the other programs were smaller and not statistically significant. (All but two of the 11 programs produced two-year earnings gains for the much larger full samples.) (See Table 2.) For COS sample mothers, impacts on total two-year earnings were found in the Riverside LFA and HCD and Grand Rapids LFA programs. The largest earnings impact occurred in the Riverside LFA program where, on average, mothers in the COS program group earned an impressive 71 percent more than their control group counterparts.
Across all 11 programs in the second year of follow-up, control group members in the client survey sample averaged between $6,055 (Oklahoma City) and $8,596 (Columbus) in combined income from earnings, welfare, and Food Stamps. Few programs substantially altered these combined income levels; in general, reductions in welfare, Food Stamps, and other benefits matched or exceeded earnings gains. (See Table 2.) Including estimates of the earned income tax credit (EITC) as income (not shown in Table 2) produced little change in this finding. Interestingly, one program (Riverside LFA) increased the proportion of COS sample members with incomes at or above the poverty level by 6 percentage points, a result not found for the Riverside LFA full sample. For COS sample members, there were no program effects on child support awards or payment.
The Grand Rapids and Riverside LFA programs increased, by 6 and 7 percentage points, respectively, the proportion of sample members living deeply in poverty, that is, below 50 percent of the poverty line. (See Table 2.) While increases on this measure are apparent for the COS samples in these two programs, they are not statistically significant.
[ Go to Contents ]
As shown in Figure 1, mandatory welfare-to-work programs can affect nontargeted outcomes (box D) through effects on the targeted outcomes (box C). For example, an increase in employment, if the increase is in jobs that do not provide health benefits, could affect the proportion of people with health care coverage. Program implementation features (box B) can also affect nontargeted outcomes (box D). For example, interactions between welfare recipients and case managers that stress the importance of quality child care, or networks among welfare recipients that develop through job search clubs or other program activities and end up providing child care provider "tips," can, in turn, change the child care environments for welfare recipients' children. This section briefly summarizes program effects on the nontargeted outcomes for which data are available in the NEWWS Evaluation. Again, the effects on nontargeted outcomes are measured by comparing outcomes for the program and control groups, and differences in impacts for the client survey and COS samples are highlighted.
In total, four programs — Riverside LFA, Portland, Columbus Integrated, and Oklahoma City — decreased health care coverage levels (as reported by parents) as of the end of the follow-up period.(18) (See Table 3, which shows these effects for several different samples for which child impacts are discussed later in this document.) Portland program impacts were not statistically significant, but were just above the 10 percent level used as the standard throughout the NEWWS Evaluation analyses. For the client survey sample, the other seven programs had no impacts on health care coverage rates for children or parents. COS sample members in the Grand Rapids and Riverside LFA programs also reported a decrease in coverage, for focal children as well as any other children. (See Box 3 for a discussion of the dynamics that may have led to health insurance coverage reductions.)
How Welfare-to-Work Programs Can Lead to a Reduction in Health Insurance Coverage At study entry, almost all NEWWS Evaluation sample members and their dependent children had health coverage because they were receiving welfare and were automatically covered under Medicaid. (In Oklahoma City, where applicants for assistance whose eligibility was not yet determined were included in the sample, initial coverage rates were lower.) Over time, coverage rates declined for both program and control group members, as some people left welfare and did not replace their Medicaid coverage with coverage from employers or other sources. By the end of the two-year follow-up period, between 81 percent (Columbus) and 88 percent (Detroit) of control group members reported having health care coverage for themselves and their children. (This range covers all sites except Oklahoma City, where the rate was 68 percent.) Three programs — Riverside LFA, Portland, and Columbus Integrated — that increased employment and decreased welfare receipt as of the end of the two-year follow-up also decreased health care coverage levels (as reported by parents). (No program increased health insurance coverage.) Reductions in coverage ranged from 4 to 7 percentage points for sample members and their children, and from 3 to 6 percentage points for children only. Although many program group members who left welfare (and automatic Medicaid coverage) found a job that provided health insurance, received transitional Medicaid benefits, or obtained alternative sources of coverage, others were not able to replace the coverage they had under Medicaid. Some of these individuals never received transitional Medicaid, and others had exhausted or had not restarted their benefits as of the end of the two-year follow-up period. Program group members in Oklahoma City reported even larger decreases in coverage: 11 percentage points for adults and children and 9 percentage points for children only. This program decreased welfare receipt and appears to have increased short-term employment that did not provide any health insurance. |
All 11 programs offered child care assistance to welfare recipients who needed this service while they were participating in program activities or while they were employed.(19) In the Atlanta LFA and HCD, Oklahoma City, Portland, and Detroit programs child care assistance was emphasized either by site staff or by the welfare department's organizational structure. In both Atlanta programs case managers actively promoted the availability of child care reimbursement as a benefit of program participation. In Oklahoma City state-wide emphasis on access to child care made assistance to clients readily available while they were in the program and after they left welfare for work. Oklahoma City had no set caps on the amount of child care assistance that clients could receive. Atlanta and Oklahoma City, unlike the other programs, reimbursed only for care given by licensed providers. In Portland case managers did not push specific types or locations of providers, but they did emphasize the necessity for individuals to make arrangements and assisted those who were unable to make arrangements on their own. In Detroit case managers placed a priority on making child care payments, and made referrals to licensed providers in the area on request, but the choice of provider (including choosing licensed or unlicensed care) was left to the parent.
Both the Grand Rapids and Columbus programs would reimburse expenses from child care in licensed as well as unlicensed care, but expected parents to make their own arrangements. Referrals to licensed providers in the area could be made at the parents' request.
Child care providers were not difficult to come by in any site except Riverside, where case managers encouraged the use of low-cost, more informal arrangements, both to contain program costs and because case managers believed that parents would be more able to afford such arrangements after program or other government supports expired.
The authorization of transitional child care benefits for those who left welfare for work was easiest in the Detroit, Portland, Columbus, and Oklahoma City programs. Few individuals who went to work in the other six programs received these benefits.(20)
Between 15 and 32 percent of control group members (depending on the site) in all 11 programs used paid child care while employed at some point during the two-year follow-up period. Nine programs — all four of the employment-focused programs and five of the seven education-focused programs — produced impacts on paid child care, with the impacts in two programs (Portland and Detroit) above 10 percentage points in magnitude. (See Table 3.) Relatively few control group members (less than 15 percent in any site) used transitional child care benefits. Five programs increased the use of such benefits, but these impacts were sizable only in the Atlanta LFA and Portland programs, where the increases in the receipt of these benefits were 7 and 11 percentage points, respectively. Among COS sample members, only the two Riverside programs increased child care during employment for the focal child (measured during the month prior to the client survey interview), and this care tended to be informal and to occur during nonstandard hours (evenings or weekends). Thus, in no site were COS focal children more likely than their control group counterparts to be in formal child care during their mother's employment. (See Box 4 for a discussion of how child care use and employment might have interacted.)
Across all 11 programs, only Columbus Traditional had any effect on fertility; this program resulted in a 3 percentage point decrease in the proportion of sample members who had a baby since study entry. (See Table 3.) Only two programs had impacts on marital status: in Portland a 5 percentage point increase in the proportion of sample members living as an unmarried couple (not shown in Table 3), and in Riverside LFA a 3 percentage point reduction in the proportion of sample members married and living with a spouse. Similarly, only two programs had impacts, small in size, on family household composition. Impacts on housing status were not extensive either, although five programs did produce impacts (not shown in Table 3). The largest of these was a 5 percentage point increase in the proportion of program group members in the Columbus Traditional program who lived with family or friends and paid rent. For COS sample members, none of the programs had an impact on fertility or marital status, and only Riverside's LFA program had an impact on families' living arrangements, with fewer fathers living with their biological focal child.
Most Programs Increased the Use of Child Care During Employment Independent of Their Effects on Increasing Employment As noted, nine programs produced increases in the use of paid child care while employed, ranging from 4 to 13 percentage points. Increases in employment do not entirely explain program impacts on child care use and on paid care while employed, since in many programs impacts on child care use were maintained even when only those who worked during the follow-up period are considered (a nonexperimental comparison). A likely explanation is that employed program group members required or preferred more stable child care arrangements than employed control group members, either because of the different characteristics of their jobs or because they more frequently heard messages from case workers regarding the importance of child care. (Keep in mind, as noted earlier, that practices related to child care assistance — access to and allowable payments for child care — were the same for control and program group members within each site.) For example, case managers in the Atlanta LFA program (as well as in the Atlanta HCD program) encouraged people to use child care and emphasized it as a reason to participate in the program, in part by strongly emphasizing the availability of reimbursement for child care costs. For the client survey sample, the LFA program did not increase employment levels but did increase child care use while employed by 4 percentage points and raised use of paid child care by nearly 6 percentage points. Similarly, case managers in Detroit placed a priority on arranging child care, and the increase in the use of paid child care (13 percentage points) exceeded the program's employment gains. |
Measures specific to the COS component of the two-year client survey were used to assess mothers' psychological well-being and parenting. Psychological well-being was assessed in three areas: time stress, locus of control or self-efficacy, and depression. Increases in mothers' feelings of time stress were found in both Atlanta programs and in the Riverside HCD program. There were no impacts on mothers' feelings of control over their own lives. Only the Grand Rapids LFA program had an impact — an unfavorable one — on mothers' depressive symptoms. Parenting was assessed on such dimensions as maternal warmth, maternal aggravation, and maternal cognitive stimulation. Impacts were found on two of the parenting measures for the Atlanta HCD program (both favorable), on four of the measures for the Atlanta LFA program (all favorable), and on one measure for the Grand Rapids LFA program (unfavorable). No impacts on parenting were found for the other programs. (The impacts discussed in this paragraph are not shown in a table.)
| Table of Contents | Sections VIII-IX |
Home Pages:
National Evaluation of Welfare-to-Work
Strategies (NEWWS)
Human Services Policy
(HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)
Last updated February 6, 2004