National Evaluation of Welfare-to-Work Strategies: Evaluating Alternative Welfare-to-Work Approaches: Two-Year Impacts for Eleven Programs: Executive Summary

National Evaluation of Welfare-to-Work Strategies

Evaluating Alternative Welfare-to-Work Approaches: Two-Year Impacts for Eleven Programs

Executive Summary

Prepared for:U.S. Department of Health and Human ServicesAdministration for Children and Families

Office of the Assistant Secretary for Planning and Evaluation

U.S. Department of EducationOffice of the Under Secretary

Office of Vocational and Adult Education

June 2000

Prepared by: Stephen Freedman Daniel Friedlander Gayle Hamilton JoAnn Rock Marisa Mitchell Jodi Nudelman Amanda Schweder Laura Storto

Manpower Demonstration Research Corporation

---

Contents

1. Findings in Brief
2. Program Approaches and Implementation Features
3. Research Designs and Samples
4. Findings
5. Conclusions

Selected publications from this evaluation

Executive Summary

The Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of August 1996⁽¹⁾  ended the Aid to Families with Dependent Children (AFDC) program, one of the nations principal safety nets for poor families.  Among its provisions, the law replaced AFDC with a block grant program, Temporary Assistance for Needy Families (TANF), and created financial incentives for states to run mandatory, work-focused welfare-to-work programs.  While these types of programs are not new, various aspects of the 1996 law increase their importance:  federal funds now may not be used to support most families on welfare for longer than five years and a number of states and localities have shorter welfare time limits; states face financial penalties if they fail to meet TANF-defined participation standards, which require large proportions of welfare recipients to be in work or work-related activities; and states must have a plan for how they will require recipients to work after two years of assistance.

To meet the new challenges of the federal welfare legislation, state and local administrators and policy makers need to know about the types of welfare-to-work program approaches that can quickly move substantial numbers of people into work and off welfare.  This report provides such guidance, by analyzing the effectiveness of 11 mandatory welfare-to-work programs operated in seven locales.  The sites included in the evaluation are Atlanta, Georgia; Columbus, Ohio; Detroit and Grand Rapids, Michigan; Oklahoma City, Oklahoma; Portland, Oregon; and Riverside, California.

The report is one in a series from an evaluation of the programs called the National Evaluation of Welfare-to-Work Strategies (NEWWS), conducted by the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. Department of Health and Human Services (HHS), with support from the U.S. Department of Education.  Child Trends, as a subcontractor, is conducting the analyses of outcomes for young children (the Child Outcomes Study).  Two other recent reports (both also published in 2000 by the U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, and the U.S. Department of Education) should be viewed as companion documents to this report:  Impacts on Young Children and Their Families Two Years After Enrollment:  Findings from the Child Outcomes Study, prepared by Sharon M. McGroder, Martha J. Zaslow, Kristin A. Moore, and Suzanne M. LeMenestrel, Child Trends; and Do Mandatory Welfare-to-Work Programs Affect the Well-Being of Children?  A Synthesis of Child Research Conducted as Part of the National Evaluation of Welfare-to-Work Strategies, prepared by Gayle Hamilton, MDRC, with Stephen Freedman, MDRC, and Sharon M. McGroder, Child Trends.

Each of the 11 studied programs operated under the federal Job Opportunities and Basic Skills Training (JOBS) program, which preceded TANF.  Unlike TANF, these programs did not impose a time limit on eligibility for welfare assistance.  However, they shared TANFs primary goal of moving welfare recipients into paid work and off assistance.  Further, among the 11 programs some are strongly employment-focused, the welfare-to-work strategy favored under TANF, and some are strongly basic education-focused, an approach possible under TANF but more prevalent during the late 1980s and early 1990s.  (Overall, the present results pertain to the period between 1991 and 1996.)  The programs varied in other ways, including how broadly the participation mandate was applied to the welfare caseload and how strictly it was enforced, the amount of child care support provided for program participation or employment, and methods of case management.  The programs also served different welfare populations and operated in a variety of labor markets.

Taking advantage of the array of programs studied as part of the evaluation, this report addresses the following critical question:  What works best, and for whom?  The report distinguishes between employment-focused and basic education-focused programs, as well as between levels of enforcement of the participation mandate.  Taking into account these two dimensions of program characteristics, plus the types of program activities to which welfare recipients were assigned, four categories of welfare-to-work program approaches emerge:

• Employment-focused programs, with first assignments made to job search and a high level of participation mandate enforcement;
• Employment-focused programs, with first assignments made to job search, basic education, or vocational skills training and a high level of participation enforcement (only one program falls into this category);
• Education-focused programs, with first assignment made to basic education or skills training and a high level of participation enforcement; and
• Education-focused programs, with first assignments made to basic education or skills training and a low level of participation enforcement.

Exhibit ES-1 categorizes the sites programs.  Notably, four of the sites (Atlanta, Grand Rapids, Riverside, and Columbus) operated two different programs simultaneously, to enable rigorous side-by-side tests of the comparative effectiveness of various approaches.  Three sites implemented a Labor Force Attachment (LFA) program as well as a Human Capital Development (HCD) program, versions of employment-focused and education-focused programs that magnified the differences between the two types of approaches.  The fourth site, Columbus, implemented a program using a traditional (TRD) case management model, in which welfare eligibility and employment program functions were performed by separate staff members, and a program using an Integrated (INT) case management model, in which these two functions were performed by the same staff member.  These eight programs in four sites, described in more detail in Section II, are referred throughout by their site name and shortened program model name (LFA, HCD, TRD, or INT).

It is important to note that the studies of the programs in the education-focused category yield information about the effects of increasing welfare recipients participation in basic education programs (including Adult Basic Education, GED preparation, and English as a Second Language classes) and, to a much lesser extent, in vocation skills training programs, but not in college.  On their own, many welfare recipients enroll in various types of education or training classes and reap benefits from them; the education-focused programs in the evaluation, however, sought to increase participation in education or training activities beyond what would normally occur.  As will be discussed below, most of the programs did indeed increase such participation, but the increases in enrollments were in basic education courses and, to some degree, in vocational training courses, and not in college-level ones.

This report analyzes the programs effects for single-parent welfare recipients, focusing on results for the two years after individuals entered the programs.  This is an important period in which to gauge whether programs moved recipients from welfare to work.  Many states and localities now terminate welfare eligibility after two years.  In addition, prior research has shown that many individuals on welfare for at least two years will likely remain on the rolls for a considerably longer time.  Under TANF, these individuals would be in jeopardy of reaching their five-year limit on federal funding for welfare benefits.  Consequently, the two-year results for these 11 programs will become a benchmark for the next generation of welfare initiatives.

The report explores the following questions:

• Which welfare-to-work program approaches were most successful in helping welfare recipients to receive the program services or attain the skills or credentials that could enhance their chances of finding employment?
• Which approaches were most successful in helping welfare recipients to find paid work and leave welfare within the two-year follow-up period and to remain off welfare?  Did any approaches help individuals to get a good job, that is, a full-time job with health benefits?
• Which approaches were most successful in increasing welfare recipients income and helping them move out of poverty?
• Did any approaches positively or negatively affect the well-being of children?
• Which approaches were most successful in achieving self-sufficiency for those who were at high risk for long stays on welfare?

The NEWWS Evaluation uses an unusually strong research design, a random assignment experiment, to estimate program effects.  In each site individuals who were required to participate in the program were assigned at random to either a program group (in some sites, one of two program groups) or a control group.  Program group members had access to program-provided services and were required to participate in the program or risk a reduction in their monthly welfare grant.  Control group members received no mandatory welfare-to-work program services but could seek similar services on their own in the community.  This random assignment design assures that within each site there are no systematic differences between the background characteristics of program and control group members when they enter the study.  In addition, within each site program and control group members are subject to the same welfare grant levels, labor market conditions, and other environmental factors.  As a result, any subsequent differences in outcomes between the groups within each site can be attributed with confidence to the effects of the program.  These differences, called impacts, can then be compared across sites, yielding a much more accurate determination of which types of programs are high and low performers than simple comparisons of statistics, such as welfare caseload reductions, across localities or states.

[ Go to Contents ]

I.  Findings in Brief

An examination of the range of effects achieved by all 11 programs yielded the following information about which welfare-to-work program strategies are more or less successful in helping welfare recipients achieve self-sufficiency:

• All programs, regardless of their approach, increased participation in activities designed to promote employment during the two-year follow-up period.  As expected, employment-focused programs increased participation primarily in job search activities, whereas education-focused programs raised participation levels primarily in basic education and vocational skills training classes.  Very different patterns of participation impacts were found for individuals who entered the study with a high school diploma or GED certificate and for those who did not have these credentials.  In most education-focused programs participation impacts were concentrated among those without a high school diploma or GED and resulted primarily from large increases in attendance in basic education; only small increases in attendance in post-secondary education or vocational training were found for the education-focused programs, and they were generally among only high school graduates or GED holders.  In contrast, large impacts on participation in job search were achieved for both groups in the employment-focused programs.
• Some education-focused programs, as well as the Portland program, were able to produce relatively large impacts (about 10 percentage points) on GED attainment among sample members who did not have a high school diploma or GED certificate at study entry.  Of the seven education-focused programs, Grand Rapids HCD, Riverside HCD, and Columbus Traditional programs had this effect.  Portlands program, in addition to boosting GED receipt, increased the rate at which those without education credentials obtained a trade license or certificate by 12 percentage points.  For sample members with a high school diploma or GED certificate at study entry, only three programs (Atlanta LFA and HCD and Grand Rapids HCD) increased receipt of a trade license or certificate.
• As expected, employment-focused programs produced larger gains in employment and earnings over the two-year follow-up period than education-focused programs, but these effects may not be sustained everywhere in the long run.  Except in Riverside, the site with the most difficult labor market, a majority of control group members found jobs on their own initiative at some point within two years of random assignment and, as a group (including zeroes for nonearners), had average earnings during the second year of follow-up ranging from $2,127 (Oklahoma City) to $3,978 (Columbus).  In Portland program group members attained the largest earnings increase among all programs, averaging more than $900 per year in earnings above control group members.  Equally important, employment and earnings gains in Portland grew larger over time and reached their highest levels at the end of year 2, the end of the short-term follow-up period available for this report.  The other employment-focused programs produced moderate earnings increases, ranging from $400 to $650 per year, that grew smaller toward the end of year 2.
• Several of the education-focused programs began to show moderate impacts in year 2.  By the end of year 2 all but two of the education-focused programs had attained increases in employment and earnings that equaled or exceeded the gains achieved by all employment-focused programs except Portlands.  The two exceptions to this pattern, the Riverside HCD and Oklahoma City programs, did not raise employment or earnings levels in year 2.  Overall, these results underscore the importance of tracking the effects of education-focused programs over a longer term.
• All programs reduced welfare dependency to some degree.  Control group members in all but one site remained on welfare for an average of 16 to 20 months during the two-year follow-up period and received payments averaging between $3,624 (Oklahoma City) and $10,302 (Riverside HCD) during this same period.  Seven of the 11 programs, a mixture of employment-and education-focused approaches, decreased cumulative welfare expenditures by more than 10 percent, a historically large effect; welfare reductions in the other four programs were smaller.  Portlands program produced a large decrease in welfare receipt that persisted at a high level throughout the follow-up period, showing a 12 percentage point decrease in welfare receipt during the last quarter of the two-year period; all other programs had reduced welfare receipt at this point by 3 to 7 percentage points.  All in all, however, at least 40 percent of sample members in the programs were still relying to some extent on welfare at the end of two years.
• Most programs increased sample members reliance on earnings, as opposed to welfare, but family net incomes were largely unchanged. As a result, the programs lifted few families above the poverty line.  Reductions in welfare, Food Stamps, and other benefits generally matched or exceeded earnings gains.  Including estimates of the Earned Income Tax Credit (EITC) as income produced little change in this finding for all programs except Portlands, which attained the largest and most consistent gain in total income ($238, or $425 including the EITC estimate, for year 2 of the follow-up) and also produced a small increase in the proportion with incomes above the poverty level (4 percentage points, or 7 percentage points including the EITC estimate, in year 2).
• Although no programs had pervasive negative effects on sample members, some individuals were adversely affected.  In year 2 of follow-up six programs (some employment-focused and some education-focused) produced small increases in the proportion of sample members with combined income from AFDC, Food Stamps, and earnings equivalent to less than 50 percent of poverty levels.  In addition, several programs (representing both types of approaches) increased the rate at which individuals left welfare without a job.  Finally, some programs that increased employment also decreased family health insurance coverage (as reported by parents) and increased out-of-pocket child care expenditures.
• The programs did not have widespread, large, or consistent effects on the children of sample members, but positive and negative effects occurred in some programs.  No programs in the evaluation provided direct services (with the exception of child care assistance) to children.  Program-produced changes in the lives of sample members (virtually all mothers) may, nevertheless, influence the well-being of children.  There is evidence that some of the programs affected the likelihood of at least one child in a family having behavioral, educational, or health and safety problems.  There was not, however, a consistent pattern of benefit or harm to children.  In addition, employment- and education-focused programs did not appear to affect children differently; there was no consistent evidence that one particular approach affected children more or less or was more likely to help or harm children.
• Several employment- and education-focused programs attained at least moderate employment and earnings gains for the most disadvantaged sample members..  Five programs (Portland, Grand Rapids LFA and HCD, and Riverside LFA and HCD) increased employment and earnings for individuals who at study entry did not have a high school diploma or GED, had not worked in the prior year, and had been on welfare cumulatively for two years or more.  These five programs and two others (Detroit and Columbus Integrated) also reduced the amount of time that the most disadvantaged individuals spent on welfare during the follow-up period.
• High enforcement programs did not produce the largest impacts, but low enforcement programs resulted in only small effects.  Programs in which staff closely monitored individuals attendance in program activities, followed up quickly when problems arose, and swiftly imposed financial sanctions when individuals did not comply with program requirements, were present among both the employment- and education-focused programs.  High enforcement programs, notably those in Grand Rapids and Columbus, did not necessarily produce the largest impacts.  However, the two low enforcement programs  Oklahoma City and, in its early stages, Detroit  yielded only small impacts.  It thus appears that a minimum level of enforcement by program staff is required to produce at least moderate earnings and welfare impacts, presumably because this extra push is needed in order to engage in program activities those who normally would not participate on their own initiative.
• While many programs achieved positive effects on employment, earnings, and reduced use of welfare, few achieved large effects, except for Portland.  The Portland program was unusually successful in substantially increasing employment and earnings, helping people to get good jobs, lowering welfare receipt, and achieving these outcomes for a cross section of sample members.  The results are probably due to a combination of factors.  While its employment message was strong, the program offered high-quality education and training services as well as job search, enforced a participation mandate, and had strong job development and placement services.  In addition, contextual factors may have contributed to the programs success.  In particular, it worked with a less disadvantaged welfare caseload (relative to the other studied programs) and operated within a good labor market with a relatively high state minimum wage.

The remainder of this summary details these findings.  First, however, it describes the key welfare-to-work program approaches contrasted in the analysis and explains the evaluations research design and samples.

[ Go to Contents ]

II.  Program Approaches and Implementation Features

As noted above, the evaluations sites implemented very different programs; in fact, the research designs in several of the sites were set up to rigorously compare the effects of specific program approaches.  This section discusses the two key implementation features used in this report to define four broad program approaches.  In addition, for context, other major program dimensions are described.

A.  Employment- or Education-Focused

Since the late 1960s welfare-to-work programs seeking to increase welfare recipients self-sufficiency have emphasized one of two strategies.  One strategy emphasizes quick employment, reflecting the belief that individuals can best build their employability, and eventually achieve self-sufficiency, through actual work, even if their initial jobs are minimum wage and without fringe benefits.  The other strategy emphasizes skill-building, particularly in the education area, reflecting the view that individuals should first invest in education or training to enable them to eventually obtain higher-wage, longer-lasting jobs with health insurance coverage.  The 11 NEWWS programs blend elements of both strategies to varying degrees.

As shown in Exhibit ES-1 four programs (Atlanta, Grand Rapids, and Riverside LFA and Portland) were employment focused.  They provided job search assistance to a large segment of their caseload and encouraged enrollees to find work as quickly as possible.  Further, both the Portland and Riverside programs employed full-time job developers to help place program enrollees in unsubsidized jobs.

The three LFA programs, however, differed from Portlands program in important ways.  The LFA programs routinely assigned individuals to job search assistance, usually job club, as their first activity, whereas Portlands program offered GED preparation classes to those deamed by case managers to have a good chance of attaining a GED certificate relatively quickly.  (Activities initially assigned are an important clue to the treatment experienced by welfare recipients, as many people leave welfare or become exempt or temporarily excused from welfare-to-work programs prior to being assigned to a second program activity.)  Further, Portland case managers encouraged enrollees to hold out for jobs that paid well above the minimum wage and offered the best chance for long-lasting and stable employment.  In contrast, case managers in the LFA programs, especially in Riverside, stressed the value of starting off with any job, even a low-paying one, and then advancing toward more stable and better-paying jobs in the future.

Seven programs (Atlanta, Grand Rapids, and Riverside HCD; Columbus Integrated and Traditional; and Detroit and Oklahoma City) can be characterized as education-focused.  (See Exhibit ES-1.)  A large percentage of enrollees in these programs were initially assigned to some type of skill-building activity.  The types of activities to which enrollees were first assigned depended, in part, on the level of educational attainment that individuals had achieved prior to entering the program.  Those who had not completed high school or received a GED certificate but who were assessed by case managers as having high school-level skills were assigned to GED preparation classes.  Those with lower reading or math levels were assigned to Adult Basic Education classes.  In addition, non-English speakers could be assigned to English as a Second Language (ESL) programs.  Finally, those who had completed high school or held a GED certificate could be assigned to vocational training or employment-oriented skills courses at local community colleges.  All in all, however, assignments to GED preparation or basic education courses were more common than assignment to vocational training programs in these education-focused programs, primarily as a result of welfare recipients low levels of educational achievement; enrollment in college played an even smaller role.

Some differences existed among the seven education-focused programs.  The three HCD programs usually assigned enrollees to education or training programs as their first activity.  Case managers in Columbus, Detroit, and Oklahoma had more discretion over activity assignments, but, in practice, most program enrollees were initially assigned to education or training activities in these sites as well.  Riversides HCD program was also unique among this group in that it did not serve high school graduates and GED holders who, at program entry, scored above minimum levels in reading and math tests.

B.  High or Low Enforcement of the Participation Mandate

The degree to which a program enforces a participation mandate can be viewed as a product of three factors:  how wide a cross section of the welfare caseload is enrolled in a program; how closely a program monitors individuals participation; and how swiftly and consistently a program imposes financial sanctions, that is, reductions in monthly welfare grants, on those who do not participate.

All four employment-focused programs, and five of the seven education-focused programs, can be considered high enforcement programs; the remaining two education-focused programs, Detroit and Oklahoma City, can be considered low enforcement programs.  While technically requiring enrollment from a cross section of their mandatory caseloads, these latter two programs put a priority on working with those individuals who expressed interest in participating in the program.  In addition, resource constraints kept staff in these sites from closely monitoring individuals participation in program activities.  Finally, staff in these two sites rarely invoked financial sanctions.  In contrast, program staff in the other programs generally enrolled and worked with a cross section of the welfare applicants and recipients who were required to participate; monitored participation more closely; and, especially in Columbus and Grand Rapids, frequently invoked sanctions for nonparticipation.

C.  Other Key Program Features

Other implementation features, beyond those discussed above, can also potentially influence a programs effectiveness.  Two of them  the level of child care support provided and the structure of program case management  are described here.

All 11 studied programs offered child care assistance to welfare recipients who needed it while they were participating in program activities or employed.  Oklahoma City, Portland, and Detroit provided the strongest staff support for arranging child care.  Staff in these programs helped to make child care arrangements and also helped those who found jobs to obtain transitional child care assistance.  In contrast, case managers for both Riverside programs did not provide much assistance in setting up child care arrangements, encouraged enrollees to use low- or zero-cost informal child care while they were participating in program activities, and did not actively promote the use of transitional child care benefits.

The programs also differed in their case management strategies.  Columbus Integrated, Portland, and Oklahoma City implemented an integrated case management staffing arrangement.  That is, case managers in these sites combined responsibilities normally performed by income maintenance staff (determining welfare eligibility, calculating welfare grants, invoking financial penalties, and arranging for transitional benefits) with responsibilities usually assigned to welfare-to-work program staff (assigning enrollees to employment-related activities, arranging for child care, and monitoring participation).  Columbus Integrated and Portland staff had sufficient resources and small enough caseloads to perform both of these roles, enabling them to promote a consistent self-sufficiency message.  In contrast, in Oklahoma City limited resources and large caseloads led case managers to put most of their overall emphasis on the financial functions of their job.

The Atlanta, Grand Rapids, and Riverside LFA and HCD, Columbus Traditional, and Detroit programs all used a traditional case management structure, in which each welfare recipient had two different case managers.  Commonly, income maintenance workers knew little about the welfare-to-work program in their site.  Among these sites, the staffing division was most pronounced in Detroit.

[ Go to Contents ]

III.  Research Designs and Samples

In Atlanta, Grand Rapids, and Riverside welfare recipients were randomly assigned to either an LFA or an HCD program group or to a control group.  (See Exhibit ES-2.)  Both types of programs operated simultaneously in these three sites.  In Columbus a three-group random assignment design was used as well.  Here, the two program groups represented two case management models:  integrated and traditional.  The remaining three sites in the evaluation  Oklahoma City, Detroit, and Portland  used random assignment to test the effectiveness of established programs, as opposed to programs designed to meet research protocols; individuals were randomly placed in either a group that entered the program or a nonprogram control group.  Note that control group members were eligible for child care assistance similar to that offered to program group members if they were participating in nonprogram activities in which they had enrolled on their own.

Individuals were randomly assigned to research groups over approximately a two-year period in each site.  Random assignment for the evaluation began in June 1991 in Riverside and ended in December 1994 in Portland.  Thus, the results presented in this report cover the calendar period from June 1991 (the first sample members entry into the study) through December 1996 (the last month of the two-year follow-up for the last sample member randomly assigned in Portland).

Differences in research design and random assignment procedures affected the composition, and thus comparability, of the samples across sites.  (See Exhibit ES-2.)  In five of the seven sites AFDC applicants and recipients were randomly assigned while attending a program orientation; in the other two sites (Columbus and Oklahoma City) individuals were randomly assigned before they were referred to a program.  Since some individuals typically exit welfare for employment or other reasons before attending a program orientation, the samples in Columbus and Oklahoma City include a larger share of individuals who quickly left welfare.⁽²⁾

The programs also differed in how broadly or narrowly they targeted enrollment.  Most notably, Oklahoma City randomly assigned only welfare applicants (that is, persons in the process of applying for welfare), including those whose application for assistance was not yet approved.  Additionally, Detroit, Grand Rapids, Oklahoma City, and Portland extended their program coverage to mothers with children as young as age 1, whereas the remaining programs exempted parents whose youngest child was under age 3.  Riverside limited enrollment in its HCD program to individuals determined by program regulations to need basic education because they lacked a high school diploma or GED certificate, attained low scores on a reading or math exam administered at program entry, or had limited proficiency in English.  Finally, other pro-grams limited enrollment (and thus those eligible for random assignment) by capping caseloads for program staff and establishing waiting lists for enrollees (Atlanta) or by excluding those who, in the judgment of program staff, had serious barriers to participation (Portland).

Because of these and other factors, the research samples differed across the seven sites in key background characteristics likely to affect individuals chances of finding employment and leaving welfare.  For instance, excluding the Riverside HCD program, the proportion of sample members who had completed high school or attained a GED certificate prior to random assignment ranged from 55 percent (Oklahoma City) to 66 percent (Portland); the proportion who had ever worked full time for at least six months for the same employer ranged from 43 percent (Columbus) to 77 percent (Portland); and, excluding Oklahoma City, between 28 and 50 percent of sample members in the sites had received welfare cumulatively for five years or more.

[ Go to Contents ]

IV.  Findings

A.  Program Participation and Enforcement

• Many control group members took part in education and training activities on their own initiative.  All programs, however, were able to increase participation levels in employment-related activities above the control groups rate of activity during the two-year follow-up.  The size of the increase was associated with the degree of enforcement of the participation mandate, but not with the program approach.

Between 19 and 42 percent of control group members surveyed in each site reported participating during the two-year follow-up period in an employment-related activity, such as basic education, skills training, post-secondary education, or formal job search.  As shown in Exhibit ES-3, all programs increased participation beyond these levels of self-initiated activity, from 9 to 40 percentage points above control group participation levels.  Overall, program participants were generally involved in activities for at least several months.

Exhibit ES-3 Impacts on Participation in Any Employment-Related Activity

SOURCE:  MDRC calculations from the Two-Year Client Survey.

All but one of the programs with high enforcement of the participation mandate (including both employment- and education-focused programs) produced large impacts on participation (above 20 percentage points).  Participation impacts were much smaller for the two low enforcement programs (Detroit and Oklahoma City).  In these two sites the programs efforts increased the number of welfare recipients who participated in activities only slightly beyond what they would have done on their own, in the absence of a mandatory welfare-to-work program.

• As expected, all of the employment-focused programs produced large increases in participation in job search activities.  Some also produced small increases in participation in education and training.

The four employment-focused programs increased job search participation by 27 (Grand Rapids LFA) to 32 percentage points (Portland and Riverside LFA), compared with control group levels.  (See Exhibit ES-4.)  The programs achieved large gains for people who entered the program with a high school diploma or GED certificate and for nongraduates.  Enrollees in the employment-focused programs could be assigned to short-term education or training if they completed job search without finding employment (or, in Portland, at program entry).  The Atlanta LFA and Portland programs produce small increases in participation in education or training.

Exhibit ES-4 Impacts on Participation in Job Search and Education or Training

SOURCE:  MDRC calculations from the Two-Year Client Survey.

• Most of the education-focused programs raised participation levels in education or training.  To a lesser extent, the programs also increased participation in job search.

As shown in Exhibit ES-4, the education-focused programs increased participation in education or training by 10 to 35 percentage points (Oklahoma and Riverside HCD, respectively) compared with control group levels.  (Detroits increase in education or training participation was not statistically significant.)

While the increases for some programs were small when all sample members are considered, most of the education-focused programs achieved large increases in participation in education or training for sample members lacking a high school diploma or GED certificate at random assignment (not shown in Exhibit ES-4).  Most of these increases are accounted for by participation in basic education.

When enrollees in the education-focused programs completed education or training, they were often assigned to job search.  As Exhibit ES-4 illustrates, all of the education-focused programs raised job search participation levels to some extent; impacts were similar for high school graduates and nongraduates.

• Most programs produced only small increases in participation in work experience or on-the-job training.

TANF participation requirements encourage states to enroll welfare recipients in unpaid work or on-the-job training.  None of the programs in the evaluation made extensive use of these activities, but most were able to produce small impacts on participation in such activities because even fewer control group members participated in them.  (Participation impacts on these activities are not shown in Exhibit ES-4.)

• The 11 programs varied widely in their use of financial sanctions, or AFDC grant reductions, to enforce mandatory participation requirements.  Sanction rates in most of the employment-focused programs were moderate, but rates in the education-focused programs ranged from very low to very high.

In three of the four employment-focused programs (Atlanta and Riverside LFA and Portland) between 11 and 18 percent of program group members reported that they were sanctioned for noncompliance with program participation requirements during the follow-up period.  Three education-focused programs (Grand Rapids HCD and Columbus Integrated and Traditional) and an employment-focused program (Grand Rapids LFA) had high sanction rates, ranging from 26 to 32 percent of program group members.  At the other extreme, almost no program group member in the low enforcement education-focused programs in Detroit and Oklahoma City reported being sanctioned.

B.  Receipt of Education or Training Credentials

• Some of the education-focused programs, as well as the Portland program, produced relatively large impacts on GED certificate attainment among sample members who entered the program without a high school diploma or GED certificate.

As noted above, most education-focused programs increased participation in basic education among nongraduates, but only three of these programs (Grand Rapids and Riverside HCD and Columbus Traditional) increased GED certificate attainment for this subgroup.  Impacts on GED receipt ranged from 8 to 11 percentage points.  Notably, Portland achieved similar gains in GED receipt.  (The other three employment-focused programs had no effect on GED attainment.)

• For those entering with a high school diploma or GED, a few programs increased the proportion who received a trade license or certificate.  One program increased the proportion of nongraduates who received a trade credential.

Two education-focused programs (Atlanta and Grand Rapids HCD) and one employment-focused program (Atlanta LFA) increased receipt of a trade license or certificate for sample members in the graduate subgroup.  Impacts ranged from 5 percentage points (Atlanta LFA) to 11 percentage points (Atlanta HCD).  Portland increased receipt of a trade license or certificate by 12 percentage points among those entering the program without a high school diploma or GED.  (Only Portlands program had this effect for the nongraduate subgroup.)

C.  Employment and Earnings

• Employment-focused programs produced larger gains in employment over the two-year follow-up period than most of the education-focused programs.

Six of the seven sites in the evaluation experienced economic growth and strong labor markets during the first years of follow-up; aided by these conditions, a majority of control group members in these sites (from 58 to 72 percent) worked for pay at some point during the two-year follow-up period.  Jobs were much harder to find in Riverside; only 45 percent of control group members were employed during the follow-up period.

As shown in Exhibit ES-5, all four employment-focused programs increased two-year employment levels, from 5 percentage points (Atlanta LFA) to 15 percentage points (Riverside LFA).  (Exhibit ES-5 shows outcomes for both program and control groups and the differences between the two groups outcomes, that is, the impacts; other exhibits present only the impacts for the various outcomes discussed.)  As described above, education-focused programs delayed job finding in the short term.  Not surprisingly, employment gains for most of these programs fell below those of the employment-focused programs.  Three of the seven education-focused pro-grams produced no statistically significant increase in employment (Columbus Integrated and Traditional and Oklahoma), and the other education-focused programs increased employment between 3 and 9 percentage points (Atlanta and Riverside HCD, respectively).

 

• Employment-focused programs produced much larger gains in earnings over the two-year follow-up period than education-focused programs.

Earnings for control group members in the seven sites averaged between $3,133 and $6,892 (including zeroes for those with no earnings) over the two-year follow-up period.  As Exhibit ES-6 illustrates, Portland increased earnings by an average of $1,842 per program group member.  This earnings gain is much larger than that of the other three employment-focused programs and exceeds that of all previously evaluated mandatory welfare-to-work initiatives, except the Riverside GAIN program of the late 1980s (another employment-focused, varied first activity program).  Earnings gains in the other employment-focused programs in the evaluation were moderate, ranging from $813 to $1,276 (Atlanta LFA and Riverside LFA, respectively).  Earnings gains in the education-focused programs were smaller; statistically significant gains ranged from $367 to $677 (earnings impacts in Riverside HCD and Oklahoma were not statistically significant).  Neither of the two low enforcement programs (Oklahoma City and Detroit) produced substantial earnings increases.

Exhibit ES-6 Impacts on Two-Year Earnings

SOURCE:  MDRC calculations from unemployment insurance (UI) earnings records.

• Over time, the employment and earnings gains diminished in most of the employment-focused programs, but increased in most of the education-focused programs.  By the end of the two-year follow-up period some of the education-focused programs had caught up to the employment-focused programs.

The earnings gains in two of the three LFA programs (Grand Rapids and Riverside) diminished over time, as increasing numbers of control group members began finding jobs on their own.  In the last quarter of year 2 the three LFA programs raised employment levels by only 4 percentage points and increased average earnings by about $100.  (Exhibit ES-7 shows employment levels over the follow-up period, averaged across programs within each approach.)

Exhibit ES-7 Quarterly Impacts on Employment Over Two Years, Averaged Across Sites by Approach

SOURCE:  MDRC calculations from unemployment insurance (UI) earnings records.

In contrast, gains increased in most of the education-focused programs.  By the last quarter of year 2, impacts on employment and earnings for five education-focused programs (Atlanta and Grand Rapids HCD, Columbus Integrated and Traditional, and Detroit) were similar to or slightly larger than impacts for the three LFA programs:  employment gains ranged from 3 to 6 percentage points and earnings gains ranged from $93 to $179.  Overall, these results underscore the importance of tracking the effects of education-focused programs over a longer period than two years.

Unlike the effects in other employment-focused programs, in Portland positive effects on employment and earnings increased over time:  in the last quarter of follow-up the program group employment level was 11 percentage points higher than the control group level, and the program group earned on average $310 more.  These impacts are far larger than those of any other program in the evaluation.

• Portlands program produced the largest, most consistent increases in employment stability and job quality during the follow-up period.

Portlands employment-focused, varied first activity program increased the proportion of people who worked all four quarters of year 2 by 8 percentage points and who earned $10,000 or more in year 2 by 6 percentage points.  At the end of year 2 (as measured from survey responses) the program increased the percentage of people working at full-time jobs and at jobs that offered health coverage.  It also increased average hourly pay for those working, but this finding, since it is based on a nonexperimental comparison (different types of individuals in the program and control groups may have been working) is more speculative.  The Riverside LFA program also increased full-time employment with health benefits and higher hourly earnings, but to a lesser extent than the Portland program.  Contrary to expectations, the education-focused programs increased job quality to only a small extent or not at all by the end of two years.

D.  Public Assistance Receipt and Payments

• All programs reduced AFDC receipt to some degree.  On average, decreases for the employment-focused programs were larger, but decreases for some education-focused programs rivaled or exceeded decreases for some employment-focused programs.

All programs lowered the proportion of welfare recipients who would have reached a two-year welfare time limit, had one been in effect.  Control group members in all but one site received AFDC for an average of 16 to 20 months during the two-year follow-up period.  (The exception was Oklahoma City, where the all-applicant sample averaged 12 months of receipt.)  The programs reduced the average number of months of AFDC receipt months (or 2 to 16 percent).  Two employment-focused programs, Grand Rapids LFA and Portland, produced the largest decreases (2.21 months and 2.41 months, respectively).  Decreases for the education-focused programs ranged from 0.48 to 1.58 months.

In the last quarter of follow-up between 41 percent (in Oklahoma City) and 74 percent (in Detroit) of control group members received an AFDC check.  Portland produced the largest reduction in the proportion of sample members receiving AFDC at this point (12 percentage points).  Among the other programs, reductions in the proportion receiving AFDC at the end of year 2 ranged from 6 to 7 percentage points for the three LFA programs and from 3 to 7 percentage points for the education-focused programs.

• All programs but one decreased average AFDC payments over the two-year follow-up period.

Control group members received AFDC payments over the two years averaging between $3,624 and $10,302 (including those who left welfare during the two-year follow-up period).  Three employment-focused programs (Grand Rapids and Riverside LFA and Portland) and one education-focused program (Riverside HCD) reduced payments by more than $1,000 (representing decreases of 10 to 19 percent, relative to payments to the control group).  (See Exhibit ES-8.)  Three other programs, all education-focused (Grand Rapids HCD and Columbus Integrated and Traditional), also reduced two-year welfare expenditures per program group member by 10 percent or more.  Detroits program produced only a slight, not statistically significant, decrease in AFDC payments over the two years.

Exhibit ES-8 Impacts on Two-Year Welfare Payments

SOURCE:  MDRC calculations from state and county AFDC records.

• Most programs reduced Food Stamp receipt and expenditures during the follow-up period.

Eight of the 11 programs decreased average Food Stamp expenditures over the two-year follow-up period and decreased the proportion of people who received Food Stamps in the last quarter of year 2.  Decreases in two-year expenditures ranged from 2 to 13 percent and decreases in receipt at the end of follow-up ranged from 4 to 8 percentage points.  One employment-focused program and two education-focused programs had no effect on Food Stamp receipt (Atlanta LFA and HCD and Oklahoma City).

E.  Employment and Welfare Status at the End of Two Years

• In all programs a substantial proportion of enrollees were receiving AFDC at the end of the two-year follow-up period.

Across all programs as many as 7 in 10 program group members (in Detroit) remained on welfare at the two-year mark.  Even in programs that moved the largest proportion of sample members off welfare, at least 4 in 10 enrollees remained on welfare.  This offers a caution to states striving to achieve very rapid self-sufficiency for virtually all welfare recipients.

• Most programs increased the proportion of people who were working and not receiving AFDC at the end of the follow-up period.

In the last quarter of year 2 between 13 and 27 percent of control group members were employed and receiving no AFDC payments.  All programs but two (Riverside HCD and Oklahoma City) increased the proportion of people in this status.  (See Exhibit ES-9.)  Impacts were generally small, with two programs (Portland and Columbus Integrated) achieving moderate increases.  The impacts, which ranged from 2 to 9 percentage points, were not associated with program approach.

 

• Several programs representing both approaches slightly increased the rate at which individuals left welfare without a job.

The proportion of control group members who were not employed and not receiving AFDC in the last quarter of year 2 ranged from 12 to 37 percent.  All of the employment-focused programs and three of the seven education-focused programs increased the percentage of sample members in this status at the end of two years.  (See Exhibit ES-9.)  Increases were small in every program, ranging from 2 to 5 percentage points.  The majority of people in this status reported having some other source of income and/or living with someone else who worked or who had another source of income.

F.  Income and Poverty

• Most programs had little or no effect on income.

In the second year of follow-up control group members averaged between $5,596 (Oklahoma City) and $9,322 (Detroit) in combined income from earnings, AFDC, and Food Stamps.  Few programs substantially altered these combined income levels; in general, reductions in AFDC, Food Stamps, and other benefits matched or exceeded earnings gains.  However, in three programs  Grand Rapids and Riverside LFA (employment-focused) and Riverside HCD (education-focused)  combined income in the second year of follow-up was reduced by $230 to $571, or 3 to 7 percent.  (See Exhibit ES-10.)  In Portland (employment-focused) and Atlanta HCD (education-focused) combined income increased in the second year by $425 and $295, or 5 and 4 percent, respectively.  This combined income measure includes estimates of the EITC; when EITC estimates are not included, losses and gains are somewhat smaller, the Portland and Atlanta HCD gains are no longer statistically significant, and a small loss in the Grand Rapids LFA program becomes statistically significant.

 

• Because income changes were minor, few programs lifted many families out of poverty.  Some programs, however, had the effect of pushing a small proportion of families deeper into poverty.

By design, the combined income from welfare and Food Stamp grants provides less than poverty-level income.  Only by working can people hope to attain enough income to escape poverty.  In the second year of follow-up between 11 and 26 percent of control group members had combined income from earnings, AFDC, Food Stamps, and estimated EITC receipt that equaled or exceeded the federal poverty level.  Five programs increased the proportion of people living at or above poverty by a small amount.  (See Exhibit ES-10.)  Portland was the most successful, producing a 7.5 percentage point gain; impacts for other programs were small, ranging from 2 to 3 percentage points.  Program-control differences for most of the other seven programs were positive but very small and not statistically significant.

In the second year of follow-up between 19 and 48 percent of control group members had combined income, including estimated EITC, totaling less than 50 percent of the poverty line.  Six programs (both employment- and education-focused) slightly increased the proportion of sample members living below 50 percent of the poverty level; they led to increases of between 2 and 6 percentage points in the proportion of individuals living deeply in poverty.

G.  Health Care Coverage and Child Care Expenses

• Some programs that increased employment levels and decreased welfare receipt also decreased reported rates of health care coverage.

At random assignment, almost all sample members in the evaluation had health coverage because they were receiving AFDC and were automatically covered under Medicaid.  (In Oklahoma City, applicants for assistance whose eligibility was not yet determined were included in the sample, so initial coverage rates there were lower.)  Over time, coverage rates declined for both program and control group members, as some people left AFDC and did not replace their Medicaid coverage with coverage from employers or other sources.  By the end of the follow-up period between 81 percent (Columbus) and 88 percent (Detroit) of control group members reported having health coverage for themselves and their children.  (This range covers all sites except Oklahoma, where 68 percent reported coverage for themselves and their children.)

Two employment-focused programs (Riverside LFA and Portland) and one education-focused program (Columbus Integrated) that increased employment and decreased welfare receipt at the end of follow-up period also lowered health care coverage levels by 4 to 7 percentage points.  (Impacts in Portland were not statistically significant, but were just beyond the .1 level of statistical significance used as the standard throughout this report.)  Although many program group members who left AFDC (and automatic Medicaid coverage) found jobs that provided health insurance, received Transitional Medicaid benefits, or obtained alternative sources of coverage, others were not able to replace the coverage they had under Medicaid.  Some of these respondents never received Transitional Medicaid, and others had exhausted or had not restarted their benefits at the end of the two-year follow-up period.

Program group members in Oklahoma City reported even larger decreases in coverage  11 percentage points.  This program decreased welfare receipt and appears to have increased short-term employment  in jobs not reported to the states unemployment insurance system  that did not provide health insurance, especially for sample members children.  The other seven programs in the evaluation did not affect health coverage rates for respondents or children.

• Some programs increased child care use while employed and out-of-pocket child care expenditures, an increase due to greater child care use among those who found jobs as well as an overall increase in employment levels.

Between 29 and 44 percent of all control group members (including those who never worked) used child care while employed at some point during the two-year follow-up period.  Seven programs  the four employment-focused programs and three of the seven education-focused programs  produced moderate to large increases in child care use while employed, ranging from 4 to 13 percentage points.  Impacts on paid child care use, that is, care paid for by either the sample member, the welfare department, the father of the child(ren), or the sample members employer, were found in nine programs and were similar in magnitude.

The increases in child care use and in paid care use while employed are not entirely explained by the programs impacts on employment; in many programs, of those who worked during the follow-up period, a greater proportion of program group members than control group members used child care (or paid care) as well.  The likely explanation for this finding is that employed program group members required or preferred more stable child care arrangements than employed control group members.  This could be partly due to differences in the characteristics of the jobs acquired by program and control group members (for example, program group members jobs were more likely to be full time).  It is also possible that program group members heeded the messages they were given by their caseworkers  messages probably delivered more frequently to program than control group members  concerning the importance of obtaining paid, stable child care.

Relatively few program and control group members used transitional child care benefits.  Five programs increased the use of such benefits, but these effects were large only in Atlanta LFA and Portland, where increases of 7 and 11 percentage points in the receipt of these benefits, respectively, were found.

H.  Well-Being of Children

• Some of the welfare-to-work programs affected children, although the effects were not large or consistent across outcome measures or programs.  Notably, the found effects on children were both positive and negative.

The NEWWS Evaluation is one of the first random assignment evaluations of mandatory welfare-to-work programs to examine programs effects on the well-being of children.  The children of sample members in the evaluation were often quite young.  As noted earlier, in three of the sites women with children as young as age 3 were required to participate in welfare-to-work programs; in the other four sites the mandate was extended to include women with children as young as age 1.  Because many of the child outcome measures used in the evaluation pertained only to children of school age, however, the child impacts discussed here are primarily for the subgroup of sample members who had no children under age 6.

Control group members in the seven sites had, on average, two to three children.  Across the sites an average of one-quarter of the control group members in the subgroup with no children under age 6 reported that at least one of their children had been suspended from school at some point during the two-year follow-up period.  A smaller share of control group members  8 percent to 23 percent, depending on the site  reported having a child who had repeated a grade in school during the follow-up period.  A relatively small proportion of all control group members  less than 8 percent in any site  reported that a child had been removed from their care during the two-year follow-up period.

On measures of childrens behavioral adjustment, such as suspension from school, eight of the programs produced at least one statistically significant effect on children among the subgroup of families with no children under age 6.  (See Exhibit ES-11.)  Three programs decreased the incidence of at least one behavioral problem, and five programs increased the frequency of at least one.  Only two programs, however, had an effect on more than one behavioral adjustment measure.  Fewer program effects were found on childrens progress in school, such as grade repetition, than on behavioral problems.  Only two programs had any effects in this area, but, notably, these effects were favorable.  Effects on childrens health and safety were also rare.  Only two programs had any effect on children being removed from their mothers care (small increases in the incidence of this event) and no programs affected the likelihood of children being taken to the hospital because of an accident, injury, or poisoning.

Exhibit 11: Impacts on Child Outcomes for Subgroup of Families with No Children Under Age 6

	Behavioral Adjustment			School Progress		Health and Safety
Site or Program	Suspended	Behavioral or Emotional Problem(a)	Attends a Special Class for Behavioral Problems	Repeated a Grade	Attends a Special Class for Learning Problems	Removed form Mother's Care	Taken to hospital for Accident, Injury, or Poisoning
Atlanta LFA	-2.5 ns	-4.4	-4.0	-1.4 ns	-1.2 ns	-1.6 ns	-1.9 ns
Grand Rapids LFA	4.9 ns	1.9 ns	9.5	4.2 ns	3.5 ns	-0.2 ns	-1.3 ns
Riverside LFA	6.8	-2.3 ns	3.7	-3.2	-0.5 ns	-0.2 ns	1.4 ns
Riverside LFA: (Lacked high school diploma or basic skills)	2.2 ns	-2.6 ns	3.9 ns	-3.9 ns	2.8 ns	-0.1 ns	-2.5 ns
Portland	-9.4 ns	-11.3 ns	-2.5 ns	-1.2 ns	-0.2 ns	-1.7 ns	3.0 ns
Atlanta HCD	0.1 ns	-2.5 ns	1.1 ns	0.1 ns	0.0 ns	-0.7 ns	0.9 ns
Grand Rapids HCD	1.9 ns	3.7 ns	8.7	-0.2 ns	4.8 ns	4.0	-1.7 ns
Riverside HCD	1.5 ns	1.1 ns	5.1	-1.8 ns	3.8 ns	1.4 ns	-0.1 ns
Columbus INT	-3.1 ns	-6.7 ns	-5.9	-3.2 ns	-10.1	1.2 ns	2.5 ns
Columbus TRD	3.7 ns	2.7 ns	-1.6 ns	-3.6 ns	-3.3 ns	6.0	4.8 ns
Detroit	-2.1 ns	1.6 ns	2.8 ns	-1.9 ns	-.8 ns	1.1 ns	9.1 ns
Oklahoma City	11.1 ns	17.3	2.0 ns	5.9 ns	-3.3 ns	1.8 ns	-0.3 ns
SOURCE: MDRC calculations from Two-Year Client Survey. NOTES:a.  Includes both respondents who reported that any of their children received help for behavioral or emotional problems and respondents who felt that any of their children needed to get this kind of help, if they were not already receiving it. ns = Not statistically significant.

• No explanations are clearly evident regarding the mechanisms through which some of the programs affected children.

Program-specific differences in employment/education focus, sanctioning practices, and impacts on adult educational attainment, employment, and household composition could not be clearly linked to the programs effects on children.  It could be that reductions in income played a role:  some evidence suggests that those few programs that raised earnings levels, but reduced welfare and Food Stamps even more, resulted in adverse effects on children.  In addition, child care policies may have made a difference in the programs effects on children.  Finally, the envi- ronments in which the programs operated (for example, their labor markets) may have been important.  Further research is needed to decide if or how these factors mediate the effects of welfare-to-work programs on children.⁽³⁾

I.  Key Subgroups of Welfare Recipients

• Interestingly, employment-focused programs were more likely than education-focused programs to achieve employment and earnings gains over the two-year follow-up period for those who entered the study without a high school diploma or GED certificate, but the difference in impacts narrowed by the end of the second year.

Many programs produced employment and earnings gains for both those with and those without a high school diploma or GED at random assignment.  (See Exhibit ES-12.)  Among non-graduates all of the employment-focused programs boosted two-year employment levels  by more than 10 percentage points in Riverside LFA and Portland  and increased average earnings per program group member in year 2.  In contrast, only three of the seven education-focused programs increased employment levels over two years, and only two programs increased year 2 average earnings.

At the end of the follow-up, however, one education-focused program (Columbus Integrated) was achieving the largest earnings gains of any program for nongraduates, and two others (Grand Rapids HCD and Columbus Traditional) attained larger earnings and/or employment impacts than two of the employment-focused programs (Atlanta and Riverside LFA).  These results suggest that additional follow-up will be necessary to determine which kind of program approach is more effective for nongraduates in the long run.

 

• Several programs produced moderate to large employment and earnings gains for the most disadvantaged sample members.

Between 5 percent (Oklahoma City) and 28 percent (Riverside HCD) of the sample members in each site were welfare recipients who at study entry did not have a high school diploma or GED, had not worked in the prior year, and had received AFDC cumulatively for two years or more.  Only a small proportion of control group members in this most disadvantaged subgroup became employed on their own during the two-year follow-up period (less than half in any site).

Five programs (Grand Rapids and Riverside LFA and HCD and Portland) increased employment and earnings for the most disadvantaged subgroup.  (See Exhibit ES-13.)  Each of these programs increased the proportion who worked for pay during the follow-up period by more than 10 percentage points.  Gains in year 2 earnings were moderate ($800 or more) in two employment-focused programs (Grand Rapids LFA and Portland) and smaller (between $605 and $667) in the three other programs.  These programs and two others (Detroit and Columbus Integrated) also reduced the amount of time that these most disadvantaged individuals received AFDC during the two-year follow-up period.

 

• Overall, both program approaches were less successful in helping people who had worked in the year before program entry, that is, a less disadvantaged subgroup of the caseload.

Between 22 percent (Riverside HCD) and 55 percent (Oklahoma City) of sample members in each site had worked for pay during the year prior to random assignment; 63 to 89 percent of control group members with this characteristic, depending on the site, became employed at some point during the two-year follow-up period.

Only two employment-focused programs (Grand Rapids LFA and Portland) and one education-focused program (Grand Rapids HCD) increased both employment and earnings beyond what would have happened in the absence of the programs, for these sample members.  Given the large proportion of control group members in this subgroup with employment and earnings in the two-year follow-up, impacts for this subgroup, when expressed as a percentage change, were rather small.

[ Go to Contents ]

V.  Conclusions

This evaluation, which used a random assignment experiment, provides solid information about the effectiveness of various types of welfare-to-work program approaches.  Its unusually strong research design isolates the effects of the programs themselves; the results reported above thus can be confidently attributed to the programs operated in the seven sites and not to improvements in the sites labor markets, population changes, or other policy reforms.

The reports findings, in conjunction with those of previous studies, suggest that strongly employment-focused programs that offer a variety of employment services are more effective than programs that offer primarily job search or education and training.  Portlands employment-focused, varied first activity program stands out as unusually successful among the 11 programs in this evaluation.  The Riverside GAIN (Greater Avenues for Independence) program of the late 1980s, often considered the benchmark for other welfare-to-work programs, was also an employment-focused, varied first activity program.  Both Portland and Riverside GAIN substantially increased employment levels, produced the largest earnings gains ever found for mandatory welfare-to-work programs, and had large impacts on welfare receipt.  Both were successful for a wide range of subgroups, including the more disadvantaged members of the caseload.  Operationally, the programs stressed the importance of finding jobs and enforced program participation requirements, but they offered many different services, including job search (along with job development), short-term education, and, in Portland, training.  In both programs people considered not ready to enter the labor market were first assigned to basic education or, in Portland, to training or life skills classes.  Although the 1996 welfare law encourages an employment focus, the available research findings indicate that states can augment the success of their programs by offering education and training as well as job search.

The report also illustrates, however, the limitations of even high-performing welfare-to-work programs:  Although all of the programs in this evaluation had some positive effects, they generally did not produce large changes in peoples lives during the follow-up period.  For example, the programs helped a substantial number of individuals replace income from AFDC and Food Stamps with income from jobs, but had not, as of two years, lifted many families out of poverty.  (Additional years of follow-up may show income gains, partly because of the increase in the value of the EITC in recent years.)  Also, although all programs reduced welfare dependency to some degree, many people were still on welfare at the end of the two-year follow-up period (between 38 and 70 percent of those subject to the programs, depending on the site).

Proponents of welfare time limits contend that the impending assistance cutoff will spur people into the labor market and promote self-sufficiency.  The programs in this evaluation, which are similar to many programs being run under the new welfare law, operated without such a welfare time limit.  (In addition, these programs did not try to meet the new laws participation goals, impose full-family financial sanctions, or put in place the generous financial work incentives of many current programs.  They also did not have available to them the recent and substantial increases in federal funding for child care or expanded eligibility for health insurance through Medicaid and the State Childrens Health Insurance Program.)  Future research will indicate whether programs run in conjunction with time limits or other recent welfare policy changes will be considerably more successful than the programs previously operated.  The present study does suggest, however, that strategies are needed to enable newly employed individuals to keep working and to help them raise their earnings.  Even in the very successful Portland program, less than one-third of all program group members worked in all four quarters of the second year of follow-up; less than one fifth of the total sample earned at least $10,000 in that same year.  Future programs will need to produce more sustained employment impacts and much bigger earnings impacts than those produced by any pre-TANF program that has been studied so far if large numbers of people are to find employment that can adequately support their children before reaching a welfare time limit.

[ Go to Contents ]

---

Footnotes

1.  Pub.L. No. 104-193.

2.  Riverside, Grand Rapids, and Portland implemented an additional random assignment study of the effects  independent of participation in welfare-to-work program activities  of referring AFDC applicants and recipients to a welfare-to-work program.  Random assignment for this study took place at income maintenance offices.  The results of this supplemental study are not included in this report.

3.  The Child Outcomes Study, conducted by Child Trends as part of the NEWWS Evaluation, also examines the effects of welfare-to-work programs on the children of LFA, HCD, and control group members in Atlanta, Grand Rapids, and Riverside.  This study uses a more comprehensive set of data about young childrens development, but only for children aged 3 to 5 at random assignment.  See Sharon M. McGroder et al., Impacts on Young Children and Their Families Two Years After Enrollment:  Findings from the Child Outcomes Study  (Washington, D.C.:  U.S. Department of Health and Human Services and U.S. Department of Education, 2000).  For a synthesis of the child research conducted thus far as part of the NEWWS Evaluation, see Gayle Hamilton, Do Mandatory Welfare-to-Work Programs Affect the Well-Being of Children?  A Synthesis of Child Research Conducted as Part of the National Evaluation of Welfare-to-Work Strategies  (Washington, D.C.:  U.S. Department of Health and Human Services and U.S. Department of Education, 2000).

[ Go to Contents ]

---

Selected Publications From This Evaluation

[See the NEWWS web site for latest list.]

From Welfare to Work.  Judith M. Gueron and Edward Pauly.  1991. New York:  Russell Sage Foundation.

Early Lessons from Seven Sites.  Gayle Hamilton and Thomas Brock.  1994.  Washington, D.C.:  U.S. Department of Health and Human Services and U.S. Department of Education.

Five Years After:  The Long-Term Effects of Welfare-to-Work Programs.  Daniel Friedlander and Gary Burtless.  1995.  New York:  Russell Sage Foundation.

Adult Education for People on AFDC:  A Synthesis of Research.  Prepared by Edward Pauly, MDRC.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education

Early Findings on Program Impacts in Three Sites.  Prepared by Stephen Freedman and Daniel Friedlander, MDRC.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

How Well Are They Faring?  AFDC Families with Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation.  Prepared by Kristin A. Moore, Martha J. Zaslow, Mary Jo Coiro, and Suzanne M. Miller, Child Trends, Inc., and Ellen B. Magenheim, Swarthmore College.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in Welfare-to-Work Programs.  Prepared by Gayle Hamilton, MDRC.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Work First:  How to Implement an Employment-Focused Approach to Welfare Reform.  [On MDRC's web site.]  Amy Brown.  1997.  New York:  MDRC.

Changing to a Work First Strategy:  Lessons from Los Angeles Countys GAIN Program for Welfare Recipients.  Evan Weissman.  1997.  New York:  MDRC.

Educating Welfare Recipients for Employment and Empowerment:  Case Studies of Promising Programs.  Prepared by Janet Quint, MDRC.  1997.  Washington, D.C.:  U.S. Department of Education, Office of the Under Secretary and Office of Vocational and Adult Education; and U.S. Department of Health and Human Services.

Evaluating Two Welfare-to-Work Program Approaches:  Two-Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites.  [On MDRC's web site.]  Prepared by Gayle Hamilton, Thomas Brock, Mary Farrell, Daniel Friedlander, and Kristen Harknett, MDRC.  1997.  Washington, D.C.:  U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Implementation, Participation Patterns, Costs, and Two-Year Impacts of the Portland (Oregon) Welfare-to-Work Program.  [On MDRC's web site.]  Prepared by Susan Scrivener, Gayle Hamilton, Mary Farrell, Stephen Freedman, Daniel Friedlander, Marisa Mitchell, Jodi Nudelman, and Christine Schwartz, MDRC.  1998.  Washington, D.C.:  U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.