Moving People from Welfare to Work. Lessons from the National Evaluation of Welfare-to-Work Strategies.

Submitted by:
Gayle Hamilton
Manpower Demonstration Research Corporation

Submitted to:
U.S. Department of Health and Human Services
Administration for Children and Families
Office of the Assistant Secretary for Planning and Evaluation

U.S. Department of Education
Office of the Under Secretary
Office of Vocational and Adult Education

"

Introduction and Overview

Over the past three decades, federal and state policymakers have created a variety of programs with the common goal of moving people from welfare to work.  How to go about increasing employment among welfare recipients, however, has long been debated.  By laying out the lessons learned from the National Evaluation of Welfare-to-Work Strategies (NEWWS) — the most ambitious welfare employment study to date — this research synthesis provides answers to critical questions in the welfare-to-work policy discussion.

NEWWS examined the long-term effects on welfare recipients and their children of 11 mandatory welfare-to-work programs, operated in seven sites, that took different approaches to helping welfare recipients find jobs, advance in the labor market, and leave public assistance.  A central question of the evaluation was:  “What program strategies work best, and for whom?”  Under study were two primary preemployment approaches — one that emphasized short-term job search assistance and encouraged people to find jobs quickly and one that emphasized longer-term skill-building activities (primarily basic education) before entering the labor market — and a third approach that mixed elements of the other two.  The strategies’ success was measured with respect to the goals and combinations of goals that policymakers and program operators have set for welfare-to-work programs, which include cutting the welfare rolls, increasing employment, reducing poverty, not worsening (or, better still, improving) the well-being of children, and saving government money.  The study examined the programs’ effects on single-parent welfare recipients, who account for the vast majority of the national welfare caseload, as well as on different subgroups thereof for example, those considered to be most disadvantaged with respect to their likelihood of finding steady employment.  The evaluation also addressed important policy questions such as how to engage a substantial proportion of people in program activities and how enforcement of welfare-to-work participation mandates influences program effectiveness.  A complete list of the questions covered in this synthesis, along with the primary sources from NEWWS that address them in detail, is provided in Table 1.

The effects of the NEWWS programs were estimated based on a wealth of data on more than 40,000 single-parent families, making NEWWS the largest study of welfare-to-work programs ever conducted.  Parents and their children were tracked over a five-year follow-up period, which, depending on the site, spanned different parts of the 1990s.  In the study’s innovative and rigorous research design, each parent was randomly assigned to a program group (in some sites, there were two program groups), whose members were eligible for program services and subject to the mandate, or a control group, whose members were not.

Table 1:
Questions Addressed in This Document and Primary Source Materials

Question	Primary Source(s)
NOTES:  Full citations of the primary NEWWS sources are provided on the inside front and back covers of this document. a This report is partly based on NEWWS data but was not produced as part of NEWWS. Full citation:  Promoting Participation:  How to Increase Involvement in Welfare-to-Work Activities (Gayle Hamilton and Susan Scrivener).  1999.  Manpower Demonstration Research Corporation.
The Status Quo and the Interventions
Control group outcomes:  How do welfare recipients fare in the absence of welfare-to-work programs?	Hamilton and Brock, 1994; Hamilton et al., 2001
Participation in education and training:  Can mandatory welfare-to-work programs engage large numbers of people in education and training? Participation in other activities:  What are typical patterns of participation in other types of program activities?	Hamilton et al., 2001; Bos et al., 2001; Freedman et al., 2000 Hamilton et al., 2001; Freedman et al., 2000; Scrivener and Walter, 2001; Farrell, 2000; Storto et al., 2000; Scrivener et al., 1998; Hamilton et al., 1997
Effects on Education Outcomes
GED and other credential receipt:  Do welfare-to-work programs’ investments in education and training result in higher rates of credential attainment? Gains in skills:  Do welfare-to-work programs’ investments in education and training result in higher skills? Adult education:  What factors enhance or diminish its beneficial effects?	Hamilton et al., 2001; Freedman et al., 2000; Bos et al., 2001 Bos et al., 2001 Bos et al., 2001
Effects on Economic Outcomes
Net impacts:  How effective are different types of welfare-to-work programs?	Hamilton et al., 2001
Relative impacts:  Which types of programs are generally most effective?	Hamilton et al., 2001
The LFA approach versus the HCD approach:  In head-to-head tests, which is more effective?	Hamilton et al., 2001; Hamilton et al., 1997
The most effective program:  What were its distinguishing features?	Hamilton et al., 2001; Freedman et al., 2000; Scrivener et al., 1998
Subgroup findings:  What types of programs work best for which groups of welfare recipients?	Hamilton et al., 2001; Michalopoulos and Schwartz, 2001
Education and training reconsidered:  Can they be made more effective?	Bos et al., 2001; Scrivener et al., 1998
Effects on Families and Children
Family circumstances:  Can programs have long-term spillover effects on family outcomes such as marriage and fertility?	Hamilton et al., 2001; Freedman et al., 2000
Children’s well-being:  How might programs that have mandates and services but leave income unchanged affect children in the long run?	Hamilton et al., 2001; Hamilton, 2000; Freedman et al., 2000; McGroder et al., 2000
Other Lessons
Income:  How can welfare-to-work programs increase family resources?	Hamilton et al., 2001; Freedman et al., 2000
Case management:  Do different strategies yield different results?	Hamilton et al., 2001; Scrivener and Walter, 2001
Participation standards:  What does it take to engage a substantial proportion of people in welfare-to-work program activities?	Hamilton, 1995; Hamilton and Scrivener, 1999a
Mandate enforcement:  What role does enforcing mandates play in program effectiveness?	Knab et al., 2001; Freedman et al., 2000
Costs and Benefits
Costs:  What contributes to the cost of welfare-to-work programs?	Hamilton et al., 2001; Scrivener and Walter, 2001; Farrell, 2000; Storto et al., 2000; Scrivener et al., 1998; Hamilton et al., 1997
Costs relative to benefits:  What is the government’s financial return on its investment in welfare-to-work programs?	Hamilton et al., 2001

Conceived and funded by the U.S. Department of Health and Human Services (HHS), NEWWS received additional support from the U.S. Department of Education.  The study was conducted by the Manpower Demonstration Research Corporation (MDRC).  Child Trends, as a subcontractor, conducted the Child Outcomes Study, the part of the evaluation that examined effects on young children.  This research synthesis is the final publication from the evaluation.

After presenting a brief description of NEWWS, this document poses a series of key questions about welfare-to-work programs and provides answers based on the evaluation’s already published findings (Table 1).  It concludes with a review of the achievements and limitations of such programs.

1. The National Evaluation of Welfare-to-Work Strategies

Program Context

The programs studied in NEWWS were initially run under the federal Family Support Act (FSA). Enacted in 1988, FSA required the government to provide education, employment, and support services to adults receiving cash welfare assistance, known at the time as Aid to Families with Dependent Children (AFDC). Recipients of AFDC, in turn, were required to participate in the Job Opportunities and Basic Skills Training (JOBS) program created under FSA. The NEWWS programs continued to operate (with some modification) after passage of the most recent federal welfare reform legislation, the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA), in 1996. Moreover, many of the goals, mandates, and program strategies first spelled out in FSA underpin PRWORA as well.

FSA, under which the NEWWS programs originated, introduced some important new features. Through its mandates and incentives, it encouraged state and local program administrators to serve welfare populations with whom they previously had little contact and to experiment with new types of services, messages, and mandates. In most states for the first time, the majority of single-parent welfare recipients with children aged 3 to 5 (or as young as age 1, at states' option) were required to enroll and participate in a welfare-to-work program, which meant that they had to work or engage in activities aimed at preparing them for work. In addition, FSA mandated that programs reserve at least 55 percent of federal welfare funds to provide services to welfare recipients who were deemed at greatest risk of long-term welfare dependency. FSA also emphasized new types of services: The services offered and supported by the states now had to include adult education -- that is, high school or General Educational Development (GED) exam preparation classes, basic and remedial education, and English as a Second Language (ESL) classes. In addition, teenage custodial parents without a high school diploma or GED had to return to classes in order to obtain one of these credentials. Finally, FSA required enrollees to participate in employment preparation activities for as long as they remained on welfare and eligible for services. Case managers were expected to monitor recipients' participation in program activities and to respond to nonparticipation using a variety of informal and formal measures, including reductions of welfare grants.

The expansion of welfare-to-work programs and the requirement to work with more disadvantaged populations intensified the long-standing debate among program administrators and policymakers concerning how best to help welfare recipients, especially those facing serious barriers to employment, move from welfare to work. Research conducted in the 1980s demonstrated that job-search-first programs sped up the entry of welfare recipients into the labor market. The jobs that people found through such programs, however, tended to be neither long-lasting nor high-paying, leaving many people with little income, living in poverty, and back on the welfare rolls. In addition, the programs did not benefit the most disadvantaged.

Realizing that in the general population people with more education and degrees tend to earn more, policymakers began to focus on the possible value of education and training in welfare-to-work programs. Proponents of education and training argued that making initial investments in building people's skills might enable them -- especially those without a high school diploma or with other employment barriers -- to get better and more stable jobs, increase their income, and become less likely to return to welfare. Critics of this approach believed that mandatory education programs for adult single parents, many of whom had left education institutions as teenagers, not only would be very costly and hard to implement on a large scale but might also delay people's labor market entry without guaranteeing that their foregone earnings would be made up by better jobs later. Proponents of job search programs countered that any job, even a low-paying or temporary one, is the best way to build skills that might lead to better jobs and is cost-effective as well. They advocated enhancing and adding services to job search programs to increase their overall effectiveness; among the new services proposed were instruction on how to find employment, peer support, time-management classes, self-esteem-building exercises, and job development (efforts to increase the pool of available jobs). Critics of the job search approach thought it still did not address the essential need -- namely, to build recipients' skills -- and that the proof of its merits would be in long-term rather than short-term results. Thus, in the wake of FSA, "What works best?" became a pivotal question for policymakers and program administrators alike.

The 1996 welfare reform law, PRWORA, built on many aspects of FSA, but it also contained new provisions. First, it replaced AFDC with a flexible, state-directed program called Temporary Assistance for Needy Families (TANF), which provided each state with a block grant -- a lump sum of money -- to spend on welfare programs and benefits. Second, for most families it put a lifetime limit of five years on federally funded cash welfare; any cash assistance beyond that point would have to be funded by the state. Third, PRWORA created financial incentives for states to run mandatory, work-focused welfare-to-work programs and required virtually all welfare recipients to work or participate in program activities. The law's time limit on welfare receipt, its focus on work, and its requirement that the entire welfare caseload work or receive work-directed services fueled the already keen interest in the question of which welfare-to-work approach is most effective at moving people from welfare to work.

Sites and Programs

The 11 programs in NEWWS were operated in seven sites across the country: Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; Columbus, Ohio; Detroit, Michigan; Oklahoma City, Oklahoma; and Portland, Oregon (for a list of the programs categorized by type, see Table 2). Because FSA gave states wide latitude to design their welfare-to-work programs and one of the aims of NEWWS was to learn about different program approaches, NEWWS planners at HHS and MDRC sought to include sites that would demonstrate a variety of programs operated in a diverse range of conditions. Although the programs were not selected to be representative of all welfare-to-work programs in the country, they varied along several important dimensions, including geographic location, labor market conditions, and welfare grant levels. To meet the demands of the research, each site had to have a relatively large welfare caseload; as a result, all seven sites include urban areas. The Appendix provides summaries of each of the 11 programs' activities, environments, and results.

Table 2
NEWWS Programs, Categorized by Approach, First Activity, and Enforcement Level

Employment-focused approach		Education-focused approach
Job search first	Varied first activity	Education or training first
High enforcement	High enforcement	High enforcement	Low enforcement
NOTES: "LFA" denotes the site's Labor Force Attachment program. "HCD" denotes the site's Human Capital Development program.
Atlanta LFA Grand Rapids LFA Riverside LFA	Portland	Atlanta HCD Grand Rapids HCD Riverside HCD Columbus Integrated Columbus Traditional	Detroit Oklahoma City

Employment- and education-focused programs operated side by side in three sites. As part of an unusual effort to determine whether the employment- or the education-focused program approach works better, each of three sites -- Atlanta, Grand Rapids, and Riverside -- operated two different welfare-to-work programs. The Labor Force Attachment (LFA) program in each site emphasized immediately assigning people to short-term job search activities with the aim of getting them into the labor market quickly. Case managers in the LFA programs stressed the value of people's taking any job, even a low-paying one, and later advancing into stabler, better-paying jobs. The Human Capital Development (HCD) program in each site emphasized first enrolling people in education or training -- primarily basic or remedial education or GED preparation (not college) -- before steering them toward the labor market. The LFA and HCD programs were designed, expressly for the purposes of this research, to magnify the differences between the employment-focused approach and the education-focused approach.

A "hybrid" program in one site. Portland operated an employment-focused program that differed from the LFA programs in using a mixed strategy for making initial acti-vity assignments. Depending on caseworkers' perception of recipients' skills and needs, different recipients were assigned to different types of initial activities. Unlike the LFA programs, the Portland program offered education or training classes to a substantial minority of its enrollees and encouraged everyone to hold out for a job that paid more than the minimum wage and offered a good chance of stable employment.

Education-focused programs in three sites, one with two types of case management. Columbus, Detroit, and Oklahoma City operated education-focused programs. Columbus simultaneously operated two education-focused programs that took different approaches to case management. In the program with traditional case management, welfare recipients interacted with two separate caseworkers: one who dealt with welfare eligibility and payment issues (often called income maintenance) and one who dealt with employment and training issues. In the program with integrated case management, in contrast, recipients worked with only one staff member, who handled both the income maintenance and employment and training aspects of the case.

Other differences among programs. Nine of the 11 programs in NEWWS were considered high enforcement in that, rather than working with recipients most motivated to participate, they worked with a broad cross-section of welfare applicants and recipients who were required to participate; monitored participation closely; and, especially in several of the programs, frequently imposed sanctions -- that is, reduced welfare grant amounts -- as a penalty for not fulfilling participation requirements. The other two programs were considered low enforcement.

It is important to note that the NEWWS programs differed in important ways from many current welfare-to-work programs. First, although several NEWWS programs required some women with children as young as age 1 to participate, none extended the participation mandate to mothers with children younger than 1, which is allowed (at states' option) under TANF. Second, none of the NEWWS programs imposed a time limit on welfare receipt. Third, none included a substantial earned income disregard, a policy that allows welfare recipients to remain eligible to receive benefits and to have earnings up to a higher level than normally allowed. Finally, none of the programs emphasized upfront practices aimed at diverting people from applying for welfare, which some programs now do. NEWWS thus reveals little about these newer policies and practices. Nevertheless, the primary goal of the NEWWS programs, like that of post-PRWORA programs, was to move welfare recipients off cash assistance and into paid employment. As a result, the NEWWS programs faced the same tensions between goals that have long shaped and challenged policies for the poor: improving families' material conditions without discouraging them from working; enforcing work and work-related requirements for parents without adversely affecting their children; and minimizing government costs when it is often cheaper in the short run simply to give low-skilled parents their welfare grants than to make more expensive investments in their education or training.

Research Design

NEWWS used a random assignment research design to estimate the effects of the studied programs. Welfare recipients were randomly assigned to one of two or three research groups, depending on the site. One of the groups was always a control group. In all the sites, control group members were eligible for welfare as usual. In addition, they were eligible for child care assistance similar to that offered to program group members, provided that they were participating in nonprogram activities in which they had enrolled on their own.

In the three sites that operated both an LFA program and an HCD program, three-way random assignment was performed. Welfare recipients in these sites were randomly assigned to one of three groups: the LFA group, whose members received LFA program services; the HCD group, whose members received HCD program services; or the control group, whose members could not receive services through a welfare-to-work program. A three-way design was also used in Columbus, except that in Columbus the program groups differed only with respect to the case management they received (integrated or traditional).

In Detroit, Oklahoma City, and Portland, two-way random assignment was used to test the effectiveness of existing programs rather than of programs designed for research purposes. In these sites, welfare recipients were randomly assigned to a group that enrolled in the program or to a control group that was not eligible for any welfare-to-work program services.

The study design allowed for many revealing comparisons. The key ones examined the programs' economic effects on adults and spillover effects on families (that is, indirect effects on noneconomic outcomes and effects on children). To determine the net effect of each program, the outcomes for each program group were compared with those for the control group in the same site. In the three-way sites, it was also possible to estimate the relative effects of alternative program approaches by comparing the outcomes for the two program groups directly. What makes the design in the three-way sites particularly robust is that, by making comparisons between programs operated in the same site, it holds constant contextual features (such as population characteristics and local economies) that might vary from site to site and affect the programs' results.

The random assignment research design used in all the sites is what makes NEWWS such a rigorous investigation of the effectiveness of various welfare-to-work approaches. Because people were assigned to groups at random within each site, one can be sure that there were no systematic differences between people in the program and control groups when they entered the study. Therefore, any subsequent differences in outcomes between groups in the same site -- whether between two program groups or between a program group and the control group -- can be confidently attributed to a particular type of program. These differences, called impacts, can relate to any type of outcome -- for instance, rates of participation in education activities, reading test scores, employment rates, earnings levels, number of months on welfare, or assessments of children's well-being (to name but a few of the outcomes examined in NEWWS). Throughout this document, statements concerning whether the NEWWS programs increased or decreased some outcome (such as earnings) refer to their impacts, that is, to differences between how program and control group members fared during the five-year follow-up period -- not to changes in any given research group's behavior over time. (For a discussion of the advantages of using impacts rather than outcomes to assess program effectiveness, see Box 1.) Unless otherwise noted, all the impacts discussed are statistically significant, meaning that they are unlikely to be due to chance.

NEWWS and Current Welfare Initiatives

The 1996 welfare reform law spawned many new welfare policies and encouraged states to experiment with new approaches. Almost all the new policies and innovations, however, take for granted the existence of and build on the quid pro quo established by FSA, namely, that welfare recipients must work or participate in some type of welfare-to-work program in order to receive welfare benefits and services from the government. The new initiatives -- which include substantial earned income disregards, welfare time limits, stricter penalties for nonparticipation, and postemployment services -- are not meant to replace welfare-to-work programs. Rather, they are intended to enhance the anticipated payoffs (such as higher employment) of the changes brought about by welfare-to-work programs. In light of this, the NEWWS results, which suggest how welfare-to-work programs can be made most effective for different groups of people, are highly relevant. As the following sections illustrate, NEWWS provides critical insights regarding how best to design and operate programs in order to maximize the payoff of such programs.

2. The Status Quo and the Interventions

As has been documented in many studies, most welfare recipients eventually find jobs, and most do not stay on welfare for long. The challenge for welfare-to-work programs is to improve on these rates of job finding and welfare exit by enabling people to find jobs and leave welfare more quickly, to keep jobs longer and avoid returning to the welfare rolls, or to build their skills while on welfare and then obtain better jobs. A key task of evaluations of such programs is to find out what is the "normal" behavior of welfare recipients over time. Only then is it clear when programs are producing true benefits for people as opposed to leading to levels of employment, earnings, and welfare leaving that would have occurred in any case.

Before examining the outcomes for the control groups in depth, this section opens by briefly summarizing the characteristics of all the adult sample members in NEWWS before they were randomly assigned to the research groups (for their average characteristics, see Table 3). Almost all of them were single women; at the time they entered the study, they were an average of 30 years old and had an average of two children. The majority had at least one child under age 6. (In four of the sites, families could include children as young as age 1; in the other three sites, families could include children as young as 3.) The racial/ethnic makeup of the samples varied from site to site, reflecting the local populations.

One of the most important points to take away from this summary is that, although welfare recipients are a diverse group, a sizeable proportion of them face one or more barriers to steady employment. Among these barriers are a lack of a high school diploma or GED, no recent employment, a long history of welfare receipt, health or emotional problems, a high risk of depression, and a reluctance to leave one's children to go to work. At study entry, about two-fifths of the sample members lacked a high school diploma or GED, having completed an average of slightly less than 10 years of school. These sample members are often referred to here as nongraduates; sample members who had at least one of these credentials are referred to as graduates. A sizeable proportion of people in the sample lacked a work history, had been on welfare for at least five years cumulatively, or both. Slightly more than one-quarter of the sample members reported at study entry that they or a family member had a health or emotional problem. About one-seventh were found to be at high risk of depression. Finally, one-quarter of sample members reported strongly preferring staying home with their children over going to work.

Table 3.

Sample Member's Characteristics at Studay Entry: Many Welfare Recipients Face Barriers to Steady Employment
SOURCE:  Hamilton et al., 2001; Michalopoulos and Schwarts, 2001.
Demographic characteristics at study entry
Female (%)	94.1
Average age (years)	30.5
Average number of children	1.9
Had at least one child under age 6	58.4
Barriers to steady employment (%)
Had no high school diploma or GED	41.0
Never worked full time for one employer for 6 months or more	36.8
Had received welfare cumulatively for 5 years or more	33.7
Had health or emotional problems	28.0
Was at high risk of depression	14.5
Was highly hesitant to leave children to go to work	24.7

Control Group Outcomes: How do welfare recipients fare in the absence of welfare-to-work programs?

The experiences of the control group members in the NEWWS sites set the standard against which the program groups' experiences were measured (for the programs' impacts, see the next sections). Through examination of control group outcomes, the following portrait of the characteristics, attitudes, and behavior of welfare recipients who are not subject to welfare-to-work programs emerges.

• Most welfare recipients eventually work, but not steadily, and their earnings are low.

About three-quarters of control group members found jobs during the five-year follow-up period. But stable employment was uncommon: About three-fourths of those who found jobs were unemployed by the end of the fourth year, although most eventually became employed again. Including all control group members -- that is, averaging in the zero earnings of those who did not work -- the control groups' average earnings over the five years ranged from $12,752 to $25,566, or about $2,500 to $5,000 a year, across the seven sites.¹ Looking only at those who were working at the end of the five years and averaging across all the sites, earnings in the last quarter were $3,110, or, annualized, about $12,500.

• Recipients with a high school diploma or GED earn more than those who lack this credential.

Control group members who had a high school diploma or GED at study entry earned an average of $24,196 over five years, whereas people who lacked these credentials at study entry earned an average of $13,231. It was this general phenomenon that led designers of FSA to emphasize education in welfare-to-work programs, in the hope that investments in education would pay off in labor market outcomes.

The positive relationship between education credentials and earnings does not prove, however, that more education leads to higher earnings. To distinguish between correlation and causation, NEWWS examined whether the welfare-to-work programs that required people to participate in adult education activities (particularly classes aimed at helping people attain a GED) boosted outcomes such as attendance in education activities, reading and math literacy skills, and the rate at which people obtained credentials -- and whether the programs thereby increased earnings -- by comparing program enrollees' outcomes with those for people were not subject to any education participation requirement. The results of this analysis are presented in the third section of this document.

• A large majority of welfare recipients leave the welfare rolls within five years.

By the end of five years, between 17 percent and 37 percent of control group members across sites were receiving welfare, some of them having remained on the rolls continuously and others having left and returned. Over the five years (60 months) of the study's follow-up period, control group members received welfare for an average of 25 months to 38 months, depending on the site.

• On average, welfare recipients' income is low.

Over the five years, control group members in most sites took in between $40,000 and $45,000 from earnings, welfare payments, food stamps, and the Earned Income Credit (EIC) -- a refundable tax credit for low-wage workers -- minus payroll taxes, or about $8,000 to $9,000 a year. This combined income typically would have been for a family of three. The proportion of family income derived from earnings ranged from 30 percent to 45 percent.

• A substantial minority of welfare recipients who leave welfare lack health care coverage five years later.

When they entered NEWWS, virtually all control group members were on welfare and had Medicaid coverage. Although the majority had health care coverage at the end of five years, about one-quarter of them did not, suggesting that many of those who left welfare to work were not able to replace Medicaid with private coverage once their post-welfare transitional health benefits expired. Among those who had coverage, most were covered by public programs such as Medicaid rather than by employer-sponsored or other private plans. Employment in no way guaranteed health care coverage: Of control group members who were working at the end of the five years, between 20 percent and 30 percent lacked coverage; of those who had coverage, only about one-third to one-half obtained it from their employer. Owing to the larger number of public health programs for low-income children than for low-income adults, children were somewhat more likely to have coverage than their parents. Still, about one-fifth of children in the control groups were not covered at the end of five years.

• The majority of welfare recipients enroll themselves in some type of employment-promoting activity even when they are not required to participate in a welfare-to-work program.

Most control group members enrolled in vocational training or postsecondary education at some point during the five years. Few enrolled themselves in organized job search activities or adult education courses.

• A small proportion of single-parent welfare mothers marry or give birth to another child within five years.

Over the five-year follow-up period, less than one-fifth of single-parent mothers in the control groups got married, and a similar proportion added a new baby to their household through birth, marriage, adoption, or foster care.

• A substantial minority of welfare recipients report having recently experienced domestic abuse.

About one-fifth of the control group members in NEWWS reported having experienced some form of domestic abuse during the fifth year of the study period. Much of this was nonphysical abuse (such as threatening, yelling, or insulting), but between 7 percent and 14 percent of control group members reported having experienced recent physical abuse (such as hitting).

• The children of welfare recipients do not fare as well on some measures of well-being as do children in national samples.

Relative to national samples, school-aged children in the NEWWS control groups were more likely to have repeated a grade or dropped out of school, and younger children were more likely to have behavior problems and were less cognitively ready for school. On measures of child health and safety, the children in the control groups were similar to those in national samples.²

Participation in Education and Training: Can mandatory welfare-to-work programs engage large numbers of people in education and training?

Since the early 1980s, welfare policymakers and program operators have debated what role adult education -- basic education, GED preparation, and ESL classes -- should play in welfare-to-work programs. Even under TANF, discussion about the potential of education to help welfare recipients make the transition from welfare to work continues. Increasingly, a minimum level of reading and math skills and the possession of an education credential are seen as crucial in the current labor market. The concern is centered on welfare recipients who have no high school diploma or GED, since many policymakers view having one of these credentials as a prerequisite for entering the work force. Recipients who have at least one of these credentials are considered to face far fewer barriers to getting jobs. Furthermore, welfare reform efforts are focusing on "hard-to-employ" recipients, many of whom have educational deficits. Finally, in an effort to target scarce resources wisely, there is great interest in determining who would benefit most from adult education.

The first-order question in this debate, however, is whether participation mandates can really induce large numbers of welfare recipients -- about one-half of whom have not finished high school -- to enroll in and attend adult education classes. More generally, there is the question of whether programs can engage more people in adult education activities or vocational training than would participate on their own in any case. The outcomes for the education-focused programs in NEWWS speak directly to these questions.

• It is possible to engage large numbers of welfare recipients in adult education and -- to a lesser extent, vocational training -- as part of mandatory welfare-to-work programs.

As shown in Figure 1, during the five-year follow-up period, 40 percent of enrollees in the three HCD programs participated for at least one day -- usually much longer -- in adult education activities, and 28 percent of them participated in vocational training. Participation rates in adult education were much higher for nongraduates (welfare recipients who entered the study without a high school diploma or GED) than for graduates (those who had at least one of these credentials at study entry). In contrast, participation rates in vocational training were higher for graduates than for nongraduates.

• Impacts on participation -- that is, differences between the program and control groups' participation rates -- are more common and larger for adult education than for vocational training.

The HCD programs increased participation in adult education by 20 percentage points and vocational training by only 5 percentage points. Part of the reason for the disparity in impacts is that, as shown in Figure 1, welfare recipients on their own are somewhat more likely to enroll in vocational training classes than in adult education, leaving programs less room to increase participation in vocational training than adult education relative to control group levels. In addition, many vocational training programs require a high school diploma or GED for entry, which largely rules out this option for nongraduates. Finally, it should be kept in mind that the HCD programs generally did not assign people to college courses.

When people enrolled in adult education as part of a welfare-to-work program, they spent more than three times as many hours participating as did control group members. In addition, the programs increased the proportion of welfare recipients who participated in adult education across a wide variety of subgroups -- for example, among those with very young children, high school dropouts who had not completed school beyond the eighth grade, and those who did not want to go back to school.

---

Figure 1.
Participation in Education and training over Five Years:
Participation in Adult Education Increased More than Participation in Vocational Training

SOURCE: Hamilton et al., 2001

NOTE: The participation rates shown are averages for the HCD and control groups in Atlanta, Grand Rapids, and Riverside

Participation in Other Activities: What are typical patterns of participation in other types of program activities?

• Employment-focused programs generally produce large increases in job search participation, while education-focused programs usually lead to large increases in adult education participation.

All the NEWWS programs raised participation relative to control group levels. Figure 2 shows the participation impacts, split by type of activity and averaged across programs within each of the four program types shown in Table 2. The employment-focused programs increased participation in job search by approximately 30 percentage points. The education-focused programs -- in which enrollees were often assigned to job search after education or training -- also increased job search participation, but to a much lesser degree. The employment-focused programs were considerably less likely to affect participation in education and training, and when impacts on participation in these activities did occur, they were smaller than the education-focused programs' impacts on job search participation. The Portland program, the only one that combined an employment focus with a mixed strategy for assigning recipients to initial activities, substantially increased job search participation but increased education and training participation as well. Notably, the participation impacts were comparable across a range of subgroups. Where the relevant data were available, participation rates for mothers with young children, for example, were similar to those for mothers with older children.

---

Figure 2.
Impacts on participation over Two Years, by Activity and Program Type:
Employment-Focused Programs Produced Large Increases in Job Search Participation,
and Education-Focused Programs Boosted Adult Education Participation

SOURCE: Freeman et al., 2000

NOTES: Participation impacts were averaged across programs within each program type.
The Riverside LFA program results include both graduates and nongraduates.
No tests of statistical significance were performed.

Endnotes

¹  Throughout this document, ranges are often presented because specific findings differed from site to site.

²  Parents with a severely ill or disabled child were generally not mandated to participate in welfare-to-work programs in the early to mid-1990s; as a result, such families were not included in the NEWWS samples. Their exclusion, however, is unlikely to have affected the overall level of assessed health for children in the control groups very much. Data available in NEWWS suggest that less than 3 percent of the exemptions from participation were granted owing to children's severe health problems.

3. Effects on Education Outcomes

The education-focused programs in NEWWS engaged large numbers of people in mandatory education or training classes -- more people than would have enrolled in such classes on their own. The next critical question iswhether higher participation in such classes enabled people in the education-focused programs to acquire the credentials or skills that might give them a better foothold in the labor market and better prospects of moving into good jobs than control group members had. On this topic, NEWWS offers a number of insights.

GED and Other Credential Receipt: Do welfare-to-work programs' investments in education and training result in higher rates of credential attainment?

• Welfare-to-work programs can increase the proportion of people who obtain a GED or high school diploma -- particularly among recipients who enter the program with literacy skills at or close to the high school level -- but the overall proportion of people who earn such a credential is likely to be low. Increases in the proportion of people who obtain a training certificate or postsecondary degree are harder to achieve.

Among nongraduates in the three sites that ran HCD programs, an average of 7 percent of those in the control group received a GED or high school diploma over the five-year follow-up period, whereas more than twice as many in the HCD group -- 17 percent -- did so (Figure 3). Overall, however, less than one-fifth of the nongraduate HCD program group members earned one of these credentials. The impact was mostly on GED (rather than high school diploma) receipt and was concentrated among people who entered the programs with high reading and math skills or at least an eighth-grade education. None of the LFA programs increased receipt of a GED or high school diploma. Examining both graduates and nongraduates, only three of the 11 NEWWS programs (one of them an HCD program) led to an increase in receipt of any other type of education or training credential, generally a trade license or certificate.

---

Figure 3.
Receipt of a High School Diploma or GED over Five Years:
Programs Can Increase the Proportion of Nongraduates Who Obtain a High School Diploma or GED, but the Overall Number Who Do So is Low

SOURCE: Hamilton et al., 2001

NOTE: The percentages shown are averages for sample members in the HCD and control groups in Atlanta, Grand Rapids, and Riverside who were nongraduates at study entry.

Gains in Skills: Do welfare-to-work programs' investments in education and training result in higher skills?

• Welfare-to-work programs that rely on adult basic education programs for the general population are unlikely to improve welfare recipients' basic reading and math skills.

In none of the NEWWS sites in which standardized reading and math literacy tests were administered two years after study entry did the programs raise test scores, even among people who at study entry wanted or planned to enroll in school. (Basic education programs are usually targeted at people whose skills are at or below the eighth-grade level.) Note, however, that most of the programs did not assess welfare recipients for learning disabilities, which could have diminished the programs' ability to improve literacy skills. Some studies have estimated that between one-quarter and one-half of welfare recipients have learning disabilities. ³

Adult Education: What factors enhance or diminish its beneficial effects?

• The gains in credential receipt and literacy skills that welfare recipients can reap from adult education programs seem to be related to the length of participation in and the quality of such programs.

How long welfare recipients participate in adult education programs can enhance or diminish such programs' beneficial impacts. Overall, the typical participant in an adult education program received the equivalent of about two-thirds of a year of high school instruction. A nonexperimental ⁴ examination of the association between credential receipt or skills improvement and length of stay suggested several patterns. Shorter stays were associated with GED receipt and gains in math skills: Enrollment in GED preparation classes for more than six months did not increase GED receipt, and most people's math skills no longer improved after six months of enrollment in basic education classes. Longer stays, in contrast, were associated with gains in reading skills: Enrollment in basic education classes for less than one year did not measurably improve reading skills.

The size of education benefits also seemed to depend on the characteristics of education providers. For instance, nonexperimental comparisons revealed that the higher the average level of teachers' experience and education, the larger the improvements in recipients' reading and math skills. The size of education benefits did not, however, seem to be affected by the fact that welfare recipients in these adult education classes were required to be there: Among adult education enrollees who went to classes, those in the program groups (almost all of whom enrolled to meet a welfare requirement) experienced gains in GED receipt and math and reading skills comparable to those experienced by adult education participants in the control groups (all of whom attended classes voluntarily).

Endnotes

³  See, for example, CLASP Update (Center for Law and Social Policy). 1998. Center for Law and Social Policy.

⁴  Nonexperimental comparisons are analyses that do not take direct advantage of the random assignment design. Differences found in some comparisons are thus not necessarily indicative of a causal relationship.

4. Effects on Economic Outcomes

This section examines the impacts on economic outcomes of the different types of welfare-to-work strategies used by the NEWWS programs. Specifically, it looks at the extent to which the programs improved on what would have happened in their absence; the relative effectiveness of employment-focused and education-focused programs and their variants; and the relative effectiveness of LFA and HCD programs.

As noted earlier, one of the key goals of education-focused programs is to increase income by improving welfare recipients' credentials or skills before they seek jobs. Short-run impacts are not expected in these programs because the programs essentially delay people's entry into the job market; rather, long-run impacts are the goal, with the hope that higher long-run earnings will make up for earnings foregone in the short run. One of the key goals of employment-focused programs, in contrast, is to reduce reliance on welfare as soon as possible. Short-run impacts are expected in these programs, with the hope that they will be sustained and even grow over time.

Net Impacts: How effective are different types of welfare-to-work programs?

• All 11 NEWWS programs increased single parents' employment and earnings and decreased their welfare receipt and payments relative to the levels found in the programs' absence.

As noted earlier, over the five-year follow-up period, approximately three-quarters of control group members in NEWWS found jobs, and more than half left the welfare rolls. Nearly all the programs improved on these statistics, causing people to work during more quarters of the follow-up period and to earn more than they would have in the absence of a program (for a discussion and illustration of the components that make up earnings impacts, see Box 2). Moreover, all the programs decreased the average number of months that people received welfare and the average number of welfare dollars they received over the five years.

The program and control group earnings levels that form the basis of the earnings impacts are shown in Figure 4. In the figure, the two groups' five-year earnings are split by program type. As can be seen, average earnings for program group members (the black bars) in all four types of programs were higher than those for control group members (the white bars).

Most of the programs increased earnings during the second and third years of the follow-up period. Their effects generally diminished, however, during the fourth and fifth years and were not statistically significant for most programs by the very end of the fifth year. Only the Portland and Riverside LFA programs continued to produce statistically significant earnings impacts at the end of the fifth year. It should be noted that, in a few sites, a small proportion of control group members (a subset of those on welfare) in a few programs received program services toward the end of the five-year follow-up period.⁵ Extensive analyses indicated that the effect of this on the impacts in the fourth and fifth years was probably small. Most of the programs' effects on earnings would likely have diminished in the later years even if no control group members had been exposed to the programs late in the study.

Notably, only a minority of program group members experienced stable employment over the five years. As an example, from 60 percent to 80 percent of program group members were unemployed for at least one quarter during the fifth year, and this situation was only slightly better than that of control group members in the same year. In addition, even after five years, most people were earning relatively low wages -- between $7 and $8 per hour -- and 70 percent to 85 percent earned less than $10,000 in the fifth year, outcomes that were not much different from those for the control groups.

The average welfare payments received by the program and control groups over the five years, which form the basis of the welfare impacts, are shown in Figure 5. Again, the results are split by program type. In all four types of programs, average welfare payments received by program group members (the black bars) were lower than those for control group members (the white bars). Most of the programs reduced five-year welfare payments relative to control group levels by 15 percent or more. All the programs also reduced the number of months that people received welfare, by 2 months to 6 months over the five-year (60-month) follow-up period. The welfare impacts were more persistent than the earnings impacts: Whereas only a few programs continued to affect earnings in the fifth year, most of the programs continued to generate welfare savings at the end of the same year.

Over five years, program group members in all 11 programs spent less time on food stamps and received smaller average food stamp payments than did control group members in the same sites. Food stamp impacts were generally smaller than welfare payment impacts, however, because some program group members continued to receive food stamps after they left welfare (as they were entitled to, provided that their earnings did not exceed a certain cutoff).

---

Figure 4.
Earnings over Five Years, by Program Type:
Program Group Members Earned More than Control Group Members

SOURCE: Hamilton et al., 2001

NOTES: Earnings for the program and control groups were averaged across programs within each program type.
The Riverside LFA program results include both graudates and nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** = 5 percent; *** = 1 percent.

---

Figure 5.
Welfare Payments over Five years, by Program Type:
Program Group Received Less Welfare than Control Group Members

SOURCE: Hamilton et al.,,2001

NOTES: Welfare payments for the program and control groups were averaged across programs within each program type.
The Riverside LFA program results include both graudates and nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** = 5 percent; *** = 1 percent.

---

• At best, the NEWWS programs led to only small increases in the low levels of income that would be expected in the absence of the programs, and some of them actually decreased income.

In NEWWS, income was calculated as the sum of earnings, welfare payments, food stamps, and the EIC minus payroll taxes. Although the programs helped people become more self-sufficient in that a larger share of their income came from earnings as opposed to welfare or food stamps, in dollar terms the decreases in welfare and food stamps (and increases in payroll taxes) largely offset the increases in earnings and the EIC. In three programs (including Portland), the five-year income of program group members was from 3 percent to 5 percent higher than that of control group members, but these impacts were slightly shy of being statistically significant by the standard used in the evaluation. Four programs had negative impacts on five-year income, decreasing it by 2 percent to 6 percent (these four impacts were all within or just slightly outside the statistical significance range). The programs that had positive impacts and those that had negative impacts range included both employment- and education-focused programs. Only one of the 11 NEWWS programs affected income in the fifth year.

Failure to increase income is not particular to the welfare-to-work programs studied in NEWWS. Results from most programs operated in the 1980s and early 1990s were similar: Even when programs increased earnings, they seldom increased income much. These findings underscore the limited ability of traditional welfare-to-work programs to improve families' material well-being.

Relative Impacts: Which types of programs are generally most effective?

• Employment-focused programs generally had larger effects on employment and earnings than did education-focused programs.

Figure 6 presents all 11 NEWWS programs' impacts on earnings, that is, the differences between the program and control groups' earnings in each site. As shown, three of the four employment-focused programs produced larger gains in earnings over the five years than did all seven education-focused programs.

The LFA programs' impacts on five-year earnings ranged from about $1,500 to $2,500. Their impacts on the number of quarters people were employed ranged from 0.7 to 1.1 (out of the 20 quarters in the study period). As is evident in the figure, the employment-focused program in Portland produced much larger effects, with an earnings impact of about $5,000 and an increase in quarters employed of 1.6. Overall, the education-focused programs' effects were smaller. Neither of the two programs with low enforcement of the participation mandate significantly raised employment, while the other five education-focused programs increased earnings by about $800 to about $2,000 and the number of quarters employed by 0.3 to 0.8.

Given the large number of programs examined in NEWWS and the diversity of the populations they served, the features of their implementation, and the labor markets in which they operated, these results strongly indicate that employment-focused programs are more effective than education-focused programs at increasing employment and earnings.

---

Figure 6.
Earnings over Five Years, by Program:
Employment-Focused Programs Generally Increased Earnings More than Education-Focused Programs

SOURCE: Hamilton et al.,,2001

NOTES: The Riverside LFA program results include both graudates and nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** = 5 percent; *** = 1 percent.

---

• Welfare and food stamp payment reductions were not consistently larger in the employment-focused programs than in the education-focused programs.

Figure 7 presents all 11 NEWWS programs' impacts on average welfare and food stamp payments. The savings were generally larger for the programs that had larger effects on earnings, but they varied for other reasons as well. For instance, as would be expected, welfare payments decreased more in sites where grant levels were relatively high than in sites where grant levels were relatively low. In addition, the programs decreased payments of welfare benefits more in sites that strictly enforced program participation mandates than in sites that did not.

---

Figure 7.
Impacts on Welfare and Food Stamp Payments over Five Years, by Program:
Welfare and Food Stamp Payment Reductions Were not Consistantly Larger in Employment-Focused Programs than in Education-Focused Programs

SOURCE:  Hamilton et al.,,2001

NOTES: The Riverside LFA program results include both graudates and nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** = 5 percent; *** = 1 percent.
Double daggers(‡) denote statistical significance levels for food stamp payments: ‡ = 10 percent; ‡‡ = 5 percent; ‡‡‡ 1 percent.;

The LFA Approach Versus the HCD Approach: In head-to-head tests, which is more effective?

• Compared with the LFA approach, the HCD approach did not produce additional long-run economic benefits.

In side-by-side comparisons in the same sites, the LFA and HCD approaches' five-year impacts on employment, earnings, months on welfare, and welfare payments were not the same, but the differences were generally not statistically significant -- that is, it could not be confidently concluded that the differences in impacts did not occur by chance. Where there were statistically significant differences between the effects of the two types of programs, however -- such differences were found for some early follow-up years and for some subgroups and outcomes -- the LFA programs always came out ahead. For example, in Grand Rapids, the LFA group worked more quarters on average than did the HCD group, and the average number of months of welfare or food stamp receipt was lower in the LFA group than in the HCD group.

• The LFA approach moved people into jobs and off welfare more quickly than did the HCD approach -- a clear advantage in an era of time limits on welfare receipt.

As is typical in welfare samples, earnings levels increased during each year in the follow-up period in the LFA and HCD programs as well as the control groups, reflecting increases in employment. Moreover, earnings were higher among both LFA and HCD program members than among control group members early in the follow-up period, but the differences between the program groups and the control group (that is, the programs' impacts) narrowed over time (for the yearly earnings impacts of the LFA and HCD programs averaged across the three sites, see Figure 8). Clearly, however, earnings rose earlier for the LFA group than the HCD group. Similarly, in the first two years of the follow-up period, the LFA programs had larger impacts on welfare receipt than did the HCD programs (for the LFA and HCD programs' yearly impacts on welfare receipt, see Figure 9). In subsequent years, however, the gap between the two lines narrows and ceases to be statistically significant.

• Relative to the LFA approach, the HCD approach did not produce more earnings growth or increase the likelihood of employment in good jobs.

Neither the HCD nor the LFA approach was generally successful in boosting earning growth or the likelihood of having a good job -- that is, a job that is stable and well-paying -- but the impacts on these measures were especially disappointing for the HCD programs. The education and training services that were part of these programs were intended to help people eventually move into stabler and higher-paying jobs (compared with control group members and LFA program group members), with the goal of more than making up for the earnings foregone early in the follow-up period while welfare recipients were enrolled in classes. However, both the LFA and HCD programs had little or no effect on earnings growth and employment stability. Furthermore, the trend lines in Figures 8 and 9 suggest that the HCD programs' lack of advantage over the LFA programs in this regard would not change if follow-up data beyond five years were available.

• The LFA approach was much cheaper to operate than the HCD approach.

The HCD programs were 40 percent to 90 percent more expensive than the LFA programs that operated in the same sites (for details on costs, see the second-to-last section of this document).

• The results of the above comparisons between the LFA and HCD programs' impacts held for nongraduates as well as graduates.

Even among nongraduates, who were expected to derive the greatest benefit from an initial investment in basic education, the employment and earnings impacts of the LFA programs were larger than those of the HCD programs.

• The LFA and HCD approaches did not differentially affect income or children's well-being in the full NEWWS sample. However, for one subgroup -- nongraduates -- the LFA programs had a larger impact on income than did the HCD programs.

Neither the LFA nor the HCD approach increased income overall. In fact, income impacts varied more by site than by program approach. But the programs did have different effects on income among nongraduates: Although in neither type of program was nongraduates' income higher than control group levels, those in the LFA programs had higher income, on average, than those in the HCD programs. Averaging the results for nongraduates across the three sites that ran LFA and HCD programs, the LFA programs resulted in almost $1,000 more in income over five years than the HCD programs. Few effects on children's well-being were found, and these did not differ consistently by program approach (for details on the effects of income on children, see the next section).

---

Figure 8.
Impacts on Earnings, by Approach and Year:
LFA Programs Increased Earnings More Quickly than HCD Programs

SOURCE: Hamilton et al., 2001

NOTES: The impacts shown are average for sample members in the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside.
Daggers(†) denote statistical significance levels for LFA-HCD differences: † = 10 percent; †† = 5 percent; ††† = 1 percent;
To ensure comparability to the Riverside HCD sample, the Riverside LFA sample includes only those who nongraduates at study entry.

---

Figure 9.
Impacts on Welfare Receipt, by Approach and Year:
LFA Program Moved People Off Welfare More Quickly than HCD Programs

SOURCE: Hamilton et al., 2001

NOTES: The impacts shown are average for sample members in the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside.
Daggers(†) denote statistical significance levels for LFA-HCD differences: † = 10 percent; †† = 5 percent; ††† = 1 percent;
To ensure comparability to the Riverside HCD sample, the Riverside LFA sample includes only those who nongraduates at study entry.

The Most Effective Program: What were its distinguishing features?

As already discussed, the most rigorous findings about the relative effectiveness of different program approaches come from the analyses that directly compare the LFA and HCD programs within each site that operated both types of program. Viewed with appropriate skepticism, however, cross-site comparisons can suggest what other program approaches or features are likely to be particularly effective.

As shown in Figure 6, the Portland program by far outperformed the other 10 programs in terms of both the size and consistency over time of its earnings gains (and its employment gains, which are not shown in the figure). The Portland program increased average five-year earnings by 25 percent and the average number of quarters employed by 21 percent. The program also increased stable employment and earnings growth more than any of the other 10 programs.

• The Portland program's success suggests that the following are key features of very effective programs: an employment focus, the use of both job search and short-term education or training, and an emphasis on holding out for a good job.

Although contextual factors may have contributed to the Portland program's success -- relative to the other NEWWS programs, Portland's worked with a less disadvantaged welfare caseload, and the state had a relatively high minimum wage -- it also differed from the other programs with respect to implementation. The Portland program had a clear employment focus. Unlike the LFA programs and the education-focused programs, however, it used a mixed strategy for matching enrollees to initial activities: Portland staff assigned some to very short-term education or training and others (the majority) to job search. Also, job search participants in Portland, unlike in the other programs, were counseled to wait for a good job (that is, one that paid at least about 25 percent higher than the minimum wage and offered a good chance for stable employment) as opposed to taking the first job they were offered. Although other aspects of the Portland program, such as its use of job developers and the considerable experience of its staff in operating job search programs, also deserve some credit for the program's beneficial effects, these features did not make the program unique in NEWWS. And although Portland's relatively strong economy may have contributed to the success of the program, other NEWWS programs in places where the demand for labor was similarly high did not have equally large earnings or employment impacts.

One implication of the Portland results -- that strongly employment-focused programs with mixed activities are more effective than programs that offer primarily job search or primarily education and training -- is buttressed by findings from previous studies. For example, the Greater Avenues for Independence (GAIN) program that was run in Riverside, California, in the late 1980s -- widely considered a paragon among welfare-to-work programs -- was also an employment-focused, mixed-strategy program. Operationally, both Portland and Riverside GAIN not only stressed the importance of finding jobs and strictly enforced program participation requirements but also offered many different services, including job search (along with job development), short-term education, and (in Portland) training. In both programs, people who were considered not ready to enter the labor market were sometimes first assigned to basic education or (in Portland) to training or life skills classes.

• The Portland program's implementation features also suggest how to make education and training in mandatory welfare-to-work programs more effective for instance, by putting an upper limit on the duration of recipients' participation in some types of adult education.

When making initial assignments to education or training activities, Portland program staff communicated to welfare recipients that improving their employability was the goal. The assignments were thus limited in duration, usually lasting 6 months or less, and participants were encouraged to complete them and then look for work rather than to "languish" in them. Decisions about who was not ready to go immediately into job search -- and thus who could be initially assigned to education, training, or other activities -- were left to program staff, who took into consideration a variety of factors that might affect recipients' employability, including work history, educational attainment, and reading and math skills. GED preparation classes, for example, were offered primarily to people who case managers thought had a good chance of obtaining a GED relatively quickly, and the program indeed raised the proportion of people who received a GED over the five years. In addition, the program led to an increase in the percentage of welfare recipients who received both a GED or high school diploma and a trade license or certificate -- a combination that nonexperimental research has suggested is particularly effective in boosting subsequent earnings.

Finally, by partnering with local community colleges to design and operate the Portland program, welfare department administrators probably increased recipients' exposure to postsecondary education. Although Portland staff did not make assignments to college courses as part of the welfare-to-work program, among recipients who entered the study with a high school diploma or GED, the Portland program produced a 21 percentage point increase in the proportion who took a course for credit at a two- or four-year college, a difference that emerged in the second half of the five-year follow-up period. The late appearance of these participation impacts and the fact that assignments to college were not made as part of the program suggest that welfare recipients' exposure to the community college system while they were participating in job search and other welfare-to-work program activities had spillover effects on college course enrollment. Portland was the only NEWWS program to increase college participation, but the timing of its impacts on this outcome makes it highly unlikely that they were related to its large earnings impacts earlier in the follow-up period.

Subgroup Findings: Which types of programs work best for which groups of welfare recipients?

Evaluations of welfare-to-work programs operated before FSA's passage found that the programs were most effective for the moderately disadvantaged and least effective for the most disadvantaged and the least disadvantaged (to learn how these groups were defined in NEWWS, read the first finding below). Partly in response to these findings, FSA required states to target welfare-to-work programs at welfare recipients who were the most likely to have long stays on welfare and the least likely to work; to offer the mix of services, including education, that they thought was most likely to benefit this hard-to-employ group; and to subsidize child care, transportation, and work-related expenses while people participated in welfare-to-work programs. This subsection describes the NEWWS findings for selected subgroups of welfare recipients, that is, for groups of sample members who shared certain characteristics when they entered the study. (These subgroup findings are corroborated by those from other evaluations of welfare-to-work programs operated before PRWORA's passage.)

• If the objective of welfare-to-work programs is to increase earnings and reduce welfare payments, the NEWWS programs worked well for a wide range of welfare recipients: Most subgroups were affected by the programs on these measures.

The NEWWS programs' effects were examined for long- and short-term welfare recipients; for people who had worked in the year prior to study entry and people who had not; for groups defined by race/ethnicity; and for the most disadvantaged (people who had long-term welfare receipt, dropped out of high school, and had been unemployed long term), the least disadvantaged (who had none of these barriers to employment), and the moderately disadvantaged (who had one of these barriers). Most of the programs led to increases in earnings and decreases in welfare payments for all of these subgroups -- except the least disadvantaged, for whom earnings impacts were small and welfare payment impacts were found for only a few programs.

• If the objective of welfare-to-work programs is to increase income, few NEWWS programs worked well for any subgroup of recipients.

The programs did not systematically change income for any subgroup. Although most of the programs changed the proportion of people's income that came from earnings as opposed to welfare and food stamps, they left people with the same income, on average, as control group members.

• Welfare-to-work programs can help people who are traditionally considered hard to employ: Almost all 11 NEWWS programs increased earnings for the disadvantaged subgroups.

Most of the programs raised earnings above control group levels for both the moderately disadvantaged people who had one or more serious barriers to employment, such as no recent work history or a lengthy history of welfare receipt and for the most disadvantaged, who had all three serious barriers.

• Neither employment-focused programs nor education-focused programs had consistently larger impacts on the earnings of the most disadvantaged.

Although neither the employment-focused approach nor the education-focused approach was clearly more effective for the most disadvantaged recipients, the employment-focused programs had slightly larger earnings impacts for this subgroup than did the education-focused programs. In two of the three sites in which LFA and HCD programs were operated side by side, for example, the LFA programs produced considerably higher impacts on the earnings of the most disadvantaged sample members than did the HCD programs.

• Although most of the NEWWS programs increased the earnings of the most disadvantaged welfare recipients, this subgroup still earned very little.

The earnings increases experienced by the more disadvantaged welfare recipients were no larger than those for the less disadvantaged recipients. As a result, even after participating in a program, the most disadvantaged program group members earned only about half as much as the moderately disadvantaged program group members, suggesting a need for policies aimed at raising the earnings of the most disadvantaged.

• Except for people at high risk of depression, all subgroups defined according to baseline measures of psychosocial well-being benefited from the programs.

There were few differences in impacts between subgroups defined by psychosocial characteristics assessed at study entry such as risk of depression, sense of control over personal destiny, work-related parental concerns, preference for work over welfare, health or emotional problems, child care problems, and transportation problems. The only exception was risk of depression. Though, surprisingly, control group members at high risk of depression and those at low risk had similar three-year earnings, overall the NEWWS programs did not boost the earnings of people at high risk of depression who also had high school diplomas, recent work experience, and little prior experience with welfare (earnings increased for people at high risk of depression who also had at least one serious barrier to work).

Education and Training Reconsidered: Can they be made more effective?

• The NEWWS findings should not be taken as a general indictment of the benefits of education and training in welfare-to-work programs.

NEWWS addressed whether programs in which welfare recipients were initially assigned to and required to participate in education and training activities produced higher average earnings than programs in which welfare recipients were initially assigned to and required to participate in job search. Although the findings indicate that the answer to this question is no, this does not mean that education and training do not pay any dividends to those who actually participate in these activities, receive a high "dosage" of instruction, complete the class sequence or program, and receive a degree or certificate or attain a certain skill level. In fact, nonexperimental work conducted as part of NEWWS suggests that people who increased their skills or obtained a GED subsequently experienced earnings gains relative to people whose skills did not improve or who did not get a credential. The biggest earnings payoff was for those few people who obtained a GED and then received some type of vocational training. Taken together, these results indicate that education and training can benefit welfare recipients. In the NEWWS sample, however, too few recipients achieved the intermediate milestones -- that is, gains in literacy skills or GED attainment -- to reap the potential rewards of education and training.

• The findings suggest ways in which the benefits of education and training in welfare-to-work programs can be heightened.

It should be kept in mind that the employment-focused programs all offered short-term education or training to people who did not find employment through job search. Furthermore, the most successful program provided a mix of job search and short-term education or training as initial activities. Thus, education and training had a role in all the programs in NEWWS.

One way to heighten the benefits of education and training would be to retain more students long enough to help them improve their literacy skills and earn a GED and access postsecondary services that would allow them to capitalize on the GED credential. Such an effort could be made while people are receiving welfare benefits or after they have left the welfare rolls. There is no guarantee, however, that increasing the duration of participation would be sufficient to help more students achieve the education milestones. Other possibly important factors include the quality of instruction, the appropriateness of the materials and technology for people with low literacy skills and possibly with learning disabilities, and the sometimes limited motivation or constricting life circumstances of program participants. In addition, consideration must be given to the current welfare environment; a long-term commitment to attending education activities, for example, might cause recipients to exhaust their welfare eligibility. Another way to enhance education and training benefits would be to encourage only those recipients who are within easy reach of earning a GED to pursue one. As already discussed, the Portland program adopted this strategy. Still another way to boost education and training impacts would be to foster links between adult education programs and postsecondary programs in an effort to encourage those who earn a GED to go further. Again, the Portland program provides an example of this approach.

Endnote

⁵  In these sites, the starting of the welfare time-limit clock necessitated allowing control group members to access welfare-to-work program services.

5. Effects on Families and Children

Family Circumstances: Can programs have long-term spillover effects on family outcomes such as marriage and fertility?

Welfare reform is often seen as a tool that can be used to do much more than raise earnings and reduce dependence on government assistance. FSA, for example, sought to bring about a sea change in people's attitudes toward welfare receipt. Some policymakers believe that reducing welfare use will have positive spillover effects on poor families, such as increases in marriage rates, reductions in out-of-wedlock childbearing, and alleviation of a range of social problems that they see as being linked to welfare use. None of the NEWWS programs were designed to change family circumstances directly. Furthermore, even the five years of follow-up data collected in NEWWS might not be sufficient to detect such changes.

All 11 NEWWS programs, however, reduced welfare receipt -- in some cases dramatically -- and increased employment and earnings. As a result, the effects that the programs had on several aspects of family circumstances can be examined to shed light on whether decreasing welfare use and increasing work can indirectly affect other aspects of family life.

• Programs that, like those in NEWWS, have participation mandates but no special financial work incentives are likely to reduce welfare dependence and increase employment and earnings with few long-term effects on marriage, fertility, or living arrangements.

At the time of study entry, all the NEWWS sample members were single mothers. Over the five-year follow-up period, roughly equal proportions of those in the control and program groups married, indicating that none of the programs had an impact on marriage. Several of the programs did, however, increase the proportion of sample members who cohabited with a partner. Similarly, most of the programs did not affect the proportion of single mothers who added a new baby to their household over the five years through birth, marriage, adoption, or foster care or the proportion whose households included extended family members or other adults. Finally, in only one site -- Grand Rapids -- were there effects on housing; compared with control group members, members of the program groups moved more often (to obtain better housing), and HCD group members were more likely to own homes.

• Welfare-to-work programs can decrease the incidence of physical domestic abuse.

At the five-year follow-up point, NEWWS program group members were less likely than control group members to report having experienced domestic abuse of a physical nature during the prior year. (There were no differences between the groups on measures of nonphysical abuse and job-related harassment.) The rates at which people reported having experienced physical abuse during the last year of the follow-up period ranged from 19 percent to 22 percent among control group members; the programs decreased these rates by 3 to 6 percentage points. There is some evidence that these reductions were fostered by increases in employment -- which may have raised people's self-esteem or self-efficacy, ameliorated family stress, or simply reduced the amount of time spent with partners -- and by caseworkers' attention to support services. Notably, NEWWS did not try to identify women who might be in imminent danger related to abuse. For some women, work may lead to greater safety. For others, especially those in imminent danger of abuse, employment may not have such positive effects, which may make it difficult for them to work or comply with welfare-to-work program requirements.

• Welfare-to-work programs like those studied in NEWWS are unlikely to have long-term effects -- positive or negative -- on the likelihood that adults or their children have health care coverage.

When people leave welfare for work, they run the risk of losing health care coverage, because they will immediately or eventually lose their Medicaid coverage and often are not offered or cannot afford private insurance -- which they would most likely get through their employer -- to replace it. Transitional Medicaid is available to families for up to one year after they leave welfare, but this benefit has been underused, probably because many newly employed welfare recipients are not aware of their eligibility for it. Some of the NEWWS programs decreased health care coverage two years into the study period, but none affected adults' or children's coverage at the end of the five years. Because the programs increased employment, however, they did lead to a shift from public to private coverage at the five-year follow-up point. Only two programs increased the use of Transitional Medicaid but, at the end of the five years, neither program had led to an overall rise in health care coverage.

Children's Well-Being: How might programs that have mandates and services but leave income unchanged affect children in the long run?

During the two decades before FSA's passage, mothers receiving welfare who had children under age 6 were generally not subject to the participation and work requirements of welfare-to-work programs. With FSA's passage came the advent of mandatory participation in welfare-to-work activities for mothers with young children. Because the new mandate's implications for young children caused considerable concern in the early 1990s, the children in NEWWS who were preschool-aged at the time of random assignment were examined especially closely. The well-being of children of other ages was also examined in NEWWS. The passage of PRWORA, which imposed participation mandates on mothers with children as young as age 1 (or even younger, at states' option), renewed concern about the effects of welfare reform on children. The NEWWS findings suggest the following conclusions.

• Programs with mandates and services that also leave income unchanged have relatively few effects on young children.

In a group of children who were preschool-aged at study entry and were studied in depth, the well-being of program group children differed from that of control group children on only a small number of measures. At the two-year follow-up point, the few impacts found occurred predominantly in the area of cognitive functioning, with some programs improving these outcomes. The impacts did not persist, however, through the later years of the follow-up period. At five years, the few impacts on young children were largely in the area of social skills and behavior (such as being sensitive to others, making friends, and fighting or arguing with others). Overall, the impacts were not consistently favorable or unfavorable -- that is, they were favorable for some programs and unfavorable for others -- and varied in size. Very few impacts were found on measures of children's health or safety at two years or five years, although most of those found were unfavorable.

• Effects on child care use -- one important route by which young children might be affected by welfare-to-work programs -- closely mirror the programs' effects on employment.

As noted earlier, program and control group members in NEWWS were eligible for similar child care benefits while they worked or participated in work-related activities. But program group members' greater participation in work-related activities and their higher employment levels were expected to create a greater need for child care. As the NEWWS programs' impacts on employment diminished over the follow-up period, so did their effects on use of child care. During the first two years, the majority of the programs produced moderate to large increases in child care use. But by the end of the five years, only the Portland program was still elevating use of child care relative to the control group level. Notably, about half of the NEWWS programs increased use of transitional child care (available to recipients for up to one year after they leave welfare), largely because they increased both the number of people leaving welfare for work and the number of people who actually received this benefit once they became eligible for it.

• Programs with mandates and services that also leave income unchanged have few effects on school-aged or very young children; the effects on adolescents, however, are likely to be unfavorable.

For children who at study entry were under 3 or over 5, whose outcomes were measured in NEWWS using a more limited set of measures than were preschool-aged children's outcomes, few impacts on well-being were found. Some impacts, however, were found on academic outcomes (such as grade repetition, dropping out of school, and being suspended or expelled from school) for children who were adolescents at study entry. Although these impacts were found in only about half of the programs for which data were available, they were predominantly unfavorable. In Riverside, for example, about 4 percent of adolescent children of control group members had ever repeated a grade in school; both programs there increased this rate by 3 to 4 percentage points. Adolescents' academic functioning may have been especially vulnerable to the employment gains and income losses found for mothers of adolescents in several of the programs.

• Effects on children do not appear to depend on whether welfare-to-work programs are employment-focused or education-focused.

Some early proponents of education-focused welfare-to-work programs hypothesized that such programs might benefit children more than employment-focused ones because parents who attended education or training classes would become more involved in their children's schoolwork and serve as role models for succeeding in school. This hope was not realized in the NEWWS programs. The impacts on children -- whether the young children studied in depth or the children of all ages studied using a more limited set of measures -- did not vary according to whether the programs had an employment focus or an education focus. Both types of programs had few effects on children.

6. Other Lessons

Income: How can welfare-to-work programs increase family resources?

The administrators of all the welfare-to-work programs studied in NEWWS hoped that their programs' preemployment services, mandates, and messages would enable welfare recipients eventually to increase their income and move out of poverty, but their relative emphasis on and methods of achieving this goal differed. The education-focused programs emphasized this goal most explicitly -- by providing education and training activities, which were seen as a gateway to high-paying, stable employment. The Portland program emphasized this goal in another way -- by encouraging welfare recipients participating in job search to accept only jobs that paid above the minimum wage and offered opportunity for advancement. Notably, however, none of the NEWWS programs provided earnings supplements to low-wage workers, as many states now do, in the form of substantial earned income disregards or work incentive payments provided outside the welfare system. (All the program and control groups in NEWWS were eligible for the federal EIC.) As a result, the NEWWS results can draw attention to gaps that the new policies might fill.

• Several factors explain why the NEWWS programs increased income so little.

First, although almost all the programs studied in NEWWS increased earnings, their impacts were relatively modest in size owing to the only modest increases in five-year job finding, the small improvement in employment stability, and the small increases in wages (Box 2). Second, there was little increase in the extent to which people worked while on welfare. Third, some program group members did not return to welfare even after they lost their jobs. Finally, some program group members failed to maintain their eligibility for food stamps after they left welfare, the reasons for which are unclear.

• The NEWWS findings underscore the value of efforts to add services and incentives aimed at increasing employment retention and advancement and encouraging low-income families to continue receiving benefits for which they are eligible.

During the 1990s, especially after passage of federal welfare reform in 1996, welfare administrators adopted a variety of services and financial incentives that were specifically intended to boost welfare recipients' incomes. These included raising the level of earned income disregards, extending child care benefits and medical coverage for welfare recipients who find employment, providing financial incentives to welfare recipients who work, and offering job search assistance to people who leave welfare for employment but then lose their jobs. During this period, the federal government joined in this effort by substantially increasing the value of the EIC. Although policy researchers have only begun to investigate the effects of recent policy changes such as these, the NEWWS findings underscore the need for such innovations.

Case Management: Do different strategies yield different results?

As mentioned in the description of the NEWWS programs operated in Columbus, there are two general approaches to welfare case management: traditional and integrated. Although each can be argued to have advantages and disadvantages, some policymakers and program operators have speculated that integrating the roles of income maintenance and employment and training might be advantageous under TANF, most importantly for its potential to change the "welfare culture" from one that emphasizes sending out checks to one that stresses employment and job preparation. In addition, some have argued that integrated case management leads welfare recipients to seek jobs or get into job preparation activities more quickly.

NEWWS provides a unique opportunity to capture the relative advantages and disadvantages of the two case management approaches in one site. Apart from the fact that one used traditional and the other integrated case management, the two welfare-to-work programs operated in Columbus were the same, and their enrollees were subject to the same public assistance eligibility and payment system. The results suggest the following lessons.

• Integrated case management, when well funded and well run, offers advantages over the traditional approach.

Integrated case management can engage more people in program activities. As shown in Table 4, which presents many of the results discussed in this subsection, welfare-to-work activity participation rates were higher in the Columbus integrated program than in the traditional one. This result seems to be attributable to the fact that the integrated staff monitored participation more closely and followed up with participants who had attendance problems more quickly than did the traditional staff. It is also possible that people with integrated case managers took the threat of sanctions for noncompliance more seriously than did people working with traditional staff because the integrated staff could enforce the sanction themselves by reducing the grant. Interestingly, the integrated approach engaged more people in the program without actually imposing more sanctions.

Engaging more people in welfare-to-work program activities is valuable for a few reasons. Most obviously, it exposes more people to the program's services and messages. It also enforces the idea that people receiving welfare should, in return, take part in employment-focused services. Finally, under PRWORA, states must meet relatively high work participation requirements or face reductions in their TANF block grant; integrated case management may help them do so.

• Relative to the traditional approach, integrated case management can foster more effective income maintenance, which in turn can reduce the time people spend receiving welfare and the total amount of welfare that they receive.

In Columbus, integrated staff closed cases more quickly than traditional staff and, through closer and more frequent contact with recipients, were better able to discover people who should not have been receiving welfare. Both programs reduced the number of months that recipients spent on welfare (relative to the control group level), but the impact of the integrated program was 1.4 months larger than that of the traditional one; similarly, whereas the five-year welfare savings generated by the integrated program were $1,523, those generated by the traditional program were $1,105.

In the current environment of time-limited welfare benefits, programs that reduce the time spent on welfare -- thus allowing people to "bank" more months of welfare eligi-bility for future use -- are particularly valuable. Moreover, decreasing welfare payments saves the government money.

• Integrated case management can increase earnings more than the traditional approach.

The Columbus program with integrated case management increased five-year earnings (relative to the control group level) more than that with traditional case management, by $2,055 compared with $1,410. The difference between the earnings outcomes for the two approaches was not, however, statistically significant. Integrated case management worked especially well for nongraduates (and the difference in earnings outcomes for the two approaches was statistically significant for this subgroup). This result suggests that integrated management especially benefits more disadvantaged groups of people, perhaps because of the closer attention and monitoring it affords.

Both case management models in Columbus were operated as part of well-funded, well-run welfare-to-work programs. Staff had extensive administrative support, including a sophisticated case records information system, a child care referral unit, and a clerical unit that tracked recipients' attendance in program activities. In addition, program administrators placed a high priority on the employment services aspect of the programs; services were plentiful; and staff training was adequate. Even with these resources and supports, integrated case managers in Columbus found the job demanding (and caseloads were high, as shown in Table 4); without them, they might have found the work overwhelming, which in turn might have diminished the positive effects of the integrated approach.

Underscoring this point are findings from another NEWWS site -- Oklahoma City -- that used integrated case management but did not have the same funding, administrative, and management advantages as Columbus. (The two-year net cost per person of the Oklahoma City program was about $1,000, much lower than that of the two Columbus programs. In addition, its caseload was higher than that in Columbus's integrated program.) Oklahoma City's program did not increase five-year earnings. Considering these sites' results together, it is clear that integrated case management alone is not sufficient to produce the added benefits found in Columbus.

Table 4.
Comparison of Case Management Approaches:
When Well Funded and Well Run, Integrated Case Management
Offered Advantages over Traditional Case Management

	Columbus
	Integrated	Traditional
SOURCES: Hamilton et al., 2001; Scrivener and Walter, 2001. NOTE: Except for the participation and sanctioning rates and the net cost per person, which cover two years, all the results shown cover five years.
Average caseload size	140	265(income maintenance)
		258(employment and training)
Participation rate (%)	53	34
Sanctioning rate (%)	36	35
Earnings impact ($)	2,055***	1,410*
Welfare payment impact ($)	-1,523***	-1,105***
Welfare receipt impact (months)	-3.9***	-2.5***
Net cost per person ($)	2,149	1,720
Return to government budget per dollar invested ($)	1.06	0.83

Participation Standards: What does it take to engage a substantial proportion of people in welfare-to-work program activities?

Despite the fact that participation in welfare-to-work activities is generally required in exchange for welfare receipt, welfare agencies often have a difficult time engaging a large share of their caseloads in program activities. Reacting in part to low participation rates in welfare-to-work programs, FSA broke new ground in requiring states to engage a specified -- and gradually increasing -- proportion of welfare recipients in work-related activities each month or to face possible reductions in federal funding.

The 1996 welfare law continued down this path but went still further, raising the percentage of welfare recipients who had to engage in work or work activities, narrowing the set of activities that could count in participation rates, and increasing the required number of hours of participation per week. States could lower their required participation rate by reducing their TANF caseload (relative to their 1995 caseload) and, owing to the welfare reform message's taking hold and unprecedented economic expansion, targets have been relatively easy to meet. But the recent economic downturn, which is likely to increase TANF caseloads, may make meeting participation standards more difficult. Furthermore, welfare time limits have increased the pressure on welfare-to-work programs to help all welfare recipients become self-sufficient before exhausting their welfare eligibility, thus magnifying the importance and urgency of increasing program participation.

The NEWWS programs, which began operating under FSA, did not put time limits on welfare receipt, generally exempted more people from participation than is now the case, imposed less severe financial penalties for nonparticipation than are now enforced in many states, and did not have enhanced earned income disregards -- which allow more people to fulfill their participation requirement by working while on welfare -- as many states now do. Nonetheless, the programs all shared PRWORA's goals of requiring welfare recipients to do something in exchange for their welfare benefits, and the NEWWS findings provide lessons and guidance concerning participation.

• Three elements strongly affect a program's participation rate and in turn how program staff behave and emphasize different procedures and outcomes in order to meet participation benchmarks: the definition of a participant, the base of people mandated to participate, and the period of time over which the rate is measured.

Table 5 shows the three main inputs to participation rate calculations and how much the participation rates of six of the NEWWS programs would have varied as a function of these components. Note that all six programs were successful in the sense that, over the five-year follow-up period, they vigorously enforced the participation mandate, increased employment, and reduced welfare. If participation is calculated using method 1, then the participation is low because only a small percentage of all welfare cases in a month included someone who participated at least 20 hours or was employed at least 15 hours every week. If, however as in method 2 recipients who are being sanctioned and recipients who participated at all during a month (that is, for any number of hours) are counted and people who qualify for exemptions are removed from the base, then the participation rate increases significantly. Methods 1 and 2 compute participation rates over a one-month period. Method 3, in contrast, lengthens the period of measurement to include everyone who participated at all over a two-year period. As the table shows, this calculation yields the highest participation rates. Method 3 is the one used most often in research (and was discussed in the previous section), but the monthly methods (particularly method 1) correspond more closely to that outlined in the 1996 federal welfare law.

Table 5.
Methods of Calculating Participation Rates: How Participation Standards Are Defined Strongly Affects What Rates Can Be Achieved

Method	Definition of a Participant		Population Base	Measurement Period	Participation Rate Range
	Statuses Counted	Level of Involvement Required
SOURCE: Hamilton et al., 1995. NOTE: The participation rate ranges shown were calculated using data for the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside.
1	Participated in JOBS activities or was employed while receiving AFDC	Participated at least 20 hours per week during every week or was employed at least 15 hours per week during every week	All AFDC cases in the month	Month	5%-10%
2	Participated in JOBS activities or was employed while receiving AFDC or sanction was imposed or requested	Participated at all or was employed at all or sanction was imposed or requested at all	All JOBS-mandatory individuals in the month	Month	35%-44%
3	Participated in JOBS activities	Participated at all	All JOBS-mandatory individuals entering program in a year	Two years after program entry	44%-74%

• Achieving high rates of participation in welfare-to-work program activities takes time and resources. In general, the higher the required participation rate, the more money is needed.

Greatly increasing program participation requires committing considerable staff time and resources. Effort must be made to reach out to potential participants, assign them quickly to program activities, monitor their participation and progress, and reengage those who drop out. In addition, penalties must be enforced in a timely manner for those who do not comply with the participation mandate. Finally, the more participants a program has, the more resources are needed to pay for program activities and support services such as child care, transportation, and other ancillary expenses. The NEWWS findings show that programs must work with almost all the welfare recipients targeted by a mandate in order to have a sufficiently large proportion of them count as participants in any given month. In fact, staff often spend almost as much time (in some cases, even more time) trying to engage nonparticipants as working with participants.

• Even if funding is plentiful and case management is flawless, in any given month some people will not count as participants for unavoidable reasons.

Figure 10 shows the participation statuses in a typical month of enrollees in the six NEWWS programs represented in Table 5. Averaged across the six programs, only a small proportion of recipients required to participate actually met FSA's definition of a participant in a typical month. What were the other recipients doing? As the figure indicates, somewhat more than one-third of them had not yet attended a program orientation; about one-fifth had attended an orientation but were not active in that particular month; and one-third participated in a program activity, but not for enough hours to meet the participation threshold. Some of these reasons for not meeting the definition of participation may be preventable, particularly through greater staffing: Tighter case monitoring and shorter gaps between activities can encourage people to enter activities more quickly and minimize the amount of time that people wait for new activities to begin. Yet other reasons, such as recipients' becoming ill or having to care for an ill child, lie outside the purview of welfare-to-work programs.

• By themselves, rates of participation may not be as critical to increasing job finding and welfare exit as are the design and implementation of program activities.

Several factors may be more important than participation rates in realizing the goals of welfare reform -- for instance, whether the types of activities in which people participate are the ones that can best help them to become self-sufficient, what people are doing while in those activities, and who is participating. In other words, there is little gain to having a lot of people participating in activities if the activities themselves are ineffective or inappropriate for those who participate in them.

---

Figure 10.
Reasons for Nonparticipation in a Typical Month:
Some Types of Nonparticipation Are Preventable, but others are Not

SOURCE: Hamilton, 1995.

NOTE: The percentage shown are averages for the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside.

Mandate Enforcement: What role does enforcing mandates play in program effectiveness?

Participation mandates in welfare-to-work programs are intended to change welfare recipients' perceptions and behavior in several ways. First, they send the message that receipt of welfare is not an unconditional entitlement. Second, owing to the time investment required by program activities, they reduce the perceived value of welfare grants relative to that of employment. Third, they compel some people who would otherwise not do so to participate in activities that program operators believe will enhance their employability.

As discussed earlier, FSA established participation mandates and means by which to enforce them. Most single parents had to participate in employment preparation activities for as long as they remained on welfare. Case managers could use a variety of informal and formal responses, including financial sanctions, when people did not cooperate. In the NEWWS programs, sanctions typically reduced the family's monthly welfare grant by approximately 20 percent until the sanctioned individual complied with the mandate; there was no minimum sanction length for the first "offense," a minimum of three months for the second, and a minimum of six months for the third. As a result, some sanctioned individuals experienced a penalty for a short time, while others were under sanction much longer. In Grand Rapids, for example, almost half of those sanctioned remained in this status for at least half of the first two years of the study period.

Like FSA, PRWORA put in place mandates and enforcement mechanisms, but it also made the compliance requirements for welfare-to-work programs stricter and gave states the option to increase the severity of penalties for noncompliance. As a result, many states now impose full-family sanctions (revocation of the family's whole grant) as opposed to the partial-family sanctions (revocation of only the adult's portion of the grant) imposed under FSA.

Participation mandates and their enforcement could contribute to a program's impacts on earnings and welfare payments by, for example, getting more people to participate or getting nonparticipants to take a job to avoid participating in the program. It is very hard to determine the size of these contributions, however, because participation in program activities and the requirement to participate are usually bundled together. An innovative substudy was conducted as part of NEWWS in an effort to assess the effects of a mandate per se. Coupled with the extensive cross-program comparisons possible in the evaluation, this investigation points to several lessons and cautions.

• Simply communicating to welfare recipients that they are required to participate in a welfare-to-work program before they actually enroll in any program can affect their employment behavior.

The substudy, which was undertaken in Grand Rapids and Riverside, involved the random assignment of recipients to one of two groups before the random assignment process described in the first section of this document began. In the substudy, some welfare recipients were randomly assigned to a group that was informed that it was subject to a welfare-to-work program participation mandate; the others were assigned to a group that was not. By comparing the employment and welfare outcomes for the two groups, it is possible to isolate the effects of simply communicating to people that they are under a mandate to enter a welfare-to-work program before they learn how the program works or receive an activity assignment.

The findings from the substudy indicate that, in the site with the healthier labor market, the mandate had a large initial effect on average earnings but left rates of employment and welfare receipt unchanged. This suggests that some recipients reacted to the mandate by finding a job more quickly, finding a job with longer hours, or working more hours at a job they already had than they would have without a mandate. The earnings impact was concentrated among the more job-ready recipients, that is, among those who had been employed at some point during the year before study entry. In the site with the weaker labor market, the mandate had no effect on employment, earnings, or welfare receipt, even among the more job-ready. These results suggest that employment opportunities play a role in how welfare recipients react to a welfare-to-work mandate: The mandate alone can have its intended effects, but it is more likely to work in situations where jobs are available and people have the recent work experience needed to obtain a job.

• High-enforcement programs lead to higher participation rates than do low-enforcement programs.

As discussed in the first section, the nine high-enforcement NEWWS programs aimed to enroll most targeted people in the program, monitored participation in program activities moderately or very closely, and stressed the mandatory nature of the program through sanctions and means such as positive encouragement. About half of these programs had high rates of sanctioning; the other half had moderate sanctioning rates. The two low-enforcement NEWWS programs tended to give preference in enrollment to those who volunteered for the program (as opposed to vigorously trying to enroll all targeted individuals), did not closely monitor participation in program activities, and rarely imposed sanctions for nonparticipation. As shown in Table 6, the low-enforcement programs produced lower participation rates (calculated using method 3 in Table 5) and smaller impacts on participation than did the high-enforcement programs. The low-enforcement programs' particularly small participation impacts (of about 10 percentage points) are partly due to the fact that some control group members engaged in activities on their own. A program that lacks a significant "push" or "pull," like the low-enforcement programs in NEWWS, will not engage many more people than would participate in activities anyway.

• Among high-enforcement programs, higher sanctioning rates do not necessarily increase participation levels.

Some of the high-enforcement programs in Table 6 sanctioned nonparticipants at very high rates, imposing partial-family sanctions on at least one-third of welfare recipients who enrolled in the programs. As shown in the table, however, these programs were no more successful in engaging people in activities than were programs in the high-enforcement category that had more moderate sanctioning rates. The programs with the highest sanctioning rates had neither the highest participation rates nor the largest participation impacts.

• More disadvantaged recipients are more likely to be sanctioned in welfare-to-work programs.

In the NEWWS programs, sanctions were more likely to be imposed on and lasted longer for the more disadvantaged welfare recipients than the less disadvantaged ones. One possible reason for this is that more disadvantaged people remain on welfare longer, lengthening the period of time during which they could fail to comply with the participation mandate and be sanctioned.

Table 6.
Sanctioning and Participation Rates and Impacts on Participation, by Enforcement Level: Higher Sanctioning Rates Did Not Increase Participation in High-Enforcement Programs

Level of Enforcement	Sanctioning Rate (%)	Participation Rate(a) (%)	Participation Impact (%)
SOURCE:  Hamilton and Scrivener, 1999. NOTE:  (a) The participation rates shown are averages of individual program rates calculated using method 3 in Table 5.
High enforcement (9 programs)	28	57	26
High sanctioning (5 programs)	38	57	23
Moderate sanctioning (4 programs)	16	58	30
Low enforcement (2 programs)	3	45	10

7. Costs and Benefits

FSA pushed administrators to operate more complex welfare-to-work programs, involve a larger share of the caseload (including the most disadvantaged), and provide more intensive and expanded services -- changes that were likely to increase the costs of the programs relative to those of programs operated in the early to mid-1980s. The legislators' hope was that the higher upfront costs would bring larger long-run benefits, but it was unclear whether the programs' benefits would surpass their costs or whether the benefit-cost trade-off would be the same for all types of program approaches or for all subgroups of welfare recipients.

Under PRWORA, the relationship between costs and benefits remains a concern. A program's cost per person largely determines how many people it can serve. With fixed resources, each state is seeking to maintain the optimal balance between involving as many people as possible in the programs and delivering appropriate, effective services. The extensive analyses conducted as part of NEWWS help reveal the parameters around these program design decisions.

Costs: What contributes to the cost of welfare-to-work programs?

Different types of five-year costs were estimated for the NEWWS programs. The gross cost per program group member is a comprehensive measure of all the costs associated with providing employment services and related support services to people while they were enrolled in a welfare-to-work program as well as after they left the program and/or the welfare rolls; similarly, the gross cost per control group member is the corresponding estimate for the control group. The net cost of the program is the difference between these two estimates; in other words, the cost for the control group is the benchmark used to determine the size of the additional cost of running the program (on a per-person basis).

The costs of providing job search, education, and training -- whether within welfare-to-work programs or when individuals enrolled in these activities on their own -- as well as the costs of case management and support services contribute to both gross and net costs. For example, if a substantial and similar proportion of program and control group members participate in high-cost activities like vocational training and postsecondary education, it is likely that the gross costs for both groups will be high but that the program's net cost will be relatively low. In contrast, if most program group members participate in low-cost job search activities and few control group members do so, it is likely that the gross costs for both groups will be low but that the program's net cost will be relatively high.

• Taking into account all the services that sample members received over the five-year follow-up period, the NEWWS programs were expensive relative to other studied programs.

Averaged across all the NEWWS programs, the five-year gross cost per program group member, which included a substantial amount of self-initiated activities that sample members engaged in after leaving welfare, was approximately $7,600. Subtracting the gross cost per control group member yields the average five-year net cost per program group member, or about $3,600. The average net cost per program group member over the first two years, during which time many sample members were still on welfare, was about $2,000. The NEWWS programs' average cost was higher than that of other welfare-to-work programs studied by MDRC and comparable to that of programs such as the Alameda and Los Angeles Greater Avenues for Independence (GAIN) programs operated in the late 1980s and early 1990s, which provided extensive education and training services. It is important to note that the NEWWS programs were not mandatory for the entire welfare caseload (as is nearly always the case now, under TANF rules). Thus, the NEWWS costs are per person for a segment of the caseload -- not averaged across the whole caseload -- and do not include women with infant children, who typically have the highest child care costs.

• Employment-promoting activities accounted for more than half of the average five-year gross cost of the NEWWS programs.

All the NEWWS programs provided job search, education, and training activities as well as child care and transportation services to support participation in these activities. The programs' operating costs -- which included expenditures for case management services, overhead, and program orientation as well as job search, education, and training program activities -- were considerably higher than their support service costs, which included expenditures for child care, transportation, and other needs such as uniforms, tools, and books. Operating costs accounted for about four-fifths of the five-year gross cost, while support services accounted for about one-fifth. Another way to examine program costs is to look solely at the cost of providing in-program activities -- employment-promoting activities (and support services) that people engaged in while on welfare and in the programs -- as opposed to including the costs of self-initiated activities in which people participated after they had left welfare. Averaged across the programs, the in-program cost was approximately 70 percent of the five-year gross cost.

• On average, welfare departments carried less than half of the gross program costs; other government and community agencies covered the remainder of program expenses.

In NEWWS, welfare departments most often directly provided and funded case management, job search activities, and support services (primarily child care and bus passes) for people in program activities. Welfare departments generally did not pay for services such as basic education or vocational training. These costs, which accounted for the majority of the gross cost of most programs, were covered by schools and community agencies.

• Education-focused programs cost more than employment-focused programs.

In all three sites where a side-by-side comparison of an employment-focused program and an education-focused program was conducted, the education-focused program was from one-third more expensive to nearly twice as expensive as the employment-focused program. This is largely because program group members in education-focused programs were more likely to participate in vocational training (a high-cost activity) and because education and training assignments typically lasted longer than job search. Both the welfare department and other agencies had to pay additional expenses for education-focused programs: Whereas the welfare department spent more on case management, nonwelfare agencies spent more on classroom instruction and related expenses.

• Gross and net costs per person were higher for graduates than nongraduates.

Recipients who entered NEWWS with a high school diploma or GED were more likely to participate in more expensive activities such as postsecondary education and vocational training, whereas those lacking these credentials were more likely to participate in less expensive activities such as basic education. Control group members' rates of (self-initiated) participation in postsecondary education and vocational training were somewhat lower than those for program group members, while their rates of participation in basic education were much lower. As a result, gross costs were much higher for those with a high school diploma or GED than for those lacking this credential, while the differences in net costs between the two subgroups were less pronounced.

Costs Relative to Benefits: What is the government's financial return on its investment in welfare-to-work programs?

The programs' five-year benefits were also calculated in NEWWS. Benefits included the increases in earnings and decreases in welfare and food stamp payments discussed earlier, as well as dollar valuations of the programs' estimated effects on Medicaid, job fringe benefits, taxes paid, and the costs of administering transfer programs such as food stamps. All of these effects were considered along with the programs' estimated net costs to ascertain the net gains and losses to government budgets as well as to program group members resulting from the programs. Some limits on the comprehensiveness of the benefit-cost analysis, however, should be recognized. The analysis did not, for example, consider education benefits that are not reflected in earnings or put a dollar value on families' or children's well-being.

• Government budgets came out ahead on their investments in about half of the NEWWS programs.

One measure of cost-effectiveness is the return to government budgets per net dollar invested, that is, the average gain to government budgets from increased tax revenues and savings in transfer programs and associated administrative costs divided by the net cost of program services. By this metric, government budgets come out ahead if a program produces more than a dollar's worth of additional revenues and savings for every extra dollar spent on services for program group members relative to control group members. About half of the NEWWS programs returned at least as much to government budgets as was invested in them. Four of the programs, however, returned only about $.40 to $.80 per dollar invested. Programs that produced larger welfare savings had a better chance of paying for themselves, and those that generated the largest welfare savings resulted in the largest returns, bringing government budgets as much as $2.83 per dollar invested. The programs that generated smaller welfare savings had a harder time breaking even and more often resulted in losses. In contrast, the programs that led to the largest employment increases did not necessarily produce the largest government returns.

• Returns to government budgets that are driven by welfare savings do not appear to improve welfare recipients' economic well-being.

Where savings to government budgets were found in NEWWS, they were driven by welfare savings, leaving the sample members no better off financially than they would have been without the programs. In general, the benefit-cost findings from the welfare recipients' perspective mirror the findings on income: In most of the NEWWS programs, program group members' financial losses from decreased welfare and food stamp payments exceeded their financial gains from increased earnings and the EIC. Taking into account the dollar value of program group members' decreased Medicaid eligibility over the whole five-year follow-up period, program group members' losses -- compared with control group members' -- were even more pronounced.

• Higher-cost programs do not necessarily result in larger returns to government budgets or in larger earnings and welfare impacts than do lower-cost programs.

To have generated a return on the government's investment as large as that generated by the employment-focused programs, the education-focused programs would have had to generate larger welfare savings and/or larger increases in tax revenues. Not surprisingly in view of the earnings and welfare impacts discussed earlier, this was not the case (for the average return to government budgets per net dollar invested for each of the four primary categories of programs examined in NEWWS, see Table 7). On average, the greater investment in education-focused programs resulted in a greater loss from the government budget perspective. Likewise, the additional cost of program services for graduates did not result in a larger return to government budgets; in fact, the return to government budgets was not consistently higher or lower for graduates than for nongraduates for any type of program. Finally, no relationship was found between the NEWWS programs' net costs and earnings impacts: For example, the Portland program had one of the lowest net costs, yet programs with much higher net costs produced much smaller effects on earnings and employment than those found in Portland.

Job search firstHigh enforcement$1.58

Education or training first

Table 7.
Returns to Government Budgets over Five Years:
The Education-Focused Programs' Higher Costs Did Not Bring Higher Returns per Government Dollar Invested

Employment-focused approach		Education-focused approach
SOURCE:  Hamilton et al., 2001.
$2.83	$0.87	$0.84

8. Conclusion

The Family Support Act of 1988 sent a strong signal to states and localities — a signal that was amplified in the 1996 welfare reform law — that it was important to move people from welfare to work. States responded by developing welfare-to-work programs that were more complex, offered a wider spectrum of services, were implemented on a broader scale, and targeted for enrollment more groups of welfare recipients. The findings from NEWWS provide compelling evidence that these programs succeeded in achieving many of FSA's and program operators' principal explicit goals. All the programs increased people's employment and earnings and decreased their receipt of welfare, thus resulting in gains in people's self-sufficiency. Notably, mothers who were single parents achieved these benefits with few concomitant indications of harm or benefit to the well-being of their children. Despite these successes, however, none of the programs met FSA's implicit goal of making people materially better off. The NEWWS programs generally did not increase income or reduce poverty. Indeed, some of the more disadvantaged program enrollees were made worse off financially.

The lessons from NEWWS remain highly relevant in the current welfare reform environment and beyond. Program initiatives now in place in some localities aim to boost incomes and improve families' and children's well-being by substantially expanding earned income disregards, which permit working recipients to remain eligible to receive welfare payments, and by providing services that help people keep and advance in their jobs after becoming employed. The NEWWS income findings suggest that these are steps in the right direction. Other reform initiatives introduced since 1996 notably, time limits on the receipt of welfare benefits and stricter penalties for recipients who fail to meet welfare-to-work programs' participation requirements focus on reducing reliance on welfare. But whether through their use of earnings supplements, job retention and advancement strategies, time limits, or participation mandates, successful reform approaches must creatively engage welfare clients before they find jobs and must maximize the number who actually obtain employment. NEWWS has helped immeasurably to show how such preemployment programs can best be designed and operated. As policymakers continue to search for the most effective ways to move people from welfare to work, they will likely glean insights from this evaluation for years to come.

---

Appendix: NEWWS Program Summaries

Appendix Table.
Summary of NEWWS Program Activities, Environments, and Results

Program	Activities	Sample and Site Characteristics	Five-Year Net Cost per Program Group Member ($)	Year	Control Group	Impact
SOURCES:  Hamilton et al., 2001; Michalopoulos and Schwartz, 2001. NOTES:  Data on race/ethnicity in each site refer to the research sample, not to the general welfare caseload or population in that side. The unemploment rate is for the year in which random assignment began. Welfare grant amount are for a family of three with no earnings. Net costs are expressed in 1999 dollars.
Atlanta LFA	Job search (typically job club) was the firstactivity for almost all participants; if no jobwas found after job search, short-term vocational training or adult education were the most common next activities	90% African-American Mandatory for parents with no child under age 3 Welfare grant: $280 (1993) Unemployment rate: 6.2%	3,974	Earnings ($)
				Year 1	1,647	396***
				Year 2	2,578	650***
				Year 3	3,313	618***
				Year 4	4,255	560**
				Year 5	5,586	235
				Welfare receipt (%)
				Year 1	82.7	-4.5***
				Year 2	70.9	-6.2***
				Year 3	61.2	-5.1***
				Year 4	50.1	-5.7***
				Year 5	36.5	-3.9**
Grand Rapids LFA	Job search (typically job club) was the first activity for almost all participants; if no job was found after job search, work experience or (for some) short-term vocational training or adult education were the most common next activities	50% white, 40% African-American Mandatory for parents with no child under age 1 Welfare grant: $474 (1993) Unemployment rate: 5.5%	1,987	Earnings ($)
				Year 1	1,759	530***
				Year 2	2,874	511***
				Year 3	4,211	447**
				Year 4	5,480	135
				Year 5	6,447	-71
				Welfare receipt (%)
				Year 1	77.8	-9.1***
				Year 2	60.8	-7.3***
				Year 3	48.0	-6.8***
				Year 4	38.0	-5.6 ***
				Year 5	27.8	-3.5**
Riverside LFA	Job search (typically job club) was the first activity for almost all participants; if no job was found after job search, continued job search or (for some) vocational training were the most common next activities search or, in some cases, life skills classes.	50% white, 35% Hispanic Mandatory for parents with no child under age 3 Welfare grant: $624 (1993) Unemployment rate: 11.7%	3,320	Earnings ($)
				Year 1	1,741	719***
				Year 2	2,482	557***
				Year 3	2,957	432***
				Year 4	3,554	313*
				Year 5	4,155	528***
				Welfare receipt (%)
				Year 1	69.6	-6.7***
				Year 2	56.4	-6.4***
				Year 3	47.6	-4.8***
				Year 4	41.1	-4.2***
				Year 5	34.6	-4.4***
Portland	Less job-ready people were assigned to adult education, vocational training, or life skills classes at the discretion of case managers; others were assigned to job search or, in some cases, life skills classes. Job search was the most common next activity for those who completed their initial activities	80% white, 20% African-American Mandatory for parents with no child under age 1 Welfare grant: $460 (1993) Unemployment rate: 6.6%	2,865	Earnings ($)
				Year 1	2,484	343*
				Year 2	3,150	1,271***
				Year 3	4,050	1,441***
				Year 4	5,112	1,207***
				Year 5	6,095	887**
				Welfare receipt (%)
				Year 1	68.0	-8.3***
				Year 2	53.3	-12.7***
				Year 3	38.6	-12.8***
				Year 4	22.5	-6.0***
				Year 5	17.1	-4.8***
Atlanta HCD	Adult education was the most common first activity for people without a high school diploma or GED; vocational training was the most common first activity for others. Job search or vocational training were the most common next activities for people who completed these initial activities	90% African-American Mandatory for parents with no child under age 3 Welfare grant: $280 (1993) Unemployment rate: 6.2%	5,480	Earnings ($)
				Year 1	1,647	199*
				Year 2	2,578	480***
				Year 3	3,313	599***
				Year 4	4,255	655***
				Year 5	5,586	84
				Welfare receipt (%)
				Year 1	82.7	-2.0
				Year 2	70.9	-4.8***
				Year 3	61.2	-3.8**
				Year 4	50.1	-3.2*
				Year 5	36.5	-2.1
Grand Rapids HCD	Adult education was the most common first activity for people without a high school diploma or GED; vocational training was the most common first activity for others, although some were first assigned to "brush-up" basic education courses or work experience. Job search or vocational training were the most common next activities for people who completed these initial activities	50% white, 40% African-American Mandatory for parents with no child under age 1 Welfare grant: $474 (1993) Unemployment rate: 5.5%	3,773	Earnings ($)
				Year 1	1,759	110
				Year 2	2,874	479***
				Year 3	4,211	303
				Year 4	5,480	56
				Year 5	6,447	-102
					Welfare receipt (%)
				Year 1	77.8	-4.2***
				Year 2	60.8	-6.4***
				Year 3	48.0	-6.3***
				Year 4	38.0	-5.6***
				Year 5	27.8	-2.0
Riverside HCD	Adult education was the most common first activity; job search or vocational training was the most common next activity for people who completed this first activity. All sample members in this program lacked a high school diploma or GED certificate,attained low scores on a reading or math test at program entry, or had limited proficiency in English	50% white, 35% Hispanic Mandatory for parents with no child under age 3 Welfare grant: $624 (1993) Unemployment rate: 11.7%	4,573	Earnings ($)
				Year 1	1,250	196*
				Year 2	1,886	139
				Year 3	2,150	411**
				Year 4	2,601	255
				Year 5	3,025	359
				Welfare receipt (%)
				Year 1	72.3	-4.1***
				Year 2	60.0	-4.1**
				Year 3	52.1	-5.9***
				Year 4	46.3	-7.5***
				Year 5	39.0	-5.7***
Columbus Integrated	Adult education was the most common first activity for people without a high school diploma or GED; "brush-up" basic education, vocational training, and postsecondary education were the most common first activities for others	50% white, 50% African-American Mandatory for parents with no child under age 3 Welfare grant: $341 (1993) Unemployment rate: 4.6%	4,183	Earnings ($)
				Year 1	2,914	80
				Year 2	3,982	595***
				Year 3	5,134	510***
				Year 4	6,260	463**
				Year 5	7,276	406
				Welfare receipt (%)
				Year 1	72.5	-4.5***
				Year 2	53.8	-6.8***
				Year 3	40.3	-7.1***
				Year 4	27.6	-5.3***
				Year 5	16.5	-3.8***
Columbus Traditional	Adult education was the most common first activity for people without a high school diploma or GED; "brush-up" basic education, vocational training, and postsecondary education were the most common first activities for others	50% white, 50% African-American Mandatory for parents with no child under age 3 Welfare grant: $341 (1993) Unemployment rate: 4.6%	3,772	Earnings ($)
				Year 1	2,914	185
				Year 2	3,982	490***
				Year 3	5,134	325*
				Year 4	6,260	298
				Year 5	7,276	113
				Welfare receipt (%)
				Year 1	72.5	-3.6***
				Year 2	53.8	-4.6***
				Year 3	40.3	-5.5***
				Year 4	27.6	-3.0**
				Year 5	16.5	-2.4**
Detroit	During the first half of the study period, adult education was the most common first activity for people without a high school diploma or GED; vocational training or postsecondary education were the most common first activities for others. During the second half of the study period, almost everyone still on welfare was assigned to job search	Mostly African-American Mandatory for parents with no child under age 1 Welfare grant: $459 (1993) Unemployment rate: 8.0%	2,053	Earnings ($)
				Year 1	1,341	57
				Year 2	2,663	313**
				Year 3	4,112	481**
				Year 4	5,570	351
				Year 5	6,823	258
				Welfare receipt (%)
				Year 1	86.5	-1.1
				Year 2	73.7	-3.6***
				Year 3	61.4	-4.8***
				Year 4	48.1	-3.0**
				Year 5	35.2	-0.2
Oklahoma City	Adult education, vocational training, and postsecondary education were the most common activities	70% white, 30% African-American Mandatory for parents with no child under age 1 Welfare grant: $324 (1993) Unemployment rate: 5.6%	N/A	Earnings ($)
				Year 1	1,387	16
				Year 2	2,127	-5
				Year 3	2,672	-40
				Year 4	3,068	45
				Year 5	3,498	100
				Welfare receipt (%)
				Year 1	53.1	-3.1***
				Year 2	41.0	-1.9
				Year 3	32.7	-2.4**
				Year 4	n/a	n/a
				Year 5	n/a	n/a

Selected Publications From This Evaluation

Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to-Work Strategies. Prepared by Gayle Hamilton, MDRC, 2002. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Prepared by Gayle Hamilton, Stephen Freedman, Lisa Gennetian, Charles Michalopoulos, Johanna Walter, Diana Adams-Ciardullo, and Anna Gassman-Pines, MDRC, and Sharon McGroder, Martha Zaslow, Surjeet Ahluwalia, and Jennifer Brooks, Child Trends, 2001. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Improving Basic Skills: The Effects of Adult Education in Welfare-to-Work Programs. Prepared by Johannes M. Bos, Susan Scrivener, Jason Snipes, and Gayle Hamilton, MDRC. 2001. Washington, D.C.: U.S. Department of Education, Office of the Under Secretary and Office of Vocational and Adult Education; and U.S. Department of Health and Human Services.

Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program. Prepared by Susan Scrivener and Johanna Walter, MDRC. 2001. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Do Mandates Matter? The Effects of a Mandate to Enter a Welfare-to-Work Program. Prepared by Jean Tansey Knab, Johannes M. Bos, Daniel Friedlander, and Joanna W. Weissman, MDRC. 2001. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

What Works Best for Whom: Impacts of 20 Welfare-to-Work Programs by Subgroup. Prepared by Charles Michalopoulos and Christine Schwartz, MDRC. 2001. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

The Experiences of Welfare Recipients Who Find Jobs. Prepared by Karin Martinson, MDRC. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Four-Year Impacts of Ten Programs on Employment Stability and Earnings Growth. Prepared by Stephen Freedman, MDRC. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Implementation, Participation Patterns, Costs, and Two-Year Impacts of the Detroit Welfare-to-Work Program. Prepared by Mary Farrell, MDRC. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Oklahoma City's ET & E Program: Two-Year Implementation, Participation, Cost, and Impact Findings. Prepared by Laura Storto, Gayle Hamilton, Christine Schwartz, and Susan Scrivener, MDRC. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Do Mandatory Welfare-to-Work Programs Affect the Well-Being of Children? A Synthesis of Child Research Conducted as Part of the National Evaluation of Welfare-to-Work Strategies. Prepared by Gayle Hamilton, MDRC. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation and Administration for Children and Families; and U.S. Department of Education.

Evaluating Alternative Welfare-to-Work Approaches: Two-Year Impacts for Eleven Programs. Prepared by Stephen Freedman, Daniel Friedlander, Gayle Hamilton, JoAnn Rock, Marisa Mitchell, Jodi Nudelman, Amanda Schweder, and Laura Storto, MDRC. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Impacts on Young Children and Their Families Two Years After Enrollment: Findings from the Child Outcomes Study. Prepared by Sharon M. McGroder, Martha J. Zaslow, Kristin A. Moore, and Suzanne M. LeMenestrel. 2000. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation and Administration for Children and Families; and U.S. Department of Education.

Implementation, Participation Patterns, Costs, and Two-Year Impacts of the Portland (Oregon) Welfare-to-Work Program. Prepared by Susan Scrivener, Gayle Hamilton, Mary Farrell, Stephen Freedman, Daniel Friedlander, Marisa Mitchell, Jodi Nudelman, and Christine Schwartz, MDRC. 1998. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Evaluating Two Welfare-to-Work Program Approaches: Two-Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites. Prepared by Gayle Hamilton, Thomas Brock, Mary Farrell, Daniel Friedlander, and Kristen Harknett, MDRC. 1997. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Educating Welfare Recipients for Employment and Empowerment: Case Studies of Promising Programs. Prepared by Janet Quint, MDRC. 1997. Washington, D.C.: U.S. Department of Education, Office of the Under Secretary and Office of Vocational and Adult Education; and U.S. Department of Health and Human Services.

Changing to a Work First Strategy: Lessons from Los Angeles County's GAIN Program for Welfare Recipients. Evan Weissman. 1997. New York: MDRC.

Work First: How to Implement an Employment-Focused Approach to Welfare Reform. Amy Brown. 1997. New York: MDRC.

Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in Welfare-to-Work Programs. Prepared by Gayle Hamilton, MDRC. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

How Well Are They Faring? AFDC Families with Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation. Prepared by Kristin A. Moore, Martha J. Zaslow, Mary Jo Coiro, and Suzanne M. Miller, Child Trends, and Ellen B. Magenheim, Swarthmore College. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Early Findings on Program Impacts in Three Sites. Prepared by Stephen Freedman and Daniel Friedlander, MDRC. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Adult Education for People on AFDC: A Synthesis of Research. Prepared by Edward Pauly, MDRC. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation; and U.S. Department of Education.

Five Years After: The Long-Term Effects of Welfare-to-Work Programs. Daniel Friedlander and Gary Burtless. 1995. New York: Russell Sage Foundation.

Early Lessons from Seven Sites. Gayle Hamilton and Thomas Brock. 1994. Washington, D.C.: U.S. Department of Health and Human Services and U.S. Department of Education.

The Saturation Work Initiative Model in San Diego: A Five-Year Follow-Up Study. Daniel Friedlander and Gayle Hamilton. 1993. New York: MDRC.

From Welfare to Work. Judith M. Gueron and Edward Pauly. 1991. New York: Russell Sage Foundation.