National Evaluation of the Welfare-to-Work Grants Program: Final Report

Contents

• Key Questions and Findings
• Conclusions
• Lessons Concerning Program Design and Implementation
• Endnotes

The Welfare-to-Work (WtW) grants program was a large federally funded effort to help the most disadvantaged welfare recipients leave the rolls and become employed.  As part of the Balanced Budget Act (BBA) of 1997, Congress provided $3 billion for WtW programs, eventually distributed to over 700 state and local grantees.⁽¹⁾  Congress appropriated funds for fiscal years 1998 and 1999, and grantees were allowed five years to spend their funds.  The intent of the grants program, administered at the national level by the U.S. Department of Labor (DOL), was to supplement the Temporary Assistance for Needy Families (TANF) block grants to states, which had been authorized as part of the welfare reforms embodied in the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA).⁽²⁾  WtW funds were to support programs  especially those in high-poverty communities  to assist the least employable welfare recipients and noncustodial parents make the transition from welfare to work.

The BBAmandated that the U.S. Department of Health and Human Services (DHHS) evaluate the WtW grants program.  This is the final in a series of reports on that evaluation, conducted by Mathematica Policy Research Inc., along with its subcontractors the Urban Institute and Support Services International.  It summarizes findings from our earlier reports on component studies under this evaluation  including the implementation study, a study of WtW initiatives undertaken by American Indian tribes, and a study of the characteristics of WtW enrollees and their outcomes one year after program entry.  It also presents new findings on enrollees' outcomes two years after entry into WtW programs in the following 11 study sites:

The evaluation involved collecting information on WtW enrollees at the time of program entry and during the following two years. These data were obtained from (1) a baseline survey of enrollees conducted in 1999-2002, (2) a 12-month follow-up survey of enrollees conducted in 2000-2003, (3) a 24-month follow-up survey of enrollees conducted in 2001-2003, and (4) state administrative records for the TANF, Food Stamp, and Unemployment Insurance (UI) programs over the two-year follow-up period for each enrollee.

We planned to evaluate program impacts based on an experimental design, but low enrollments in local programs funded by WtW grants made doing so unfeasible. An experimental evaluation would have required that the number of eligible individuals referred to a program exceed its capacity, thus allowing use of random assignment to create a control group; but actual referrals were too low. As a result, the findings presented here are descriptive, and should not be interpreted as indications of program impacts. Neither should differences in findings across the study sites be interpreted as differences in program efficacy.

Key Questions and Findings

The findings from this evaluation's outcomes study that are presented in this executive summary are organized around four key questions about the individuals who enrolled in WtW-funded programs in the 11 study sites:

• Were WtW enrollees hard to employ, compared with the overall TANF population?
• What services did enrollees receive?
• How successful were WtW enrollees in the labor market?
• How did enrollees' well-being evolve over the two years following program entry?

Were WtW Enrollees Particularly Hard to Employ? (Exhibit ES.1)

WtW enrollees came primarily from the welfare rolls, but most had not been there long. Given the nature of the WtW grants program, most enrollees had received welfare benefits  TANF or its precursor, Aid to Families with Dependent Children (AFDC)  prior to program entry. In all of the study sites except Milwaukee, 87 percent or more of WtW enrollees received welfare benefits as adults sometime before entering the program. InChicago, Nashville, and St. Lucie County, virtually all WtW enrollees had received TANF/AFDC at some point. However, long-term welfare receipt was the exception rather than the rule. In most of the study sites, only about one-third of WtW enrollees reported in the baseline survey that they had received TANF/AFDC for more than five years.

WtW enrollees in the study sites resembled TANF recipients nationwide. In most sites, they were predominantly female, unlikely to be married, and typically members of racial or ethnic minority groups. The Boston site illustrates this pattern. There, 93 percent of enrollees were women, 95 percent were unmarried, and 93 percent were minorities. About equal numbers in most sites were above and below age 30. In sharp contrast to the typical study site, Milwaukee's Nontraditional Opportunities for Work (NOW) program  which served noncustodial parents who were on probation or parole  had a clientele that was 95 percent male. Enrollees in St. Lucie County, West Virginia, and Yakimawere less likely than their counterparts in the study's more urbanized sites to be women or members of a minority group, and somewhat more likely to be married. In West Virginia, for example, only 17 percent of enrollees were minorities and more than a quarter were married.

Many of the WtW enrollees faced significant barriers to employment. In most sites, more than one-third lacked a high school diploma or GED. Most had weak employment histories. In eight of the nine sites for which UI data were available, just one-third to one-half of enrollees were employed in the second quarter prior to program entry, and in the ninth site only one in five were employed. Very few enrollees in any of the study sites had been steadily employed. The share of enrollees with any recorded employment in all four quarters prior to program entry ranged from just 7 percent in West Virginia to 32 percent in Baltimore County  work histories that were similar to those of the general TANF population. About one-third of enrollees resided with a child under the age of 3, which may have presented a barrier to employment.

What Services Did WtW Enrollees Receive? (Exhibit ES.2)

PRWORA emphasized rapid employment, and the evaluation showed that WtW enrollees in the study sites received services consistent with this emphasis. In most of the sites, 80 percent or more of enrollees received some type of employment preparation service during the year following program entry. Such services are typically designed to quickly address barriers to employment and move enrollees into jobs. These are distinct from skill enhancement services, which generally provide longer-run solutions to human capital deficits.

Job readiness preparation and help with finding a job were at the core of WtW services. Job readiness training and job search assistance were received by more than half of enrollees in seven of the study sites (Boston, Chicago,Nashville, Philadelphia,Phoenix, West Virginia, and Yakima). The Phoenixand Yakima enrollees typically received brief job readiness training followed by job search assistance  a combination of services that was highly consistent with rapid transition to employment. In contrast, enrollees in Boston,Nashville, Philadelphia, and West Virginia typically received extended job readiness training (or, in the case of Nashville, skill enhancement services), followed by job search assistance. It generally took enrollees in these sites longer to become employed than their counterparts in Phoenixand Yakima. Enrollees in Chicagoreceived heterogeneous services due to the diverse programs in that site; however, most of the Chicagoenrollees entered programs that emphasized "rapid attachment" to jobs.

In the other four sites, job readiness training and job search assistance were less consistently dominant. Fewer than half of the enrollees in Milwaukee, Ft. Worth, Baltimore County, and St. Lucie County received job readiness training and job search assistance. In Ft. Worth, these low rates were not offset by other services, resulting in the lowest rate of receipt of any employment preparation services among the study sites (68 percent). Perhaps as a result, enrollees in Ft. Worth, along with those in Milwaukee, required more than five months on average to find their first post-entry job. Many ex-offender/noncustodial parents served by Milwaukee's NOWprogram received services that were less common in the other sites: peer support/discussion groups, legal assistance, and substance abuse treatment. Most of the enrollees in Baltimore County and St. Lucie County  the two Johns Hopkins University (JHU) sites  were already employed and therefore had lower need for job readiness training and job search assistance. Instead, they had relatively high rates of receipt of counseling and mediation services.

As federal policymakers intended, skill enhancement services (education and training) were not prominent in programs funded by WtW grants. Enrollees in Baltimore County, St. Lucie County, and Nashville were the most likely to have received these services, but even in these sites fewer than half of enrollees received them. The federal legislation that initially authorized the WtW grants program (the BBA) permitted skill enhancement services to be provided to enrollees only after they had obtained jobs. Subsequent amendments eased this restriction by allowing such services prior to employment for a maximum of six months.⁽³⁾ Still, it is clear that federal policymakers intended for most WtW program investment in human capital to occur after, rather than before, an enrollee obtained employment.⁽⁴⁾

How Successful Were WtW Enrollees in the Labor Market? (Exhibit ES.3)

Two years after entering WtW, enrollees were doing better overall in the labor market, but success was still bypassing many of them. They were much more likely to be employed two years after entering WtW than at the time of entry; nevertheless, most were not working at the 24-month follow-up survey. In the non-JHU sites, about four in ten enrollees were employed two years after entering WtW, whereas no more than about one-fourth were employed when they entered the program.⁽⁵⁾ The typical pattern is illustrated at the Transitional Work Corporation in Philadelphia, where 36 percent of enrollees were employed at the end of the second post-entry year, compared with only 7 percent at entry. Despite the widespread increases in employment, majorities of enrollees were employed at the time of the 24-month follow-up survey in only three of the study sites  Baltimore County, St. Lucie County, and West Virginia.

In many cases, however, employment was spotty, so most enrollees were employed at some time during the second year after program entry, even if they were not employed at the 24-month follow-up survey. For example, 61 percent of the Boston enrollees were employed sometime during the second year, although only 41 percent were employed at the end of that year. Except in Phoenix, about 60 percent or more of enrollees worked sometime during the second year. The low rate in Phoenixreflects, in part, a sharp drop in employment between the first and second post-entry years. This is the most extreme example of a general pattern; enrollees in all of the study sites except Ft. Worthwere less likely to have been employed during the second post-entry year than during the first. The reductions in employment during the second year were especially large in Phoenix and Yakima  where the local WtW programs emphasized the rapid placement of enrollees into jobs, perhaps resulting in poorer matches of enrollees with jobs  and in Philadelphia, where many of the first-year jobs were temporary subsidized jobs.

When they worked, WtW enrollees put in substantial hours on the job. Those who were employed two years after program entry tended to work full-time, or nearly so, on their principal job. Their mean hours of work in a typical week ranged from 32 to 38 per week across the study sites; however, they were not necessarily employed consistently week after week over the course of a month. The mean wage varied more widely, from a low of $6.40 per hour in West Virginiato around $10 in Baltimore County,Boston, and Milwaukee. It was about $8 per hour in the other seven sites. Only about one in every five enrollees who was employed at the end of the second year received health insurance benefits on the principal job, except in Baltimore County, where nearly half had such coverage.

Work hours were stable over time following program entry, but wages and benefits improved from the first to second year in some sites. In six sites, the mean wage rate was higher and/or health insurance coverage was more prevalent on the principal job held at the end of the second year. For example, employed enrollees in Chicagowere 8 percentage points more likely to be covered by health insurance and were paid an average of $0.43 per hour more on the principal job held two years after program entry than on the principal job held one year after entry.

How Were WtW Enrollees Faring Two Years After Program Entry? (Exhibit ES.4)

WtW enrollees were less dependent on TANF as time went on, but not necessarily because they found jobs. Most enrollees were on TANF when they entered WtW. Two years later, the rate of TANF receipt was lower  generally dramatically so  in every study site except Milwaukee. The Yakimasite is typical; there, the rate of TANF receipt fell from 85 percent at program entry to just 36 percent two years later. However, in all but the two JHU study sites, only about one-third of enrollees were employed and off TANF two years after they entered WtW.

Moreover, WtW enrollees generally remained poor. Household incomes were low and poverty rates were high in all study sites at the end of the second year following program entry. Enrollees in Nashville, Philadelphia, and West Virginia had mean monthly incomes of less than $1,200 and among the highest poverty rates (at least 83 percent) of enrollees in all of the study sites. Enrollees in Baltimore County and Milwaukeehad the highest mean monthly incomes, $1,606 and $1,816, respectively, and the lowest poverty rates, which, at 54 and 59 percent were nevertheless high in an absolute sense. Excepting only these two study sites, the incidence of poverty at the two-year follow-up point exceeded 60 percent. Furthermore, the incidence of severe poverty  income below 50 percent of the poverty threshold  among all of the sites ranged from a low of 25 percent in Baltimore County to a high of 57 percent in Philadelphia. Following the Census Bureau's methodology for determining poverty status, these findings are based on a measure of income that does not include food stamps, the earned-income tax credit (EITC), income taxes, or payroll taxes.

Employment was clearly associated with lower poverty rates. Although poverty was pervasive two years after program entry, its incidence was typically about 30 percentage points lower among those who were employed. The difference in poverty rates between employed and not-employed enrollees was greatest in sites like Baltimore Countyand Boston where wage rates were high, and smallest in sites like Philadelphiaand West Virginia where wage rates were low. The fact that poverty rates even among employed enrollees were high in an absolute sense (greater than 50 percent in all of the study sites except Baltimore County and Milwaukee) is a reflection of both low wages and instability in employment over the course of a month.

Conclusions

The findings from this evaluation's study of the characteristics of WtW enrollees and their outcomes during the two years following program entry lead to the following conclusions:

Most WtW enrollees were TANF recipients who faced significant barriers to employment. This is consistent with the WtW grants program's objective of serving hard-to-employ welfare recipients. However, a comparison of the work histories of WtW enrollees with those of all TANF recipients in the study sites indicates that the enrollees were neither more nor less hard to employ, on average, than the general TANF population.

WtW enrollees were much more likely to receive employment preparation services than skill enhancement services. In accordance with the legislation that authorized the WtW grants program, more than two-thirds of enrollees in each of the 11 study sites received employment preparation services designed to get them ready for and move them into jobs. There was considerable variability across the sites in the types and duration of these services, but the most common by far were job readiness training and job search assistance. With the exception of Baltimore County, Nashville, and St. Lucie County, no more than about one-third of enrollees received skill enhancement services designed to increase their human capital.

Most enrollees were employed sometime during the second year after they entered WtW, but their employment tended to be unstable. With the exception of enrollees in the two JHU sites that primarily served employed persons, very few enrollees were employed when they entered WtW. But most  about 60 percent or more  were employed sometime during the second year following entry. However, that employment often proved to be unstable; only about 40 percent of enrollees were employed at the end of the second year in all sites except Baltimore County and St. Lucie County, where about 70 percent of enrollees were employed at the end of the second year.

Employment fell between the first and second years after program entry. In every study site except Ft. Worth, WtW enrollees were less likely to have been employed sometime during the second year following program entry than during the first year. Among these sites, the median reduction in enrollee employment sometime during the second year was about 10 percentage points.

Enrollees who were employed two years after program entry typically worked about the same number of hours as those who were employed at the end of the first year, but for somewhat greater compensation. As they had at the end of the first year after entering WtW, enrollees who were employed two years after entry worked nearly full-time, on average. Their hourly wages tended to be low  averaging about $8 per hour in seven of the study sites  and only about one in five enrollees was participating in an employer-sponsored health insurance plan. However, the mean wage rate and/or the health insurance coverage rate were modestly higher in six sites at the end of the second year following program entry relative to the end of the first year.

The incidence of poverty was high among WtW enrollees two years after program entry, but it was lower among those who were employed. Two years after they entered WtW, the poverty rate for WtW enrollees exceeded 60 percent in all of the study sites except Baltimore County (54 percent) and Milwaukee(59 percent). However, the rate for employed enrollees was 16 to 43 percentage points lower than for those who were not employed. But even among the employed, more than half of enrollees were living in poverty at the end of the second year in 9 of the 11 study sites.

Lessons Concerning Program Design and Implementation

The WtW grants program encouraged grantees to find creative ways to move enrollees quickly into the labor force and provide supports for that transition  an approach whose full consequences remain somewhat unclear. Our non-experimental outcomes analysis did not allow us to draw a firm conclusion regarding whether WtW enrollees made better employment progress than they would have without the program. Although most enrollees worked at some time during the evaluation's two-year follow-up period, many faced employment problems at the end of that period, and the jobs they held often left them in poverty.

Whether a more comprehensive approach, with greater attention to skills development before employment, would work better remains an open issue. Congress, responding to views on this issue expressed by grantees, amended the program in 1999 to allow greater use of job training before job placement. Notwithstanding this change, the outcomes observed in the 11 study sites suggest that there remains room for considerable improvement in our ability to move welfare recipients into sustainable employment that lifts them out of poverty. Further experimentation could clarify the contributions to that goal that could be made by greater use of job training, as well as the effects of other factors such as health care, child care, other support services, and help with family relationships.

Even in the absence of impact estimates, this evaluation's implementation study provides a basis for six lessons regarding the design and implementation of employment programs for TANF recipients and individuals with significant labor market liabilities.

Effective inter-agency partnerships are important. The WtW legislation required local programs to be implemented within a framework of partnership with local TANF agencies. However, effective partnerships were often slow to develop. In combination with falling welfare caseloads, this often resulted in low numbers of referrals of welfare recipients by TANF agencies to WtW programs, thereby exacerbating the difficulties that many local WtW programs experienced in achieving their enrollment targets. In sites where effective partnerships ultimately did develop, they resulted in improved access for welfare recipients to the workforce development system.

Increased Service Capacity is an Important Legacy. WtW grants afforded many nonprofit community-based organizations their initial opportunity to serve TANF recipients and/or noncustodial parents. Thus, the program increased the pool of qualified organizations with which TANF agencies can contract for employment services in the post-WtW era.

Program Flexibility Encourages Innovative Programming. Flexible rules allowed WtW grantees and their service providers to develop creative program service approaches and administrative practices. These included partnerships with employers, transitional and supported employment, and post-employment case management and job retention services. Some grantees pressed for additional flexibility to provide a broader range of pre-employment services, and Congress responded in 1999 with amendments to the program that permitted up to six months of pre-employment skill-enhancement training.

Stringent eligibility criteria and fiscal requirements can result in low program enrollment. WtW grantees had to spend at least 70 percent of their grant funds on services for enrollees who met detailed, restrictive eligibility requirements. This requirement contributed to the widespread problems that grantees experienced in achieving enrollment targets during the early years of the WtW program.

Correcting the rules governing a temporary program may be ineffective. The 1999 amendments to the BBA loosened the criteria that defined the enrollees on whom at least 70 percent of grant funds had to be spent. They also expanded the list of allowable pre-employment program activities to include more use of vocational education and job training. However, these changes had limited effect, because the final rules reflecting the amendments were published late in the life of the program, and grantees were reluctant to revise existing procedures and referral agreements with local TANF agencies.

Finally, the very use of such temporary funding may accentuate program design and implementation problems. The BBAoriginally gave grantees three years to spend their funds; the timeframe was extended to five years under the 1999 amendments. Despite the extension, some local WtW administrators continued to believe that the temporary nature of the funding compounded difficulties in program design and implementation, such as the reluctance of TANF and other agencies to refer clients to WtW rather than to service providers with whom they had long-term relationships. The administrators also viewed short-term funding as an impediment to identifying and correcting program design problems.

Endnotes

(1) BBA: Public Law 105-33, section 5001, August 5, 1997.

(2) PRWORA: Public Law 104-103, section 103, August 22, 1996.

(3) Public Law 106-118, Title VIII, sections 801-807, November 29, 1999.

(4) This interpretation is consistent with DOL's final rule for the program (DOL 2001, page 2715, Sec. 645.220).

(5) The increases in employment rates from program entry to the end of the second year after entry should not be interpreted as impacts of the WtW-funded programs, as external factors may have contributed to the changes.