Prepared for:
U.S. Department of Health and Human Services
Administration for Children and Families
Office of the Assistant Secretary for Planning and Evaluation
U.S. Department of Education
Office of the Under Secretary
Office of Vocational and Adult Education
Prepared by:
Charles Michalopoulos
Christine Schwartz
with
Diana Adams-Ciardullo
Manpower Demonstration Research Corporation
| This document was prepared as part of the National Evaluation of Welfare-to-Work Strategies (NEWWS). The Manpower Demonstration Research Corporation (MDRC) is conducting the NEWWS Evaluation under a contract with the U.S. Department of Health and Human Services (HHS), funded by HHS under a competitive award, Contract No. HHS-100-89-0030. Child Trends, as a subcontractor, is conducting the analyses of outcomes for young children (the Child Outcomes Study). HHS is also receiving funding for the evaluation from the U.S. Department of Education. The study of one of the sites in the evaluation, Riverside County (California), is also conducted under a contract from the California Department of Social Services (CDSS). CDSS, in turn, is receiving funding from the California State Job Training Coordinating Council, the California Department of Education, HHS, and the Ford Foundation. Additional funding to support the Child Outcomes Study portion of the evaluation is provided by the following foundations: the Foundation for Child Development, the William T. Grant Foundation, and an anonymous funder. The findings and conclusions presented herein do not necessarily represent the official positions or policies of the funders. |
Contents
- Findings in Brief
- Research Questions
- Pooled Results Across Subgroups
- Impacts for the More Disadvantaged Subgroups
- Outcomes for the More Disadvantaged Subgroups
- Evidence on Which Approaches Work Best
- Policy Implications
Executive Summary
In 1996, Congress radically transformed the nation's cash assistance welfare program when it passed the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA). The legislation replaced the 60-year-old Aid to Families with Dependent Children (AFDC) entitlement program with Temporary Assistance for Needy Families (TANF), a funding mechanism that provides states with block grants and considerable flexibility in designing their welfare programs. In addition to making other changes, many states responded by expanding their employment and training programs or changing the focus of their existing programs. A number of states replaced voluntary welfare-to-work programs that emphasized education and training with mandatory programs that stressed quick employment. While many aspects of the 1996 legislation and the state policies that followed were untested, the use of mandatory welfare-to-work programs was not. During the ten years prior to PRWORA, large-scale rigorous studies of welfare-to-work programs were launched in many states and counties. This report investigates results from 20 of these programs to determine who has benefited from welfare-to-work programs (and who has not) and whether some practices appear more effective than others at increasing the employment and earnings of single-parent welfare recipients.
The programs studied in this report share two key characteristics. They all required some portion of the welfare caseload to participate in a welfare-to-work program or risk losing some or all of their welfare benefits through sanctions. And they were all studied by the Manpower Demonstration Research Corporation (MDRC) using a rigorous experimental research design in which individuals were assigned at random either to a program group, which was required to participate in an employment or training program, or to a control group, which did not have access to the program.
In other ways, the 20 programs are quite diverse (see Table 1 for a summary of the programs). They operated in many states and counties across the country, with programs in Atlanta, Georgia; Columbus, Ohio; Detroit, Michigan; Grand Rapids, Michigan; Oklahoma City, Oklahoma; Escambia County (Pensacola), Florida; Portland, Oregon; six counties in California (Riverside, Los Angeles, San Diego, Alameda, Butte, and Tulare); and seven counties in Minnesota. While all began operating prior to the passage of PRWORA, the earliest began in 1985 and the latest are still in operation. The programs also vary in origin; most were part of state welfare-to-work programs funded under the Job Opportunity and Basic Skills Training (JOBS) program of the Family Support Act of 1988; however, one was a federal demonstration to test how high participation could be among individuals who were supposed to enroll in the program, and two were begun under waivers of the AFDC program when it was still in place. Finally, the programs vary in their approach to helping welfare recipients find work; five programs encouraged or required nearly all individuals to look for work, seven focused on basic education for most participants, and eight used a mix of the two approaches, encouraging or requiring more job-ready participants to look for work but allowing others to build skills through basic education. Although welfare-to-work programs have changed in response to welfare reform, these programs are relevant to the current policy debate; many of the 20 programs are still being operated, two contain other features of states' TANF programs such as financial incentives and time limits, and most enforced the mandate to participate in their programs by using tough sanctions (although most sanction policies were not as tough as those used by many states today).
| San Diego's Saturation Work Initiative Model (SWIM) operated between July 1985 and September 1987. SWIM provided a fixed sequence of services: job-search workshop, unpaid work experience, and education and training for those still jobless. |
| Alameda, Butte, Los Angeles, Riverside, San Diego, and Tulare County, California, ran versions of the Greater Avenues for Independence (GAIN) program beginning in the mid 1980s. Operated statewide, GAIN directed individuals considered "in need of basic education" to basic education, but required others to enroll in a job search activity. |
| Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California, operated two welfare-to-work programs each as part of the JOBS program authorized by the Family Support Act of 1988. In each site, some individuals were assigned to a Labor Force Attachment (LFA) program that required most participants to initially look for work; other individuals were assigned to a Human Capital Development (HCD) program that placed most participants in basic education. |
| Columbus, Ohio, tested two approaches to case management as part of the state's JOBS program. In the Columbus Traditional program, two different workers handled income maintenance and employment and training case management. In the Columbus Integrated program, one staff member handled both. Both programs were education-focused, placing most participants into basic education and some into post-secondary programs. |
| Detroit, Michigan, and Oklahoma City, Oklahoma, ran education-focused JOBS programs that assigned most individuals to basic education. These are the only programs studied in this report in which the mandate to participate was not strongly enforced. |
| Portland, Oregon's JOBS program was employment-focused; staff told clients that their goal should be to get a job. Participants were told to wait until they found a "good" job and those in need of more skills were encouraged to enroll in short-term education or training initially and look for a job later. |
| The Minnesota Family Investment Program (MFIP) was begun in 1994 in seven rural and urban counties in Minnesota. The MFIP policy combined a mandatory employment and training program for long-term welfare recipients with financial incentives to encourage them to work. MFIP's welfare-to-work program was an employment-focused program that encouraged participants to take a job quickly. |
| Florida's Family Transition Program (FTP) was implemented in 1994 in Escambia County, Florida. Participants who were considered not job-ready were allowed to participate in education and skills development; others were required to look for work. In addition to the welfare-to-work program, FTP offered financial incentives to work and imposed a time limit on receipt of welfare benefits. As a result, FTP has the key components of many states' TANF policies. |
The results analyzed in this report may be particularly important at this time. In addition to giving states flexibility in designing their welfare programs, PRWORA also required a growing percentage of the welfare caseload to be working or participating in work-related activities and it imposed a five-year time limit on how long most families could receive federal support. States may be better able to meet their obligation and help welfare recipients become self-sufficient before they reach the time limit if they understand what has worked in the past and if they know which groups may require more or different types of help because they have not benefited from previous efforts.
I. The Findings in Brief
As mentioned above, people in each site were assigned at random to either a program group or a control group. Since random assignment ensured that the groups were similar at the time of random assignment, any differences that emerged between them could reliably be attributed to the mandatory welfare-to-work programs. Comparing outcomes for the program and control groups therefore reveals the effects of the program. The key findings follow.
- For most subgroups, people in the program groups had higher earnings and lower welfare payments than people in the control groups, but generally had the same combined income from earnings, AFDC, and Food Stamps. When samples from the 20 programs were combined, effects on annual earnings were similar for most subgroups; they exceeded $1,000 per year for only one group and were close to zero for only one group. The programs also reduced annual AFDC payments by similar amounts for all groups, with the effects ranging between $200 and $600. As a result of increased earnings and reduced welfare payments, the programs generally neither increased nor decreased combined income from earnings, welfare, and Food Stamps.
- Measures of psychosocial well-being and barriers to work were typically not strongly related to impacts on earnings. Private Opinion Survey data were used to define subgroups based on risk of depression, mastery, work-related parental concerns, preference for work, health or emotional problems, child care problem, and transportation problems, all measured at the time of random assignment. In general, there was little relationship between these measures and impacts. The one exception was risk of depression. The programs did not affect earnings for people at high risk of depression when they entered the study, and the programs had smaller effects for those at high risk than for those at low risk.
- The programs increased earnings about as much for the more disadvantaged groups as for the less disadvantaged groups. Nevertheless, the more disadvantaged groups earned much less than others. The programs increased earnings for long-term recipients, high school nongraduates, families with three children or more, and people with no recent work experience. In particular, the programs increased earnings for the most disadvantaged group: long-term recipients who did not have a high school diploma and had not worked in the year prior to random assignment. Although the programs increased earnings across the board, they typically increased earnings no more for the more disadvantaged groups than for the less disadvantaged groups. As a result, earnings for the more disadvantaged groups remained far below earnings for other groups even after participating in these programs.
- Employment-focused programs tended to be more effective than education-focused programs for the more disadvantaged groups. Programs that provided a mix of first activities tended to help the broadest range of people. For the more disadvantaged groups, most of the programs with the largest effects on earnings were employment-focused. Programs with an education focus rarely had large effects for these groups. In a rigorous comparison of employment-focused and education-focused programs that magnified the differences between these two types of models, programs that required nearly all participants initially to look for work had larger effects on earnings for the more disadvantaged groups than programs that enrolled most people initially into basic education. However, the two program models had similar effects for the less disadvantaged groups. A number of programs that provided a mix of first activities (some of which were employment-focused) produced large earnings gains for the more disadvantaged groups and the less disadvantaged groups. Thus, programs with a mix of first activities were effective for the broadest range of individuals.
II. Research Questions
This report tries to answer the question of "what works best for whom" in mandatory welfare-to-work programs for single-parent welfare recipients. Implicit in this question are three broad research issues.
- Which groups were affected the most and the least?
To answer the "for whom" part of the question, the report examines subgroups of single-parent families based on a number of characteristics, including educational attainment; work and welfare history; race, ethnicity, and sex; number and age of children; barriers to work because of child care, transportation, and health or emotional problems; preference for work over welfare; parental concerns about leaving family for work; and depression and feeling of mastery over life circumstances. To investigate results for a group of individuals expected to be especially hard to help, a most disadvantaged subgroup was defined to include long-term recipients (those who had ever been on welfare two years or more prior to random assignment) who had not graduated from high school and who had no earnings in the year prior to random assignment. Likewise, a least disadvantaged group was defined as individuals with none of these barriers, while individuals were considered moderately disadvantaged if they had one or two barriers. To search for an even more disadvantaged group, the most disadvantaged group was further divided by some of the psychosocial measures and barriers to work, such as risk of depression, mastery, and child care problems.
Understanding what happened to various groups will require looking at both outcomes how much groups earned on average or what their average income was, for example and impacts how much average earnings or other outcomes increased or decreased because of the programs. Some groups with low earnings may not have benefited from the programs studied in this report. Likewise, some groups may be benefiting from welfare-to-work programs, but still be left without enough earnings to move completely off welfare. For those groups, policymakers may need to use new strategies such as offering post-employment services or help in overcoming substance abuse or domestic violence.
- In what dimensions are the programs succeeding?
In studying outcomes and impacts, the report investigates three dimensions: earnings, welfare benefits, and income. Policymakers may want to encourage welfare recipients to work; for them, the "best" program may be the one that increases employment and earnings the most. Other policymakers may be primarily interested in reducing spending on welfare; for them, the best program may be the one that reduces cash assistance the most. Welfare recipients and policymakers concerned about child and family poverty may care most about total income; for them, the best program may be the one that increases income the most.
- Which programs or program models work best?
These programs vary in a number of ways, including how they helped clients make the transition from welfare to work, who was enrolled in the programs, how the programs were implemented, where the programs were implemented, and the economic conditions under which they were implemented. If programs with one set of characteristics consistently outperformed others for some subgroups, policymakers might want to repeat those programs for some welfare recipients.
III. Pooled Results Across Subgroups
Published results show that most of these programs increased earnings and reduced welfare receipt overall, but led to no change in combined income from earnings, welfare, and Food Stamps. This study produced similar results for a wide range of subgroups. Overall, the programs increased earnings and reduced welfare payments for most subgroups, an encouraging finding that suggests that few groups were left behind. Table 2 summarizes these impacts for a variety of subgroups when samples from the 20 programs are combined.
| Subgroup, at baseline | Sample Size | Average Total Earnings per Year, Years 1-3 ($) | Average Total AFDC Payments per Year, Years 1-3 ($) | Average Total Income per Year, Years 1-3 ($) | |||
|---|---|---|---|---|---|---|---|
| Control Group | Impact | Control Group | Impact | Control Group | Impact | ||
| Total earnings in past 12 months | |||||||
| No earnings | 41,434 | 1,754 | 571*** | 4,675 | -416*** | 8,082 | 41 |
| $5,000 or less | 20,554 | 3,425 | 399*** | 3,696 | -359*** | 8,707 | -58 |
| More than $5,000 | 9,944 | 6,957 | 548*** | 2,967 | -305*** | 11,200 | 143 |
| Welfare History(a) | # | # | ## | ||||
| Long-term recipient | 43,339 | 2,480 | 544*** | 4,791 | -433*** | 9,027 | 4 |
| Short-term recipient | 21,333 | 3,708 | 534*** | 3,400 | -337*** | 8,463 | 94 |
| New applicant | 6,853 | 3,025 | 1,106*** | 2,611 | -218** | 6,819 | 773*** |
| Education Credential receipt | ## | ## | |||||
| No high school diploma/GED | 31,139 | 1,867 | 430*** | 4,708 | -395*** | 8,282 | -66 |
| High school diploma/GED | 40,793 | 3,751 | 627*** | 3,749 | -389*** | 8,989 | 123* |
| Number of children | ### | # | |||||
| Three or more | 18,179 | 2,523 | 682*** | 5,604 | -458*** | 10,412 | 93 |
| Two | 22,950 | 2,957 | 663*** | 4,185 | -408*** | 8,769 | 128 |
| One | 30,562 | 3,196 | 328*** | 3,268 | -326*** | 7,589 | -65 |
| Level of Disadvantage(b) | ## | ||||||
| Most Disadvantaged | 14,393 | 983 | 404*** | 5,570 | -411*** | 8,426 | -116 |
| Moderately Disadvantaged | 47,113 | 2,955 | 599*** | 4,066 | -414*** | 8,591 | 79 |
| Least Disadvantaged | 10,019 | 5,664 | 421*** | 2,677 | -282*** | 9,558 | 41 |
| Risk of depression(c) | # | # | |||||
| High | 2,507 | 3,071 | 289 | 3,308 | -392*** | 8,281 | -236 |
| Moderate | 4,157 | 3,138 | 460*** | 3,381 | -442*** | 8,510 | -121 |
| Low | 10,588 | 3,049 | 769*** | 3,496 | -596*** | 8,529 | -22 |
| Work-Related Parental Concerns scale | # | ||||||
| High | 4,786 | 1,973 | 748*** | 4,012 | -551*** | 8,067 | -76 |
| Low | 15,796 | 3,524 | 588*** | 3,099 | -407*** | 8,424 | 51 |
| Mastery Scale | |||||||
| Low | 7,680 | 2,554 | 623*** | 3,480 | -378*** | 8,016 | 100 |
| High | 12,911 | 3,503 | 672*** | 3,196 | -465*** | 8,495 | 33 |
| Barriers to work or participation | |||||||
| Health or emotional problem(d) | |||||||
| Yes | 5,507 | 2,097 | 552*** | 3,518 | -355*** | 7,619 | 28 |
| No | 15,181 | 3,525 | 663*** | 3,220 | -468*** | 8,563 | 36 |
| Transportation problem | |||||||
| Yes | 7,212 | 2,026 | 725*** | 3,736 | -457*** | 7,753 | 110 |
| No | 13,252 | 3,742 | 616*** | 3,078 | -430*** | 8,616 | 20 |
| Cannot afford/arrange for child care | |||||||
| Yes | 12,478 | 2,827 | 648*** | 3,591 | -452*** | 8,397 | 9 |
| No | 7,832 | 3,666 | 651*** | 2,863 | -439*** | 8,222 | 71 |
| SOURCES: MDRC calculations from unemployment insurance (UI) records, AFDC records, Food Stamp, Private Opinion Survey (POS) data and Baseline Information Forms. NOTES: A two-tailed t-test was applied to differences between outcomes for the program and control groups. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. An F-test was applied to differences among subgroups for each characteristic. Statistical significance levels are indicated as: # = 10 percent; ## = 5 percent; and ### = 1 percent. FOOTNOTES: b. Individuals were classified as most disadvantaged if they had no earnings in the year prior to random assignment, did not have a high school diploma or GED at random assisgnment, and had received welfare two years or more years prior to random assignment and as least disadvantaged if they had none of these characteristics. All other sample members were classified as moderately disadvantaged. c. Risk of depression subgroups include only sample members from NEWWS sites in Atlanta, Grand Rapids, Riverside, and Portland. d. Sample members in the "yes" category on this measure could have had a health or emotional problem themselves which they reported as a barrier to work or participation at random assignment or one of their family members couild have had such a problem. | |||||||
- If the objective of welfare-to-work programs is to increase earnings, this set of programs worked well for almost every group.
The primary purpose of welfare-to-work programs is to help recipients go to work and increase their earnings. Overall, the 20 programs studied in this report succeeded in this regard. On average, they increased annual earnings by about $500 per person; that is, program group members earned about $500 more per year on average than control group members.(1) Moreover, the programs increased earnings by a similar amount across a wide range of subgroups (see Table 2). Only for new applicants did the effect on earnings exceed $1,000 and only for the group at high risk of depression did the programs not significantly increase earnings. (See the accompanying box for a discussion of statistical significance.)
Defining Statistical SignificanceStatistical significance is used to determine whether estimated differences between two groups are real or due to chance. Usually, statistical significance is defined at a certain level. Thus, if a diference is statistically significant at the 5 percent level, the implication is that there is only a 5 percent chance that the difference is due to chance. In this report (which follows generally accepted practices), the minimum acceptable level of statistical significance is 10 percent. Any difference with a significance level less than or equal to 10 percent is described as being statistically significant (or not likely to be due to chance). Any difference with a significance level greater than 10 percent is described as not statistically significant (or possibly due to chance). |
- Measures of psychosocial well-being and barriers to work were typically not strongly related to impacts on earnings.
Private Opinion Survey (POS) data from some of the programs were used to define subgroups based on risk of depression, mastery, work-related parental concerns, preference for work; and health or emotional, child care, and transportation barriers to work, all measured at the time of random assignment. In general, there was little relationship between these measures and impacts (see Table 2). The one exception was risk of depression. The programs did not affect earnings for people at high risk of depression when they entered the study, and had significantly smaller effects for those at high risk than for those at low risk. These results are consistent in some ways with the programs that were studied. While most provided assistance with child care and transportation, few explicitly tried to address psychological problems.
- If the objective of welfare-to-work programs is to reduce welfare payments, this set of programs succeeded for most subgroups.
A second objective of welfare-to-work programs is to reduce the use and cost of welfare programs. This may occur directly through sanctioning or by creating a burden that makes people want to leave welfare. However, the primary mechanism for reducing welfare payments is the work that results from the programs' services. In all programs studied in this report, an individual's welfare benefit was reduced by some amount if she earned above a threshold known as the earnings disregard. Since the programs significantly increased earnings, they should also have reduced welfare benefit amounts, and they did. On average, they reduced annual welfare payments by nearly $400 and reduced Food Stamp payments by another $100 (not shown in Table 2).
Just as the programs increased earnings by about the same amount for a broad range of subgroups, they tended to reduce welfare payments by similar amounts for most subgroups (see Table 2). In fact, impacts on welfare payments were, if anything, more similar across subgroups than were impacts on earnings. For no subgroup did the annual impact on welfare payments fall below $200 or rise above $600.
- If the objective of welfare-to-work programs is to increase income from earnings and public assistance, welfare-to-work programs succeeded for few groups, but were more likely to have increased income for the less disadvantaged groups.
As described above, the programs' effects on earnings were about the same as their effects on welfare plus Food Stamps. As a result, the programs did not significantly increase combined income from earnings, welfare, and Food Stamps. A few subgroups were exceptions to this result, although all of the exceptions occurred for the less disadvantaged subgroups (see Table 2). The programs increased annual income by nearly $800 for new applicants but barely changed income for long-term recipients, and they increased income by more than $100 for high school graduates but did not significantly change income for nongraduates.
Although the programs did not increase income for most subgroups, they also did not decrease income for most subgroups. This might be viewed as a positive result for two reasons. First, the programs might have reduced income because individuals were either sanctioned or lost their job and decided not to reapply for welfare benefits. Although this probably happened for some individuals, there is no evidence that it occurred so frequently that the average income of entire groups was reduced. Second, the income amounts shown in Table 2 reflect only welfare, Food Stamps, and earnings. In particular, they exclude income from the federal Earned Income Credit (EIC), a source of considerable income for working poor families, and the programs' impacts on income would have been bigger if the EIC had been included.(2) At the same time, the calculation of income also ignores a number of work-related expenses, such as payroll and income taxes, child care costs, and transportation costs.
IV. Impacts for the More Disadvantaged Subgroups
All but one of the programs being studied met the provisions of the JOBS program, which were designed to benefit those most likely to be long-term recipients. An important question, therefore, is whether the programs succeeded for their targeted groups. The broad answer is that they did. As discussed above, the programs increased earnings for most groups, including the more disadvantaged groups. However, several important results warrant further discussion.
- The programs increased earnings about as much for the most disadvantaged groups as for the moderately and least disadvantaged groups.
As discussed above, impacts on earnings were spread fairly evenly across subgroups. Earnings gains due to the programs were as large for long-term recipients as for short-term recipients; almost as large for high school graduates as for nongraduates; slightly larger for families with three children or more than for families with one child; and larger for people with no recent work experience than for those with some recent work experience. An especially encouraging finding is that impacts on earnings for the group classified as the most disadvantaged were about as large as those for the least disadvantaged group and almost as large as those for the moderately disadvantaged group.
- The programs reduced welfare payments more for the more disadvantaged groups than for the less disadvantaged groups.
As discussed above, reductions in welfare payments were fairly similar across subgroups. However, there is a hint that reductions were slightly greater for the more disadvantaged groups. For example, welfare payments were reduced by twice as much for long-term recipients as for new welfare applicants even though the programs' impact on earnings was twice as large for new applicants as for long-term recipients. Likewise, welfare reductions were nearly identical for high school graduates and nongraduates, even though high school graduates had significantly larger earnings impacts. Welfare reductions were also almost twice as much for the most disadvantaged sample members as for the least disadvantaged sample members; however, earnings impacts were also higher for the most disadvantaged group.
- The programs did not increase earnings for sample members at high risk of depression but increased earnings substantially for those at low risk.
Welfare-to-work programs have been designed to help people with few job skills and little work experience. However, a disproportionate number of welfare recipients also exhibit symptoms of depression, and depression may keep them from taking advantage of welfare-to-work programs and from working. As indicated above, this report finds reason to be concerned. Overall, the programs did not increase the earnings of sample members at high risk of depression, but increased the earnings of those at low risk by a substantial amount. At the same time, the programs decreased welfare payments to those at high and at low risk by a similar amount. Regardless of risk of depression, however, the programs neither significantly increased or decreased combined income from earnings, AFDC, and Food Stamps.
- The effects of the programs depended on the kind of disadvantage an individual suffered from.
In an analysis not shown in Table 2, individuals who were receiving welfare at the time of random assignment were divided into eight groups according to whether they were long-term recipients, whether they had graduated from high school, and whether they had recent work experience. Earnings impacts were larger for more disadvantaged groups if the disadvantages included lack of prior work experience, but smaller if the disadvantages included lack of a high school diploma. They were about the same for long-term recipients as for others. This analysis suggests impacts are related not to the number but to the kind of disadvantage.
- Measures of psychosocial well-being did not help define a new group of the hard to serve who were not being helped by the programs.
As welfare rolls decline, states are being left with a caseload that is harder to serve than the individuals who were randomly assigned in these programs. To try to define an extremely disadvantaged group, the most disadvantaged group shown in Table 2 was further divided according to the psychosocial measures described above (risk of depression, mastery, and so on.). In general, the psychosocial measures did not help define a new group of the extremely disadvantaged who were not benefiting from the programs. For example, the programs significantly increased earnings for members of the most disadvantaged group who were also at high risk of depression. Moreover, this impact on earnings was about as large for the most disadvantaged sample members at low risk of depression. (Although the programs did not significantly increase earnings for the group at high risk of depression overall, this was due to low earnings impact for the least disadvantaged sample members at high risk of depression.)
V. Outcomes for the More Disadvantaged Subgroups
One objective of welfare-to-work programs is to increase the earnings of welfare recipients. A related objective is to help welfare recipients earn enough to end their reliance on public assistance. This is an especially important goal under time-limited welfare. Even if welfare-to-work programs increase earnings levels, those levels might remain too low to eliminate a family's need for welfare. For families who eventually reach the time limit and lose their welfare benefits, their income might then be insufficient to meet even basic needs such as food and housing.
- Despite positive effects on earnings for the more disadvantaged welfare recipients, absolute levels of earnings remained particularly low for these groups.
During the three-year follow-up period studied in this report, the more disadvantaged members of the control group earned substantially less on average than others (see Table 2). Individuals with no earnings in the year prior to random assignment earned only one-fourth as much as those with $5,000 or more in prior-year earnings.(3) The same was true for other subgroups. Sample members who had not graduated from high school earned only half as much as those who had graduated. Long-term recipients also earned substantially less than short-term recipients. The most troublesome outcome, however, is the average earnings level for the most disadvantaged group (long-term recipients who have not graduated from high school and who have no recent work experience). For control group members in this subgroup, average annual earnings over the three-year follow-up period were less than $1,000 compared with almost $6,000 for the least disadvantaged group. Although the welfare-to-work programs increased earnings across the board, they typically increased earnings no more for the more disadvantaged groups than for the less disadvantaged groups. As a result, earnings for the more disadvantaged groups were as far below earnings for other groups after participating in these programs as they were before, and new policies may be needed to raise their earnings.
- The sample members at high risk of depression were financially as well off as those at low risk.
As described above, individuals at high risk of depression were one of the few subgroups that did not have significant earnings impacts from these mandatory welfare-to-work programs. In terms of economic well-being, however, depression might not be as important as work experience, education, and welfare history. Although the programs did not increase earnings for those at high risk of depression, Table 2 shows that the average annual earnings and income were similar for control group members at high and at low risk. In contrast, earnings for high school nongraduates fell far below earnings for graduates, and earnings for people with no recent work experience were much lower than earnings for people with substantial recent work experience.
VI. Evidence on Which Approaches Work Best
The previous sections argued that the welfare-to-work programs as a group increased earnings for the more disadvantaged and the less disadvantaged groups by similar amounts. Although the pooled results show few differences across subgroups, it is possible that some program models performed better than others for some subgroups. The four categories shown in Table 3 provide one means of classifying the program models. Although program model is an important dimension on which to compare the programs, it is important to remember that the programs differed in a number of other dimensions, including who was enrolled, when and where programs took place, and the economic conditions at the time they took place.
| Education-Focused | Mix of First Activities Without Employment Focus | Employment-Focused With Mix of First Activity | Employment-Focused With Job Search as First Activity |
|---|---|---|---|
| Atlanta HCD Grand Rapids HCD Riverside HCD Columbus Integrated Columbus Traditional Detroit Oklahoma City | Alameda GAIN Butte GAIN Los Angeles GAIN San Diego GAIN Tulare GAIN | Riverside GAIN Portland Florida FTP Minnesota MFIP | Atlanta LFA Grand Rapids LFA Riverside LFA San Diego SWIM
|
The largest of the four categories shown in Table 3 contains the education-focused programs which sought to place most participants initially in basic education (the three HCD programs, the two Columbus programs, Detroit, and Oklahoma City). At the other extreme are the four employment-focused programs with job search as the first activity for most participants (the three LFA programs and SWIM). Four other programs (Riverside GAIN, Portland, FTP, and MFIP) were also employment-focused, but they used a mix of first activities by enrolling more job-ready individuals in job search and allowing or directing others to enroll in basic education. Finally, the remaining five GAIN sites used a mix of activities without an employment focus. Even though the six GAIN sites followed the same policy, Riverside differed from the other five in that nearly all staff emphasized quick employment to participants; in the other five sites, most staff did not.
- Employment-focused programs tended to be more effective than education-focused programs for the more disadvantaged groups. Portland and Riverside GAIN, two of the employment-focused programs that allowed some individuals to build skills through basic education, were especially effective.
Over the three-year follow-up period, employment-focused programs produced four of the five largest earnings impacts for individuals with no earnings in the year prior to random assignment, for long-term welfare recipients, and for the most disadvantaged group and three of the five largest earnings impacts for high school nongraduates (see Table 4). Programs with an education focus are listed only once. Even in the third year of follow-up (not shown), after individuals initially enrolled in basic education had time to gain some skills and then find work, most of the programs with the largest effects on earnings were employment-focused, and education-focused programs barely made the list of the most effective programs for the more disadvantaged groups. Two programs in particular stand out from the rest. Riverside GAIN produced the second or third largest average earnings impact for each group of the more disadvantaged people shown in the upper part of Table 4. Portland's JOBS program likewise produced some of the largest impacts for each group. Both programs were employment-focused, but both also used a mix of job search and basic education as first activities.
| More Disadvantaged Groups | ||||
|---|---|---|---|---|
| No Earnings in Year Prior to Random Assignment | Without High School Diploma or GED | Long-Term Welfare Recipient | Most Disadvantaged | |
| Largest Impact | Portland ($1,476) | Butte GAIN ($1,257) | Butte GAIN ($1,445) | Minnesota MFIP ($1,115) |
| 2nd largest impact | Riverside GAIN ($1,262) | Riverside GAIN ($1,029) | Riverside GAIN ($1,296) | Grand Rapids ($1,035) |
| 3rd largest impact | Minnesota MFIP ($1,074) | Grand Rapids LFA ($838) | Portland ($1,222) | Riverside GAIN ($1,026) |
| 4th largest impact | Riverside LFA ($782) | Columbus Integrated ($808) | Riverside ($742) | Portland ($701) |
| 5th largest impact | Alameda GAIN ($659) | Portland ($767) | Atlanta ($586) | Riverside LFA ($668) |
| Less Disadvantaged Groups | ||||
| Earnings > $5,000 in Year Prior to Random Assignment | With High School Diploma or GED | Short-Term Welfare Recipients | Least Disadvantaged | |
| Largest Impact | Butte GAIN ($3,670) | Riverside GAIN ($1,780) | Riverside GAIN ($1,409) | Riverside GAIN ($1,976) |
| 2nd largest impact | Riverside GAIN ($1,917) | Alameda GAIN ($1,203) | San Diego GAIN ($1,022) | Butte GAIN ($1,593) |
| 3rd largest impact | San Diego GAIN ($1,471) | Portland ($1,202) | Portland ($1,012) | San Diego GAIN ($1,549) |
| 4th largest impact | Detroit ($1,260) | San Diego GAIN ($1,030) | Butte GAIN ($885) | SWIM ($1,504) |
| 5th largest impact | Grand Rapids HCD ($970) | Florida FTP ($833) | Florida FTP ($840) | Florida FTP ($698) |
| SOURCE: MDRC calculations from unemployment insurance (UI) earnings records and Baseline Information Forms. | ||||
- Programs with a mix of activities tended to help the widest range of individuals.
Programs with a mix of activities dominate the list of the most effective programs for the less disadvantaged participants (the lower part of Table 4) GAIN programs were especially effective for the less advantaged participants, but FTP and Portland's JOBS program were also effective for some of these groups. Programs with a mix of first activities were also frequently effective for the more disadvantaged participants. This is largely because Riverside GAIN and Portland were so successful two programs that were also employment-focused but MFIP and the GAIN program in Butte also produced large earnings impacts for these groups (as did FTP and the GAIN program in San Diego in the third year of follow-up; not shown in Table 4). Thus, programs with a mix of first activities were effective for the broadest mix of individuals.(4)
It is interesting that programs with a mix of first activities did better than education-focused programs for the more disadvantaged groups even though both emphasized basic education for the more disadvantaged. Likewise, it is interesting that they did better than job search programs for the less disadvantaged groups even though both emphasized job search for job-ready participants. The broad success of the mixed programs may indicate that determining whether individuals need basic education is more difficult than determining whether they have graduated from high school or worked recently. In fact, the programs with a mix of first activities used other criteria, such as scores on tests of basic skills and English proficiency. Thus, programs with a mix of first activities may have been more effective at increasing earnings because they effectively determined who would benefit from job search and who would benefit from basic education.
- Programs that required most individuals to immediately look for work increased earnings faster than programs that directed most toward basic education, but those differences dissipated over time. Nevertheless, for the more disadvantaged groups, programs that emphasized job search increased earnings overall more than programs that emphasized basic education.
Post-AFDC welfare-to-work programs have primarily used a "work-first" approach that encourages recipients to look for work immediately. However, many welfare recipients and advocates for welfare recipients decry the lack of opportunities to augment skills through education. Atlanta, Grand Rapids, and Riverside provide the best comparison of the two approaches. In each site, two programs operated side by side. While one program emphasized quick job entry (labor force attachment, or LFA) by requiring most participants to initially look for work, the other emphasized basic education (human capital development, or HCD) and enrolled most individuals initially in basic education. People were randomly assigned to one of the two programs, so that any differences in impacts of the programs were due to differences in the programs themselves, particularly the different emphases.
For several subgroups that were examined, the LFA programs initially produced larger earnings impacts than the HCD programs (see Table 5), but differences in earnings impacts were no longer statistically significant for any of the subgroups by the third year of the follow-up period. Over the three-year period, however, the LFA programs produced significantly higher earnings impacts than the HCD programs for four groups of the more disadvantaged recipients: those without a high school diploma or GED, those at high risk of depression, those with no earnings in the year prior to random assignment, and those considered the most disadvantaged. In comparison, the LFA and HCD programs produced essentially the same earnings impacts over the three-year period for the less disadvantaged counterparts of these groups. Five years of follow-up information will eventually be available for people in all of these programs, and it will be interesting to see how the two approaches compare over a longer period.
| Program and Subgroup | Year 1 | Year 3 | Years 1-3 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| LFA | HCD | Difference | LFA | HCD | Difference | LFA | HCD | Difference | |
| By high school credential | |||||||||
| No high school diploma or GED | 658*** | 160* | 498### | 625*** | 504*** | 121 | 636*** | 319*** | 317### |
| High school diploma or GED | 415*** | 183 | 232# | 314* | 431** | -116 | 366** | 373** | -7 |
| By risk of depression | |||||||||
| High risk | 675** | -193 | 869### | 175 | -369 | 544 | 417 | -201 | 618## |
| Moderate risk | 476** | 110 | 366# | 553* | 667** | -114 | 462** | 363* | 99 |
| Low risk | 547*** | 271** | 276## | 499*** | 730*** | -231 | 540*** | 536*** | 4 |
| By prior earnings | |||||||||
| No earnings | 522*** | 158** | 364### | 566*** | 522*** | 44 | 535*** | 377*** | 157# |
| Earned less than $5,000 | 540*** | 36 | 504### | 445** | 264 | 182 | 468*** | 166 | 302# |
| Earned $5,000 or more | 525 | 272 | 253 | -135 | 341 | -477 | 284 | 289 | -6 |
| By level of disadvantage | |||||||||
| Most disadvantaged | 500*** | 120 | 380### | 690*** | 493*** | 198 | 602*** | 316*** | 286## |
| Moderately disadvantaged | 624*** | 161 | 463### | 449*** | 478*** | -29 | 540*** | 374*** | 167 |
| Least disadvantaged | n/a | n/a | n/a | n/a | n/a | n/a | |||
| SOURCES: MDRC calculations from unemployment insurance (UI) earnings records. NOTES: Two-tailed t-test was applied to differences between outcomes for the program and control groups. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Two-tailed t-test was applied to differences between outcomes for the two programs. Statistical significance levels are indicated as: # = 10 percent; ## = 5 percent; and ### = 1 percent. N/a = not applicable. | |||||||||
VII. Policy Implications
For a policymaker or program administrator, the results in this report yield several important implications.
- It is possible to help the most disadvantaged participants if resources are targeted toward them and programs are developed to meet their needs.
The Family Support Act of 1988 required states to target welfare-to-work programs toward welfare recipients who were the most likely to have a very long stay on welfare and the least likely to work. States were also required to offer a mix of services that were thought most likely to benefit this hard-to-serve group and to subsidize child care, transportation, and work-related expenses for participants in their welfare-to-work programs. Most of the programs studied in this report were either operated under the Family Support Act or anticipated the key requirements of the act. As described above, the programs did increase earnings for the more disadvantaged groups.
In studying a group of mandatory but lower-cost welfare-to-work programs from the early 1980s, Daniel Friedlander (Subgroup Impacts and Performance Indicators for Selected Welfare-to-Work Programs. New York: MDRC, 1988) found, in contrast, that earnings impacts were small for the more disadvantaged. Since the programs studied by Friedlander preceded the FSA in both time and character, the comparison suggests that the approach of the FSA was more successful in increasing earnings of the more disadvantaged. More broadly, it suggests that it is possible to help the more disadvantaged participants.
- A mix of job search and education increases earnings the most for the broadest range of individuals.
Most of the programs with the largest effects on earnings used a mix of job search and basic education as first activities. People who appeared to be ready to work were required to look for work, but participants who lacked basic skills were allowed to enroll in basic education. For the more disadvantaged groups, programs with a mix of first activities were especially effective if they were also employment-focused, suggesting that program administrators may want to build programs that have a mix of services. Some caution should be used in interpreting this result, however. There has been no direct, rigorous comparison of a program with a mix of first activities with a program that emphasized primarily job search or basic education. The success of the mixed programs could stem from other factors such as the state of the economy or program location (most of the programs that used a mix of first activities were in California, for example).
- Job search rather than education increases earnings quickly.
If resources limit a program to one activity for most participants, that activity should be job search if the objective is to increase employment and earnings quickly. This makes sense, since people who are in school have less time to work and earn. By the third year of follow-up, for example, the two approaches were about equally effective at increasing earnings. Over a three-year period of time, however, job search appeared to increase earnings more than basic education for the more disadvantaged participants (but not for the less disadvantaged participants).
- Psychological problems may still be an impediment to the success of welfare-to-work programs.
This report investigated the impact of welfare-to-work programs by risk of depression and feelings of self-efficacy. Although individuals at high risk of depression in the control group fared as well in the labor market as those at low risk, the former group was less able to capitalize on the ability of welfare-to-work programs to increase earnings. These results suggest that welfare administrators may need to implement different or more intensive interventions for the depressed. It also suggests that further research is needed to understand whether other psychological problems limit the effectiveness of welfare-to-work programs.
Footnotes
1. All dollar amounts were inflation-adjusted to 1997 dollars.
2. This measure of income also excludes other income sources and income from other household members. In the studies in which the information has been collected through surveys, however, the impact on other income sources has generally been small.
3. Since average earnings includes zero earnings for people who are not working, some of the differences across subgroups are due to lower employment rates. For example, people with no earnings in the year prior to random assignment were only half as likely to work as those with $5,000 or more in prior-year earnings (not shown in Table 2). Even among those who worked, however, people with no earnings in the year prior to random assignment earned about half as much as those with $5,000 or more in prior-year earnings (not shown in Table 2).
4. A number of programs did not randomly assign new applicants (including Los Angeles and Tulare in GAIN, and most of the programs evaluated as part of NEWWS). In addition, this report includes only long-term welfare recipients from MFIP because others in MFIP were not immediately required to participate in employment and training services. Therefore, only 8 of the 20 programs being studied were among the most effective for new applicants.