Effects on Economic Outcomes
[ Main Page of Report |
Contents of Report ]
Endnote
This section examines the impacts on economic outcomes of the different types
of welfare-to-work strategies used by the NEWWS programs. Specifically, it
looks at the extent to which the programs improved on what would have happened
in their absence; the relative effectiveness of employment-focused and
education-focused programs and their variants; and the relative effectiveness
of LFA and HCD programs.
As noted earlier, one of the key goals of education-focused programs is to
increase income by improving welfare recipients' credentials or skills before
they seek jobs. Short-run impacts are not expected in these programs because
the programs essentially delay people's entry into the job market; rather,
long-run impacts are the goal, with the hope that higher long-run earnings
will make up for earnings foregone in the short run. One of the key goals
of employment-focused programs, in contrast, is to reduce reliance on welfare
as soon as possible. Short-run impacts are expected in these programs, with
the hope that they will be sustained and even grow over time.
[Go To Content]
-
All 11 NEWWS programs increased single parents' employment and earnings
and decreased their welfare receipt and payments relative to the levels found
in the programs' absence.
As noted earlier, over the five-year follow-up period, approximately
three-quarters of control group members in NEWWS found jobs, and more than
half left the welfare rolls. Nearly all the programs improved on these
statistics, causing people to work during more quarters of the follow-up
period and to earn more than they would have in the absence of a program
(for a discussion and illustration of the components that make up earnings
impacts, see Box 2). Moreover, all the programs
decreased the average number of months that people received welfare and the
average number of welfare dollars they received over the five years.
The program and control group earnings levels that form the basis of the
earnings impacts are shown in Figure 4. In
the figure, the two groups' five-year earnings are split by program type.
As can be seen, average earnings for program group members (the black bars)
in all four types of programs were higher than those for control group members
(the white bars).
Most of the programs increased earnings during the second and third years
of the follow-up period. Their effects generally diminished, however, during
the fourth and fifth years and were not statistically significant for most
programs by the very end of the fifth year. Only the Portland and Riverside
LFA programs continued to produce statistically significant earnings impacts
at the end of the fifth year. It should be noted that, in a few sites, a
small proportion of control group members (a subset of those on welfare)
in a few programs received program services toward the end of the five-year
follow-up period.(5) Extensive analyses indicated
that the effect of this on the impacts in the fourth and fifth years was
probably small. Most of the programs' effects on earnings would likely have
diminished in the later years even if no control group members had been exposed
to the programs late in the study.
Notably, only a minority of program group members experienced stable employment
over the five years. As an example, from 60 percent to 80 percent of program
group members were unemployed for at least one quarter during the fifth year,
and this situation was only slightly better than that of control group members
in the same year. In addition, even after five years, most people were earning
relatively low wages -- between $7 and $8 per hour -- and 70 percent to 85
percent earned less than $10,000 in the fifth year, outcomes that were not
much different from those for the control groups.
The average welfare payments received by the program and control groups over
the five years, which form the basis of the welfare impacts, are shown in
Figure 5. Again, the results are split by program
type. In all four types of programs, average welfare payments received by
program group members (the black bars) were lower than those for control
group members (the white bars). Most of the programs reduced five-year welfare
payments relative to control group levels by 15 percent or more. All the
programs also reduced the number of months that people received welfare,
by 2 months to 6 months over the five-year (60-month) follow-up period. The
welfare impacts were more persistent than the earnings impacts: Whereas only
a few programs continued to affect earnings in the fifth year, most of the
programs continued to generate welfare savings at the end of the same year.
Over five years, program group members in all 11 programs spent less time
on food stamps and received smaller average food stamp payments than did
control group members in the same sites. Food stamp impacts were generally
smaller than welfare payment impacts, however, because some program group
members continued to receive food stamps after they left welfare (as they
were entitled to, provided that their earnings did not exceed a certain cutoff).
Box 2
What Makes Up Impacts on Earnings?
A welfare-to-work program's impacts on earnings are likely to consist of
several components. First, a program can result in more people becoming employed
than would normally have been the case (an impact on job finding).
Second, among people who would have become employed in any case, a program
can shorten the time that passes until they find a job (an impact on time
to first job), lengthen the time they stay in a job (an impact on
employment stability), or raise wage rates (an impact on earnings
on the job).
The graph below shows the relative contributions of job finding, time to
first job, employment stability, and earnings on the job to the five-year
earnings impacts of four of the NEWWS programs (the black bars). Each impact
was made up of a different configuration of these four contributing factors.
SOURCE: Freeman,2000, upadte to reflect data collected over five
years. |
Figure 4.
Earnings over Five Years, by Program Type:
Program Group Members Earned More than Control Group Members
SOURCE: Hamilton et al., 2001
NOTES: Earnings for the program and control groups were averaged across programs
within each program type.
The Riverside LFA program results include both graudates and nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** =
5 percent; *** = 1 percent.
Figure 5.
Welfare Payments over Five years, by Program Type:
Program Group Received Less Welfare than Control Group Members
SOURCE: Hamilton et al.,,2001
NOTES: Welfare payments for the program and control groups were averaged
across programs within each program type.
The Riverside LFA program results include both graudates and nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** =
5 percent; *** = 1 percent.
-
At best, the NEWWS programs led to only small increases in the low levels
of income that would be expected in the absence of the programs, and some
of them actually decreased income.
In NEWWS, income was calculated as the sum of earnings, welfare payments,
food stamps, and the EIC minus payroll taxes. Although the programs helped
people become more self-sufficient in that a larger share of their income
came from earnings as opposed to welfare or food stamps, in dollar terms
the decreases in welfare and food stamps (and increases in payroll taxes)
largely offset the increases in earnings and the EIC. In three programs
(including Portland), the five-year income of program group members was from
3 percent to 5 percent higher than that of control group members, but these
impacts were slightly shy of being statistically significant by the standard
used in the evaluation. Four programs had negative impacts on five-year income,
decreasing it by 2 percent to 6 percent (these four impacts were all within
or just slightly outside the statistical significance range). The programs
that had positive impacts and those that had negative impacts range included
both employment- and education-focused programs. Only one of the 11 NEWWS
programs affected income in the fifth year.
Failure to increase income is not particular to the welfare-to-work programs
studied in NEWWS. Results from most programs operated in the 1980s and early
1990s were similar: Even when programs increased earnings, they seldom increased
income much. These findings underscore the limited ability of traditional
welfare-to-work programs to improve families' material well-being.
[Go To Content]
-
Employment-focused programs generally had larger effects on employment
and earnings than did education-focused programs.
Figure 6 presents all 11 NEWWS programs' impacts
on earnings, that is, the differences between the program and control groups'
earnings in each site. As shown, three of the four employment-focused programs
produced larger gains in earnings over the five years than did all seven
education-focused programs.
The LFA programs' impacts on five-year earnings ranged from about $1,500
to $2,500. Their impacts on the number of quarters people were employed ranged
from 0.7 to 1.1 (out of the 20 quarters in the study period). As is evident
in the figure, the employment-focused program in Portland produced much larger
effects, with an earnings impact of about $5,000 and an increase in quarters
employed of 1.6. Overall, the education-focused programs' effects were smaller.
Neither of the two programs with low enforcement of the participation mandate
significantly raised employment, while the other five education-focused programs
increased earnings by about $800 to about $2,000 and the number of quarters
employed by 0.3 to 0.8.
Given the large number of programs examined in NEWWS and the diversity of
the populations they served, the features of their implementation, and the
labor markets in which they operated, these results strongly indicate that
employment-focused programs are more effective than education-focused programs
at increasing employment and earnings.
Figure 6.
Earnings over Five Years, by Program:
Employment-Focused Programs Generally Increased Earnings More than
Education-Focused Programs
SOURCE: Hamilton et al.,,2001
NOTES: The Riverside LFA program results include both graudates and
nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** =
5 percent; *** = 1 percent.
-
Welfare and food stamp payment reductions were not consistently larger
in the employment-focused programs than in the education-focused programs.
Figure 7 presents all 11 NEWWS programs' impacts
on average welfare and food stamp payments. The savings were generally larger
for the programs that had larger effects on earnings, but they varied for
other reasons as well. For instance, as would be expected, welfare payments
decreased more in sites where grant levels were relatively high than in sites
where grant levels were relatively low. In addition, the programs decreased
payments of welfare benefits more in sites that strictly enforced program
participation mandates than in sites that did not.
Figure 7.
Impacts on Welfare and Food Stamp Payments over Five Years, by Program:
Welfare and Food Stamp Payment Reductions Were not Consistantly Larger in
Employment-Focused Programs than in Education-Focused Programs
SOURCE: Hamilton et al.,,2001
NOTES: The Riverside LFA program results include both graudates and
nongraduates.
Asterisks(*) denote statistical signficance levels: * = 10 percent; ** =
5 percent; *** = 1 percent.
Double daggers(‡) denote statistical significance levels for food stamp
payments: ‡ = 10 percent; ‡‡ = 5 percent; ‡‡‡
1 percent.;
[Go To Content]
-
Compared with the LFA approach, the HCD approach did not produce additional
long-run economic benefits.
In side-by-side comparisons in the same sites, the LFA and HCD approaches'
five-year impacts on employment, earnings, months on welfare, and welfare
payments were not the same, but the differences were generally not statistically
significant -- that is, it could not be confidently concluded that the
differences in impacts did not occur by chance. Where there were statistically
significant differences between the effects of the two types of programs,
however -- such differences were found for some early follow-up years and
for some subgroups and outcomes -- the LFA programs always came out ahead.
For example, in Grand Rapids, the LFA group worked more quarters on average
than did the HCD group, and the average number of months of welfare or food
stamp receipt was lower in the LFA group than in the HCD group.
-
The LFA approach moved people into jobs and off welfare more quickly than
did the HCD approach -- a clear advantage in an era of time limits on welfare
receipt.
As is typical in welfare samples, earnings levels increased during each year
in the follow-up period in the LFA and HCD programs as well as the control
groups, reflecting increases in employment. Moreover, earnings were higher
among both LFA and HCD program members than among control group members early
in the follow-up period, but the differences between the program groups and
the control group (that is, the programs' impacts) narrowed over time (for
the yearly earnings impacts of the LFA and HCD programs averaged across the
three sites, see Figure 8). Clearly, however,
earnings rose earlier for the LFA group than the HCD group. Similarly, in
the first two years of the follow-up period, the LFA programs had larger
impacts on welfare receipt than did the HCD programs (for the LFA and HCD
programs' yearly impacts on welfare receipt, see
Figure 9). In subsequent years, however, the
gap between the two lines narrows and ceases to be statistically significant.
-
Relative to the LFA approach, the HCD approach did not produce more earnings
growth or increase the likelihood of employment in good jobs.
Neither the HCD nor the LFA approach was generally successful in boosting
earning growth or the likelihood of having a good job -- that is, a job that
is stable and well-paying -- but the impacts on these measures were especially
disappointing for the HCD programs. The education and training services that
were part of these programs were intended to help people eventually move
into stabler and higher-paying jobs (compared with control group members
and LFA program group members), with the goal of more than making up for
the earnings foregone early in the follow-up period while welfare recipients
were enrolled in classes. However, both the LFA and HCD programs had little
or no effect on earnings growth and employment stability. Furthermore, the
trend lines in Figures 8 and
9 suggest that the HCD programs' lack of advantage
over the LFA programs in this regard would not change if follow-up data beyond
five years were available.
-
The LFA approach was much cheaper to operate than the HCD approach.
The HCD programs were 40 percent to 90 percent more expensive than the LFA
programs that operated in the same sites (for details on costs, see the
second-to-last section of this document).
-
The results of the above comparisons between the LFA and HCD programs'
impacts held for nongraduates as well as graduates.
Even among nongraduates, who were expected to derive the greatest benefit
from an initial investment in basic education, the employment and earnings
impacts of the LFA programs were larger than those of the HCD programs.
-
The LFA and HCD approaches did not differentially affect income or children's
well-being in the full NEWWS sample. However, for one subgroup -- nongraduates
-- the LFA programs had a larger impact on income than did the HCD
programs.
Neither the LFA nor the HCD approach increased income overall. In fact, income
impacts varied more by site than by program approach. But the programs did
have different effects on income among nongraduates: Although in neither
type of program was nongraduates' income higher than control group levels,
those in the LFA programs had higher income, on average, than those in the
HCD programs. Averaging the results for nongraduates across the three sites
that ran LFA and HCD programs, the LFA programs resulted in almost $1,000
more in income over five years than the HCD programs. Few effects on children's
well-being were found, and these did not differ consistently by program approach
(for details on the effects of income on children, see the next section).
Figure 8.
Impacts on Earnings, by Approach and Year:
LFA Programs Increased Earnings More Quickly than HCD Programs
SOURCE: Hamilton et al., 2001
NOTES: The impacts shown are average for sample members in the LFA and HCD
programs in Atlanta, Grand Rapids, and Riverside.
Daggers(†) denote statistical significance levels for LFA-HCD differences:
† = 10 percent; †† = 5 percent; ††† = 1
percent;
To ensure comparability to the Riverside HCD sample, the Riverside LFA sample
includes only those who nongraduates at study entry.
Figure 9.
Impacts on Welfare Receipt, by Approach and Year:
LFA Program Moved People Off Welfare More Quickly than HCD Programs
SOURCE: Hamilton et al., 2001
NOTES: The impacts shown are average for sample members in the LFA and HCD
programs in Atlanta, Grand Rapids, and Riverside.
Daggers(†) denote statistical significance levels for LFA-HCD differences:
† = 10 percent; †† = 5 percent; ††† = 1
percent;
To ensure comparability to the Riverside HCD sample, the Riverside LFA sample
includes only those who nongraduates at study entry.
[Go To Content]
As already discussed, the most rigorous findings about the relative effectiveness
of different program approaches come from the analyses that directly compare
the LFA and HCD programs within each site that operated both types of program.
Viewed with appropriate skepticism, however, cross-site comparisons can suggest
what other program approaches or features are likely to be particularly
effective.
As shown in Figure 6, the Portland program
by far outperformed the other 10 programs in terms of both the size and
consistency over time of its earnings gains (and its employment gains, which
are not shown in the figure). The Portland program increased average five-year
earnings by 25 percent and the average number of quarters employed by 21
percent. The program also increased stable employment and earnings growth
more than any of the other 10 programs.
-
The Portland program's success suggests that the following are key features
of very effective programs: an employment focus, the use of both job search
and short-term education or training, and an emphasis on holding out for
a good job.
Although contextual factors may have contributed to the Portland program's
success -- relative to the other NEWWS programs, Portland's worked with a
less disadvantaged welfare caseload, and the state had a relatively high
minimum wage -- it also differed from the other programs with respect to
implementation. The Portland program had a clear employment focus. Unlike
the LFA programs and the education-focused programs, however, it used a mixed
strategy for matching enrollees to initial activities: Portland staff assigned
some to very short-term education or training and others (the majority) to
job search. Also, job search participants in Portland, unlike in the other
programs, were counseled to wait for a good job (that is, one that paid at
least about 25 percent higher than the minimum wage and offered a good chance
for stable employment) as opposed to taking the first job they were offered.
Although other aspects of the Portland program, such as its use of job developers
and the considerable experience of its staff in operating job search programs,
also deserve some credit for the program's beneficial effects, these features
did not make the program unique in NEWWS. And although Portland's relatively
strong economy may have contributed to the success of the program, other
NEWWS programs in places where the demand for labor was similarly high did
not have equally large earnings or employment impacts.
One implication of the Portland results -- that strongly employment-focused
programs with mixed activities are more effective than programs that offer
primarily job search or primarily education and training -- is buttressed
by findings from previous studies. For example, the Greater Avenues for
Independence (GAIN) program that was run in Riverside, California, in the
late 1980s -- widely considered a paragon among welfare-to-work programs
-- was also an employment-focused, mixed-strategy program. Operationally,
both Portland and Riverside GAIN not only stressed the importance of finding
jobs and strictly enforced program participation requirements but also offered
many different services, including job search (along with job development),
short-term education, and (in Portland) training. In both programs, people
who were considered not ready to enter the labor market were sometimes first
assigned to basic education or (in Portland) to training or life skills classes.
-
The Portland program's implementation features also suggest how to make
education and training in mandatory welfare-to-work programs more effective
for instance, by putting an upper limit on the duration of recipients'
participation in some types of adult education.
When making initial assignments to education or training activities, Portland
program staff communicated to welfare recipients that improving their
employability was the goal. The assignments were thus limited in duration,
usually lasting 6 months or less, and participants were encouraged to complete
them and then look for work rather than to "languish" in them. Decisions
about who was not ready to go immediately into job search -- and thus who
could be initially assigned to education, training, or other activities --
were left to program staff, who took into consideration a variety of factors
that might affect recipients' employability, including work history, educational
attainment, and reading and math skills. GED preparation classes, for example,
were offered primarily to people who case managers thought had a good chance
of obtaining a GED relatively quickly, and the program indeed raised the
proportion of people who received a GED over the five years. In addition,
the program led to an increase in the percentage of welfare recipients who
received both a GED or high school diploma and a trade license or certificate
-- a combination that nonexperimental research has suggested is particularly
effective in boosting subsequent earnings.
Finally, by partnering with local community colleges to design and operate
the Portland program, welfare department administrators probably increased
recipients' exposure to postsecondary education. Although Portland staff
did not make assignments to college courses as part of the welfare-to-work
program, among recipients who entered the study with a high school diploma
or GED, the Portland program produced a 21 percentage point increase in the
proportion who took a course for credit at a two- or four-year college, a
difference that emerged in the second half of the five-year follow-up period.
The late appearance of these participation impacts and the fact that assignments
to college were not made as part of the program suggest that welfare recipients'
exposure to the community college system while they were participating in
job search and other welfare-to-work program activities had spillover effects
on college course enrollment. Portland was the only NEWWS program to increase
college participation, but the timing of its impacts on this outcome makes
it highly unlikely that they were related to its large earnings impacts earlier
in the follow-up period.
[Go To Content]
Evaluations of welfare-to-work programs operated before FSA's passage found
that the programs were most effective for the moderately disadvantaged and
least effective for the most disadvantaged and the least disadvantaged (to
learn how these groups were defined in NEWWS, read the first finding below).
Partly in response to these findings, FSA required states to target
welfare-to-work programs at welfare recipients who were the most likely to
have long stays on welfare and the least likely to work; to offer the mix
of services, including education, that they thought was most likely to benefit
this hard-to-employ group; and to subsidize child care, transportation, and
work-related expenses while people participated in welfare-to-work programs.
This subsection describes the NEWWS findings for selected subgroups of welfare
recipients, that is, for groups of sample members who shared certain
characteristics when they entered the study. (These subgroup findings are
corroborated by those from other evaluations of welfare-to-work programs
operated before PRWORA's passage.)
-
If the objective of welfare-to-work programs is to increase earnings and
reduce welfare payments, the NEWWS programs worked well for a wide range
of welfare recipients: Most subgroups were affected by the programs on these
measures.
The NEWWS programs' effects were examined for long- and short-term welfare
recipients; for people who had worked in the year prior to study entry and
people who had not; for groups defined by race/ethnicity; and for the most
disadvantaged (people who had long-term welfare receipt, dropped out of high
school, and had been unemployed long term), the least disadvantaged (who
had none of these barriers to employment), and the moderately disadvantaged
(who had one of these barriers). Most of the programs led to increases in
earnings and decreases in welfare payments for all of these subgroups --
except the least disadvantaged, for whom earnings impacts were small and
welfare payment impacts were found for only a few programs.
-
If the objective of welfare-to-work programs is to increase income, few
NEWWS programs worked well for any subgroup of recipients.
The programs did not systematically change income for any subgroup. Although
most of the programs changed the proportion of people's income that came
from earnings as opposed to welfare and food stamps, they left people with
the same income, on average, as control group members.
-
Welfare-to-work programs can help people who are traditionally considered
hard to employ: Almost all 11 NEWWS programs increased earnings for the
disadvantaged subgroups.
Most of the programs raised earnings above control group levels for both
the moderately disadvantaged people who had one or more serious barriers
to employment, such as no recent work history or a lengthy history of welfare
receipt and for the most disadvantaged, who had all three serious barriers.
-
Neither employment-focused programs nor education-focused programs had
consistently larger impacts on the earnings of the most disadvantaged.
Although neither the employment-focused approach nor the education-focused
approach was clearly more effective for the most disadvantaged recipients,
the employment-focused programs had slightly larger earnings impacts for
this subgroup than did the education-focused programs. In two of the three
sites in which LFA and HCD programs were operated side by side, for example,
the LFA programs produced considerably higher impacts on the earnings of
the most disadvantaged sample members than did the HCD programs.
-
Although most of the NEWWS programs increased the earnings of the most
disadvantaged welfare recipients, this subgroup still earned very little.
The earnings increases experienced by the more disadvantaged welfare recipients
were no larger than those for the less disadvantaged recipients. As a result,
even after participating in a program, the most disadvantaged program group
members earned only about half as much as the moderately disadvantaged program
group members, suggesting a need for policies aimed at raising the earnings
of the most disadvantaged.
-
Except for people at high risk of depression, all subgroups defined according
to baseline measures of psychosocial well-being benefited from the
programs.
There were few differences in impacts between subgroups defined by psychosocial
characteristics assessed at study entry such as risk of depression, sense
of control over personal destiny, work-related parental concerns, preference
for work over welfare, health or emotional problems, child care problems,
and transportation problems. The only exception was risk of depression. Though,
surprisingly, control group members at high risk of depression and those
at low risk had similar three-year earnings, overall the NEWWS programs did
not boost the earnings of people at high risk of depression who also had
high school diplomas, recent work experience, and little prior experience
with welfare (earnings increased for people at high risk of depression who
also had at least one serious barrier to work).
[Go To Content]
-
The NEWWS findings should not be taken as a general indictment of the
benefits of education and training in welfare-to-work programs.
NEWWS addressed whether programs in which welfare recipients were initially
assigned to and required to participate in education and training activities
produced higher average earnings than programs in which welfare recipients
were initially assigned to and required to participate in job search. Although
the findings indicate that the answer to this question is no, this does not
mean that education and training do not pay any dividends to those who actually
participate in these activities, receive a high "dosage" of instruction,
complete the class sequence or program, and receive a degree or certificate
or attain a certain skill level. In fact, nonexperimental work conducted
as part of NEWWS suggests that people who increased their skills or obtained
a GED subsequently experienced earnings gains relative to people whose skills
did not improve or who did not get a credential. The biggest earnings payoff
was for those few people who obtained a GED and then received some type of
vocational training. Taken together, these results indicate that education
and training can benefit welfare recipients. In the NEWWS sample, however,
too few recipients achieved the intermediate milestones -- that is, gains
in literacy skills or GED attainment -- to reap the potential rewards of
education and training.
-
The findings suggest ways in which the benefits of education and training
in welfare-to-work programs can be heightened.
It should be kept in mind that the employment-focused programs all offered
short-term education or training to people who did not find employment through
job search. Furthermore, the most successful program provided a mix of job
search and short-term education or training as initial activities. Thus,
education and training had a role in all the programs in NEWWS.
One way to heighten the benefits of education and training would be to retain
more students long enough to help them improve their literacy skills and
earn a GED and access postsecondary services that would allow them to capitalize
on the GED credential. Such an effort could be made while people are receiving
welfare benefits or after they have left the welfare rolls. There is no
guarantee, however, that increasing the duration of participation would be
sufficient to help more students achieve the education milestones. Other
possibly important factors include the quality of instruction, the
appropriateness of the materials and technology for people with low literacy
skills and possibly with learning disabilities, and the sometimes limited
motivation or constricting life circumstances of program participants. In
addition, consideration must be given to the current welfare environment;
a long-term commitment to attending education activities, for example, might
cause recipients to exhaust their welfare eligibility. Another way to enhance
education and training benefits would be to encourage only those recipients
who are within easy reach of earning a GED to pursue one. As already discussed,
the Portland program adopted this strategy. Still another way to boost education
and training impacts would be to foster links between adult education programs
and postsecondary programs in an effort to encourage those who earn a GED
to go further. Again, the Portland program provides an example of this approach.
5. In these sites, the starting of the welfare
time-limit clock necessitated allowing control group members to access
welfare-to-work program services.
Where to?
Top of Page |
Contents
Main Page of Report | Contents
of Report
Home Pages:
National Evaluation of Welfare-to-Work Strategies
(NEWWS)
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)