May 1998
| The Manpower Demonstration Research Corporation (MDRC) is conducting the National Evaluation of WelfaretoWork Strategies under a contract with the U.S. Department of Health and Human Services (HHS), funded by HHS under a competitive award, Contract No. HHS-100-89-0030. HHS is also receiving funding for the evaluation from the U.S. Department of Education. The study of one of the sites in the evaluation, Riverside County (California), is also conducted under a contract from the California Department of Social Services (CDSS). CDSS, in turn, is receiving funding from the California State Job Training Coordinating Council, the California Department of Education, HHS, and the Ford Foundation. Additional funding to support the Child Outcomes portion of the study is provided by the following foundations: the Foundation for Child Development, the William T. Grant Foundation, and an anonymous funder. |
Other Publications from This Evaluation
This evaluation could not have taken place without the unwavering cooperation and commitment of the Oregon Adult and Family Services (AFS) and the Steps to Success staff. The top management staff set the tone for this support: Kevin Concannon and Gary Weeks, the present and past Directors of the Department of Human Resources (DHR); Sandie Hoback and Stephen Minnich, the present and past Administrators of the AFS Division of DHR, and Jerry Burns and Debbie White, the former Managers of the AFS JOBS Unit. Their willingness to submit their welfaretowork program to an ambitious, longterm study, with its elaborate research design and rigorous data requirements, merits special recognition. Special thanks go also to Susan Blanche-Kappler, the site liaison with MDRC and the U.S. Department of Health and Human Services, who has overseen the implementation of the evaluation and has provided ongoing guidance and input.
A number of AFS management information systems personnel facilitated the access and utilization of state administrative and cost datasets: Margaret Armontrout, Edward Buckner, Ward Kent, Larry Morris, and Bob Putman.
At the AFS District Two office, thanks are due to the current regional manager, Maureen Casterline, and her predecessor, Jean Stryker, as well as their staff for their support in implementing the evaluation research design: Judith Brown, David Flock, Frank Gembinski, Erma Hepburn, Jean Pullen, and Marge Reinhart.
Many AFS branch office managers in Multnomah and Washington counties provided assistance over the course of the evaluation: Pat Adair, Rod Brown, June Cook, Cathy Craner, Oren Cyphers, Hilda Davis, Bob Earnest, Carol Eckel, Dorothy Fuller, Angel Grogen, Veda Latin, Bruce Lowry, Linda Montgomery, Ann Pickar, Kei Quitevis, Will Reinhart, Pam Ruddell, C. L. Thames, and Roger Zwemke.
Thanks also go to Jodi Davich, Mardica Hicks, Maureen Judge-Morris, Nan Poppe, and Julie Wyckoff-Byers, the Steps to Success program managers at the Mt. Hood and Portland Community College Districts, as well as their staff for conducting the random assignment procedures and participating in a multitude of research activities.
The contribution of these people, plus many more too numerous to mention, made the evaluation in Oregon a success.
Few domestic issues have generated as much attention over the past decade as welfare reform. Persistent dissatisfaction with the Aid to Families with Dependent Children (AFDC) program — the nation’s principal safety net for poor families — spurred the enactment in August 1996 of the Personal Responsibility and Work Opportunity Reconciliation Act.1 Among its provisions, the law replaced AFDC with a block grant program, Temporary Assistance for Needy Families (TANF), and created financial incentives for states to run mandatory, workfocused welfaretowork programs.
Welfaretowork programs provide services such as job search assistance, education, and training to help welfare recipients prepare for and find jobs. States have run versions of these programs to serve part of the welfare caseload for the past three decades. Various facets of the 1996 law, however, magnify the need for effective strategies to move people more quickly into jobs and off welfare. First, states may not use federal funds to support most families for longer than five years and may impose even shorter time limits on assistance (they may also elect to use state funds to support families beyond five years). Second, to prevent reductions in their block grants, states must meet demanding "participation standards" by engaging large proportions of TANF recipients in work or workrelated activities. To meet these standards, most states will have to engage a wider cross section of the caseload in work or program activities than they did previously. Third, states’ TANF plans must include how the states will require recipients to work after two years of assistance.
As states and localities transform their welfaretowork programs in response to the federal legislation, the need to learn about programs that have moved substantial numbers of people into work and off welfare increases. The twoyear findings presented in this report show the Portland, Oregon, welfaretowork program run between early 1993 and mid 1996 to be among the most successful largescale mandatory welfaretowork programs studied, producing large increases in employment and earnings and equally large reductions in welfare receipt for a broad cross section of the welfare caseload. The positive effects remained very strong at the end of the twoyear period studied, and preliminary data suggest they will continue into the third year.
This report is the latest from an evaluation of mandatory welfaretowork programs in seven sites called the National Evaluation of welfaretowork Strategies (NEWWS Evaluation), conducted by the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. Department of Health and Human Services, with support from the U.S. Department of Education. The report examines the mandatory welfaretowork program run in Portland (Multnomah and Washington counties). Through the program, Portland provided employment and support services to a broad cross section of the AFDC caseload, including parents with children as young as one year old. These people were required to participate in program activities or face reductions in their welfare grants. Although the program studied was designed and implemented prior to the 1996 reform, its overarching goal was similar to that of the new law: to foster the selfsufficiency of adult recipients through increased employment and decreased welfare receipt. (The program that Portland is running under the 1996 welfare reform law includes some key features of the program studied in this report.)2
This report describes the implementation, participation patterns, and cost of the Portland program, and presents estimates of the effects of the program on employment, earnings, and welfare receipt during the two years following people’s entry into the program. To determine the effects of Portland’s program, 5,547 singleparent AFDC applicants and recipients aged 21 and over who attended a program orientation between February 1993 and December 1994 were randomly assigned to either a program group, eligible for program services and subject to participation requirements, or a control group, not eligible for services and not subject to participation requirements (although they could participate in other services in the community). Because randomization makes the two groups similar at the start, any differences in average subsequent outcomes (such as twoyear earnings) can be confidently attributed to the effects of the program. These differences, known as program impacts, will be discussed later in the summary and are statistically significant unless otherwise noted.3
The Portland program was run through a cooperative partnership between the welfare department and various local service providers. Most program services were provided by the local community colleges and were of high quality. The program was strongly employmentfocused: staff communicated that the primary program goal was to help people move into jobs, and job search was the most common activity. However, in contrast to many employmentfocused programs, participants were encouraged to look for and take "good" jobs — fulltime, paying above the minimum wage, with benefits and potential for advancement. Also, Portland’s program utilized a more mixed services strategy than is typically implemented by strongly employmentfocused programs. Staff assigned many people to shortterm education, vocational training, work experience, and life skills training to improve their employability. Some people were deferred from program participation, although the program did work with at least some people traditionally defined as the most disadvantaged portion of the caseload. The per person cost of the program was moderate, relative to other welfaretowork programs of the 1990s.
Followup of more than two years is needed to fully assess the success of a welfaretowork program, but at the twoyear mark Portland’s program produced effects (impacts) on employment, earnings, and welfare receipt that were among the largest ever found for largescale mandatory programs.
Portland’s program was unusually successful in moving people into jobs, increasing their earnings, and moving them off welfare. This success occurred in a specific context: Portland’s caseload was predominantly white, minimizing the chances of racial discrimination in the labor market; a high percentage of the caseload entered the program with a high school diploma or GED certificate; and Portland’s economy was very strong during the study period, with low unemployment and substantial job growth.
To illustrate the magnitude of Portland’s accomplishments, this report makes a number of comparisons. The primary comparison — to show the net effects or impacts of the program — is between people in the program group and those in the control group. In addition, Portland’s program is compared with other programs to show the relative effectiveness of Portland’s approach. Specifically, this summary makes three types of explicit crossprogram comparisons:
| of welfaretowork Strategies A key issue in welfare reform throughout the last decade has been how best to move welfare recipients into the workforce, toward selfsufficiency, and out of poverty. One approach, commonly referred to as the "work first," or "labor force attachment," approach, aims to get people to work quickly, even at low wages, by requiring and helping them to look for work, reflecting a view that welfare recipients can best build their work habits and skills in the workplace. A second approach emphasizes skillsbuilding, or "human capital development," through education and training as a precursor to employment, based on the belief that an upfront investment in the skills levels of welfare recipients will allow them to obtain higherpaying and more secure jobs. The program run from early 1993 to mid 1996 in Portland can be considered to be a blend of strong work first elements and moderate skillsbuilding elements. Most programs across the nation have blended the two approaches, although in response to the 1996 welfare reform law most states are shifting toward a work first approach. To determine the strengths and limitations of each approach, three of the seven sites studied as part of the NEWWS Evaluation — Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California — simultaneously operated two different programs: a work first program and a skillsbuilding program. The goal of the work first programs was rapid employment, and job search was the prescribed first activity for virtually the entire caseload. In contrast, most people in the skillsbuilding programs were first assigned to education or training; basic education was the most common activity because of the generally low educational attainment of the enrollees at program entry. In addition to Portland’s program and the six programs discussed above, the NEWWS Evaluation includes four other programs. In one site — Columbus, Ohio — two different case management approaches were compared side by side. The study in the other two sites — Detroit, Michigan, and Oklahoma City, Oklahoma — tested the net effects of the sites’ welfaretowork programs (similar to the study in Portland). The Columbus, Detroit, and Oklahoma City programs primarily utilized a skillsbuilding approach. In total, the 11 evaluation programs range from strongly work firstfocused to strongly skillsbuildingfocused and from somewhat voluntary to highly mandatory. The program sites offer diverse geographic locations, caseload demographics, labor markets, and AFDC grant levels. These programs, while not representing all welfaretowork programs in the nation, represent a wide range of welfaretowork options. |
The results presented in this report should be considered in the context of Portland’s research sample and program environment. Compared to the samples in the other NEWWS Evaluation sites, the Portland sample had relatively high prior lifetime employment levels, but average prior recent employment (defined as work in the year before program entry). More sample members in Portland than in the other sites had a high school diploma or GED at program entry, but the average highest grade completed in school was similar across all the sites (about 11th grade). Portland’s sample fell in the middle of the range of prior AFDC receipt. The sample in Portland was predominantly white, nonHispanic, as is the general population in the Pacific Northwest, minimizing the chances of racial discrimination in the labor market.
AFDC grant levels in Oregon were significantly higher than the national average. Portland’s labor market was strong, with relatively low unemployment rates, decreasing through most of the followup period, and high employment growth. The caseload and labor market can affect the results of a welfaretowork program in many ways. Throughout the summary, these effects will be noted when relevant.
Although state legislators and administrators defined the basic parameters of Oregon’s welfaretowork program, each district in the state was given the freedom to implement the program as it saw fit. The Portland area welfare offices designed the program in cooperation with local community agencies; major budgeting and programming decisions were reached jointly.
Job search, education, training, and work experience services were provided by the local community colleges (with job search provided by the chambers of commerce in one study county), under contract with the welfare agency, and case management services were provided by both the welfare agency and the community colleges. Field researchers and program participants rated Portland’s program services highly compared to services in other programs.
In Portland, integrated case managers, employed by the welfare department, were responsible for all aspects of program participants’ cases and had sole authority for all decisions, such as making activity referrals, authorizing support services, and imposing financial penalties for program noncompliance. In many instances, case managers employed by the community colleges worked in cooperation with the integrated staff and helped recommend services, checked up on attendance, and provided quick intervention if problems in attendance or progress arose.
The primary goal of Portland’s program was to move people into the labor market. For applicants, the employment message was apparent even before they entered the program: at AFDC application, people were assigned to an initial work search activity to be completed prior to the welfaretowork program orientation, which took place about a month later. (Since this initial activity occurred prior to random assignment, which took place in conjunction with orientation sessions, any effects it may have had are not reflected in the program group/control group differences discussed in this report.) Among postorientation activities, job search (including job club and other job search activities) was the one most commonly assigned and utilized. Program staff told people in all activities, including education and training, that their goal should be to achieve selfsufficiency through paid work.
Job club, job development, and job placement activities in Portland were particularly well supported. Fulltime job developers used various methods to link participants to job opportunities, such as working closely with employers to discover unadvertised openings and sending them flyers "advertising" their linking service. Other staff worked to connect program group members with existing leads through the state Employment Department.
Portland also had an individualized work experience component in which program staff customdesigned work experience positions based on people’s skills and interests. Positions were in both nonprofit organizations and forprofit companies. (Participation was voluntary in the forprofit work slots.) Staff reported that many work experience positions led to unsubsidized jobs.
In Portland, "good" jobs, rather than "any" jobs, were considered the preferable path to selfsufficiency. One standard used to measure the welfare district’s and service providers’ performance was an average wageofplacement that was always much higher than the minimum wage (for example, in 1994 Oregon’s minimum wage was $4.75 and the average wageofplacement target was $6.00). Another standard concerned "AFDC recidivism" — the percentage of individuals who returned to the welfare rolls — and encouraged staff to promote jobs that were likely to last. If a service provider’s performance was more than 20 percent below any of the standards in a quarter, provider and welfare staff were required to write a corrective plan describing how they would meet the standard in the future; ultimately the provider risked losing its contract with the welfare department. (During the followup period for this report, standardsrelated issues did not lead to written corrective plans or loss of contracts by providers.) Job developers actively sought positions that paid above the minimum wage and provided room for advancement. Some staff did not discourage participants from accepting lowpaying jobs, but usually only when there was potential for advancement. It is important to note that "good" jobs would probably have been more difficult to develop and find in a poor economy, and a more disadvantaged caseload would have been less able to be selective when choosing work.
There was no single uniform path through the program. Although Portland aimed to move people into the labor market, at oneonone meetings directly following program orientation, case managers evaluated some people — based on a confluence of factors including work history, educational status, and reading and math skills — as not ready to go immediately into job search. During the period studied, about half of first activity assignments were to job search and half were to other activities (over time, an increasing proportion of people were first assigned to job search). Some people who completed job search without finding work subsequently took part in education or training, and many people who first participated in a nonjob search activity subsequently participated in job search (if they remained on the welfare rolls). Activities were especially varied for those who entered the program without a high school diploma or GED certificate, with many participating in basic education.
The goal of education, training, and work experience activities was to prepare individuals relatively quickly for unsubsidized employment; thus, Portland staff encouraged shortterm participation. Program group members who took part in program activities participated for about five months over the twoyear followup, similar to the average length of stay in the three work first programs in the NEWWS Evaluation and shorter than the average length of stay in the three skillsbuilding programs.
About onethird of program group members were not assigned to an activity in the three months following random assignment. During this period, about half of these nonassigned individuals were "deferred" from participation by a case manager; most others became nonmandatory for the program because they left AFDC or experienced a status change such as pregnancy. Most of those initially deferred were never assigned to an activity during the twoyear followup (although case managers encouraged many of these people to seek services outside the program, such as remedial education or mental health counseling).
During field research, case managers reported that people with very low skills, serious physical or mental health problems, or exceptionally low motivation levels often were not referred to program activities. However, as mentioned, Portland’s program produced employment and AFDC impacts for all subgroups of the sample, including the most disadvantaged portion of the caseload, defined using education, employment, and AFDC receipt indicators (as noted below, the program increased participation for this group). Thus, while case managers were somewhat selective in whom they assigned to activities, they were, in fact, working with at least some of those who are traditionally viewed, according to objective measures, as the most disadvantaged portion of the caseload.
Control group members’ level of selfinitiated activity represents what would have happened if program group members had had no exposure to Portland’s welfaretowork program. As Figure 1 shows, most control group participation was in basic education and vocational training or college. Program group members were seven times more likely to engage in job search than their control group counterparts during the twoyear followup period (for the program group, this "job search" measure included some participation in life skills training classes, which covered issues such as career exploration, résumé preparation, time management, and problemsolving). Program group members were also more likely to participate in basic education, including GED preparation classes, vocational training or college, and work experience.
The program substantially increased participation in activities for a broad cross section of the caseload — both those with a high school diploma or GED ("graduates") and those without ("nongraduates"), and the most disadvantaged.
Staff closely monitored those assigned to or engaged in program activities. Most Portland staff tried to encourage participation and compliance with program requirements using positive encouragement, emphasizing the potential benefits for the individual and her family. If, after lengthy cajoling, people did not eventually comply, staff imposed financial sanctions (AFDC grant reductions). Twentyone percent of program group members were sanctioned during the twoyear followup period and the average sanction lasted about five months. In the six programs run in Atlanta, Grand Rapids, and Riverside, sanction rates ranged from 9 to 42 percent and sanctions lasted for an average of eight months.
The costs estimated in this report consist of all costs associated with providing employment services and related support services to sample members; costs associated with authorizing and processing welfare payments are not included. The gross cost per program group member during the twoyear followup period consists of costs paid by the welfare department and nonwelfare agencies while sample members were enrolled in Portland’s program as well as for employment and support services after they exited the program and, in some cases, left AFDC. The twoyear gross cost per Portland program group member was $4,027. The welfare department paid about twothirds of this cost; the remainder was paid by schools and other agencies. The proportion of costs covered by the welfare department is high compared to the other programs studied in the NEWWS Evaluation. The welfare department in Portland contracted out, and thus paid for, most of the program services, whereas most programs rely more on noncontracted outside agencies.
The net cost per program group member during the twoyear followup period is the gross cost per program group member minus what would have been spent in the absence of a mandatory welfaretowork program, as measured by the cost per control group member. Control group members were not eligible to take part in program activities, but could enroll on their own in other employmentrelated activities in the community and were eligible for activity and employmentrelated welfare department support services. Thus, control group costs include expenditures for all of the nonprogram activities and support services utilized by control group members during the twoyear followup period. As Table 1 shows, Portland’s twoyear net cost per program group member was $2,017. This net cost is higher than the average net cost per work first sample member and much lower than the average net cost per skillsbuilding sample member. (A fiveyear benefitcost analysis will eventually be completed as part of the NEWWS Evaluation.)
Support services accounted for 38 percent of Portland’s gross costs (see Table 1). A large proportion of the support service costs — $1,422 per program group member — was for child care. Total child care costs in the six work first and skillsbuilding programs ranged from $88 to $794 per person. In Portland, 50 percent of program group members received payments for child care services while they were enrolled in the program, compared to 17 to 31 percent in the six work first and skillsbuilding programs. Twentysix percent in Portland received employmentrelated child care payments (including transitional child care, provided for up to one year for those who left AFDC for work, and EmploymentRelated Day Care, provided with no time limit to those not eligible for transitional care) compared to no more than 7 percent in any of the other programs studied.
The high rates of child care use for both program participation and employment in Portland probably reflect the strong emphasis that staff placed on securing child care arrangements and having a backup option. The high rates of employmentrelated child care use also reflect Portland’s large impact on employment. Furthermore, it is possible that Portland’s integrated case managers were more likely than the traditional case managers in the other sites to know the two facts that qualify an individual for transitional benefits: namely, that the individual has received AFDC for at least three of the past six months and is leaving AFDC for employment. Staff who perform a dual role may also be more inclined to authorize benefits automatically, without a special request from a parent or other staff member.
Child care costs in Portland were particularly high for parents with younger children. As mentioned, Portland required parents with children as young as age 1 to participate in the program. The child care costs for those entering the program with a child under age 3 were more than twice the costs for parents entering the program with no children under age 3 ($2,144 compared to $936). Parents with younger children at program entry had higher average monthly child care payments and higher rates of child care receipt and received payments for a longer time than those without young children. It is interesting to note that although Grand Rapids also required participation from parents with children as young as age 1, per person child care costs in the Grand Rapids work first and skillsbuilding programs ($367 and $542, respectively) were much lower than in Portland, owing to lower monthly payments and much lower rates of child care use.
Of individuals who entered Portland’s program without a high school diploma or GED certificate ("nongraduates"), 16 percent of program group members received a GED during the two years following random assignment compared to 5 percent of the control group, an 11 percentage point increase. As mentioned earlier, many nongraduates attended basic education classes as part of the program. It should be noted, however, that case managers tended not to refer people with very low skills levels to basic education class. The program also increased the receipt of a trade license or certificate for nongraduates by 12 percentage points.
As shown in Table 2, over two years of followup, 72 percent of program group members were employed compared to 61 percent of control group members, a difference of 11 percentage points. This impact represents the effect of the program in promoting paid work among sample members who would not have worked on their own. Said another way, more than one out of every four welfare recipients who normally would not have worked in an unsubsidized job during the twoyear followup period did so as a result of the program. The employment gains persisted through the followup period: in the last quarter of the followup period, 46 percent of program group members worked for pay compared to 35 percent of control group members, an increase of 11 percentage points.
Program group members earned, on average, $7,133 over the twoyear period, and control group members earned $5,291. (These averages include those who did and did not work during the followup period.) This difference of $1,842, a 35 percent increase above the control group average, is the largest earnings impact measured among the programs studied as part of the NEWWS Evaluation (twoyear impacts have been found in 8 of the other 10 programs and range from $367 to $1,276), and it approaches the magnitude of the largest earnings impact attained by a largescale mandatory welfaretowork initiative (the Riverside GAIN program of the late 1980s increased twoyear earnings by $2,103, or 56 percent).
In Portland, just over onehalf of the earnings impact resulted from job finding (increases in employment levels), about onequarter came from increased employment duration (program group members finding jobs sooner and/or finding jobs that last longer than do control group members), and oneseventh resulted from increased earnings on the job (higher hourly wages, more hours of work per week, and/or more weeks of work in a quarter).6 In other words, better job quality (the combination of longer employment duration and higher earnings on the job) accounted for nearly 40 percent of the total impact on earnings. This result was relatively uncommon for welfaretowork programs of the 1980s and has not been found for all employmentfocused programs of the 1990s.
The quarterly average earnings for the program and control groups are plotted in the upper panel of Figure 2. The strong early impact is similar to that found for the three work first programs studied as part of the NEWWS Evaluation. However, in contrast to the work first programs in Atlanta, Grand Rapids, and Riverside, the earnings gains in Portland did not diminish during the followup period (the impacts in the work first programs peaked between one and one and a half years after random assignment, and then grew smaller as increasing numbers of control group members got jobs). In Portland, the quarterly impact increased throughout most of the two years. Program group members earned an average of $1,155 in quarter 9, $309 (37 percent) more than the control group.
Survey results provide further evidence that the program increased job quality. At the end of two years, 40 percent of program group members were working at fulltime jobs (providing at least 30 hours of work) compared to 27 percent of control group members, an increase of 13 percentage points. Twentyfour percent of program group members had employerprovided health benefits at the end of two years compared to 14 percent of control group members, a 10 percentage point increase. (These results include zeros for those not working at the end of two years.)
Portland’s program achieved these gains by helping more people find work, but also by helping program group members find jobs at higher pay and with more benefits than control group members found on their own initiative. Program group members employed at the end of two years averaged $7.34 per hour in wages, $0.86 above the average wage levels earned by employed control group members — the biggest gain of any program in the NEWWS Evaluation. Employed program group members were also more likely to be working full time and receiving health benefits from their employers. These results suggest that the program’s emphasis on waiting for "good" jobs paid off.
As shown in Table 2, over the twoyear followup period, the program group received $1,196 less in AFDC payments than the control group, a 17 percent reduction. This impact on AFDC is among the largest found in the 11 NEWWS Evaluation programs and is similar in magnitude to the decrease produced by the 1980s Riverside GAIN program ($1,399, or 14 percent). Portland’s AFDC reduction resulted almost exclusively from program group members receiving AFDC for fewer months than control group members, as opposed to lower monthly grants for those on assistance.
As the lower panel of Figure 2 illustrates, the impact on AFDC payments remained large throughout the followup period. By the end of the followup period, only 41 percent of program group members were receiving welfare compared to 53 percent of control group members, a decrease of 12 percentage points.
Table 3 shows cumulative twoyear earnings and AFDC payment information for high school graduates and nongraduates, as well as the most disadvantaged sample members — as mentioned earlier, a subgroup of nongraduates who did not work for pay in the year prior to program entry and had received AFDC for more than two years prior to program entry. Earnings and AFDC payments for control group members reveal how each subgroup would have fared without the program intervention. Graduates in the control group had the highest average earnings and the lowest average AFDC payments, while the most disadvantaged control group members had the lowest earnings and highest AFDC payments. Portland’s program substantially increased earnings and decreased AFDC payments for all three groups, and in each case earnings gains were larger than AFDC losses.7
Earnings and AFDC impacts were produced for all three subgroups during the first year of followup and remained strong through the end of the second year (quarterly trends by subgroup are not illustrated here). However, impacts for graduates were most immediate, achieving statistical significance in the second quarter of followup. Impacts for nongraduates and the most disadvantaged first became statistically significant, in the third and fourth quarters, respectively. These differences likely reflect the fact that a higher proportion of graduates were first assigned to job search, which has been found to produce more immediate impacts than other activities. Also, it is typically easier for graduates than nongraduates to find jobs (especially "good" jobs).
Threeyear employment, earnings, and AFDC payment data are available for individuals who entered the program during the first year of random assignment. For this early cohort, the program group earned an average of $1,217 more than the control group in the second year of followup and $1,402 more in the third year; the early cohort program group received an average of $737 less in AFDC payments than the control group in the second year of followup and $756 less in the third year.
Over the twoyear followup period, although earnings gains ($1,842) exceeded AFDC losses ($1,196) by $646, program group members also lost $456 in Food Stamp benefits. The average twoyear combined income from earnings, AFDC, and Food Stamp benefits (calculated using administrative records) for program group members was $16,886 compared to a combined income for control group members of $16,696. The resulting net gain of $191 (a 1 percent increase) is not statistically significant.
In the second year of followup, the program moved a small portion of program group members out of poverty (measured using combined income, as defined above). Twentyone percent of program group members had a combined income at or above the federal poverty level compared to 17 percent of control group members. However, the program also moved a small portion of program group members below 50 percent of the poverty level (33 percent of program group members had a combined income below 50 percent of the poverty level compared to 31 percent of control group members).
Program group/control group differences in combined income grew larger over time and first achieved statistical significance at the end of two years; in the last quarter of followup, program group members’ average combined income from earnings, AFDC, and Food Stamps totaled $2,051, $80 higher than the average combined income for control group members. The analysis also draws on client survey data that include a much wider range of sources for estimating income, including regular or odd jobs, AFDC, Food Stamps, Supplemental Security Income, unemployment insurance, and child support (not including Earned Income Tax Credit [EITC] receipt or other tax credits available to lowincome families in Oregon). These data show that average individual and household income in the last month of followup was higher for program group members than for control group members, although the increases are not statistically significant. Similar impacts were found when average EITC receipt and outofpocket child care payments were included to estimate total net individual and household income.
The increase in income measured at the end of the two years suggests that program group income may be higher than control group income in the third year of followup. Threeyear findings for the early cohort offer additional evidence: for the early cohort, the impact on combined income from earnings, AFDC, and Food Stamps (calculated using administrative records) in the third year of followup ($369) surpassed that of the second year ($215 and not statistically significant).
At random assignment, all program and control group members had health care coverage because they were receiving AFDC and thus were automatically covered under Medicaid. Two years after program entry, coverage rates had decreased as some people had left welfare and did not replace their Medicaid coverage with coverage from employers or other sources. At the end of the followup period, 81 percent of program group members and their children had health care coverage from some source compared to 86 percent of control group members and their children. This 5 percentage point difference is not statistically significant, but is just beyond the statistical significance benchmark applied to the other impacts in this report.
The decrease was generated by those who left AFDC (and automatic Medicaid coverage) for work. As mentioned above, the program increased the proportion of people with employerprovided health coverage, but some people in the program group moved into jobs that did not provide health insurance. Some received coverage from another source, such as Transitional Medicaid or the Oregon Health Plan (the statewide initiative that offers health care coverage for lowincome families). However, some people found jobs that did not provide health insurance and did not obtain alternative coverage.
The Riverside GAIN (Greater Avenues for Independence) program of the late 1980s produced large employment and earnings gains and welfare reductions and is often considered the benchmark for other welfaretowork programs. Portland’s twoyear results are similar in magnitude to those in Riverside; however, the people served and the local environments differed in a number of ways. A higher proportion of Portland’s sample had a high school diploma or GED at program entry and were white (and thus likely faced less racial discrimination in the labor market). Portland mandated participation for parents with children as young as age 1; 68 percent of the sample had children under age 6. Riverside mandated participation only for parents with children aged 6 or over. As mentioned earlier, Portland’s labor market was strong, with low and decreasing unemployment and steadily growing employment. Riverside’s employment growth was high, but unemployment rates were high and increased throughout the followup period.
Despite these differences, impacts were quite similar. Both programs substantially increased employment levels and produced the largest earnings gains ever found for mandatory welfaretowork programs. The Portland program increased twoyear earnings by $1,842, a 35 percent increase, and the Riverside GAIN program increased twoyear earnings by $2,103, a 56 percent increase. Both programs had large impacts on AFDC payments as well: the Portland program reduced payments by 17 percent over two years ($1,196), and Riverside GAIN reduced payments by 14 percent ($1,399).8 In both programs, earnings gains, in dollars, were about one and a half times the amount of AFDC reductions. Both programs produced impacts for a wide range of subgroups, including the more disadvantaged members of the caseload.
The Portland and Riverside GAIN programs both communicated a strong and pervasive message that employment was the chief program goal. Both programs enforced this message through extensive use of job search activities and through the use of job developers to assist recipients in gaining access to job opportunities. Although job search was the most common activity in both programs, many people participated in other activities, including basic education, and, in Portland, vocational training, work experience, and life skills training. Overall participation patterns were similar for the two programs. About threefifths of the program group in each site ever participated in a program activity. Both programs substantially increased participation in job search and basic education (compared to the activity levels of control group members), and Portland’s program also increased participation in work experience. Only Portland increased the proportion of people who received a GED certificate or a trade license or certificate.
The programs communicated differing levels of job selectivity. As previously discussed, Portland staff encouraged participants to wait for "good" jobs. In sharp contrast, staff in the Riverside GAIN program consistently communicated that any job was worthwhile; they encouraged quick entry into the labor market and communicated that lowpaying or parttime jobs should not be turned down. This difference was both reflected in and fostered by the ways each program used performance standards. Portland’s standards encouraged placing participants in jobs paying higher than minimum wage that were likely to last; in contrast, Riverside’s individual monthly job placement standards for case managers reflected and emphasized the importance of quickly placing people in any job.
These divergent messages likely contributed to the programs’ somewhat different composition of the earnings impacts. Earnings impacts in both programs resulted from increases in the proportion of individuals employed and the number of quarters that individuals were employed. Portland’s impacts, however, were also due to program group members earning more on the job than their control group counterparts. Survey results from the two programs corroborate this difference: according to client responses, if one considers only those who were working at the end of the followup period, Portland’s program substantially increased average hourly pay, but Riverside program group members earned slightly less per hour than their control group counterparts.
Research has shown that a variety of program approaches can produce positive effects. While it is difficult to distinguish which of many program features and environmental factors may have fueled Portland’s unusual success, it is likely to be due to a combination of factors. Portland’s program may have worked particularly well in the specific environment in which it was implemented: Portland’s caseload was not as disadvantaged as some of the populations in the other six NEWWS Evaluation sites, and the labor market in the Portland area during the followup period was strong, allowing people to find and obtain "good" jobs. Furthermore, Oregon has a history of progressive social legislation, including generously funded welfare programs and a state minimum wage that is higher than the federal standard. Although this study cannot prove the causality of any program feature or set of features, the report suggests a number of influential factors.
Portland’s strong employment message and extensive use of job search, coupled with a willingness to impose sanctions for program noncompliance, affected how much and how seriously people looked for a job. Portland also offered strong job development and placement services, which provided a direct link to employers and job openings not otherwise available or apparent. Although, in previous studies, increasing participation in job search alone has not been found to be consistently related to large impacts, this combination of message, willingness to impose sanctions, and program services (similar to the combination in Riverside GAIN) probably increased participants’ incentive and opportunity to get a job, and thus increased job finding.
Programs employing a mixed services strategy upon program entry (again, such as Riverside GAIN) also have been found to be more effective than programs that offer only job search or programs that rely mainly on longterm skillsbuilding education and training services; the findings for Portland corroborate this. The nonjob search services, designed to improve employability, were targeted, and participants were not allowed to "languish" in activities, but were encouraged to complete activities and then look for work.
Staff closely monitored people in program activities and followed up quickly when attendance problems arose. Staff communicated the importance of participating in activities, emphasizing the potential benefits for recipients and their families. When people did not eventually comply with program requirements, staff imposed financial sanctions. (However, the sanction rate in Portland was substantially lower than the rate in some other programs.)
As discussed earlier, Portland’s program was designed and implemented through a strong partnership between the welfare agency and various contractor service agencies. Typically, welfare departments that contract out various portions of their welfaretowork programs do so after the program model is chosen and planned; the contractors simply implement the program as the welfare department envisioned. Portland’s arrangement allowed the contractors to participate in defining the program from the ground up, which fostered a sense of "ownership" and a strong commitment to the program in the contractor agencies.
Overall, Portland’s services were of high quality: the job club and job development and placement services were unusually well supported and creative; the basic education classes were more highly rated by participants than classes in the other programs in the NEWWS Evaluation; and staff had relatively more experience in employmentrelated positions.
It is possible that combining the income maintenance and the employment and training aspects of welfare recipients’ cases yields some positive results. Compared to traditional case management, which requires people to interact with two separate workers, integrated case management may allow staff to work sooner with recipients on employment preparation and minimize communication breakdowns between different groups of staff. Integrated case management has also been hypothesized to help change the "eligibilitycompliance culture" of the average welfare office to a "selfsufficiency culture" that focuses more on moving people into the labor market and off welfare than on validating individuals’ credentials to remain on welfare.9 In Portland, once people were active in the program, all of their interactions were with staff who dealt with employment and training issues (integrated case managers, supplementary case managers at the community colleges, and activity instructors). Various indicators suggest that most of these staff communicated a selfsufficiency message to clients. Furthermore, in surveys, an overwhelming majority of Portland staff (more than in the other 10 NEWWS Evaluation programs) expressed confidence that the program could help people become selfsupporting.10
For those entering the program without a high school diploma or GED, Portland’s program increased the receipt of a GED certificate and a trade certificate or license. Although no consistent relationship has been found between acquisition of a GED or trade certificate and impacts, some research suggests that it may contribute to a program’s success.
States must engage a large portion of the caseload in work or workrelated activities to meet the 1996 welfare law’s increased participationlevel targets and must reduce the number of people who return to the welfare rolls and eventually reach a time limit. Thus, the 1996 law increases the importance of providing adequate supports, including child care benefits, both for those participating in program services and for those working. The Portland program provided child care benefits for a large portion of the caseload, including parents of younger children. Although the resulting average child care costs were quite high, it is possible that the support enabled some program group members to participate, and to accept and keep jobs, who would not have been able to do so otherwise.
Typically, programs that produce immediate increases in employment do so by causing some welfare recipients to find jobs faster than they otherwise would have, but participants usually find the same kinds of lowwage jobs as control group members. Portland succeeded in moving many recipients quickly into jobs, as well as changing the types of jobs that they had at the end of the twoyear followup period. This may be a result of Portland’s simultaneous focus on moving people into the labor market and encouraging them to find "good" jobs (in the context of a strong labor market). Portland’s approach may serve as a model of how to rethink an employmentfocused model in the context of a strong labor market.
The 1996 welfare reform law includes an expectation that all TANF recipients work after receiving two years of federal assistance. Also, some states have imposed twoyear time limits on TANF receipt. Although Portland did not impose a work requirement or time limits during the period studied, the program findings can shed some light on how an employmentfocused, mixed services program might fare in the short term.
Portland’s substantial impacts remained at the end of the twoyear followup period: 46 percent of the program group worked for pay in the last quarter of followup compared to 35 percent of the control group, an 11 percentage point increase. Similarly, 33 percent of the program group worked for pay and did not receive any AFDC payments in the last quarter compared to 24 percent of the control group, a 9 percentage point increase.
However, despite the positive effects of the program, about twofifths of program group members were receiving AFDC at the end of the twoyear period; moreover, about onequarter were both receiving AFDC benefits and not working. Although these proportions are lower than in many previously studied programs, these results offer a caution as states strive to achieve very rapid selfsufficiency for virtually all welfare recipients.
The 1996 law specifies that eventually at least onehalf of all recipients of federal welfare benefits must be participating intensively (20 to 30 hours per week) in subsidized or unsubsidized work or in employmentrelated activities. Although Portland’s program was not operated under the new law’s rules or designed to meet its standards, the extent to which Portland imposed a welfare obligation on sample members can highlight the challenges of these participation standards.
Sample members in Portland were "covered" by the program — participating in an activity (for at least one hour, but usually much more), employed, or sanctioned for nonparticipation — in 38 percent of the followup months in which they were receiving AFDC and subject to a participation requirement. In contrast to the standards in the 1996 law, this calculation considers only those who were "mandatory" for program participation, not all AFDC recipients. Furthermore, it does not take into account the number of hours each week in which people were participating or employed, but simply counts people as fulfilling a welfare obligation if they were participating or employed at all, or sanctioned, at any point in a month.
"Coverage" rates in Atlanta, Grand Rapids, and Riverside were similar to or higher than the rate in Portland. However, extensive analysis of participation in these three sites showed that monthly participation rates calculated similarly to those contained in the 1996 welfare law probably would have been quite low.11 This suggests that monthly participation rates would also have been quite low in Portland by the new federal standard.
This report contains valuable information on an unusually effective welfaretowork program. Additional followup and information will offer a more complete picture of Portland’s effects. Future documents — as part of the full, sevensite evaluation — will provide up to five years of followup on the Portland sample members, analyze the program’s impacts on a wider array of outcomes and for more subgroups, examine the extent to which the program had "spillover" effects on sample members’ children, and compare the program’s fiveyear costs with the fiveyear benefits.
From Welfare to Work. Judith M. Gueron and Edward Pauly. 1991. New York: Russell Sage Foundation.
The Saturation Work Initiative Model in San Diego: A FiveYear Followup Study. Daniel Friedlander and Gayle Hamilton. 1993. New York: MDRC.
Early Lessons from Seven Sites. Gayle Hamilton and Thomas Brock. 1994. Washington, D.C.: U.S. Department of Health and Human Services and U.S. Department of Education.
Five Years After: The LongTerm Effects of WelfaretoWork Programs. Daniel Friedlander and Gary Burtless. 1995. New York: Russell Sage Foundation.
Adult Education for People on AFDC: A Synthesis of Research. U.S. Department of Education and U.S. Department of Health and Human Services. Prepared by Edward Pauly, MDRC. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.
Early Findings on Program Impacts in Three Sites. U.S. Department of Health and Human Services and U.S. Department of Education. Prepared by Stephen Freedman and Daniel Friedlander, MDRC. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.
How Well Are They Faring? AFDC Families with PreschoolAged Children in Atlanta at the Outset of the JOBS Evaluation. U.S. Department of Health and Human Services and U.S. Department of Education. Prepared by Kristin A. Moore, Martha J. Zaslow, Mary Jo Coiro, Suzanne M. Miller, Child Trends, Inc., and Ellen B. Magenheim, Swarthmore College. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.
Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in WelfaretoWork Programs. U.S. Department of Health and Human Services and U.S. Department of Education. Prepared by Gayle Hamilton, MDRC. 1995. Washington, D.C.: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.
Work First: How to Implement an EmploymentFocused Approach to Welfare Reform. Amy Brown, 1997. New York: MDRC.
Changing to a Work First Strategy: Lessons from Los Angeles County’s GAIN Program for Welfare Recipients. Evan Weissman. 1997. New York: MDRC.
Educating Welfare Recipients for Employment and Empowerment: Case Studies of Promising Programs. U.S. Department of Education and U.S. Department of Health and Human Services. Prepared by Janet Quint, MDRC. Forthcoming.
Evaluating Two WelfaretoWork Program Approaches: TwoYear Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites. U.S. Department of Health and Human Services and U.S. Department of Education. Prepared by Gayle Hamilton, Thomas Brock, Mary Farrell, Daniel Friedlander, and Kristen Harknett, MDRC. 1997. Washington, D.C.: U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation.
Return to ASPE Home Page or HHS Home Page