Implementation, Participation Patterns, Costs, and Two-Year Impacts of the Portland (Oregon) Welfare-to-Work Program: Executive Summary

Prepared for:
U.S. Department of Health and Human Services
Administration for Children and Families
Office of the Assistant Secretary for Planning and Evaluation

U.S. Department of Education
Office of the Under Secretary
Office of Vocational and Adult Education

Prepared by:
Susan Scrivener
Gayle Hamilton
Mary Farrell
Stephen Freedman
Daniel Friedlander
Marisa Mitchell
Jodi Nudelman
Christine Schwartz

Manpower Demonstration Research Corporation

Acknowledgments

This evaluation could not have taken place without the unwavering cooperation and commitment of the Oregon Adult and Family Services (AFS) and the Steps to Success staff.  The top management staff set the tone for this support:  Kevin Concannon and Gary Weeks, the present and past Directors of the Department of Human Resources (DHR); Sandie Hoback and Stephen Minnich, the present and past Administrators of the AFS Division of DHR, and Jerry Burns and Debbie White, the former Managers of the AFS JOBS Unit.  Their willingness to submit their welfare­to­work program to an ambitious, long­term study, with its elaborate research design and rigorous data requirements, merits special recognition.  Special thanks go also to Susan Blanche-Kappler, the site liaison with MDRC and the U.S. Department of Health and Human Services, who has overseen the implementation of the evaluation and has provided ongoing guidance and input.

A number of AFS management information systems personnel facilitated the access and utilization of state administrative and cost datasets:  Margaret Armontrout, Edward Buckner, Ward Kent, Larry Morris, and Bob Putman.

At the AFS District Two office, thanks are due to the current regional manager, Maureen Casterline, and her predecessor, Jean Stryker, as well as their staff for their support in implementing the evaluation research design:  Judith Brown, David Flock, Frank Gembinski, Erma Hepburn, Jean Pullen, and Marge Reinhart.

Many AFS branch office managers in Multnomah and Washington counties provided assistance over the course of the evaluation:  Pat Adair, Rod Brown, June Cook, Cathy Craner, Oren Cyphers, Hilda Davis, Bob Earnest, Carol Eckel, Dorothy Fuller, Angel Grogen, Veda Latin, Bruce Lowry, Linda Montgomery, Ann Pickar, Kei Quitevis, Will Reinhart, Pam Ruddell, C. L. Thames, and Roger Zwemke.

Thanks also go to Jodi Davich, Mardica Hicks, Maureen Judge-Morris, Nan Poppe, and Julie Wyckoff-Byers, the Steps to Success program managers at the Mt. Hood and Portland Community College Districts, as well as their staff for conducting the random assignment procedures and participating in a multitude of research activities.

The contribution of these people, plus many more too numerous to mention, made the evaluation in Oregon a success.

Executive Summary

Few domestic issues have generated as much attention over the past decade as welfare reform.  Persistent dissatisfaction with the Aid to Families with Dependent Children (AFDC) program — the nation’s principal safety net for poor families — spurred the enactment in August 1996 of the Personal Responsibility and Work Opportunity Reconciliation Act.¹  Among its provisions, the law replaced AFDC with a block grant program, Temporary Assistance for Needy Families (TANF), and created financial incentives for states to run mandatory, work­focused welfare­to­work programs.

Welfare­to­work programs provide services such as job search assistance, education, and training to help welfare recipients prepare for and find jobs.  States have run versions of these programs to serve part of the welfare caseload for the past three decades.  Various facets of the 1996 law, however, magnify the need for effective strategies to move people more quickly into jobs and off welfare.  First, states may not use federal funds to support most families for longer than five years and may impose even shorter time limits on assistance (they may also elect to use state funds to support families beyond five years).  Second, to prevent reductions in their block grants, states must meet demanding "participation standards" by engaging large proportions of TANF recipients in work or work­related activities.  To meet these standards, most states will have to engage a wider cross section of the caseload in work or program activities than they did previously.  Third, states’ TANF plans must include how the states will require recipients to work after two years of assistance.

As states and localities transform their welfare­to­work programs in response to the federal legislation, the need to learn about programs that have moved substantial numbers of people into work and off welfare increases.  The two­year findings presented in this report show the Portland, Oregon, welfare­to­work program run between early 1993 and mid 1996 to be among the most successful large­scale mandatory welfare­to­work programs studied, producing large increases in employment and earnings and equally large reductions in welfare receipt for a broad cross section of the welfare caseload.  The positive effects remained very strong at the end of the two­year period studied, and preliminary data suggest they will continue into the third year.

This report is the latest from an evaluation of mandatory welfare­to­work programs in seven sites called the National Evaluation of welfare­to­work Strategies (NEWWS Evaluation), conducted by the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. Department of Health and Human Services, with support from the U.S. Department of Education.  The report examines the mandatory welfare­to­work program run in Portland (Multnomah and Washington counties).  Through the program, Portland provided employment and support services to a broad cross section of the AFDC caseload, including parents with children as young as one year old.  These people were required to participate in program activities or face reductions in their welfare grants.  Although the program studied was designed and implemented prior to the 1996 reform, its overarching goal was similar to that of the new law:  to foster the self­sufficiency of adult recipients through increased employment and decreased welfare receipt.  (The program that Portland is running under the 1996 welfare reform law includes some key features of the program studied in this report.)²

This report describes the implementation, participation patterns, and cost of the Portland program, and presents estimates of the effects of the program on employment, earnings, and welfare receipt during the two years following people’s entry into the program.  To determine the effects of Portland’s program, 5,547 single­parent AFDC applicants and recipients aged 21 and over who attended a program orientation between February 1993 and December 1994 were randomly assigned to either a program group, eligible for program services and subject to participation requirements, or a control group, not eligible for services and not subject to participation requirements (although they could participate in other services in the community).  Because randomization makes the two groups similar at the start, any differences in average subsequent outcomes (such as two­year earnings) can be confidently attributed to the effects of the program.  These differences, known as program impacts, will be discussed later in the summary and are statistically significant unless otherwise noted.³

Overview of the Findings

The Portland program was run through a cooperative partnership between the welfare department and various local service providers.  Most program services were provided by the local community colleges and were of high quality.  The program was strongly employment­focused:  staff communicated that the primary program goal was to help people move into jobs, and job search was the most common activity.  However, in contrast to many employment­focused programs, participants were encouraged to look for and take "good" jobs — full­time, paying above the minimum wage, with benefits and potential for advancement.  Also, Portland’s program utilized a more mixed services strategy than is typically implemented by strongly employment­focused programs.  Staff assigned many people to short­term education, vocational training, work experience, and life skills training to improve their employability.  Some people were deferred from program participation, although the program did work with at least some people traditionally defined as the most disadvantaged portion of the caseload.  The per person cost of the program was moderate, relative to other welfare­to­work programs of the 1990s.

Follow­up of more than two years is needed to fully assess the success of a welfare­to­work program, but at the two­year mark Portland’s program produced effects (impacts) on employment, earnings, and welfare receipt that were among the largest ever found for large­scale mandatory programs.

• The Portland program substantially increased employment and produced unusually large increases in earnings.  The program raised employment levels by 11 percentage points over two years (relative to the control group).  More than one out of every four welfare recipients who normally would not have worked in an unsubsidized job during the two­year follow­up period did so as a result of the program.  In addition, two­year earnings were increased by over $1,800 per sample member, a 35 percent increase over the control group’s earnings.  These earnings gains are the largest found in the National Evaluation of welfare­to­work Strategies and approach the largest gains found for a large­scale mandatory program (those in the Riverside, California, GAIN program of the late 1980s).
• Unlike many programs that produce immediate impacts on employment and earnings, the Portland program increased job quality.  At the end of two years, the program increased the proportion of people working at full­time jobs by 13 percentage points and, among those employed (a nonexperimental comparison), increased average hourly pay by $0.86.  It increased the proportion of people with employer­provided health benefits by 10 percentage points.
• The program reduced welfare expenditures by 17 percent over the two­year follow­up period.  Relative to the average total welfare payments that people in the control group received over the two years, the program reduced per person expenditures by almost $1,200.  By the end of the follow­up period, only 41 percent of program group members were receiving welfare compared to 53 percent of control group members, a decrease of 12 percentage points.
• Portland’s impacts were widespread:  both recipients with relatively few barriers to employment and those typically considered very hard to place achieved employment and earnings gains and AFDC reductions.  Few other programs have attained such consistent impacts.  Employment, earnings, and AFDC impacts were produced for those who entered the program with a high school diploma or GED (high school equivalency certificate) and those who had neither credential, as well as for the "most disadvantaged" (sample members who entered without a high school diploma or GED, had not worked during the prior year, and had received AFDC for at least two years prior to program entry).
• Over the two­year follow­up period, program group members’ average combined income from earnings, AFDC, and Food Stamps was not substantially higher than that of control group members.  However, more positive results at the end of the follow­up period suggest that the program group may become financially better off in the future.  Program group members’ two­year earnings gains were largely offset by losses in AFDC and Food Stamps.  Quarterly impact trends suggest that income gains may emerge in the third year of follow­up.

Portland’s program was unusually successful in moving people into jobs, increasing their earnings, and moving them off welfare.  This success occurred in a specific context:  Portland’s caseload was predominantly white, minimizing the chances of racial discrimination in the labor market; a high percentage of the caseload entered the program with a high school diploma or GED certificate; and Portland’s economy was very strong during the study period, with low unemployment and substantial job growth.

Portland’s Evaluation Context

A.  Interpreting the Results

To illustrate the magnitude of Portland’s accomplishments, this report makes a number of comparisons.  The primary comparison — to show the net effects or impacts of the program — is between people in the program group and those in the control group.  In addition, Portland’s program is compared with other programs to show the relative effectiveness of Portland’s approach.  Specifically, this summary makes three types of explicit cross­program comparisons:

• Between Portland’s program and the three work first and three skills­building programs studied in Atlanta, Grand Rapids, and Riverside as part of the NEWWS Evaluation.  (See the accompanying text box for a brief description of the programs in the evaluation.)  Analyses similar to those reported here for Portland have been completed for these six programs; thus, rigorous comparisons can be made and Portland’s place on the work first/skills­building continuum can be estimated.
• Between Portland’s program and the other 10 programs in the NEWWS Evaluation (which include the three work first and three skills­building programs) on measures for which data have been collected for all programs.  This places Portland in the context of a wide range of welfare­to­work programs.⁴
• Between Portland’s program and the Greater Avenues for Independence (GAIN) program run in Riverside, California, in the late 1980s, which produced very large increases in employment and earnings, and large decreases in welfare receipt, and is often considered the benchmark for other programs.  (This program is distinct from the work first and skills­building programs run in Riverside for the NEWWS Evaluation discussed in the text box.)⁵

B.  Sample and Program Environment

The results presented in this report should be considered in the context of Portland’s research sample and program environment.  Compared to the samples in the other NEWWS Evaluation sites, the Portland sample had relatively high prior lifetime employment levels, but average prior recent employment (defined as work in the year before program entry).  More sample members in Portland than in the other sites had a high school diploma or GED at program entry, but the average highest grade completed in school was similar across all the sites (about 11th grade).  Portland’s sample fell in the middle of the range of prior AFDC receipt.  The sample in Portland was predominantly white, non­Hispanic, as is the general population in the Pacific Northwest, minimizing the chances of racial discrimination in the labor market.

AFDC grant levels in Oregon were significantly higher than the national average.  Portland’s labor market was strong, with relatively low unemployment rates, decreasing through most of the follow­up period, and high employment growth.  The caseload and labor market can affect the results of a welfare­to­work program in many ways.  Throughout the summary, these effects will be noted when relevant.

Findings on Program Implementation and Participation

• Portland’s program was designed and implemented through an unusually strong partnership between the welfare department and various local service providers, including the local community colleges, chambers of commerce, JTPA agencies, the state Employment Department, and others.  Program services were of high quality.

Although state legislators and administrators defined the basic parameters of Oregon’s welfare­to­work program, each district in the state was given the freedom to implement the program as it saw fit.  The Portland area welfare offices designed the program in cooperation with local community agencies; major budgeting and programming decisions were reached jointly. 

Job search, education, training, and work experience services were provided by the local community colleges (with job search provided by the chambers of commerce in one study county), under contract with the welfare agency, and case management services were provided by both the welfare agency and the community colleges.  Field researchers and program participants rated Portland’s program services highly compared to services in other programs.

• People active in the program were assigned to "integrated" case managers responsible for both welfare eligibility and employment, training, and social services.  The integrated staff were often supplemented by case managers employed by the community colleges.  Caseload sizes were at the low end of the 11 programs in the NEWWS Evaluation.

In Portland, integrated case managers, employed by the welfare department, were responsible for all aspects of program participants’ cases and had sole authority for all decisions, such as making activity referrals, authorizing support services, and imposing financial penalties for program noncompliance.  In many instances, case managers employed by the community colleges worked in cooperation with the integrated staff and helped recommend services, checked up on attendance, and provided quick intervention if problems in attendance or progress arose.

• Portland’s program was focused on employment; however, rather than urging people to take "any" jobs (the approach typical of most work first programs), staff encouraged people to seek and accept "good" jobs — full­time jobs paying more than minimum wage with benefits and potential for advancement.

The primary goal of Portland’s program was to move people into the labor market.  For applicants, the employment message was apparent even before they entered the program:  at AFDC application, people were assigned to an initial work search activity to be completed prior to the welfare­to­work program orientation, which took place about a month later.  (Since this initial activity occurred prior to random assignment, which took place in conjunction with orientation sessions, any effects it may have had are not reflected in the program group/control group differences discussed in this report.)  Among post­orientation activities, job search (including job club and other job search activities) was the one most commonly assigned and utilized.  Program staff told people in all activities, including education and training, that their goal should be to achieve self­sufficiency through paid work.

Job club, job development, and job placement activities in Portland were particularly well supported.  Full­time job developers used various methods to link participants to job opportunities, such as working closely with employers to discover unadvertised openings and sending them flyers "advertising" their linking service.  Other staff worked to connect program group members with existing leads through the state Employment Department.

Portland also had an individualized work experience component in which program staff custom­designed work experience positions based on people’s skills and interests.  Positions were in both nonprofit organizations and for­profit companies.  (Participation was voluntary in the for­profit work slots.)  Staff reported that many work experience positions led to unsubsidized jobs.

In Portland, "good" jobs, rather than "any" jobs, were considered the preferable path to self­sufficiency.  One standard used to measure the welfare district’s and service providers’ performance was an average wage­of­placement that was always much higher than the minimum wage (for example, in 1994 Oregon’s minimum wage was $4.75 and the average wage­of­placement target was $6.00).  Another standard concerned "AFDC recidivism" — the percentage of individuals who returned to the welfare rolls — and encouraged staff to promote jobs that were likely to last.  If a service provider’s performance was more than 20 percent below any of the standards in a quarter, provider and welfare staff were required to write a corrective plan describing how they would meet the standard in the future; ultimately the provider risked losing its contract with the welfare department.  (During the follow­up period for this report, standards­related issues did not lead to written corrective plans or loss of contracts by providers.)  Job developers actively sought positions that paid above the minimum wage and provided room for advancement.  Some staff did not discourage participants from accepting low­paying jobs, but usually only when there was potential for advancement.  It is important to note that "good" jobs would probably have been more difficult to develop and find in a poor economy, and a more disadvantaged caseload would have been less able to be selective when choosing work.

• The Portland program used a mixed services strategy:  most people participated in job search, but many also participated in short­term education, vocational training, work experience, and life skills training.

There was no single uniform path through the program.  Although Portland aimed to move people into the labor market, at one­on­one meetings directly following program orientation, case managers evaluated some people  — based on a confluence of factors including work history, educational status, and reading and math skills — as not ready to go immediately into job search.  During the period studied, about half of first activity assignments were to job search and half were to other activities (over time, an increasing proportion of people were first assigned to job search).  Some people who completed job search without finding work subsequently took part in education or training, and many people who first participated in a non­job search activity subsequently participated in job search (if they remained on the welfare rolls).  Activities were especially varied for those who entered the program without a high school diploma or GED certificate, with many participating in basic education.

The goal of education, training, and work experience activities was to prepare individuals relatively quickly for unsubsidized employment; thus, Portland staff encouraged short­term participation.  Program group members who took part in program activities participated for about five months over the two­year follow­up, similar to the average length of stay in the three work first programs in the NEWWS Evaluation and shorter than the average length of stay in the three skills­building programs.

• Many people were not assigned by a case manager to a program activity, although the program worked with at least some individuals traditionally defined as the most disadvantaged.

About one­third of program group members were not assigned to an activity in the three months following random assignment.  During this period, about half of these nonassigned individuals were "deferred" from participation by a case manager; most others became nonmandatory for the program because they left AFDC or experienced a status change such as pregnancy.  Most of those initially deferred were never assigned to an activity during the two­year follow­up (although case managers encouraged many of these people to seek services outside the program, such as remedial education or mental health counseling).

During field research, case managers reported that people with very low skills, serious physical or mental health problems, or exceptionally low motivation levels often were not referred to program activities.  However, as mentioned, Portland’s program produced employment and AFDC impacts for all subgroups of the sample, including the most disadvantaged portion of the caseload, defined using education, employment, and AFDC receipt indicators (as noted below, the program increased participation for this group).  Thus, while case managers were somewhat selective in whom they assigned to activities, they were, in fact, working with at least some of those who are traditionally viewed, according to objective measures, as the most disadvantaged portion of the caseload.

• Compared to what would have happened in the absence of a welfare­to­work program, Portland’s program dramatically increased participation in job search and, to a lesser extent, increased participation in basic education, vocational training or college, and work experience.

Control group members’ level of self­initiated activity represents what would have happened if program group members had had no exposure to Portland’s welfare­to­work program.  As Figure 1 shows, most control group participation was in basic education and vocational training or college.  Program group members were seven times more likely to engage in job search than their control group counterparts during the two­year follow­up period (for the program group, this "job search" measure included some participation in life skills training classes, which covered issues such as career exploration, résumé preparation, time management, and problem­solving).  Program group members were also more likely to participate in basic education, including GED preparation classes, vocational training or college, and work experience.

The program substantially increased participation in activities for a broad cross section of the caseload — both those with a high school diploma or GED ("graduates") and those without ("nongraduates"), and the most disadvantaged.

• Overall, Portland’s program was strongly mandatory.  Sanctioning rates were high compared to rates in programs run in the 1980s, but fall in the middle of the range of rates for the six 1990s work first and skills­building programs in Atlanta, Grand Rapids, and Riverside studied as part of the NEWWS Evaluation.

Staff closely monitored those assigned to or engaged in program activities.  Most Portland staff tried to encourage participation and compliance with program requirements using positive encouragement, emphasizing the potential benefits for the individual and her family.  If, after lengthy cajoling, people did not eventually comply, staff imposed financial sanctions (AFDC grant reductions).  Twenty­one percent of program group members were sanctioned during the two­year follow­up period and the average sanction lasted about five months.  In the six programs run in Atlanta, Grand Rapids, and Riverside, sanction rates ranged from 9 to 42 percent and sanctions lasted for an average of eight months.

Findings on Program Costs

• Excluding spending that would have occurred without any special welfare­to­work program, the two­year net per person cost of Portland’s program was $2,017.  This net cost is higher than the average net cost for the three work first programs studied as part of the NEWWS Evaluation ($1,550) and much lower than the average net cost for the three skills­building programs ($3,077).

The costs estimated in this report consist of all costs associated with providing employment services and related support services to sample members; costs associated with authorizing and processing welfare payments are not included.  The gross cost per program group member during the two­year follow­up period consists of costs paid by the welfare department and non­welfare agencies while sample members were enrolled in Portland’s program as well as for employment and support services after they exited the program and, in some cases, left AFDC.  The two­year gross cost per Portland program group member was $4,027.  The welfare department paid about two­thirds of this cost; the remainder was paid by schools and other agencies.  The proportion of costs covered by the welfare department is high compared to the other programs studied in the NEWWS Evaluation.  The welfare department in Portland contracted out, and thus paid for, most of the program services, whereas most programs rely more on noncontracted outside agencies.

The net cost per program group member during the two­year follow­up period is the gross cost per program group member minus what would have been spent in the absence of a mandatory welfare­to­work program, as measured by the cost per control group member.  Control group members were not eligible to take part in program activities, but could enroll on their own in other employment­related activities in the community and were eligible for activity­ and employment­related welfare department support services.  Thus, control group costs include expenditures for all of the nonprogram activities and support services utilized by control group members during the two­year follow­up period.  As Table 1 shows, Portland’s two­year net cost per program group member was $2,017.  This net cost is higher than the average net cost per work first sample member and much lower than the average net cost per skills­building sample member.  (A five­year benefit­cost analysis will eventually be completed as part of the NEWWS Evaluation.)

• Compared to the six programs studied in Atlanta, Grand Rapids, and Riverside, the cost of support services in Portland was very high because of high monthly child care payments and extensive use of child care benefits both while people were participating in program activities and during employment.  For parents with younger children, these factors were even more pronounced.

Support services accounted for 38 percent of Portland’s gross costs (see Table 1).  A large proportion of the support service costs — $1,422 per program group member — was for child care.  Total child care costs in the six work first and skills­building programs ranged from $88 to $794 per person.  In Portland, 50 percent of program group members received payments for child care services while they were enrolled in the program, compared to 17 to 31 percent in the six work first and skills­building programs.  Twenty­six percent in Portland received employment­related child care payments (including transitional child care, provided for up to one year for those who left AFDC for work, and Employment­Related Day Care, provided with no time limit to those not eligible for transitional care) compared to no more than 7 percent in any of the other programs studied.

The high rates of child care use for both program participation and employment in Portland probably reflect the strong emphasis that staff placed on securing child care arrangements and having a backup option.  The high rates of employment­related child care use also reflect Portland’s large impact on employment.  Furthermore, it is possible that Portland’s integrated case managers were more likely than the traditional case managers in the other sites to know the two facts that qualify an individual for transitional benefits:  namely, that the individual has received AFDC for at least three of the past six months and is leaving AFDC for employment.  Staff who perform a dual role may also be more inclined to authorize benefits automatically, without a special request from a parent or other staff member.

Child care costs in Portland were particularly high for parents with younger children.  As mentioned, Portland required parents with children as young as age 1 to participate in the program.  The child care costs for those entering the program with a child under age 3 were more than twice the costs for parents entering the program with no children under age 3 ($2,144 compared to $936).  Parents with younger children at program entry had higher average monthly child care payments and higher rates of child care receipt and received payments for a longer time than those without young children.  It is interesting to note that although Grand Rapids also required participation from parents with children as young as age 1, per person child care costs in the Grand Rapids work first and skills­building programs ($367 and $542, respectively) were much lower than in Portland, owing to lower monthly payments and much lower rates of child care use.

Findings on Program Impacts

A.  Impacts on Receipt of Education or Training Credentials

• For those entering the program without a high school diploma or GED certificate, the program substantially increased the number who obtained a GED certificate or a trade license or certificate.  The increase in GED receipt rivals increases found for skills­building programs offering extensive, long­term education; the increase in trade license receipt exceeds that of any similar program studied.

Of individuals who entered Portland’s program without a high school diploma or GED certificate ("nongraduates"), 16 percent of program group members received a GED during the two years following random assignment compared to 5 percent of the control group, an 11 percentage point increase.  As mentioned earlier, many nongraduates attended basic education classes as part of the program.  It should be noted, however, that case managers tended not to refer people with very low skills levels to basic education class.  The program also increased the receipt of a trade license or certificate for nongraduates by 12 percentage points.

B.  Impacts on Employment and Earnings

• Over two years, Portland’s program substantially increased employment and produced unusually large increases in earnings relative to what would have happened in the absence of a mandatory welfare­to­work program.

As shown in Table 2, over two years of follow­up, 72 percent of program group members were employed compared to 61 percent of control group members, a difference of 11 percentage points.  This impact represents the effect of the program in promoting paid work among sample members who would not have worked on their own.  Said another way, more than one out of every four welfare recipients who normally would not have worked in an unsubsidized job during the two­year follow­up period did so as a result of the program.  The employment gains persisted through the follow­up period:  in the last quarter of the follow­up period, 46 percent of program group members worked for pay compared to 35 percent of control group members, an increase of 11 percentage points.

Program group members earned, on average, $7,133 over the two­year period, and control group members earned $5,291.  (These averages include those who did and did not work during the follow­up period.)  This difference of $1,842, a 35 percent increase above the control group average, is the largest earnings impact measured among the programs studied as part of the NEWWS Evaluation (two­year impacts have been found in 8 of the other 10 programs and range from $367 to $1,276), and it approaches the magnitude of the largest earnings impact attained by a large­scale mandatory welfare­to­work initiative (the Riverside GAIN program of the late 1980s increased two­year earnings by $2,103, or 56 percent).

In Portland, just over one­half of the earnings impact resulted from job finding (increases in employment levels), about one­quarter came from increased employment duration (program group members finding jobs sooner and/or finding jobs that last longer than do control group members), and one­seventh resulted from increased earnings on the job (higher hourly wages, more hours of work per week, and/or more weeks of work in a quarter).⁶  In other words, better job quality (the combination of longer employment duration and higher earnings on the job) accounted for nearly 40 percent of the total impact on earnings.  This result was relatively uncommon for welfare­to­work programs of the 1980s and has not been found for all employment­focused programs of the 1990s.

• The earnings gains began immediately after program entry and remained strong throughout the follow­up period.

The quarterly average earnings for the program and control groups are plotted in the upper panel of Figure 2.  The strong early impact is similar to that found for the three work first programs studied as part of the NEWWS Evaluation.  However, in contrast to the work first programs in Atlanta, Grand Rapids, and Riverside, the earnings gains in Portland did not diminish during the follow­up period (the impacts in the work first programs peaked between one and one and a half years after random assignment, and then grew smaller as increasing numbers of control group members got jobs).  In Portland, the quarterly impact increased throughout most of the two years.  Program group members earned an average of $1,155 in quarter 9, $309 (37 percent) more than the control group.

• Program group members in Portland got better jobs, on average, than control group members:  the program substantially increased full­time employment and average hourly pay and increased the proportion of people with employer­provided health insurance.

Survey results provide further evidence that the program increased job quality.  At the end of two years, 40 percent of program group members were working at full­time jobs (providing at least 30 hours of work) compared to 27 percent of control group members, an increase of 13 percentage points.  Twenty­four percent of program group members had employer­provided health benefits at the end of two years compared to 14 percent of control group members, a 10 percentage point increase.  (These results include zeros for those not working at the end of two years.)

Portland’s program achieved these gains by helping more people find work, but also by helping program group members find jobs at higher pay and with more benefits than control group members found on their own initiative.  Program group members employed at the end of two years averaged $7.34 per hour in wages, $0.86 above the average wage levels earned by employed control group members — the biggest gain of any program in the NEWWS Evaluation.  Employed program group members were also more likely to be working full time and receiving health benefits from their employers.  These results suggest that the program’s emphasis on waiting for "good" jobs paid off.

C.  Impacts on AFDC Receipt and Payments

• The program produced substantial decreases in AFDC receipt and payments over the two­year follow­up period.  AFDC savings occurred immediately after program entry and continued throughout the follow­up period.

As shown in Table 2, over the two­year follow­up period, the program group received $1,196 less in AFDC payments than the control group, a 17 percent reduction.  This impact on AFDC is among the largest found in the 11 NEWWS Evaluation programs and is similar in magnitude to the decrease produced by the 1980s Riverside GAIN program ($1,399, or 14 percent).  Portland’s AFDC reduction resulted almost exclusively from program group members receiving AFDC for fewer months than control group members, as opposed to lower monthly grants for those on assistance.

As the lower panel of Figure 2 illustrates, the impact on AFDC payments remained large throughout the follow­up period.  By the end of the follow­up period, only 41 percent of program group members were receiving welfare compared to 53 percent of control group members, a decrease of 12 percentage points.

D.  Impacts for Subgroups and Three­Year Impacts for an Early Cohort

• Portland’s program produced substantial impacts for a broad cross section of the caseload, both recipients with relatively few barriers to employment and those typically considered very hard to place.  Few programs have attained the level of consistency of impacts attained by the Portland program.

Table 3 shows cumulative two­year earnings and AFDC payment information for high school graduates and nongraduates, as well as the most disadvantaged sample members — as mentioned earlier, a subgroup of nongraduates who did not work for pay in the year prior to program entry and had received AFDC for more than two years prior to program entry.  Earnings and AFDC payments for control group members reveal how each subgroup would have fared without the program intervention.  Graduates in the control group had the highest average earnings and the lowest average AFDC payments, while the most disadvantaged control group members had the lowest earnings and highest AFDC payments.  Portland’s program substantially increased earnings and decreased AFDC payments for all three groups, and in each case earnings gains were larger than AFDC losses.⁷

Earnings and AFDC impacts were produced for all three subgroups during the first year of follow­up and remained strong through the end of the second year (quarterly trends by subgroup are not illustrated here).  However, impacts for graduates were most immediate, achieving statistical significance in the second quarter of follow­up.  Impacts for nongraduates and the most disadvantaged first became statistically significant, in the third and fourth quarters, respectively.  These differences likely reflect the fact that a higher proportion of graduates were first assigned to job search, which has been found to produce more immediate impacts than other activities.  Also, it is typically easier for graduates than nongraduates to find jobs (especially "good" jobs).

• Impact results for an early cohort, for whom more follow­up is available, suggest that the strong employment, earnings, and AFDC impacts will persist and possibly grow during the third year of follow­up.

Three­year employment, earnings, and AFDC payment data are available for individuals who entered the program during the first year of random assignment.  For this early cohort, the program group earned an average of $1,217 more than the control group in the second year of follow­up and $1,402 more in the third year; the early cohort program group received an average of $737 less in AFDC payments than the control group in the second year of follow­up and $756 less in the third year.

E.  Impacts on Measured Income

• Measured over the two­year follow­up period, average combined income from earnings, AFDC, and Food Stamps was not substantially higher for program group members than for control group members.  However, more positive results at the end of the follow­up period suggest that the program group may become financially better off in the future.

Over the two­year follow­up period, although earnings gains ($1,842) exceeded AFDC losses ($1,196) by $646, program group members also lost $456 in Food Stamp benefits.  The average two­year combined income from earnings, AFDC, and Food Stamp benefits (calculated using administrative records) for program group members was $16,886 compared to a combined income for control group members of $16,696.  The resulting net gain of $191 (a 1 percent increase) is not statistically significant.

In the second year of follow­up, the program moved a small portion of program group members out of poverty (measured using combined income, as defined above).  Twenty­one percent of program group members had a combined income at or above the federal poverty level compared to 17 percent of control group members.  However, the program also moved a small portion of program group members below 50 percent of the poverty level (33 percent of program group members had a combined income below 50 percent of the poverty level compared to 31 percent of control group members).

Program group/control group differences in combined income grew larger over time and first achieved statistical significance at the end of two years; in the last quarter of follow­up, program group members’ average combined income from earnings, AFDC, and Food Stamps totaled $2,051, $80 higher than the average combined income for control group members.  The analysis also draws on client survey data that include a much wider range of sources for estimating income, including regular or odd jobs, AFDC, Food Stamps, Supplemental Security Income, unemployment insurance, and child support (not including Earned Income Tax Credit [EITC] receipt or other tax credits available to low­income families in Oregon).  These data show that average individual and household income in the last month of follow­up was higher for program group members than for control group members, although the increases are not statistically significant.  Similar impacts were found when average EITC receipt and out­of­pocket child care payments were included to estimate total net individual and household income.

The increase in income measured at the end of the two years suggests that program group income may be higher than control group income in the third year of follow­up.  Three­year findings for the early cohort offer additional evidence:  for the early cohort, the impact on combined income from earnings, AFDC, and Food Stamps (calculated using administrative records) in the third year of follow­up ($369) surpassed that of the second year ($215 and not statistically significant).

F.  Impacts on Total Health Insurance Coverage

• The program’s success in moving large numbers of people off welfare and into the labor market may have had an unintended negative consequence:  although the program increased the proportion of people with employer­provided health insurance at the end of the two­year follow­up period, some people who left AFDC (and automatic Medicaid coverage) for work did not find alternative sources of health insurance.  Overall, program group members and their children incurred a small, not statistically significant, reduction in health insurance coverage.

At random assignment, all program and control group members had health care coverage because they were receiving AFDC and thus were automatically covered under Medicaid.  Two years after program entry, coverage rates had decreased as some people had left welfare and did not replace their Medicaid coverage with coverage from employers or other sources.  At the end of the follow­up period, 81 percent of program group members and their children had health care coverage from some source compared to 86 percent of control group members and their children.  This 5 percentage point difference is not statistically significant, but is just beyond the statistical significance benchmark applied to the other impacts in this report.

The decrease was generated by those who left AFDC (and automatic Medicaid coverage) for work.  As mentioned above, the program increased the proportion of people with employer­provided health coverage, but some people in the program group moved into jobs that did not provide health insurance.  Some received coverage from another source, such as Transitional Medicaid or the Oregon Health Plan (the statewide initiative that offers health care coverage for low­income families).  However, some people found jobs that did not provide health insurance and did not obtain alternative coverage.

Discussion and Implications of the Findings

A.  Comparisons Between the Portland and Riverside GAIN Programs

• Portland’s program shared two key features with the successful GAIN program run in Riverside, California, in the late 1980s.  Both were employment­focused, mixed services programs; however, Portland emphasized job quality, whereas Riverside encouraged participants to take any job.

The Riverside GAIN (Greater Avenues for Independence) program of the late 1980s produced large employment and earnings gains and welfare reductions and is often considered the benchmark for other welfare­to­work programs.  Portland’s two­year results are similar in magnitude to those in Riverside; however, the people served and the local environments differed in a number of ways.  A higher proportion of Portland’s sample had a high school diploma or GED at program entry and were white (and thus likely faced less racial discrimination in the labor market).  Portland mandated participation for parents with children as young as age 1; 68 percent of the sample had children under age 6.  Riverside mandated participation only for parents with children aged 6 or over.  As mentioned earlier, Portland’s labor market was strong, with low and decreasing unemployment and steadily growing employment.  Riverside’s employment growth was high, but unemployment rates were high and increased throughout the follow­up period.

Despite these differences, impacts were quite similar.  Both programs substantially increased employment levels and produced the largest earnings gains ever found for mandatory welfare­to­work programs.  The Portland program increased two­year earnings by $1,842, a 35 percent increase, and the Riverside GAIN program increased two­year earnings by $2,103, a 56 percent increase.  Both programs had large impacts on AFDC payments as well:  the Portland program reduced payments by 17 percent over two years ($1,196), and Riverside GAIN reduced payments by 14 percent ($1,399).⁸  In both programs, earnings gains, in dollars, were about one and a half times the amount of AFDC reductions.  Both programs produced impacts for a wide range of subgroups, including the more disadvantaged members of the caseload.

The Portland and Riverside GAIN programs both communicated a strong and pervasive message that employment was the chief program goal.  Both programs enforced this message through extensive use of job search activities and through the use of job developers to assist recipients in gaining access to job opportunities.  Although job search was the most common activity in both programs, many people participated in other activities, including basic education, and, in Portland, vocational training, work experience, and life skills training.  Overall participation patterns were similar for the two programs.  About three­fifths of the program group in each site ever participated in a program activity.  Both programs substantially increased participation in job search and basic education (compared to the activity levels of control group members), and Portland’s program also increased participation in work experience.  Only Portland increased the proportion of people who received a GED certificate or a trade license or certificate.

The programs communicated differing levels of job selectivity.  As previously discussed, Portland staff encouraged participants to wait for "good" jobs.  In sharp contrast, staff in the Riverside GAIN program consistently communicated that any job was worthwhile; they encouraged quick entry into the labor market and communicated that low­paying or part­time jobs should not be turned down.  This difference was both reflected in and fostered by the ways each program used performance standards.  Portland’s standards encouraged placing participants in jobs paying higher than minimum wage that were likely to last; in contrast, Riverside’s individual monthly job placement standards for case managers reflected and emphasized the importance of quickly placing people in any job.

These divergent messages likely contributed to the programs’ somewhat different composition of the earnings impacts.  Earnings impacts in both programs resulted from increases in the proportion of individuals employed and the number of quarters that individuals were employed.  Portland’s impacts, however, were also due to program group members earning more on the job than their control group counterparts.  Survey results from the two programs corroborate this difference:  according to client responses, if one considers only those who were working at the end of the follow­up period, Portland’s program substantially increased average hourly pay, but Riverside program group members earned slightly less per hour than their control group counterparts.

B.  Factors Influencing Portland’s Impacts

Research has shown that a variety of program approaches can produce positive effects.  While it is difficult to distinguish which of many program features and environmental factors may have fueled Portland’s unusual success, it is likely to be due to a combination of factors.  Portland’s program may have worked particularly well in the specific environment in which it was implemented:  Portland’s caseload was not as disadvantaged as some of the populations in the other six NEWWS Evaluation sites, and the labor market in the Portland area during the follow­up period was strong, allowing people to find and obtain "good" jobs.  Furthermore, Oregon has a history of progressive social legislation, including generously funded welfare programs and a state minimum wage that is higher than the federal standard.  Although this study cannot prove the causality of any program feature or set of features, the report suggests a number of influential factors.

• Three key features of the Portland program have previously been found to be associated with successful welfare­to­work programs:  a strong employment focus, a mixed services strategy, and close monitoring of participation in mandated activities.

Portland’s strong employment message and extensive use of job search, coupled with a willingness to impose sanctions for program noncompliance, affected how much and how seriously people looked for a job.  Portland also offered strong job development and placement services, which provided a direct link to employers and job openings not otherwise available or apparent.  Although, in previous studies, increasing participation in job search alone has not been found to be consistently related to large impacts, this combination of message, willingness to impose sanctions, and program services (similar to the combination in Riverside GAIN) probably increased participants’ incentive and opportunity to get a job, and thus increased job finding.

Programs employing a mixed services strategy upon program entry (again, such as Riverside GAIN) also have been found to be more effective than programs that offer only job search or programs that rely mainly on long­term skills­building education and training services; the findings for Portland corroborate this.  The non­job search services, designed to improve employability, were targeted, and participants were not allowed to "languish" in activities, but were encouraged to complete activities and then look for work.

Staff closely monitored people in program activities and followed up quickly when attendance problems arose.  Staff communicated the importance of participating in activities, emphasizing the potential benefits for recipients and their families.  When people did not eventually comply with program requirements, staff imposed financial sanctions.  (However, the sanction rate in Portland was substantially lower than the rate in some other programs.)

• Other important aspects of the Portland program included a strong partnership with community agencies, high­quality services, and an integrated case management structure; the program also increased receipt of GEDs and trade certificates.  Less is currently known, however, about the link between these factors and welfare­to­work program impacts.

As discussed earlier, Portland’s program was designed and implemented through a strong partnership between the welfare agency and various contractor service agencies.  Typically, welfare departments that contract out various portions of their welfare­to­work programs do so after the program model is chosen and planned; the contractors simply implement the program as the welfare department envisioned.  Portland’s arrangement allowed the contractors to participate in defining the program from the ground up, which fostered a sense of "ownership" and a strong commitment to the program in the contractor agencies.

Overall, Portland’s services were of high quality:  the job club and job development and placement services were unusually well supported and creative; the basic education classes were more highly rated by participants than classes in the other programs in the NEWWS Evaluation; and staff had relatively more experience in employment­related positions.

It is possible that combining the income maintenance and the employment and training aspects of welfare recipients’ cases yields some positive results.  Compared to traditional case management, which requires people to interact with two separate workers, integrated case management may allow staff to work sooner with recipients on employment preparation and minimize communication breakdowns between different groups of staff.  Integrated case management has also been hypothesized to help change the "eligibility­compliance culture" of the average welfare office to a "self­sufficiency culture" that focuses more on moving people into the labor market and off welfare than on validating individuals’ credentials to remain on welfare.⁹  In Portland, once people were active in the program, all of their interactions were with staff who dealt with employment and training issues (integrated case managers, supplementary case managers at the community colleges, and activity instructors).  Various indicators suggest that most of these staff communicated a self­sufficiency message to clients.  Furthermore, in surveys, an overwhelming majority of Portland staff (more than in the other 10 NEWWS Evaluation programs) expressed confidence that the program could help people become self­supporting.¹⁰

For those entering the program without a high school diploma or GED, Portland’s program increased the receipt of a GED certificate and a trade certificate or license.  Although no consistent relationship has been found between acquisition of a GED or trade certificate and impacts, some research suggests that it may contribute to a program’s success.

C.  Issues of Particular Relevance in the Current Welfare Environment

• Portland’s program offered supports for those participating in activities and those working, including parents with young children.  The extensive use of child care benefits resulted in a substantial cost, but also may have enabled parents to move from welfare to work.

States must engage a large portion of the caseload in work or work­related activities to meet the 1996 welfare law’s increased participation­level targets and must reduce the number of people who return to the welfare rolls and eventually reach a time limit.  Thus, the 1996 law increases the importance of providing adequate supports, including child care benefits, both for those participating in program services and for those working.  The Portland program provided child care benefits for a large portion of the caseload, including parents of younger children.  Although the resulting average child care costs were quite high, it is possible that the support enabled some program group members to participate, and to accept and keep jobs, who would not have been able to do so otherwise.

• Given the 1996 welfare law’s focus on increasing work and promoting self­sufficiency, it is notable that the Portland program both produced immediate impacts on employment and earnings and increased full­time employment and hourly wages.

Typically, programs that produce immediate increases in employment do so by causing some welfare recipients to find jobs faster than they otherwise would have, but participants usually find the same kinds of low­wage jobs as control group members.  Portland succeeded in moving many recipients quickly into jobs, as well as changing the types of jobs that they had at the end of the two­year follow­up period.  This may be a result of Portland’s simultaneous focus on moving people into the labor market and encouraging them to find "good" jobs (in the context of a strong labor market).  Portland’s approach may serve as a model of how to rethink an employment­focused model in the context of a strong labor market.

• While Portland’s program was very successful in reducing joblessness and decreasing the proportion of people on AFDC, many people were receiving AFDC benefits and not employed at the end of two years.

The 1996 welfare reform law includes an expectation that all TANF recipients work after receiving two years of federal assistance.  Also, some states have imposed two­year time limits on TANF receipt.  Although Portland did not impose a work requirement or time limits during the period studied, the program findings can shed some light on how an employment­focused, mixed services program might fare in the short term.

Portland’s substantial impacts remained at the end of the two­year follow­up period:  46 percent of the program group worked for pay in the last quarter of follow­up compared to 35 percent of the control group, an 11 percentage point increase.  Similarly, 33 percent of the program group worked for pay and did not receive any AFDC payments in the last quarter compared to 24 percent of the control group, a 9 percentage point increase.

However, despite the positive effects of the program, about two­fifths of program group members were receiving AFDC at the end of the two­year period; moreover, about one­quarter were both receiving AFDC benefits and not working.  Although these proportions are lower than in many previously studied programs, these results offer a caution as states strive to achieve very rapid self­sufficiency for virtually all welfare recipients.

• Like the work first and skills­building programs studied as part of the NEWWS Evaluation, Portland probably would have failed to meet the participation rates called for in the 1996 welfare law, even though the program achieved many of the law’s aims:  it engaged many people in employment­related activities or imposed financial sanctions on them, increased the number of people who worked during the follow­up period, and decreased welfare expenditures.

The 1996 law specifies that eventually at least one­half of all recipients of federal welfare benefits must be participating intensively (20 to 30 hours per week) in subsidized or unsubsidized work or in employment­related activities.  Although Portland’s program was not operated under the new law’s rules or designed to meet its standards, the extent to which Portland imposed a welfare obligation on sample members can highlight the challenges of these participation standards.

Sample members in Portland were "covered" by the program — participating in an activity (for at least one hour, but usually much more), employed, or sanctioned for nonparticipation — in 38 percent of the follow­up months in which they were receiving AFDC and subject to a participation requirement.  In contrast to the standards in the 1996 law, this calculation considers only those who were "mandatory" for program participation, not all AFDC recipients.  Furthermore, it does not take into account the number of hours each week in which people were participating or employed, but simply counts people as fulfilling a welfare obligation if they were participating or employed at all, or sanctioned, at any point in a month.

"Coverage" rates in Atlanta, Grand Rapids, and Riverside were similar to or higher than the rate in Portland.  However, extensive analysis of participation in these three sites showed that monthly participation rates calculated similarly to those contained in the 1996 welfare law probably would have been quite low.¹¹  This suggests that monthly participation rates would also have been quite low in Portland by the new federal standard.

Future Research

This report contains valuable information on an unusually effective welfare­to­work program.  Additional follow­up and information will offer a more complete picture of Portland’s effects.  Future documents — as part of the full, seven­site evaluation — will provide up to five years of follow­up on the Portland sample members, analyze the program’s impacts on a wider array of outcomes and for more subgroups, examine the extent to which the program had "spillover" effects on sample members’ children, and compare the program’s five­year costs with the five­year benefits.

Notes

1. Pub. L. No. 104-193.
2. In July 1996, through waivers of the law in effect prior to the 1996 reform, Oregon expanded the participation mandate to parents with children as young as 90 days old; expanded the definition of program participation to include some nontraditional activities, such as mental health counseling; and increased the ultimate penalty for noncompliance with the program to closure of a family’s grant.  The program continued to provide the wide array of employment­related activities discussed in this report.  The 1996 reform law allows states to continue any waiver plans in effect on the day the act became law and remain exempt from any provisions of the law inconsistent with that waiver until the end of the waiver period; Oregon chose to do this.  While most of the data for this report cover early 1993 through mid 1996, the report includes impact data through the end of 1996 for some sample members.
3. Statistical significance indicates the probability that the program actually produced the observed difference.
4. For two­year results for the work first and skills­building programs, see Gayle Hamilton, Thomas Brock, Mary Farrell, Daniel Friedlander, and Kristen Harknett, Evaluating Two Welfare­to­Work Program Approaches:  Two­Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites (Washington, DC:  U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation, 1997).  For a description of all 11 programs, see Gayle Hamilton and Thomas Brock, The JOBS Evaluation:  Early Lessons from Seven Sites (Washington, DC:  U.S. Department of Health and Human Services and U.S. Department of Education, 1994).  The two­year impacts for all programs will appear in a forthcoming report.
5. For presentation of findings, see James Riccio, Daniel Friedlander, and Stephen Freedman, GAIN:  Benefits, Costs, and Three­Year Impacts of a Welfare­to­Work Program (New York:  MDRC, 1994); and Stephen Freedman, Daniel Friedlander, Winston Lin, and Amanda Schweder, Five­Year Impacts on Employment, Earnings, and AFDC Receipt (New York:  MDRC, 1996).
6. The sum of the three contributions does not equal 100 percent because a small portion of the earnings impact is attributable to interactions among the components.
7. Analysis not presented in this report shows that substantial impacts were also found for subgroups with and without young children (those with children age 2 or under, 3 to 5, and 6 or over).
8. Owing to inflation and differing costs­of­living, a dollar change had a different value in the two sites.
9. Mary Jo Bane and David T. Ellwood, Welfare Realities:  From Rhetoric to Reform (Cambridge:  Harvard University Press, 1994).
10. Two­year results from Columbus — the site in the NEWWS Evaluation that tested integrated and traditional case management side by side — indicate that recipients with integrated case managers had significantly higher rates of participation in program activities and significantly lower welfare payments than recipients with traditional case managers.  However, there was no difference between the two groups’ employment rates and earnings.  See Thomas Brock and Kristen Harknett, Welfare­to­Work Case Management:  A Comparison of Two Models (New York:  MDRC, 1998).
11. Many AFDC recipients in the three sites did not participate or work for 20 hours in every week of a month for a variety of reasons.  For example, they were waiting for an activity to begin, were having child care or transportation problems, or were sick or had a sick family member.  In some instances, staff had temporarily "lost track" of them.

Other Publications from This Evaluation

From Welfare to Work.  Judith M. Gueron and Edward Pauly.  1991.  New York:  Russell Sage Foundation.

The Saturation Work Initiative Model in San Diego:  A Five­Year Follow­up Study.  Daniel Friedlander and Gayle Hamilton.  1993.  New York:  MDRC.

Early Lessons from Seven Sites.  Gayle Hamilton and Thomas Brock.  1994.  Washington, D.C.:  U.S. Department of Health and Human Services and U.S. Department of Education.

Five Years After:  The Long­Term Effects of Welfare­to­Work Programs.  Daniel Friedlander and Gary Burtless.  1995.  New York:  Russell Sage Foundation.

Adult Education for People on AFDC:  A Synthesis of Research.  U.S. Department of Education and U.S. Department of Health and Human Services.  Prepared by Edward Pauly, MDRC.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.

Early Findings on Program Impacts in Three Sites.  U.S. Department of Health and Human Services and U.S. Department of Education.  Prepared by Stephen Freedman and Daniel Friedlander, MDRC.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.

How Well Are They Faring?  AFDC Families with Preschool­Aged Children in Atlanta at the Outset of the JOBS Evaluation.  U.S. Department of Health and Human Services and U.S. Department of Education.  Prepared by Kristin A. Moore, Martha J. Zaslow, Mary Jo Coiro, Suzanne M. Miller, Child Trends, Inc., and Ellen B. Magenheim, Swarthmore College.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.

Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in Welfare­to­Work Programs.  U.S. Department of Health and Human Services and U.S. Department of Education.  Prepared by Gayle Hamilton, MDRC.  1995.  Washington, D.C.:  U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.

Work First:  How to Implement an Employment­Focused Approach to Welfare Reform.  Amy Brown, 1997.  New York:  MDRC.

Changing to a Work First Strategy:  Lessons from Los Angeles County’s GAIN Program for Welfare Recipients.  Evan Weissman.  1997.  New York:  MDRC.

Educating Welfare Recipients for Employment and Empowerment:  Case Studies of Promising Programs.  U.S. Department of Education and U.S. Department of Health and Human Services.  Prepared by Janet Quint, MDRC.  Forthcoming.

Evaluating Two Welfare­to­Work Program Approaches:  Two­Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites.  U.S. Department of Health and Human Services and U.S. Department of Education.  Prepared by Gayle Hamilton, Thomas Brock, Mary Farrell, Daniel Friedlander, and Kristen Harknett, MDRC.  1997.  Washington, D.C.:  U.S. Department of Health and Human Services, Administration for Children and Families and Office of the Assistant Secretary for Planning and Evaluation.