Targeted Help for the Hard-to-Employ: Outcomes of Two Philadelphia Welfare-to-Work Programs

Michelle VanNoy and Irma Perez-Johnson

Submitted to:
U.S. Department of Health and Human Services
Office of the Assistant Secretary for Planning and Evaluation

Project Officer: Alana Landey

Submitted by:
Mathematica Policy Research, Inc.
Project Director: Alan Hershey

Contract No.: 100-98-0009
MPR Reference No.: 8550-097

"

Acknowledgements

Mathematica Policy Research, Inc. (MPR) conducted the Philadelphia outcomes report as part of the congressionally mandated evaluation of the National Welfare-to-Work (WtW) grants program. The overall study is being conducted by MPR and its subcontractors-the Urban Institute (UI) and Support Services International, Inc. (SSI)-under a contract from the U.S. Department of Health and Human Services (DHHS). It would not have been possible to complete this report without the hard work of many people at MPR and other organizations, and the authors wish to acknowledge their contributions here.

We could not have completed the study without the ongoing support of the dedicated WtW program staff in Philadelphia. Linda Blanchette at the Philadelphia Workforce Development Corporation gave us background on Philadelphia's complex WtW program structure. Richard Greenwald and Lili-An Elkins of the Transitional Work Corporation (TWC) gave us background on the program, supported the study's sample enrollment, and welcomed us on our many visits to observe program operations. We appreciate the assistance of Caryn May in working with TWC management information systems data. We gratefully acknowledge the time and effort of the numerous staff at TWC and the Regional Service Centers (RSCs) who assisted in sample enrollment for the follow-up survey and who shared information with us on their program operations during our visits. We also are grateful to staff at Pennsylvania's Department of Public Welfare, Bureau of Program Evaluation, including Gayle Brignoni, Fred Mast, Scott Sanson, and Dennis Stamm, for their help in providing us with administrative records data.

At DHHS, Alana Landey oversaw the study and provided valuable guidance and feedback. Pam Holcomb from UI led site visits to TWC and the RSCs and provided key information on program operations.

At MPR, Daisy Ewell and Mark Brinkley expertly coordinated the process of securing and processing administrative data from the state of Pennsylvania. Rita Stapulonis, Shawn Marsh, and Jason Markesich oversaw sample enrollment at the RSCs and TWC, as well as the collection of 12-month follow-up survey data. Julie Sykes provided careful analysis of follow-up survey data and program management information systems data. Dan Levy, Rob Olsen, and Thomas Fraker oversaw the initial analyses of these data and the development of survey weights. Taryn Eckstein and Elizabeth McClintock provided invaluable research assistance for the analysis of these data. Anne Gordon, Alan Hershey, and Peter Schochet reviewed drafts of this report and provided many helpful comments in guiding the analysis. Patricia Ciaccio edited the report, and Bryan Gustus and William Garrett produced it.

We thank all these organizations and individuals, as well as any others we may have omitted unintentionally, for their important contributions to this study. The opinions and conclusions expressed herein are solely those of the authors and should not be construed as representing the opinions or policy of MPR or of any agency of the federal government. Any errors or omissions are strictly the responsibility of the authors.

How to Obtain a Printed Copy

To obtain a printed copy of this report, send the title and your mailing information to:

Human Services Policy, Room 404E
Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Av, SW
Washington, DC 20201

Fax:  (202) 690-6562

Introduction

The five-year time limits that the 1996 federal welfare reforms placed on cash assistance, or Temporary Assistance for Needy Families (TANF), heightened the need for programs to help the hardest-to-employ people move into jobs and become self-sufficient. To address this need, Congress authorized the Welfare-to-Work (WtW) grants program as part of the Balanced Budget Act of 1997. Because the WtW grants program gave states and local areas much flexibility in how they could use these funds, WtW grantees developed a variety of program approaches to help hard-to-employ TANF recipients move to work.

This report examines the outcomes for participants in two WtW programs in Philadelphia: (1) the Regional Service Centers (RSCs), and (2) the Transitional Work Corporation (TWC). These programs represent an important contrast in approaches to serving the hard to employ and in target populations. The RSCs offered 30 days of basic job search assistance services to the broad WtW-eligible population, while the TWC provided paid work experience for up to six months and targeted WtW-eligible people who had little or no work experience. The main objective of this study was to examine the employment, earnings, and TANF receipt outcomes of enrollees in these two WtW programs. Table I.1 summarizes our main findings.

 

Table I.1
Philadelphia WtW Outcomes Study:
Key Findings

TWC and RSC participants worked more, earned more, and received less TANF after program entry.

Participants in both programs had increases in employment immediately after program entry, followed by declines. At TWC, some of this increase, in the short term, was associated with placement in a subsidized job as part of the program intervention. In the longer term, one and a half years after program entry, participants from both programs still had higher employment rates than before program entry. They also had higher earnings and lower rates of TANF receipt than before program entry.

Consistent with the targeting and sequencing of the programs, RSC and TWC participants differed in their outcomes over time.

RSC participants had higher rates of employment, higher earnings, and lower rates of TANF receipt than TWC participants one and a half years after program entry. However, RSC and TWC participants also differed in their employment, earnings, and TANF receipt prior to program entry. Hence, this finding could reflect the way the two programs were created, with the RSCs offering a basic intervention for the general WtW population and TWC offering more intensive services for people facing greater employment challenges.

Observable factors explained RSC and TWC participants difference in employment and some of their differences in earnings and TANF receipt.

Controllingfor demographic characteristics, prior work and TANF receipt, and economic conditions accounts for the simple observed differences in the percentage of RSC and TWC participants employed one and a half years after program entry. Differences in earnings and TANF receipt remained, with about half the difference explained by these observable factors.

Further research is needed to clarify how programs like the RSCs and TWC contribute to participant outcomes.

The results offer a hint that the intensive TWC intervention might have partially made up for the greater employment challenges faced by TWC participants. However, the study raises questions that only a more rigorous random assignment evaluation can answerВ  most notably, how did TWC participants outcomes compare to how they would have fared in the absence of this intervention? Further research could determine the most appropriate targeting and cost-effective pairing of similar interventions.

In this chapter, we first describe the national WtW grants program, Philadelphias WtW program, and the local context of welfare reform in Philadelphia. We then provide background information on this studyВ  its research questions, sample, and data sources. In subsequent chapters, we describe the outcomes of RSC and TWC participants, as well as potential factors associated with these outcomes, and give an interpretation of findings and study conclusions.

Program Background

WtW programs operated based on federal guidelines that the WtW legislation specified. These guidelines allowed Philadelphia to develop WtW programs that addressed the particular needs of its WtW-eligible population and that operated alongside the citys welfare reform initiatives.

National WtW Grants Program

The WtW grants program allocated a total of $2.85 billion in federal funds through formula and competitive grants. WtW formula funds were allocated to states based on their share of the national poverty population and TANF caseload.^[1] States, in turn, distributed these funds to local areas. WtW competitive funds were available directly to states and local areas that applied for funds for specific projects. The WtW funds could be used for a range of program approaches, as long as the program emphasized employment.

The WtW legislation contained strict guidelines for program eligibility that initially slowed enrollment. The legislation required that 70 percent of funds be used for long-term TANF recipients who had two of three employment barriers: (1) no high school diploma or GED and low reading or math skills, (2) a substance abuse problem, and (3) a poor work history. The remaining 30 percent of funds could be used for recipients who had characteristics typical of long-term TANF recipients but did not meet the 70 percent criteria. In 2000, Congress relaxed the eligibility criteria, allowing WtW programs to serve more flexibly those long-term TANF recipients who faced barriers that were likely to make their transition to employment difficult.

Philadelphia's WtW Program

The Philadelphia Workforce Development Corporation (PWDC) was the local recipient of some of Pennsylvanias formula WtW funds. PWDC used WtW funds to launch a citywide initiative called Greater Philadelphia Works (GPW) to serve the needs of the hardest-to-employ TANF recipients in Philadelphia. central components of GPW were the RSC and TWC programs. GPW also included support services and programs that targeted teenagers, noncustodial parents, and homeless people. PWDC used $15.8 million in WtW formula funding to operate the RSCs and TWC. In addition, the Pew Foundation provided $4.3 million to cover TWCs administrative costs.PWDC also received $4.3 million in WtW competitive funding to operate WtW programs for teenage and noncustodial parents.

As originally designed, the RSC and TWC programs were expected to differ in the populations they targeted and the intensity and duration of their service approaches. The RSCs, designed to serve the broad population of WtW-eligible clients, operated in seven locations across the city, providing clients with up to 30 days of job search and placement assistance. Clients attended job search readiness sessions, conducted directed job search, and met regularly with an employment adviser. To promote clients job retention after placement, employment advisers followed up with them regularly for up to one year. RSC contractors received bonuses for their participants continuous employment at 30, 60, and 90 days and at six months.

In contrast, the TWC program was designed to provide services to the hardest-to-employ among the WtW-eligible population: those who had little or no work experience. The TWC provided clients with 25 hours a week of paid transitional employment for up to six months, followed by placement in unsubsidized jobs.^[2] TWC career advisers monitored clients progress and helped resolve problems at work. In addition, TWC clients attended 10 hours a week of wraparound training, which included such topics as GED preparation, basic skills, job readiness, and life skills. After clients obtained unsubsidized employment, the TWC offered up to $800 in job retention bonuses and six to nine months of retention-focused case management.Table I.2 provides more information on the services of the RSC and TWC programs.

Table I.2
Philadelphia WTW Outcomes Study:
RSC and TWC Program Descriptions

Regional Service Centers: Program Description
Program Structure	Provided short-term work readiness and job search assistance services. Provided job placement and retention services for 12 months after placement.
Target Population	The RSCs served long-term welfare clients who were nearing, or who had reached, two years of TANF receipt. RSC services targeted more job-ready WtW-eligible TANF clients.
Employment-Related Services	After attending a brief general orientation, clients participated in job readiness workshops and directed job search activities. The programs objective was for clients to find unsubsidized jobs within 30 days. Each RSC had job developers who identified work opportunities by working directly with employers. RSC participants who failed to secure employment within 30 days from enrollment were placed in paid community service positions (while continuing to search for work).Alternatively, they could be referred to the TWC program or back to their County Assistance Office (CAO) caseworker for reevaluation and assignment to another program or exemption from work requirements (as appropriate).
Transitional Work Corporation: Program Description
Program Structure	Provides up to six months of subsidized work experience employment, followed by assistance securing unsubsidized employment and job retention services for 12 six to nine months after placement.
Target Population	Targets hard-to-serve WtW-eligible TANF recipients who have participated in a mandatory job search required by the TANF agency but did not find a job after basic employment assistance at the RSCs; have limited educational attainment and also lack work experience and work history; or are otherwise considered hard to place. The TWC typically serves long-term welfare clients who are nearing, or who have reached, two years of TANF receipt.
Employment-Related Services	Referred individuals are immediately placed on TWCs payroll, receiving minimum wage ($5.15 per hour) for 25 hours per week for up to six months. Program participation begins with a two-week orientation, which provides an overview of TWC and covers job readiness and behavioral topics. During the second week of orientation, participants interview for, and are placed in, transitional work assignments at government agencies or nonprofit organizations. While in transitional work, TWC participants must attend 10 hours of career development training each week, including modules on literacy, math skills, computer skills, GED preparation, job readiness, and general life skills. While in transitional work, participants receive intensive supervision and support from on-site work partners and their TWC career advisers. The work partner is a regular employee, who mentors and supervises the TWC participant daily and provides assessments of the participants job performance to TWC career advisers every other week. When TWC participants are judged work-ready (based on their work partners assessments) or are close to completing their six months of transitional employment, TWC placement staff help them obtain an unsubsidized job. Participants are offered up to $800 in job-retention bonuses and 12 months of retention-focused case management.

The relationship and client flow between the RSC and TWC evolved over time. First, the referral process for the TWC changed in order to address underenrollment in the TWC program. Initially, staff from the County Assistance Office (CAO)  the welfare agency in Philadelphia  referred WtW-eligible clients to the RSCs, and at which time the RSCs identified appropriate clients for referral to TWC. If clients were determined not to be job-ready or were unable to find a job after the 30 days of services the RSCs provided, the RSCs referred them to the more intensive TWC program. In 2000, because enrollment in TWC was low, the program began to conduct its own direct outreach, and the CAO began to refer WtW-eligible clients directly to TWC. Thus, the program evolved to serve a somewhat more general WtW-eligible population, rather than the hardest-to-serve, as originally intended. Second, the process for placing TWC participants in unsubsidized work changed. Originally, TWC referred participants back to the RSC for placement after they had completed their TWC experience. In an effort to focus more attention on the placement of TWC participants, program operations changed in 2001 so that the TWC handles the placement of its participants in unsubsidized jobs after they complete their transitional work. Since WtW funds were time-limited, these WtW initiatives could not continue without additional sources of support and funding. In September 2001, the RSCs ceased operations as their funding ended. The TWC operations continued with ongoing support from state WtW funds through February 2004. State TANF funds support the TWC operations for the rest of the 2004 fiscal year.

Welfare Reform in Philadelphia

Two distinct features of Pennsylvanias welfare reform initiative set it apart from those of other states and thus created a unique context for the operation of Philadelphias WtW program. First, Pennsylvania has a two-year work-trigger time limit that requires TANF recipients to participate in work-related activities for a minimum of 20 hours a week by the time they have been receiving TANF for two years, or face a full family sanction. In anticipation of the first cohort of TANF recipients to reach this time limit in March 1999, PWDC developed programs, including the WtW programs, to help work-mandatory persons meet this requirement.

Second, in keeping with the states client choice philosophy, the CAO offered work-mandatory clients a broad menu of programs and allowed them to decide where to participate. The city offered at least six work activity programs that would meet the two-year work requirement. In addition to the WtW-funded programs, the Philadelphia CAO administered several TANF-funded programs offering similar services.^[3] TANF recipients could choose to enroll in any of the programs, and welfare staff did not usually conduct an assessment or recommend which program to attend. In this context, the TWC and RSC programs were just two options among a wide range of choices available to work-mandatory TANF recipients.^[4] Because clients could move easily between the work activity programs the CAO and GPW offered, their outcomes over time may have been affected by services they received from more than one program.

The Philadelphia Outcomes Study

The WtW Philadelphia outcomes study is part of a congressionally mandated, national evaluation of the WtW grants program, conducted by Mathematica Policy Research, Inc. (MPR), the Urban Institute, and Support Services International under a contract to the U.S. Department of Health and Human Services (DHHS). The main evaluation includes three key components: (1) a descriptive assessment of grantees based on two surveys of all WtW grantees nationwide, (2) a process analysis based on visits to 11 in-depth study sites and a program cost analysis in most of these sites, and (3) participant outcomes analysis in most of the in-depth study sites.^[5] In addition to these three components in the core evaluation, a special process and implementation study focuses on programs operated by American Indian tribes and Alaska Native villages.

The WtW Philadelphia outcomes study builds on and expands an examination of outcomes for the TWC program, which is included as an in-depth study site in the national WtW evaluation (Fraker et al. 2004). However, the examination of the TWC alone does not provide a full understanding of Philadelphias WtW strategy, since the RSCs were an important part of that strategy. To document more fully the outcomes of Philadelphias WtW efforts, MPR conducted this study to examine the employment experience of both RSC and TWC participants. Findings from the larger evaluation illustrate the employment experiences of participants in the 11 in-depth study sites. These sites offered similar kinds of services to a similar range of populations as the TWC and RSC programs. Thus, the larger evaluation findings provide a broader context in which to view these findings.

Study Design and Research Questions

This study is designed to describe the outcomes for RSC and TWC participants. Since it is not based on the random assignment of clients to these programs, differences in the outcomes for RSC and TWC participants do not provide evidence of program impacts or the relative effectiveness of these two program models. The study does, however, provide an overall description of Philadelphias WtW participants outcomes after program entry and a comparison of the outcomes for different populations served with different program approaches.

The study examines RSC and TWC participant outcomes  employment, earnings, and TANF receipt over time  and addresses two sets of related questions:

What were the outcomes of RSC and TWC enrollees?Were they able to find employment? To what extent did they retain and advance in their employment? What were their earnings over time? Were they able to move off TANF? How different were these outcomes for enrollees in the RSCs and TWC?

What factors were associated with RSC and TWC enrollees outcomes?Were there notable differences in the characteristics of enrollees in the RSC and TWC programs before and at program enrollment? To what extent were such factors as enrollees characteristics before and at program enrollment, enrollees program participation, and economic conditions after program enrollment associated with their outcomes?

Study Sample and Data Sources

The study sample included people enrolled from September 1999 to January 2001 at the RSCs and from September 1999 to April 2001 at TWC. Sample members were enrolled in the study upon their entry into either the RSC or the TWC program during these periods. To enter either program, clients typically were referred by the CAO after screening for WtW eligibility. They could also be referred to the TWC by the RSCs. The TWC sample enrollment period was longer than the RSC one because of TWCs initially slow enrollment and smaller scale. The final study samples for the RSCs and the TWC are substantial, including more than 2,300 RSC program enrollees and more than 2,500 TWC program enrollees.^[6]

Study sample members were identified in different ways. All enrollees who entered the TWC at some point in the sample enrollment period were considered part of the TWC sample. However, if a study sample member enrolled in the RSC study sample but later enrolled in the TWC study sample or was found in the TWC program database, that person was coded as a TWC sample member only, regardless of the persons participation in the RSC program. Constructing the analysis sample required a substantial effort to sort out which sample members were enrolled in the TWC. Because this study focuses on how the different populations were served differently based on their identified needs, all sample members who received the TWCs intensive services are considered TWC sample members. About half of those in the TWC sample participated in both the RSC and the TWC. The RSC sample is limited to enrollees who entered only the RSC program.

The study draws on data from four main sources, documented in Table I.3:

1. A baseline information form (BIF) completed at sample enrollment
2. A follow-up survey conducted 12 months after sample enrollment
3. State administrative data on earnings and TANF
4. Management information systems (MIS) data containing information about program participation

 

Table I.3
Philadelphia WTW Outcomes Study:
Main Data Sources

Data Source	Timing of Data	Key Measures	Sample Definition	RSC Sample Size	TWC Sample Size a
Baseline Information Forms (BIFs)	Program enrollment: September 1999 to January 2001 (for RSCs) and September 1999 to April 2001 (for TWC)	-Demographics - Household structure - Health problems - Education - Employment history - Public assistance receipt	Program enrollees who completed a BIF	1,109	1,279
12-Month Follow-up Survey	12 months after program enrollment	- Household structure - Income - Employment history - Child care - Education/training - Criminal activity - Material well-being	Program enrollees who completed a BIF and responded to follow-up survey	944	1,110
Administrative Data	Four quarters before and up to eight quarters after program enrollment	- Employment - Earnings - TANF receipt	Program enrollees who completed a BIF or were in MIS	2,338	2,543
Management Information System (MIS) Data	Ongoing after-program enrollment	Full program participation	Program enrollees in MIS with enrollment date during sample enrollment period	2,248	2,320
a  Enrollees who participated in both RSC and TWC programs are included in the TWC sample only.

Because of lapses in sample enrollment procedures at the program sites, BIF data are not available for all program enrollees during the sample enrollment time period. To compensate for these missing data, and to capture data on all enrollees during the sample enrollment period, MPR collected administrative data either on those who completed a BIF or who were in the MIS data with an enrollment date during the sample enrollment period. Since the follow-up survey sample included only enrollees who had completed BIFs, we weighted survey data using MIS data to adjust for differences among enrollees who completed and did not complete BIFs.

We collected administrative data on TANF receipt and earnings on sample members for four quarters before, and up to eight quarters after, program entry. Because participants entered the programs over time, the number of quarters of data available varies by individual; data for later quarters are available for fewer sample members. The data in the report are restricted to quarters in which data are available for the most sample members. Eight quarters of postenrollment data are available for the RSC sample, and six quarters are available for the TWC sample, because TWC participants enrolled later on average.

This report is organized into four chapters. Chapter II describes RSC and TWC enrollees employment, earnings, and TANF receipt over time. Chapter III discusses factors that may be related to the differences in RSC and TWC outcomes, such as enrollees characteristics, program participation, and economic conditions. Chapter IV discusses the study findings and presents study conclusions and implications.

Endnotes

[1] In January 2004, Congress rescinded unspent fiscal year 1999 WtW state formula funds.

[2] Throughout the report we use the terms paid transitional work experience and paid work experience interchangeably.

[3] Participants in the RSC and TWC programs could receive support services funded through TANF; however, these programs did not initially receive TANF funding.

[4] To participate in the RSC or TWC programs, participants had to be eligible for WtW. However, not all WtW-eligible people were necessarily referred to the RSCs or the TWC.

[5] The evaluation originally planned to use an experimental design to examine the net impacts of the WtW grants program on participants and to analyze the programs costs and benefits. Because of low enrollment in WtW programs, however, it was not feasible to randomly assign participants to treatment and control groups. As a result, MPR revised the study design to an outcomes analysis.

[6] Since the programs were newly created because of the WtW funding and evolved over time based on their experiences in serving the WtW-eligible population, they were not in a steady state of operations over the course of the study. Thus, the outcomes observed in this study may not be representative of the true potential of the programs, but rather reflect the outcomes achieved during this initial experimental stage.

Outcomes

The main goal of Philadelphias WtW programs was to help the hardest-to-employ TANF recipients become self-sufficient through increased employment, increased earnings, and reduced TANF receipt. This study examines RSC and TWC participants outcomes over time after WtW enrollment. Because the study was not experimental, differences between RSC and TWC participants should not be interpreted as relative impacts of the programs; they have differences other than the program services they received. Rather, this study is designed to provide an overall description of Philadelphias WtW program participants outcomes. While we cannot determine whether, or to what extent, the WtW programs are responsible for the outcomes observed over time, we can examine the extent to which the programs intended outcomes were being achieved. In this chapter, we examine outcomes for RSC and TWC participants over time after program entry. We first discuss participants employment patterns, then their earnings, and finally their TANF receipt.

Trends in employment and TANF receipt for the Philadelphia TANF caseload at the time study sample enrollment began provide context for the examination of outcomes for RSC and TWC participants. To provide this context, we used the entire Philadelphia TANF caseload as of September 1999 as a reference sample. This is the month when sample enrollment began at the RSCs and TWC. Administrative records data for the September 1999 Philadelphia TANF caseload indicate that, over time, TANF receipt declined sharply, while employment increased only slightly (Figure II.1).

Figure II.1
Philadelphia WTW Outcomes Study:
Trends in Outcomes for TANF Caseload in Philadelphia, as of September 1999

Source: Administrative Data from State of Pennsylvania

However, although TANF receipt did decline, nearly half the September 1999 TANF caseload (48 percent) was still receiving TANF two years later. In addition, the majority of the caseload were not employed two years laterВ  only 43 percent were employed in September 2001, a slight increase from 39 percent in September 1999.

While these overall trends provide a context in which to place the RSC and TWC participants, the September 1999 TANF caseload is different from the RSC and TWC study samples in two important ways. First, the September 1999 TANF caseload includes a mix of people with different patterns of TANF receipt and program participation, such as short-term TANF recipients, long-term TANF recipients enrolled in WtW or other programs, and long-term TANF recipients with deferrals (who were not required to participate in work activities). In contrast, the RSC and TWC study samples included only long-term TANF recipients enrolled in these WtW programs. Second, the September 1999 caseload included TANF recipients at one point in time, whereas the study sample were enrolled over a period of time. Therefore, study sample outcomes may reflect differences from September 1999 in economic or other conditions at the time of enrollment and during the follow-up period.

Employment

While their approaches differed, a central goal of both the RSC and the TWC programs was to help participants become employed. To help participants find jobs quickly, the RSCs emphasized short-term job search and placement. In contrast, TWC sought to enhance participants' overall employability by providing them with work experience in paid transitional jobs for up to six months before helping them find permanent, unsubsidized jobs. Here, we examine the trends over time in employment for RSC and TWC participants.

Employment rates initially increased. The participants in both programs had sharp increases in employment soon after program entry (Figure II.2). TWC participants' employment rates rose to a high of 79 percent during the quarter of program enrollment, reflecting their immediate placement in paid transitional work positions.^[1] RSC participants employment rate rose to a high of 59 percent in the first quarter after program enrollment, as they quickly obtained jobs through the program's job search and placement services or through their own efforts. Nevertheless, because participants in both programs were likely to be at a relatively low point in their employment just before program enrollment, their postenrollment outcomes are likely to reflect some natural recovery from these lows.^[2]

Figure II.2
Philadelphia WTW Outcomes Study:
Employment Rates Over Time

Source: Administrative Data from State of Pennsylvania

After these initial increases, employment declined steadily over time. By six quarters after program enrollment, only 39 percent of TWC participants, and 49 percent of RSC participants, were employed (Figure II.2). TWC participants experienced a markedly steep decline in employment within the first three quarters of program enrollment, suggesting that many did not complete their transitional employment or move into unsubsidized employment. RSC participants experienced a smaller, yet consistent, decline in employment over the quarters after program entry.^[3]

Because TWC participants primarily held transitional jobs in the first quarters after program enrollment while RSC participants held unsubsidized jobs, we do not focus on comparisons across these early quarters. The administrative data do not provide information on whether the employment was subsidized or not. Instead, we examine outcomes at least four quarters after program enrollment, when both TWC and RSC participants could have completed the program and reached unsubsidized employment (this is also true of subsequent analyses of earnings and TANF receipt).^[4]

Despite these declines, overall employment success improved after program entry. In general, for both RSC and TWC participants, employment rates six quarters after program entry were higher than the highest preenrollment rates. Moreover, a higher proportion of both RSC and TWC participants were employed consistently during all four of the quarters after program entry (31 and 25 percent), compared to the four quarters before entry (15 and 10 percent; Table II.1). Further, RSC and TWC participants were more likely to be at least somewhat engaged in the labor market after program entry. More than three-quarters of RSC and TWC participants (76 and 84 percent, respectively) were employed at some point in the four quarters after program entry, compared with about two-thirds (65 and 64 percent, respectively) in the four quarters prior to program entry (Table II.1).^[5]

Table II.1
Philadelphia WTW Outcomes Study:
Consistency of Employment and Average Annual Earnings, Before and After Program Entry
(Percentages, Unless Otherwise Indicated)

	RSC	TWC	Significance
Four Quarters Before Program Entry
Ever employed	65.4	64.1	***
Employed in all four quarters	15.4	10.0	***
Employed in at least one quarter	50.0	54.1	***
Never employed	34.6	35.9
Average annual earnings (dollars)	2,204	1,561	***
Four Quarters After Program Entry
Ever employed	75.5	84.0	***
Employed in all four quarters	31.3	25.4	***
Employed in at least one quarter	44.2	58.6	***
Never employed	24.5	16.0	***
Average annual earnings (dollars)	4,501	3,389	***
Sample Size	2,338	2,543
Administrative records data from state of Pennsylvania. */**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

Nevertheless, employment was unstable. Most RSC and TWC participants surveyed 12 months after sample enrollment reported that they had had at least one spell without employment in the year after program entry (90 and 95 percent, respectively; Table II.2). Their spells without employment made up a substantial proportion of the year after program entry. RSC and TWC participants who worked at some time in the year after program entry reported that, on average, they worked for only 61 and 49 percent of the year, respectively (Table II.2).

Table II.2
Philadelphia WTW Outcomes Study:
Employment of Welfare-to-Work Enrollees During the Year After Program Entry
(Percentages, Unless Otherwise Indicated)

Employment Measure	RSC	TWC	Significance
Number of Employment Spells
0	22.6	26.2	*
1	52.4	51.2
2	20.1	17.3
3 or more	4.9	5.3
Number of Spells Without Employment
0	10.3	5.4	***
1	53.9	56.1
2	27.6	31.3	*
3 or more	8.2	7.2
Proportion of YearAfter Program Entry Employed
All enrollees	47.0	36.0	***
If employed sometime during year	61.0	49.0	***
Sample Size	944	1,110
Source: 2001-2003 12-month follow-up survey of Welfare-to-Work enrollees.
Note 1: The survey data have been weighted to be representative of all WtW enrollees in the respective sites. Survey item nonresponse may cause the sample sizes for specific variables to be smaller than those shown. Rounding may cause percentages to sum to something other than 100.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

For TWC participants, some instability is likely to reflect movement from transitional jobs to unsubsidized jobs.  The employment rate of TWC participants declines most sharply in the first three quarters after program entry, when they would be leaving their transitional jobs to look for unsubsidized work. In the follow-up survey, more than one-third of TWC participants reported that they left their first job because the work period ended (Table II.3).^[6]

Table II.3
Philadelphia WTW Outcomes Study:
Circumstances of Departure from the First Job Held After Enrolling in Welfare-to-Work
(Percentages)

Circumstance of Departure	RSC	TWC	Significance
Reason for Departure from Job
Quit	44.7	31.9	***
Laid off	12.7	8.4	**
Fired	13.3	10.2
Work period ended	15.0	35.2	***
Self-employed job ended	9.5	6.0	**
Other reason	4.9	8.4	**
Sample Size	439	605
2001-2003 12-month follow-up survey of Welfare-to-Work enrollees.
Notes: The survey data have been weighted to be representative of all WtW enrollees in the respective sites. Survey item nonresponse may cause the sample sizes for specific variables to be smaller than those shown. Rounding may cause percentages to sum to something other than 100. The statistics presented in this table are for WtW enrollees who left the first job that they held on or following their entry into WtW. If there was more than one first job, then the principal job  the job with the most hours worked in a typical week  was selected. In the event of a tie on hours worked, several additional criteria were applied in sequence.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

The employment instability among Philadelphia WtW participants is typical of that of similar groups. The employment instability observed for the RSC and TWC participants is consistent with findings of other research on the employment of former TANF recipients. Andersson et al. (2003) indicate that the retail and service industries  where RSC and TWC participants most commonly worked (Table II.4)  provided unstable employment. Moreover, an analysis of the Philadelphia caseload from 1997 to 1999 indicates that, while employment increased, much of it was short-term and unstable (Michalopoulos 2003).

Table II.4
Philadelphia WTW Outcomes Study:
Industry of Welfare-to-Work Enrollees Who Worked One Year After Program Entry
(Percentages)

Industry	RSC	TWC	Significance
Services	58.0	67.2	**
Retail Trade	21.0	17.7
Manufacturing	4.7	2.2	*
Transportation and Utilities	6.5	2.7	***
Wholesale Trade	1.7	0.2	**
Finance, Insurance, and Real Estate	3.7	4.1
Public Administration	3.4	4.8
Other	1.2	1.2
Sample Size	476	420
Source: 2001-2003 12-month follow-up survey of Welfare-to-Work enrollees.
Notes: The survey data have been weighted to be representative of all WtW enrollees in the respective sites. Survey item nonresponse may cause the sample sizes for specific variables to be smaller than those shown. Rounding may cause percentages to sum to something other than 100. The statistics presented in this table pertain to the principal job held by a sample member at the time of the survey interview. If the sample member held more than one job at that time, then the principal job was identified as the job on which the most hours were worked in a typical week. In the event of a tie on hours worked, the job with the earliest starting date.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

RSC participants were more likely to move successfully from one job to another. Most RSC and TWC participants who found a job in the year after program enrollment left that job within the year (60 and 73 percent, respectively; Table II.5). Among those who left their first job, RSC participants were more likely than TWC participants to find another job by one year after program enrollment (63 versus 53 percent, respectively).^[7] However, because their transitional jobs ended, more TWC participants had to search for another job during the year following program entry.

Table II.5
Philadelphia WTW Outcomes Study:
Work Status One Year After Program Entry
(Percentages)

Work Status	RSC	TWC	Significance
Employed During Year After Program Entry
Still employed at first job 1 year after program entry	39.9	27.4	***
Left first job during year after program entry	60.1	72.5	***
Left first job during year after program entry, employed at another job 1 year after program entry	37.9	38.2
Left first job during year after program entry, not employed 1 year after program entry	22.2	34.3	***
Sample Size	733	826
Source: 2001-2003 12-month follow-up survey of Welfare-to-Work enrollees.
Notes: The survey data have been weighted to be representative of all WtW enrollees in the respective sites. Survey item nonresponse may cause the sample sizes for specific variables to be smaller than those shown. Rounding may cause percentages to sum to something other than 100. The statistics presented in this table pertain to the principal job a sample member held at the time of the survey interview. If the sample member held more than one job at that time, then the principal job was identified as the job on which the most hours were worked in a typical week. In the event of a tie on hours worked, the job with the earliest starting date.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

 

Earnings

While helping participants become employed was a central goal of the WtW programs, the ultimate goal was to help participants become self-sufficient.To the extent that WtW participants were employed and had increased their earnings, they moved closer to self-sufficiency. Next, we describe the earnings of RSC and TWC participants after program entry.

Earnings improved over time after program enrollment. The average quarterly earnings across all RSC and TWC participants (including those with no earnings) increased after program entry and remained substantially higher than those of preprogram levels (Figure II.3).^[8] One year after program enrollment, both RSC and TWC participants had average quarterly earnings ($1,232 and $841, respectively) that were about two times higher than one year before program enrollment ($520 and $429, respectively).^[9] Improved employment success, evident in the increased employment rates in the quarters after program enrollment, is likely associated with higher earnings.

Figure II.3
Philadelphia WTW Outcomes Study:
Earnings Over Time

Source: Administrative Data from State of Pennsylvania

Increases in wages and/or hours worked played a role in increased earnings. RSC and TWC participants who changed jobs during the first year after program entry, on average, reported higher hourly wages at their most recent job than at their first one (Table II.6). In addition, TWC participants who changed jobs reported working more hours each week. Hours worked and wage rates may also have increased for participants who stayed in the first job over the year after program entry; however, the follow-up survey data do not provide this information for these participants.

In addition, benefits improved over time.As RSC and TWC participants moved into jobs with increased wages and hours worked, they also received better benefits. Both RSC and TWC participants who switched jobs were more likely to report receiving such benefits in their most recent job as participation in their employers health or dental insurance plan, availability of a pension plan, and receipt of sick leave, vacation leave, or paid holidays (Table II.6).

TANF Receipt

WtW programs ultimately sought to help participants move off TANF. Time limits on TANF receipt encouraged participants to leave TANF. At the same time, Pennsylvania offers generous earnings disregards to TANF recipients that would allow WtW participants who became employed to continue to receive TANF up to a certain earnings threshold. Here, we discuss RSC and TWC participants' TANF receipt before and after program entry, as well as the extent to which program participants combined employment and TANF receipt after program entry.

TANF receipt declined steadily. Four quarters before program entry, most RSC and TWC participants received TANF (81 and 89 percent, respectively; Figure II.4). By six quarters after program entry, the rate of TANF receipt among RSC and TWC participants had declined to 49 and 64 percent, respectively. Given that WtW-eligible participants were all long-term TANF recipients, these declines represent a major reduction in TANF receipt, although about half of them continued to receive TANF. Fewer TWC and RSC participants received TANF during all four quarters after program entry (47 and 64 percent, respectively) than received TANF during all four quarters before program entry (70 and 80 percent, respectively; Table II.7).

Figure II.4
Philadelphia WTW Outcomes Study:
Rates of TANF Receipt Over Time

Source: Administrative Data from State of Pennsylvania

Table II.7
Philadelphia WTW Outcomes Study:
Consistency of TANF Receipt, Before and After Program Entry
(Percentages, Unless Otherwise Indicated)

	RSC	TWC	Significance
Four Quarters Before Program Entry
Ever on TANF	92.3	97.6	***
On TANF in all four quarters	70.0	80.2	***
On TANF in at least one quarter	22.3	17.4	***
Never employed	7.7	2.4	***
Four Quarters After Program Entry
Ever on TANF	89.1	68.4	***
On TANF in all four quarters	47.3	63.7	***
On TANF in at least one quarter	41.8	31.7	***
Never employed	11.0	4.6	***
Sample Size	2,338	2,543
Source: Administrative records data from State of Pennsylvania.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

TWC participants were more likely to remain on TANF, regardless of their employment status. TWC participants were more likely than RSC participants to combine work with TANF receipt after program entry (Figures II.5 and II.6). Since most TWC participants were placed in transitional jobs that paid minimum wage, the earnings disregard would likely allow them to continue receiving TANF. Over the quarters after program entry, the percentage of TWC participants who combined work and TANF receipt declined, while the percentage of TWC participants who received TANF and did not work increased. As TWC participants completed or dropped out of their transitional jobs, many continued to receive TANF but did not find another job.

Figure II.5
Philadelphia WTW Outcomes Study:
RSC-Combination of Work and TANF

Source: Administrative Data from State of Pennsylvania

Figure II.6
Philadelphia WTW Outcomes Study:
TWC-Combination of Work and TANF

Source: Administrative Data from State of Pennsylvania

In contrast, RSC participants were more likely to use employment to move off TANF. The percentage of both TWC and RSC participants who worked and did not receive TANF (white portion of bar graphs) increased over time. However, a higher proportion of RSC participants than TWC participants worked and did not receive TANF after program entry. At the same time, the proportion of RSC participants who received TANF and worked declined over time; the proportion who received TANF and did not work remained fairly stable.

Over time, an increasing percentage of TWC and RSC participants neither received TANF nor worked. During the quarters after program entry, the proportion of both TWC and RSC participants not on TANF and not employed increased (black portion of bar graphs). Some of this population may have been living on sources of support, such as supplemental security income, unemployment insurance, saved earnings, or partners' income (Wood and Rangarajan 2003).^[10] Others, however, may have been among the least stable, with no formal source of income.

Summary

Employment and earnings for RSC and TWC participants increased over time after program entry, while TANF receipt declined. Although participants' employment success improved after program entry, their employment often was unstable. Ultimately, RSC participants were more able than TWC participants to move from one job to another during the year after program entry. However, both RSC and TWC participants experienced increased earnings over time after program entry. At the same time, both RSC and TWC participants' receipt of TANF declined overall. RSC participants were more likely to leave TANF for work, whereas TWC participants were more likely to continue to receive TANF. Several factors-including participant characteristics upon program entry, program services, and economic conditions-may be related to these RSC and TWC participants' outcomes, as well as to the differences in outcomes. In Chapter III, we discuss these factors in detail.

Endnotes

[1] Some TWC participants may have dropped out of the program very soon after enrollment, before placement in transitional work.

[2] This pattern of recovery from a preprogram low is typically referred to as "Ashenfelter's dip," for his observation that adult participants in job training programs often have a dip in earnings prior to their decision to participate (Ashenfelter and Card 1985).

[3] The declines in the employment rate could be due in part to participants' taking out-of-state jobs, which would not be captured in the state wage data (Corson 1989).

[4] To account for differences in participants' entry into unsubsidized employment, we compared RSC participant outcomes to "lagged" outcomes for TWC participants (see Appendix). That is, we compared TWC participants' outcomes from quarter three after program entry to RSC participants' outcomes from quarter one after program entry. At these quarters, participants from both programs should have exited the program and entered unsubsidized work. This examination of outcomes reveals that the overall pattern does not change.

[5] The percentage of TWC participants who worked after program entry is particularly high because they participated in paid transitional work positions.

[6] The survey was designed as a broad instrument to collect data over all WtW programs included in the outcomes analysis for the national evaluation. Thus, it does not identify TWC participants' first unsubsidized job.

[7] We calculated these percentages by dividing the percentage who left their first job after program entry and were employed at another job one year after program entry by the percentage who left their first job after program entry.

[8] To make consistent comparisons across quarters, we include the full sample in these estimates, not just those with earnings who are working.

[9] In addition, participants in both programs had higher employment rates one year after program entry compared to one year before program entry.

[10] Some of these people may be employed outside the state.

Factors Associated with Differences in RSC and TWC Outcomes

The employment rates, average earnings, and rates of TANF receipt for RSC and TWC participants are likely to reflect a wide range of influences. Individual, family, and social factors all influence whether people work and whether they are able to keep their jobs over time, as well as whether they use public assistance and how long they remain on it. Differences in observable characteristics (such as educational attainment, prior work experience, and household structure) and unobservable characteristics (such as personal motivation or innate ability) of RSC and TWC participants could also be reflected in their outcomes. Since RSC and TWC participants were enrolled in these programs over a 20-month period, trends in economic conditions could have influenced participant outcomes. In their design, the RSC and TWC programs reflected different philosophies about how best to support transitions to employment for the hard to employ. The services the programs offeredВ  and how intensely or completely people participated in those servicesВ  also could have played a role in participant outcomes.

In this chapter, we explore three sets of factors that may have influenced the outcomes of RSC and TWC participants: (1) individual characteristics, (2) economic conditions, and (3) the intensity of program participation. This exploration is intended to help us identify factors we should control for as we examine RSC and TWC outcomes further. That is, once we identify substantive differences of statistical significance among RSC and TWC participants, we can control for these differences in multivariate models and explore the extent to which they help explain the differences in outcomes for these two groups.

Characteristics of RSC and TWC Participants

The RSC and TWC programs were designed to serve different groups of WtW-eligible people. Under the original client-flow process, the RSCs provided basic reemployment assistance to most WtW-eligible people in Philadelphia. The TWC program would then provide more intensive assistance to those WtW-eligible people who, after a short time at the RSCs, were found to have limited work experience or other noTable barriers to employment or who had not found a job after 30 days of RSC work search activities.^[1] We can examine a number of participant characteristics to clarify how different the RSC and TWC participant populations ultimately were. These characteristics were not all necessarily used to determine the appropriate program referral and evaluate their participants employment needs. Rather, they reflect the mix of participants who ultimately participated in each program (for example, educational attainment and prior work experience were used, but marital status and number of children were not). Because of Pennsylvanias client choice orientation, even though the programs were to focus on serving different populations, little formal assessment took place to sort clients to the appropriate program. Initially, the RSCs were to refer their unsuccessful participants to the TWC, but because of underenrollment, TWC conducted its own outreach. As a result, the differences in the characteristics of participants referred to each program may be less clear than the original distinctions in program intent might have suggested. Baseline information and administrative data for the preenrollment period suggest that, while RSC and TWC participants were similar in many demographic characteristics, the average RSC participant was somewhat less disadvantaged. We found statistically significant differences in the household structure, educational attainment, work history, and public assistance receipt of RSC and TWC participants. In general, however, these differences were small, suggesting that both programs served groups that were fairly disadvantaged.

Nearly all RSC and TWC participants were young, minority, single women. Both programs served primarily women, but the RSCs were more likely to serve some men (5 percent, compared with 1 percent among TWC participants; see Table III.1). On average, participants in both programs were 32 years old. The majority of both RSC and TWC participants were African American, but RSC participants were somewhat more likely to be Hispanic (15 percent, compared with 7 percent of TWC participants) or white (6 percent, compared with 2 percent of TWC participants). Few participants in either the RSC or the TWC programs were married at baseline, but TWC participants were more likely to have never been married (79 percent, compared with 70 percent among RSC participants).

 

Table III.1
Philadelphta WTW Study:
Baseline Characteristcs of RSC and TWC Enrollees

Baseline Characteristics	RSC Enrollees	TWC Enrollees	Significance
Demographic Characteristics
Age Category
Younger than 20 years	0	0	ns
20 to 29 years	43	44	ns
30 to 39 years	42	41	ns
40 years or older	16	15	ns
Average Age	32.3	32.1	ns
Gender
Female	95	99	***
Race/Ethnicity
Hispanic	15	7	***
White non-Hispanic	6	2	***
Black non-Hispanic	78	89	***
Other non-Hispanic	2	2	ns
Household Structure
Marital Status
Married	6	3	***
Cohabiting	7	6	ns
Separated/divorced/widowed	17	13	***
Never married	70	79	***
Number of Children in Household
0	5	5	ns
1 to 2	56	49	***
3 to 5	35	41	***
6 or more	4	5	ns
Average Number of Children in Household	2.4	2.7	ns
Age of Youngest Child in Household
Younger than 3 years	13	23	***
Younger than 5 years	39	46	***
Average Age of Youngest Child in Household	6.0	6.8	ns
Education
Educational Attainment
High school dropout	43	47	ns
Still attending high school or GED program	3	6	***
GED	5	4	ns
High school diploma	17	15	ns
Postsecondary degree	3	2	ns
Vocational/technical certificate	29	26	ns
At Least High School Diploma or GED	54	47	**
Employment History
Employment at Baseline
Currently employed	8	8	ns
Employed within the past year	55	47	***
Employed more than a year ago	29	34	**
Never employed in the past	7	10	**
Welfare Receipt
TANF Receipt at Baseline
Currently receiving	90	91	ns
Received in the past but not currently receiving	6	5	ns
Never received	4	3	ns
Total Time on TANF or AFDC
Never on TANF or AFDC	4	3	ns
1 to 24 months	30	23	***
25 to 60 months	28	33	**
More than 60 months	37	41	*
Health Problems
Work-Limiting Problem
Own	21	20	ns
Other household member	12	13	ns
Type of Work-Limiting Problem (Own)
Medical condition	9	10	ns
Physical disability	3	2	ns
Mental health or substance abuse problem	4	5	ns
Sample Size	1,109	1,282
Source: Baseline information forms of Welfare-to-Work participants, MPR. ns = not significant. */**/*** Significantly different from zero at the .10/.05/.01 level, two-tailed test.

RSC participants had fewer young children. While the average number of minor children living with RSC and TWC participants at baseline was similar (2.4 and 2.7 children, respectively), TWC participants were more likely to have three or more minor children living with them (46 percent, compared with 39 percent of RSC participants). The average age of the youngest child in RSC and TWC participants households was similar (6 and 7 years old, respectively), but TWC participants were significantly more likely to have a child under age 5 living with them (46 percent, compared with 39 percent of RSC participants). This suggests that, on average, TWC participants had more minor children living with them and that these children included children who were both younger and older than the children of RSC participants.

RSC participants had higher educational attainment.  RSC participants were more likely to have at least a high school diploma or GED (54 percent, compared with 47 percent for TWC participants).  Further, RSC participants were somewhat more likely to have a postsecondary degree or vocational/technical certificate (32 percent, compared with 28 percent for TWC participants), although this difference was not statistically significant.

RSC participants were more likely to report some work history and to have worked consistently during the year before program entry.  At baseline, 7 percent of RSC enrollees reported never having worked at a job for pay, compared with 10 percent of TWC enrollees.  In addition, RSC participants were more likely to report having been employed at some point in the year before enrollment (55 percent, compared with 47 percent for TWC participants).  TWC participants were more likely to report that their last job had ended more than a year ago (34 percent, compared with 29 percent for RSC participants).  Administrative data show comparable proportions of RSC and TWC participants, with some earnings in the four quarters before program enrollment (65 and 64 percent, respectively; Table II.1).  Nevertheless, a larger share of RSC participants (15 percent, compared with 10 percent for TWC participants) had earnings in all four preenrollment quarters.

RSC participants were less likely to report long-term receipt of public assistance.  Most RSC and TWC participants (90 and 91 percent) reported receiving TANF at baseline.  The TANF administrative data show even higher rates of receipt of TANF at baseline among RSC and TWC participants (94 and 98 percent).^[2]  Both groups also displayed steady TANF receipt in the year before enrolling in WtW  70 percent of RSC participants and 80 percent of TWC participants received TANF in all four quarters prior to the quarter of WtW enrollment, based on administrative data (Table II.7).  At baseline, however, slightly more TWC participants reported having received public assistance for five or more years (41 percent, compared with 37 percent for RSC participants).

Similar proportions of RSC and TWC participants reported having work-limiting health problems.  About one-fifth of both RSC and TWC participants reported having a health problem that limited their ability to work, including medical conditions, physical disabilities, and mental health or substance abuse problems (Table III.1).  Another 12 percent of RSC participants and 13 percent of TWC participants reported that they were responsible for another household member with a health problem and that this responsibility limited their ability to work.  Hence, although work-limiting health problems may have been an important factor in the employment outcomes of RSC and TWC participants, they are likely to have influenced both groups similarly.

Economic Conditions

Since the RSC and TWC programs operated on different scales, and our study examines outcomes for a similar number of participants for each program, the study includes participants who enrolled in these programs over slightly different periods of time.  The seven RSCs that operated throughout the city of Philadelphia had the capacity to enroll as many as 1,400 new WtW-eligible clients each month.  In contrast, the TWC program was set up to serve about 1,500 clients each year.  Because of these different scales of operation, the RSCs reached our sample goal of 2,000 WtW participants sooner than the TWC program.  As Figure III.1 shows, RSC sample enrollment began in September 1999 and ended in January 2001, although most RSC sample members had been enrolled by June 2000.  In contrast, TWC sample enrollment also began in September 1999 but progressed more gradually and continued until April 2001.^[3]  Hence, differences in economic conditions at the time of enrollment and during the follow-up period could have contributed to the differences for outcomes of RSC and TWC participants.

Figure III.1
Philadelphia WTW Outcomes Study:
Cumulative Enrollment into RSC and TWC Study Samples

Source: Administrative Data from State of Pennsylvania

Rising unemployment may have contributed to poorer employment outcomes for TWC participants. As Figure III.2 shows, unemployment rates for Philadelphia had a pronounced upward trend from January 2001 through August or September 2002.  Rising unemployment could have made it more difficult for later sample enrollees  mostly TWC participants  to find and keep jobs.^[4]  To test this hypothesis, we divided our TWC study sample into two groups  based on their date of enrollment  and compared their employment outcomes.  TWC participants enrolled by June 30, 2000, were labeled early entrants, and TWC participants enrolled on or after July 1, 2000, were labeled late entrants.  As Figure III.3 shows, for early TWC enrollees, employment rates after the subsidized-employment portion of the TWC program (that is, in quarter 3 and beyond) are lower than for RSC enrollees but follow a similar pattern. Compared with early TWC enrollees, late TWC enrollees appeared to have more success during the transitional work component of the program but poorer employment outcomes afterward.^[5]  Since late enrollees make up about half our TWC participant sample, their poorer employment outcomes during the unsubsidized portion of the program could have contributed to lower average employment rates for TWC participants overall.

Figure III.2
Philadelphia WTW Outcomes Study:
Monthly Unemployment for Philadelphia

Source: Local Area Unemployment Statistics, BLS

 

Figure III.3
Philadelphia WTW Outcomes Study:
Employment Rates Over Time for RSC, TWC-Early, and TWC-Late Enrollees

Source: Administrative Data from State of Pennsylvania

Service Receipt and Program Completion

Differences in the services that the RSC and TWC programs offered and in how intensely or completely participants engaged in these services could also have influenced participant outcomes.  Differences in RSC and TWC services were largely by design.  That is, the RSC and TWC programs targeted somewhat different groups of WtW-eligible clients and, consistent with such targeting, emphasized different services (placement assistance versus work experience) and offered different levels of services (30 days versus six months).  In contrast, participants levels of engagement and success in completing program services commonly reflect a combination of both program and participant factors.  For example, participants can become more or less engaged in a program because of the types or convenience of services offered.  The overall duration and intensity of services offered can also influence participants likelihood of completion.  Program engagement levels can also reflect differences in both observable and unobservable participant characteristics (that is, self-selection).  That is, motivated clients usually participate willingly and seek services, while harder-to-serve clients may avoid services or resist participation requirements.  Next, we examine the types and levels of services that RSC and TWC participants received in the year after WtW enrollment, as well as the likelihood of their completing the program.

Most RSC and TWC participants reported receiving some labor market services.At followup, a majority of both RSC and TWC participants reported having received some type of labor market service during the year after enrollment. These services included job readiness training, job search or placement services, and life skills or self-management training.  More TWC participants (85 percent, compared with 79 percent of RSC participants) said they had received labor market services (Table III.2).

Consistent with differences in program design, TWC participants reported receiving more intensive labor market services.  Based on survey responses, we estimated the median number of days of job readiness training for TWC service recipients at 44 days, compared with 24 days for RSC service recipients.  TWC service recipients also reported receiving more job search or placement assistance and more life skills or self-management training (Table III.2).

Table III.2
Philadelphia WTW Outcomes Study:
Receipt of Employment Preparation Services
During the Year after Enrolling in Welfare-to-Work

	RSC Enrollees	TWC Enrollees	Significance
Receipt of Labor Market Services (Percentages)
Job readiness training	70.7	80.1	***
Job search or placement services	68.6	72.6	*
Life skills or self-management training	39.7	51.6	***
Any Labor Market Service (Any of the Above)	78.7	85.1	***
Duration of Labor Market Services for Those Who Received Them  (Median Number of Days)
Job readiness training	23.8	44.0	NA
Job search or placement services	3.5	5.5	NA
Life skills or self-management training	14.0	23.8	NA
Source: 2001-2003 12-month follow-up survey of Welfare-to-Work participants, MPR. NA = not available. */**/*** Significantly different from zero at the .10/.05/.01 level, two-tailed test.

Placement success serves as a proxy for completion.  We used program MIS records to examine the proportions of RSC and TWC participants who were successfully placed in unsubsidized employment through the programs and, hence, appeared to complete program services.  Our approach was driven largely by limitations in the available MIS data.^[6]  Nevertheless, this seemed a reasonable approach in the context of the RSC and TWC programs, since neither terminated clients because staff determined they could not be successfully placed in unsubsidized employment.  That is, both programs were committed to placing all enrolled WtW participants in unsubsidized jobs and would terminate a client only if he or she stopped participating.  Moreover, lack of placement success through the programs does not necessarily predict poorer outcomes for RSC or TWC participants.  The participants who were terminated from either program without being placed in unsubsidized employment could have been more motivated and committed to finding employment on their own, they could have been the hardest-to-serve, or they could have been a combination of these two types.

TWC participants were less likely to reach the point of unsubsidized job placement.  About 36 percent of TWC participants were placed in unsubsidized employment through the program, compared with 59 percent of RSC participants (Figure III.4).^[7]  This finding is not surprising, given the longer duration and higher intensity of the TWC program.  TWC participants may have faced a higher bar for program completion, that is, reaching placement in an unsubsidized job after successfully completing their transitional position.  In contrast, RSC participants immediately focused on securing placement in unsubsidized work.  Given their somewhat greater disadvantage, this objective may have been hard for TWC participants to achieve.

Figure III.4
Philadelphia WTW Outcomes Study:
Rates of Completion of the RSC and TWC Programs

Source: Administrative Data from State of Pennsylvania

Outcomes varied by participants completion of either program.  People who complete program interventions typically have better outcomes than those who do not, and, as discussed, differences in program completion tend to reflect both participant and program factors.  To explore whether the different rates of completion observed for the RSC and TWC programs could have made a difference in participant outcomes, we examined separately the employment rates, quarterly earnings, and rates of TANF receipt for RSC and TWC participants placed in unsubsidized jobs through the programs (program completers) and for RSC and TWC participants who had no program record of unsubsidized job placement (noncompleters).  Completers in both the RSC and TWC programs had higher employment rates, higher quarterly earnings, and lower rates of TANF receipt than their noncompleter counterparts (Figures III.5, III.6, and III.7).  These findings suggest (not surprisingly) that remaining sufficiently engaged in either program to reach the point of unsubsidized job placement may have benefited people more than merely participating.  The findings also suggest that differences in rates of program completion may have contributed to the observed differences in average RSC and TWC outcomes.

Figure III.5
Philadelphia WTW Outcomes Study:
Employment Rates Over Time for RSC and TWC "Completers" Versus "Noncompleters"

Source: Administrative Data from State of Pennsylvania

Figure III.6
Philadelphia WTW Outcomes Study:
Earnings Over Time for RSC and TWC "Completers" Versus "Noncompleters"

Source: Administrative Data from State of Pennsylvania

Figure III.7
Philadelphia WTW Outcomes Study:
Rates of TANF Receipt Over Time for RSC and TWC "Completers" Versus "Noncompleters"

Source: Administrative Data from State of Pennsylvania

For both programs, there were important differences in the characteristics of completers versus noncompleters.  Relative to RSC completers, RSC noncompleters were more likely to be African American and unmarried, have a child under age 3 in their household, lack a high school diploma or GED, lack recent work experience (within the past year), be long-term recipients of public assistance (60 months or more), and be responsible for another person with a health or other condition that limits their ability to work (Table III.3).  There were fewer significant differences in the baseline characteristics of TWC completers versus noncompleters.  Relative to the completers, TWC noncompleters were more likely to lack a high school diploma or GED.^[8]

Table III.3
Philadelphia WTW Outcomes Study:
Baseline Characteristics of RSC and TWC enrollees,
by Program Completion

Baseline Characteristics	RSC Completers	RSC Noncompleters	Significance	TWC Completers	TWC Noncompleters	Significance
Demographic Characteristics
Age Category
Younger than 20 years	0	0	ns	0	0	ns
20 to 29 years	45	43	ns	44	43	ns
30 to 39 years	41	41	ns	41	41	ns
40 years or older	14	16	ns	15	16	ns
Average Age	31.8	32.3	ns	32.4	32.2	ns
Gender
Female	96	98	***	98	97	ns
Race/Ethnicity
Hispanic	12	10	ns	9	12	ns
White non-Hispanic	6	3	***	2	4	ns
Black non-Hispanic	79	85	***	88	82	**
Other non-Hispanic	3	2	ns	1	2	ns
Household Structure
Marital Status
Married	7	3	***	3	5	*
Cohabiting	5	6	ns	6	6	ns
Separated/divorced/widowed	17	14	*	14	15	ns
Never married	71	76	**	77	74	ns
Number of Children in Household
0	4	5	ns	4	5	ns
1 to 2	57	50	***	48	54	*
3 to 5	33	40	***	40	37	ns
6 or more	5	5	ns	8	4	***
Average Number of Children in Household	2.4	2.6	ns	2.9	2.4	ns
Age of Youngest Child in Household
Younger than 3 years	13	20	***	24	17	***
Younger than 5 years	42	43	ns	47	41	**
Average Age of Youngest Child in Household	6.6	6.2	ns	6.0	6.5	ns
Education
Educational Attainment
High school dropout	38	45	***	35	45	***
Still attending high school or GED program	3	6	***	5	5	ns
GED	5	4	ns	5	5	ns
High school diploma	20	15	**	19	16	ns
Postsecondary degree	3	2	ns	3	3	ns
Vocational/technical certificate	32	28	*	34	27	**
At Least High School Diploma or GED	59	49	***	60	50	***
Employment History
Employment at Baseline
Currently employed	9	8	ns	7	8	ns
Employed within the past year	58	48	***	53	51	ns
Employed more than a year ago	28	33	***	33	32	ns
Never employed in the past	5	10	***	8	9	ns
Welfare Receipt
TANF Receipt at Baseline
Currently receiving	92	91	ns	94	90	**
Received in the past but not currently receiving	5	5	ns	3	6	ns
Never received	3	3	ns	2	4	*
Total Time on TANF or AFDC
Never on TANF or AFDC	3	3	ns	2	4	*
1 to 24 months	33	24	***	22	28	*
25 to 60 months	29	31	ns	35	30	**
More than 60 months	35	41	**	40	39	ns
Health Problems
Work-Limiting Problem
Own	19	21	ns	18	22	ns
Other household member	7	11	**	12	13	ns
Type of Work-Limiting Problem (Own)
Medical condition	7	11	**	8	10	ns
Physical disability	4	2	*	2	3	ns
Mental health or substance abuse problem	3	5	*	5	5	ns
Sample Size	633	386		427	636
Source: Baseline information forms of Welfare-to-Work participants, Mathematica Policy Research. ns = not significant. */**/*** Significantly different from zero at the .10/.05/.01 level, two-tailed test.

Both TWC completers and TWC noncompleters were similar to RSC noncompleters.  As noted, there were few observable differences of statistical significance among TWC completers and noncompleters.  In addition, we found that the average TWC participant  regardless of program completion  was similar along observable characteristics to the average RSC noncompleter.  For instance, 43 percent of RSC noncompleters lacked recent work history (that is, within the past year; Table III.3), compared with 44 percent of TWC participants overall (Table III.1) and 41 percent of both TWC completers and TWC noncompleters (Table III.3).^[9]  This suggests that the TWC program was well targeted  that it served people with a strong likelihood of failure at the RSC intervention.

Summary

In this chapter, we explored a variety of factors that could have contributed to the difference in outcomes for RSC and TWC participants.  We found some statistically significant differences in the background characteristics of RSC and TWC participants.  These differences were generally small, however, suggesting that both programs worked with fairly disadvantaged populations.  We also found differences in the periods of sample enrollment and in deteriorating economic conditions over time, which could have played a role in the deteriorating employment outcomes, especially for later TWC participants.  Finally, we found important differences in the rates of program completion among RSC and TWC participants, in the characteristics of completers versus noncompleters of both programs, and in the outcomes of participants in either program according to whether or not they completed the programs.  These differences could reflect differences in observable and unobservable participant characteristics, as well as differences in the services offered by, and received by participants from, the RSC and TWC programs. In the next chapter, we examine the extent to which differences in observable characteristics of RSC and TWC participants, economic conditions, and program completion account for differences in the outcomes for RSC and TWC participants.

Endnotes

[1] As noted in Chapter I, because of insufficient referrals to the TWC program, this two-stage client-flow process eventually was modified so that WtW-eligible TANF recipients could be referred directly to TWC.  To be referred directly to TWC, however, WtW-eligible clients still had to have limited or no work experience or other severe barriers to employment.

[2] The proportion of the sample who do not receive TANF are most likely noncustodial parents, as males make up 5 percent of RSC participants and 1 percent of TWC participants.

[3] As already noted, the TWC program encountered enrollment challenges early in its operations that affected enrollment into our study sample.  This can be seen in the lower rate of enrollment into our study through January 2000 (Figure III.1).

[4] January 2001 was the month when TWC participants who were enrolled in the program in June 2000 and later would typically be moving to unsubsidized employment (after their six-month subsidized work experience assignment).  As noted, most RSC sample members had enrolled in the program by June 2000.

[5] Higher employment rates during the first two quarters after program enrollment for the late TWC enrollees suggest improved program success with placement in transitional work, possibly due to maturation of the program.  Worsening economic conditions could also have contributed to this pattern, since qualifying employers may have become more receptive to hosting free TWC workers.

[6] Neither the RSC nor the TWC data contained information on completion of specific program components.  The programs defined completion as a participant obtaining an unsubsidized job and retaining employment for their stipulated retention support period.  However, the available data did not include information that would allow the definition of program completion in this way.

[7] The placement rate we estimate for the TWC program is somewhat lower than those reported in other reports, including a cost analysis of WtW programs (48 percent, Perez-Johnson et al. 2002) and a study of transitional employment programs (48.5 percent, Kirby et al. 2002).  These differences mainly reflect differences in the participant samples examined.  Both earlier studies examined outcomes for TWC participants enrolled through December 2000, while this study includes participants enrolled as late as May 2002.  The placement rate we estimate could be lower because of (1) some truncation of employment records for participants enrolled later in the program, and (2) weakening economic conditions over time.

[8] We found other significant, yet counterintuitive, differences in the characteristics of TWC completers versus noncompleters.  Specifically, TWC completers were more likely to have six or more children, to have a child under age 5, and to have been on TANF for more than 24 months. Higher rates of program completion among TWC participants with six or more children and participants with younger children could reflect special efforts on the part of TWC staff to attend to the needs of such participants.  Higher rates of program completion among people on TANF for more than 24 months could reflect work participation requirements.

[9] The difference in rates for TWC participants overall (44 percent) versus TWC completers and noncompleters (41 percent) is due to missing data for participants who were still active in the intervention at the time the program provided MIS records.

Interpretation of Findings and Study Conclusions

The differences in overall characteristics of RSC and TWC participants, economic conditions at the time of participation, and other observable factors discussed in the previous chapter are likely to accountВ  at least in partВ  for the differences in employment, earnings, and TANF receipt outcomes found for participants in these two programs.В  We use multivariate statistical analysis techniques to help explain the differences in the outcomes of RSC and TWC participants.В  Only an experimental design evaluation could determine conclusively the extent to which the RSC and TWC programs contribute to participant outcomes.В  Nevertheless, given the evidence available from our analyses, we also discuss possible implications of our study findings for future programs.В 

We have organized the discussions in this chapter around two broad sets of questions:

• What factors help explain the differences in RSC and TWC outcomes? В What factors are most important in explaining the differences in outcomes between RSC and TWC participants?В  How much of the difference can be explained?
• How can our findings help inform future programs?В  What factors are associated with program success?В  How might programs such as the TWC and RSCs better identify and serve participants needs?

What Factors Help Explain RSC-TWC Differences in Outcomes

To identify the factors that contributed to differences in RSC-TWC outcomes, we regressed key outcomes on participants demographic characteristics, prior work experience, prior TANF receipt, economic conditions after program entry, and an indicator of RSC/TWC status.  The primary goal of this analysis was to assess the extent to which the parameter estimate on the RSC/TWC indicator variable could be reduced when the observable participant characteristics and other factors were included as explanatory variables in the models.  We used three multivariate statistical analysis techniques:  ordinary least squares (OLS) regression, fixed-effects regression, and propensity scoring.  Using these techniques, we examined participant outcomes six quarters after the quarter of program enrollment  the latest point for which we have consistent follow-up data for most RSC and TWC participants. 

The results from these three techniques were fairly consistent (Table IV.1).  For example, the predicted difference in TWC-RSC employment rates six quarters after enrollment was 4.9 percent and insignificant in the OLS model, 6.7 percent and only marginally significant in the fixed-effects model, and 3.8 percent and insignificant in the propensity scoring model.  This consistency across techniques suggests that the results are robust.  To simplify the discussion, Chapter IV focuses on the OLS regression results.  Our main results can be summarized as follows:

Observable factors account for most of the difference in TWC-RSC employment rates.  The simple difference (that is, before taking into account differences in observable factors) in employment rates a year and a half after program enrollment for TWC and RSC participants was a statistically significant 14 percentage points.  After demographics, prior employment, and economic conditions are taken into account in the OLS model, however, the predicted difference in TWC-RSC employment rates becomes smaller (4.9 percent) and statistically insignificant (Table IV.1).

 

Table IV.1
Philadelphia WTW Outcomes Study:
Estimated Difference Between TWC and RSC Outcomes Six Quarters after Program Entry

Statistical Method	Employment	Earningsa	TANF Receipt
	Difference in the Percentage Employed (TWC-RSC)	Difference in Dollars (TWC-RSC)	Difference in the Percentage Receiving TANF (TWC-RSC)
Simple Difference in Means	14.1***	598.24***	16.7***
OLS Regression	4.9	367.62***	11.6***
Fixed-Effects	6.7*	-320.31***	11.7***
Propensity Scoring	3.8	347.93***	13.8***
Source:  Baseline information forms of Welfare-to-Work participants, Mathematica Policy Research, Inc.; state administrative records data; and RSC and TWC Management Information Systems data.
Note:  All models include demographics, prior work or prior TANF receipt, and unemployment rate as explanatory variables.  Prior work is included in employment and earnings models; prior TANF receipt is included in TANF receipt model.
a The earnings models include participants who were not employed and had zero earnings.
*/**/*** Significantly different from zero at the .05/.01/.001 level, two-tailed test.

Even after observable factors are controlled for, however, about half the TWC-RSC differences in earnings and TANF receipt remain. Six quarters after program enrollment, TWC participants earned, on average, about $600 less than RSC participants (Table IV.1).  After observable factors are controlled for, the predicted difference in postprogram earnings between TWC and RSC participants declines to $368 and remains statistically significant.  Similarly, the difference in TANF receipt rates six quarters after enrollment between TWC and RSC participants is reduced from 17 to 12 percent and remains statistically significant.

Thus, participation in TWC does not appear to lead to a full catching up to the outcomes of RSC participants.  Despite being about equally likely to be employed six quarters after program enrollment, TWC participants had lower earnings and were more likely to receive TANF than comparable RSC participants.  These differences could be due to unobserved factors.  Another possible interpretation of this finding is that subsequent employers do not value the time TWC participants spent in transitional work as highly as time spent in unsubsidized employment.  Thus, when TWC participants finally moved into unsubsidized jobs, they still entered jobs comparable to those first entered by RSC participants.^[1]  To the extent that TWC participation delayed participants entry into unsubsidized employment, this would mean that, compared with RSC participants, TWC participants may have been at an earlier point in the development of their employment capabilities.  They may have had less time to achieve gains in earnings due to advancement within jobs or to progress to better-paying jobs.^[2]  The lower earnings of TWC participants, in turn, could have contributed to their higher rates of TANF receipt.

Educational attainment, prior earnings, and prior TANF receipt were key factors in explaining outcomes. Both educational attainment and prior earnings provide a good indication of peoples skills and prior workplace performance, and thus their ability to succeed in the labor market.  Not surprisingly, having a high school diploma or a GED was a highly significant factor related to employment, earnings, and TANF receipt (Table IV.2).  Average earnings in the four quarters before program entry were also significantly related to both postprogram employment and earnings.  Similarly, TANF receipt in all four quarters before program enrollment was significantly related to TANF receipt six quarters after program enrollment.

 

Table IV.2
Philadelphia WTW Outcomes Study:
Factors Related to Outcomes Six Quarters
After Program Entry, Based on OLS Regression Results

Factor Associated with Outcomes	Employment	Earnings	TANF Receipt
Program Status
Participated in TWC	.05	367.62***	.116***
Baseline Characteristics
Age Is Less than 30	.028	45.46	.073*
Age Is Greater than 30 and Less than 40	.015	98.77	.016
Female	.038	605.51*	.261***
Hispanic	.049	57.80	.099**
White	.007	131.72	.105*
Other Race/Ethnicity	.043	159.08	.120
Married or Cohabiting	.021	19.17	.040
Number of Children	.002	20.01	.027***
Age of Youngest Child Less than 5	.033	67.57	.048*
Has High School Diploma or GED	.085***	513.64***	.130***
Own Health Problem Limits Ability to Work	.066*	120.50	.030
Family Members Health Problem Limits Ability to Work	.019	70.00	.010
Economic Conditions
Unemployment Rate in Quarter 6 After Program Entry	.101***	191.50*	.016
Prior Employment
Never Employed Before Program Entry	.051	123.31
Employed in All Four Quarters Before Program Entry	.029	29.50
Average Earnings in Four Quarters Before Program Entry	.273***	763.01***
Prior TANF Receipt
Received TANF in All Four Quarters Before Program Entry			.121***
Received TANF Two to Five Years			.000
Received TANF Five or More Years			.020
Sources:  Baseline information forms of Welfare-to-Work participants, Mathematica Policy Research, Inc.; state administrative records data; and RSC and TWC Management Information Systems data.
Note:  Missing values for race/ethnicity, health problem, family members health problem, and length of TANF receipt were imputed.  Dummy variables for imputed cases, included in the model, were not significant.
*/**/*** Significant at the .05/.01/.001 level, two-tailed test.

Job placement success also was an important factor in explaining later employment, earnings, and TANF receipt. In some analyses, we included as an explanatory variable an indicator of program completion  that is, whether the RSC or TWC participant had successfully reached the point of unsubsidized job placement through the program  to capture unmeasured characteristics, such as greater motivation or a positive attitude, likely to have made participants more job ready.  When included in the final OLS regression model, program completion further reduces the predicted difference in TWC-RSC participant outcomes.  For example, the predicted difference in earnings declines from $368 to $248.  Similarly, the predicted difference in rates of TANF receipt declines from 10.4 to 8.9 percent.  Hence, differences between TWC and RSC participants along the unmeasured characteristics captured by program completion may be another factor contributing to their differences in outcomes.  At the same time, program completion may also measure the programs ability to engage participants in activities and help them find unsubsidized jobs.  Thus, its inclusion in our regression models may make the remaining difference between RSC and TWC participant outcomes an understatement of real differences in program effects.

RSC noncompleters offer a further comparison group for TWC participants.  Failure to complete the program was one way the RSCs identified participants likely to need the more intensive services offered by the TWC program.  It is unclear why some RSC noncompleters were not referred to TWC  in theory, all of them should have been  but, as discussed in Chapter III, we do know that, consistent with the programs targeting, RSC noncompleters were very similar in their overall characteristics to TWC participants.  To the extent that RSC noncompleters truly resembled TWC participants (that is, along both observed and unobserved characteristics), their outcomes offer some suggestion of the outcomes TWC participants might have achieved without this intervention.^[3]

On average, TWC participants had outcomes similar to those of RSC noncompleters. After we control for observable factors, there are only small, insignificant differences in employment, earnings, and TANF receipt between TWC participants (both completers and noncompleters) and RSC noncompleters (Table IV.3).  Therefore, regardless of which program they were involved in, TWC participants (in general) and RSC noncompleters fared similarly over time.  As discussed in Chapter III, there were important differences in the outcomes of TWC completers and TWC noncompleters (although their baseline characteristics are similar).  As a result, comparing the average outcomes of TWC completers and noncompleters to the outcomes of RSC noncompleters is likely to mask important relationships.  Thus, we compare the RSC noncompleters to TWC completers and TWC noncompleters separately.

 

Table IV.3
Philadelphia WTW Outcomes Study:
Estimated Difference in Outcomes Between TWC Participants
and RSC Noncompleters Six Quaters after Program Entry

	Employment	Earningsa	TANF Receipt
Statistical Method	Difference in Percentage Employed	Difference in Dollars	Difference in the Percentage Receiving TANF
All TWC Participants vs. RSC Noncompleters
Simple Difference in Means	0.4	93.23	6.6*
OLS Regression	5.0	20.1	1.8
TWC Noncompleters vs. RSC Noncompleters
Simple Difference in Means	11.3***	487.27***	12.4***
OLS Regression	1.8	395.98***	10.9**
TWC Completers vs. RSC Noncompleters
Simple Difference in Means	11.7***	460.30***	3.8
OLS Regression	11.1**	471.27***	7.9*
Source: Baseline information forms of Welfare-to-Work participants, Mathematica Policy Research, Inc.; state administrative records data; and RSC and TWC Management Information Systems data.
Note:   All models include demographics, prior work or prior TANF receipt, and unemployment rate as explanatory variables.  Prior work is included in the employment and earnings models.  Prior TANF receipt is included in the TANF receipt models.
aThe earnings models include participants who were not employed and had zero earnings.

The postprogram outcomes of TWC completers are significantly better than the outcomes of RSC noncompleters.  Six quarters after program enrollment, TWC completers were 11 percent more likely to be employed, earned about $470 more, and were 8 percent less likely to receive TANF than comparable RSC noncompleters Table IV.3).  To the extent that TWC served people unlikely to succeed in the RSC program, this suggests that the program may have helped these participants achieve better outcomes.  Because the original RSC-TWC referral process eventually broke down, it is also possible that TWC completers include people who enrolled directly in this program but could have succeeded in securing unsubsidized employment through the RSCs.  To the extent this happened, apparent differences between the outcomes of TWC completers and RSC noncompleters would be an overstatement of TWCs success.  Given the lack of an experimental design, we cannot determine which TWC completers may have succeeded in getting unsubsidized jobs with the less intensive help of the RSC programs, nor the extent to which TWC may have helped convert actual or potential RSC failures into successes.

However, TWC noncompleters fared much worse than RSC noncompleters.  TWC noncompleters were as likely as RSC noncompleters to be employed six quarters after program referral, but they earned about $400 less and were 11 percent more likely to receive TANF (Table IV.3).  This suggests that TWC noncompleters may have been the most disadvantaged of the WtW population  unable to complete either the TWC or the RSC program.  Their poor outcomes highlight the importance of identifying and addressing factors contributing to participants lack of success in these types of programs.  The marked differences in outcomes between TWC noncompleters and RSC noncompleters further suggest that important, unobserved differences among TWC and RSC participants remain unaccounted for in our study.

Summary

We used multiple methods  OLS regression, fixed-effects regression, and propensity scoring  to attempt to control for differences in the characteristics of TWC and RSC participants(both observed and unobserved) and other factors likely to have contributed to their differences in outcomes.  These regression adjustments reduced the TWC-RSC differences in employment, earnings, and TANF receipt but did not erase them completely.  The inclusion of a program completion term  to capture additional unobserved participant characteristics  further reduced the difference in outcomes.  In a strategy analogous to propensity scoring, we also restricted our analysis to RSC noncompleters, who were very similar to TWC participants along observable characteristics and who, in theory, should have been referred to the TWC.  This analysis revealed that, regardless of which program they were involved in, TWC participants (in general) and RSC noncompleters fared similarly over time.  Marked differences in the regression-adjusted outcomes of RSC noncompleters, TWC completers, and TWC noncompleters suggest, however, that important, unmeasured differences remain unaccounted for in our study.  Hence, we are unable to reach definitive conclusions about the effects of these programs.  The potential benefits of subsidized work experience relative to direct placement in unsubsidized employment for the hard to employ can be assessed only through a randomized trial of such programs.

Implications for Future Programs and Study Conclusions

Although our study of outcomes of RSC and TWC participants cannot offer definitive conclusions, it suggests themes that could contribute to the further development of programs aimed at helping the hard to employ succeed in their transition from welfare to work.  This section assembles these themes and presents our broad conclusions.

Intensive services can be targeted to the most disadvantaged.  The original design of the RSC and TWC programs and, in particular, their sequencing within the larger GPW initiative represented an innovative, commendable approach to program development.  Believing that failure at the RSCs should not be the only way to secure referral to TWC, program developers specified that staff would have discretion to identify people likely to need more intensive services and refer them directly to the TWC program.  This would accomplish several important objectives.  It would avert the costly waste of resources in delivering RSC services to participants unlikely to succeed with their basic assistance, minimize the time participants spent in services before successfully transitioning off time-limited TANF, and avoid the potential discouragement of participants required to fail at one program before gaining access to more appropriate services.  We can reasonably assume that most of the TWC participants in our study enrolled directly in the program, since enrollments increased markedly after direct TWC outreach was allowed and RSC referrals had been limited to that point.  Thus, our finding that TWC participants, in general, were similar along observable characteristics to RSC noncompleters suggests that the intended targeting was both feasible and successful in these programs.

Programs targeting the hard to employ may be more effective if they devote attention to identifying and addressing factors that contribute to participants lack of success.  As discussed, TWC participants who did not achieve job placement through the TWC program fared worse than any other RSC or TWC participants.  These people may have been the most disadvantaged among the WtW-eligible population.  Intensive programs aimed at serving this population need to identify and address the barriers they face.  One clue that our study offers regarding factors that may contribute to lack of success is that TWC participants without a high school diploma were less likely to complete the program.^[4]  However, simply focusing on education is unlikely to lead to improved outcomes for these people, since earlier studies have shown that providing education services alone does not generally lead to improved employment outcomes (Michalopoulos and Schwartz 2000; and Burghardt et al. 1992).  Those who did not succeed at TWC are likely to have a variety of complex barriers that, unfortunately, remain unmeasured in our study.^[5]

The hardest-to-employ participants in intensive programs like TWC may be especially vulnerable during periods of high unemployment.  The only other highly significant factor in predicting program completion among TWC participants was economic conditions, as measured by the local unemployment rate.^[6]  This suggests that programs like TWC may need to offer even more intensive placement help to participants in times when there is more competition for available unsubsidized jobs.  Because experience in transitional work may not be as highly valued as unsubsidized work experience, transitional jobs may need to include more skill building and training (to make participants more attractive to prospective employers), and placements may need to be longer.  In addition, program staff may need to take on an even more active role in unsubsidized job placement than in a time of more favorable economic conditions.

Services related to retention and advancement remain important in helping participants build on their employment experience and achieve further gains.  Our study confirms that those who maintain employment continue to build on these experiences and increase their earnings over time.  In addition, the RSC and TWC participants who switched jobs tended to move to jobs with better wages, hours, and benefits.  Thus, both job retention and advancement services, including ongoing job search and placement services, are potentially important components to help participants build a strong employment history leading them to further employment success.  

Further research is needed to clarify how programs like the RSCs and the TWC contribute to participant outcomes.  Shortfalls in program enrollment made it impossible to implement the original random-assignment design planned for this evaluation.  Our results hint that the intensive TWC intervention may have partially, but not completely, made up for the greater employment challenges TWC participants faced.  Nevertheless, our study leaves unanswered questions that only a more rigorous evaluation can answer.  Large scale experiments provide evidence that programs promoting rapid attachment while allowing for some education and training are particularly effective in helping welfare recipients increase earnings and reduce welfare receipt (Hamilton 2002).  Transitional work programs, like the TWC, have a similar approach in that they promote rapid entry to work while incorporating ongoing skill-building.  Further study is needed to determine the actual effects of transitional work on participants outcomes and the most appropriate targeting and sequencing of programs like the TWC and the RSCs.

Endnotes

[1] As supporting evidence for this explanation, note that the average earnings of TWC participants three quarters after program entry  when they would have transitioned fully out of unsubsidized employment  are comparable to the earnings of RSC participants one quarter after program entry (Figure III.6).

[2] This interpretation would be consistent with findings from our follow-up survey of WtW participants.  As discussed in Chapter II, survey results show that RSC and TWC participants who changed jobs during the first year after program enrollment had higher earnings and worked more hours, or both, and that these gains played a role in their increased earnings.

[3] There are several possible explanations for why RSC noncompleters were not referred to the TWC program.  As we noted in Chapter I, Pennsylvania is a client choice state.  Hence, when referred to the Philadelphia welfare agencies for referral to another program, RSC noncompleters could have opted out of TWC (for example, because of location or other preferences) and chosen a different employment program.  They could also have been exempted from work requirements or could have left TANF altogether.  If RSC noncompleters tend to be people who were systematically excluded from, or opted out of, participation in TWC, their outcomes would not necessarily provide a good representation of the likely outcomes of TWC participants in the absence of this intervention.

[4] For further information on factors associated with program success, see Appendix Table A.6.

[5] For example, a study of barriers to completion in Philadelphias Single Point of Contact program  one of the other employment assistance programs available to work-mandatory TANF recipients in Philadelphia  suggests that noncompleters often faced many barriers, including child care concerns, domestic violence, and low self-efficacy (Kinnevy et al. 2003).

[6] Since most RSC participants enrolled before the economic downturn in 2000, they had little variation in economic conditions.  Therefore, an association with economic conditions was less likely to emerge for these participants.

References

Agodini, Roberto, and Mark Dynarski .Are Experiments the Only Option? A Look at Dropout Prevention Programs. Princeton, NJ: Mathematica Policy Research, Inc., August 2001.

Andersson, Fredrik, Harry Holtzer, and Julia Lane. Worker Advancement in the Low-Wage Labor Market: The Importance of Good Jobs. Washington, DC: Center of Urban and Metropolitan Policy, Brookings Institution, October 2003.

Ashenfelter, Orley, and David Card. Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs.Review of Economics and Statistics, vol. 67, 1985.

Burghardt, John, Anu Rangarajan, Anne Gordon, and Ellen Kisker. Evaluation of the Minority Single Parent Demonstration: Volume One: Summary Report. Princeton, NJ: Mathematica Policy Research, Inc., October 1992.

Corson, Walter, Paul Decker, and Anne Gordon. New Jersey Unemployment Insurance Reemployment Demonstration Project Final Evaluation Report. Princeton, NJ: Mathematica Policy Research, Inc., 1989.

Fraker, Thomas, Dan Levy, Robert Olsen, and Rita Stapulonis. The Welfare-to-Work Grants Program: Enrollee Outcomes One Year After Program Entry: Revised Draft Report. Washington, DC: Mathematica Policy Research, Inc., November 2003.

Hamilton, Gayle. Moving People from Welfare to Work: Lessons from the National Evaluation of Welfare-to-Work Strategies. New York: Manpower Demonstration Research Corporation, July 2002.

Kinnevy, Susan, Caroline Wong, and Lisa Colby. Barriers to Completion of Welfare-to-Work Programs: The Single Point of Contact Program in Philadelphia. Philadelphia: Center for Research on Youth and Social Policy, School of Social Work, University of Pennsylvania, November 2003.

Kirby, Gretchen, Heather Hill, LaDonna Pavetti, Jon Jacobson, Michelle Derr, and Pamela Winston. Transitional Jobs: Stepping Stones to Unsubsidized Employment. Washington, DC: Mathematica Policy Research, Inc., April 2002.

Michalopoulos, Charles. The Effects of Welfare Reform in Philadelphia. New York: Manpower Demonstration Research Corporation, November 2003.

Michalopoulos, Charles, and Christine Schwartz. National Evaluation of Welfare-to-Work Strategies: What Works Best for Whom: Impacts of 20 Welfare-to-Work Programs by Subgroup. New York: Manpower Demonstration Research Corporation, August 2000.

Wood, Robert, and Anu Rangarajan. Issue Brief: Whats Happening to TANF Leavers Who Are Not Employed? Princeton, NJ: Mathematica Policy Research, Inc., October 2003

Appendix A: Appendix Tables and Figures

Table A.1
Philadelphia WTW Outcomes Study:
Trends in Employment Rates for RSC and TWC Participants

Quarter Relative to Program Entry	RSC (Percentages)	TWC (Percentages)	Significance
-4	37.7	35.2	*
-3	39.4	35.2	***
-2	43.7	36.2	***
-1	40.3	33.7	***
0	47.8	78.8	***
1	58.6	71.3	***
2	54.4	55.8
3	51.3	45.6	***
4	52.1	44.7	***
5	50.3	41.6	***
6	49.4	38.9	***
7	47.3	NA	n.a.
8	45.5	NA	n.a.
Sample Size	2,338	2,543
Source:  State administrative records data.
NA = not available.
n.a. = not applicable.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

 

Table A.2
Philadelphia WTW Outcomes Study:
Average Earnings

Quarter Relative to Program Entry	RSC (Dollars)	TWC (Dollars)	Significance
-4	519.71	8.66	***
-3	556.52	422.82	***
-2	620.52	414.54	***
-1	506.93	294.93	***
0	455.24	532.50	***
1	1,001.16	889.67	***
2	1,132.00	828.67	***
3	1,136.10	829.12	***
4	1,231.50	841.28	***
5	1,236.33	827.81	***
6	1,274.78	777.74	***
7	1,264.30	NA	n.a.
8	1,235.26	NA	n.a.
Sample Size	2,338	2,543
Source:  State administrative records data.
NA = not available.
n.a. = not applicable.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

 

Table A.3
Philadelphia WTW Outcomes Study:
TANF Participation Rate

Quarter Relative to Program Entry	RSC (Percentage)	TWC (Percentage)	Significance
-4	81.2	88.5	***
-3	81.5	88.8	***
-2	82.3	90.1	***
-1	86.0	93.7	***
0	94.3	98.2	***
1	85.6	93.3	***
2	68.9	84.2	***
3	60.8	75.9	***
4	56.1	71.1	***
5	52.7	66.4	***
6	49.3	64.0	***
7	46.9	NA	n.a.
8	45.4	NA	n.a.
Sample Size	2,338	2,543
Source: State administrative records data.
NA = not available.
n.a. = not applicable.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

 

Table A.4
Philadelphia WTW Outcomes Study:
Combination of Employment and TANF Receipt

	Employed on TANF			Employed, Not on TANF			Not Employed, on TANF			Not Employed, Not on TANF
Quarter Relative to Program Entry	RSC (Percentages)	TWC (Percentages)	Sig.	RSC (Percentages)	TWC (Percentages)	Sig.	RSC (Percentages)	TWC (Percentages)	Sig.	RSC (Percentages)	TWC (Percentages)	Sig.
-4	28.8	29.0		8.9	6.1		52.4	59.5	***	9.8	5.4	***
-3	29.7	29.1		9.6	6.1		51.8	59.7	***	8.9	5.1	***
-2	34.1	31.0	**	9.6	5.2	**	48.2	59.1	***	8.1	4.7	***
-1	33.2	30.6	*	7.1	3.1	*	52.8	63.1	***	7.0	3.2	***
0	45.9	78.2	***	1.9	0.6	***	48.4	20.1	***	3.8	1.2	***
1	50.1	67.4	***	8.5	3.9	***	35.5	25.9	***	5.9	2.8	***
2	33.9	45.8	***	20.5	10.0	***	35.0	38.4	**	10.6	5.8	***
3	26.4	30.8	***	25.0	14.8	***	34.4	45.0	***	14.2	9.4	***
4	24.8	27.4	**	27.3	17.3	**	31.3	43.7	***	16.6	11.6	***
5	20.4	23.4	**	29.9	18.2	**	32.2	43.0	***	17.5	15.4	*
6	18.4	22.7	***	31.0	17.9	***	30.9	41.2	***	19.7	18.1
7	16.7	NA	n.a.	30.6	NA	n.a.	30.2	NA	n.a.	22.6	NA	n.a.
8	15.0	NA	n.a.	30.5	NA	n.a.	30.4	NA	n.a.	24.1	NA	n.a.
Sample Size	2,338	2,543		2,338	2,543		2,338	2,543		2,338	2,543
Source:  State administrative records data.
NA = Not Available.
n.a. = not applicable.
*/**/*** Difference between RSC and TWC estimates is statistically significant at the .10/.05/.01 level, two-tailed test.

 

Table A.5
Philadelphia WTW Outcomes Study:
Estimated Difference Between TWC and RSC Outcomes Six Quarters after
Program Entry, Including Program Completion Status as Explanatory Variable

	Employment	Earningsa	TANF Receipt
Statistical Method	Difference in the Percentage Employed (TWC-RSC)	Difference in Dollars (TWC-RSC)	Difference in the Percentage Receiving TANF (TWC-RSC)
OLS Regression
Without program completion	4.9	367.62***	10.4***
With program completion	2.4	247.62***	8.9***
Fixed-Effects Regression
Without program completion	6.7*	320.31***	11.7***
With program completion	7.3*	338.66***	11.8***
Propensity Scoring Model
Without program completion	3.8	347.93***	13.8***
With program completion	5.0	308.69***	10.0***
Source: Baseline information forms of Welfare-to-Work participants, Mathematica Policy Research, Inc.; state administrative records data; and RSC and TWC Management Information Systems data.
Note:  All models include demographics, prior work or prior TANF receipt, and unemployment rate as explanatory variables.  Prior work is included in employment and earnings models; prior TANF receipt is included in TANF receipt model.
a The earnings models include participants who were not employed and had zero earnings.
*/**/*** Significantly different from zero at the .05/.01/.001 level, two-tailed test.

 

Table A.6
Philadelphia WTW Outcomes Study:
Factors Associated with Placement Success,
Based on OLS Regression Results

Factors Associated with Placement Success	TWC	RSC
Baseline Characteristics
Age Is Less than 30	.046	.042
Age Is Greater than 30 and Less than 40	.011	.025
Female	.405*	.007
Hispanic	.119*	.110*
White	.151	.039
Other Race/Ethnicity	.097	.152
Married or Cohabiting	.049	.029
Number of Children	.018*	.002
Age of Youngest Child Less than 5	.057	.080*
Has High School Diploma or GED	.168***	.057
Own Health Problem Limits Ability to Work	.081	.054
Family Members Health Problem Limits Ability to Work	.059	.035
Economic Conditions
Unemployment Rate in Quarter 6 After Program Entry	.457***	.125
Prior Employment
Never Employed Before Program Entry	.064	.146*
Employed in All Four Quarters Before Program Entry	.012	.112
Average Earnings in Four Quarters Before Program Entry	.143*	.086
Prior TANF Receipt
Received TANF in All Four Quarters Before Program Entry	.041	.092*
Received TANF Two to Five Years	.056	.019
Received TANF Five or More Years	.064	.028
Source:  Baseline information forms of Welfare-to-Work participants, Mathematica Policy Research, Inc.; state administrative records data; and RSC and TWC Management Information Systems data.
Note:  Missing values for race/ethnicity, health problem, family members health problem, and length of TANF receipt were imputed.  Dummy variables for imputed cases, included in the model, were not significant.
*/**/*** Significant at the .05/.01/.001 level, two-tailed test.

Figure A.1
Philadelphia WTW Outcomes Study:
Employment Rates Over Time after Estimated Placement in Unsubsidized Work

Source: Administrative Data from State of Pennsylvania

Figure A.2
Philadelphia WTW Outcomes Study:
Earnings Over Time after Estimated Placement in Unsubsidized Work

Source: Administrative Data from State of Pennsylvania

Figure A.3
Philadelphia WTW Outcomes Study:
Employment Rates of TANF Receipt Over Time after Estimated Placement in Unsubsidized Work

Source: Administrative Data from State of Pennsylvania

Appendix B: Methodology

We employed multiple analytic methods to estimate the regression models in order to test the robustness of study findings. In particular, we used three statistical techniques to control for differences between RSC and TWC participants when modeling their postprogram outcomes:

1. Ordinary Least Squares (OLS) Regressions.В  This method estimated the relationship of each observable factor to outcomes while holding all other factors constant. The basic form of the OLS model is: y = ₀ + ₁*TWC + x + , where y is the outcome measure, TWC is an indicator variable that equals 1 for TWC participants and 0 for RSC participants, x is a vector of observable participant characteristics measured at baseline, the Greek letters are parameters to be estimated, and is a mean-zero error term. In this formulation, the estimate of ₁ represents the regression-adjusted TWC-RSC difference. We then compare these estimates to the simple differences in mean outcomes between the two participant groups.^[1]

2. Fixed-Effects Regressions.В  This approach used longitudinal data on outcomes over time to examine whether changes in outcomes between the post- and preintervention periods differed across the TWC and RSC groups. This difference-in-difference method attempts to correct for unobserved differences between the two participant groups that remain constant over time and are captured by the preintervention outcome measures. We estimated these models by stacking quarterly outcome data and including time indicators and time*TWC interaction terms as explanatory variables.

3. Propensity Scoring.В  This method matched RSC participants to TWC participants using observable characteristics. The matching was performed in three stages. First, we estimated a logit model where the dependent variable (equaling 1 for TWC participants and 0 for RSC participants) was regressed on the full set of explanatory variables. Second, using the logit results, we calculated a predicted probability of being in the TWC group (that is, a propensity score) for each sample member. Finally, we matched to each TWC sample member that RSC participant with the closest propensity score. The matching was done with replacement, so that an RSC participant could match to more than one TWC participant. We then compared mean outcomes of TWC participants to those of their matched comparison group. This method yielded a comparison group (from among RSC participants) that is very similar to the program group (in this case, TWC participants) on a wide range of characteristics. Thus, our hope is that the two groups also match on unobservables that are correlated with outcomes. Some evidence suggests that this method may be able to replicate experimental findings, but results can be biased to the extent that participants motivation and interest in the program are not measured (Agodini and Dynarski 2001).

Endnote

[1] Note that this simple difference is the estimate of ₁ when no explanatory variables are included in the models.