Oklahoma City's Education, Training, and Employment Program, Two Year Implementation, Participation, Cost, and Impact Findings: Overview and Summary

Prepared for:
U.S. Department of Health and Human Services
Administration for Children and Families
Office of the Assistant Secretary for Planning and Evaluation

U.S. Department of Education
Office of the Under Secretary
Office of Vocational and Adult Education

Prepared by:
Laura Storto
Gayle Hamilton
Christine Schwartz
Susan Scrivener

Manpower Demonstration Research Corporation

Oklahoma City's Education, Training, and Employment (ET & E) program was designed to promote self-sufficiency among applicants for and recipients of Aid to Families with Dependent Children (AFDC). The program (1) advocated participation in education, training, and job search classes to enhance individuals' employability and (2) granted child care assistance to support participation in the program and employment. However, ET & E was hampered by limited funding, and administrators and staff did not strongly enforce the program's mandate to participate. (Owing to statewide budget cuts and caps, caseloads were high; when case workers faced a time crunch, income maintenance functions took priority over employment and training functions.) As a result, overall, ET & E produced only small increases in the percentage of individuals who participated in basic education, vocational training, and job search classes, compared with the participation levels of a control group. For those who entered the program without a high school diploma or GED, ET & E produced larger increases in participation. The program did not increase enrollees' employment and earnings, compared with a control group's, but it did produce moderate welfare savings. Though the program's mandate to participate was not strongly enforced, it is possible that the welfare effects resulted from individuals deciding to forego cash assistance after they heard the mandate stated at application. Another possibility is that case managers were better able to discover AFDC ineligibility information with ET & E enrollees. Oklahoma City has since changed its program substantially to emphasize the mandate for welfare applicants and recipients to look for work as a first activity.

These findings come at a time when state and local welfare-to-work programs are being changed across the country in response to a major overhaul of the welfare system that was mandated by the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996. Oklahoma City's results provide program administrators with valuable lessons on how to improve programs' short-term effectiveness when implementing a welfare-to-work program in a tight funding environment. The main lessons are discussed at the end of this report.

ET & E is being assessed as part of the National Evaluation of Welfare-to-Work Strategies (NEWWS), a comprehensive study of welfare-to-work programs in seven sites. The evaluation is being conducted by the Manpower Demonstration Research Corporation (MDRC), under contract to the U.S. Department of Health and Human Services (HHS) with support from the U.S. Department of Education (ED). The evaluation in Oklahoma City and in the other six sites uses random assignment to rigorously test programs' effects.⁽¹⁾  Applicants for welfare in Oklahoma City between 1991 and 1993 were randomly assigned to two research groups and, for this report, were followed for two years. To determine the effects of ET & E, outcomes are compared between a program group, which was required to participate, and a control group, which could not participate in ET & E but could seek out services in the community. This comparison thus tests whether special welfare-to-work programs improve outcomes for welfare applicants over and above what they would have achieved on their own. The evaluation does not test the merit of individual services but, rather, how much a program can increase the use of those services and whether the increases can make a difference in raising employment rates and speeding welfare exits.

This report's data on implementation, participation, costs, and impact findings measure ET & E's operation before it was overhauled in late 1995, partly in response to early results from other evaluations of welfare-to-work programs which indicated that mandatory "work first" approaches have large effects in the short term. Oklahoma City's program shifted at that time from one that encouraged individuals to build skills through formal education and that put great emphasis on participants' choice to a program that is mandatory, employment-focused, and requires individuals to look for a job first, both before and after their application for welfare is approved. Future NEWWS documents will follow Oklahoma City sample members for up to five years; it is possible that longer follow-up will reflect Oklahoma City's shift to a program type that has produced large effects in other locales.

The following are the key two-year findings about how ET & E affected welfare applicants:

• ET & E administrators and staff did not strongly enforce the stated mandate to participate.  Staff universally told applicants for welfare in Oklahoma City about ET & E's mandate to participate, but after individuals enrolled in the program, staff did not strictly enforce it. High caseloads  created by limited funding  and the higher priority that administrators placed on eligibility functions cut into the time that staff had to monitor participation, to cajole reticent individuals to participate, or to sanction enrollees who failed to comply. The administrators' and staff's philosophies about the desirability of honoring participants' choices and about the undesirability of sanctioning also undermined enforcement.
• ET & E only slightly increased participation among welfare applicants in education and training activities above what they would have accessed on their own within a two-year period. ET & E was highly committed to a skill-building approach to self-sufficiency. Staff almost universally advocated that enrollees return to school to enhance their employability. However, administrators' and staff's decisions to focus limited resources on individuals who wanted to participate  and their weak enforcement of the mandate to participate  kept ET & E from engaging many more individuals than would normally have participated on their own. Thus, the program group's participation rates in employment-related activities were not much higher than the control group's. One exception was for those who entered the evaluation without a high school diploma or GED. For these individuals, who tended to stay on welfare longer, the program produced a 22 percentage point increase in the proportion who attended basic education classes, a 10 percentage point increase in participation levels in vocational training programs, and a 9 percentage point increase in job search activities. ET & E did not generate statistically significant increases in college attendance or in the receipt of any educational credential, such as a high school diploma or trade certificate, for either subgroup.
• Disregarding the costs that the government would have incurred without ET & E, just $951 was spent on each program group member, the lowest found for a NEWWS program.  Oklahoma City's welfare department generated this low cost by spending less on ET & E case management and program activities than did any other NEWWS program for which these data are available. In addition, although the welfare department did spend slightly more on child care and other support services than in most other programs, much of the cost was for non-ET & E-related child care.
• ET & E produced no impacts on employment or earnings within two years.  Relative to the control group, the program group's increased participation in basic education, job search, and vocational training did not lead to increases in employment or earnings either for the full sample or for any subgroups of individuals. There are a number of reasons for this result. First, other studies suggest that programs that primarily provide job search and basic education do not work as well for the sample studied in Oklahoma City  applicants for welfare  as for other, more disadvantaged members of the welfare caseload. Second, ET & E did not create a large treatment difference between the program and control groups. Third, programs that encourage enrollees to invest in education or training before entering the labor market are not expected to show immediate employment gains; payoffs are expected to emerge in later years.
• ET & E did generate moderate AFDC savings.  Relative to the total welfare payments that the control group received, Oklahoma City's ET & E program reduced expenditures by 6 percent. These AFDC savings were found only for a subgroup of individuals who had a high school diploma or GED at the time that they applied for welfare. The absence of impacts on employment and earnings suggests that the welfare savings are not the result of enrollees' achieving self-sufficiency. It is possible that applicants chose alternatives to welfare because of the stated participation requirement or that case managers were better able to discover AFDC ineligibility information for ET & E enrollees.
• These findings suggest several lessons. First, for a welfare-to-work program to achieve gains for enrollees over and above what they can do on their own, it is important to engage individuals who would not otherwise participate in education and employment activities. Though education and training may have increased the employment potential of program and control group members who participated in them, ET & E did not largely increase the use of these services among those who were required to enroll in the program. Second, the cost findings suggest that the welfare department must make some minimum per-person resource investment in order to have effects. ET & E case management may have been spread too thinly over the program group to make a difference. Third, it is important for welfare-to-work program administrators to clarify the priority of their program within the welfare department. Oklahoma City's experiences demonstrate that unless administrators stress the importance of a welfare-to-work program to staff, the program can suffer when underfunded welfare departments use integrated case management. This type of case management, where case workers have responsibility for welfare eligibility duties as well as employment and training functions, has been suggested as one way to move the culture of a welfare department toward promoting self-sufficiency. When caseloads are high and time is scarce, however, ensuring that cash assistance is quickly and accurately delivered to families in need can take precedence over the task of trying to move individuals from welfare to work. These lessons will be discussed in detail at the end of this report.

[Note, the following references are not available online yet.]  The following pages will first provide some context for the results obtained   by describing Oklahoma City, the sample studied, and the research design used to gauge ET & E's effectiveness (Chapter 1). Next, a description of the program treatment and its implementation is presented (Chapter 2). Findings on the per-person cost of ET & E and the impacts on employment, earnings, and welfare receipt then follow (Chapters 3 and 4). The report concludes with lessons that the evaluation of Oklahoma City's program can provide for the future implementation of welfare-to-work programs (Chapter 5).

Footnotes

1.  The present study draws its sample and data from Oklahoma, Cleveland, and Pottawatomie Counties, Oklahoma. For ease of reference, the name of the urban area that encompasses these counties, Oklahoma City, will be used throughout this report.