| [Section E] | [Table of Contents] | [Appendix] |
The success of WtW programs in carrying out their plans and affecting participants' employment outcomes remains to be determined. Most grantees are still at very early stages of recruiting participants and delivering services, and additional grants remain to be made with currently appropriated funds. In many states, TANF time limits are just now beginning to result in actual termination of benefits; how increasing numbers of recipients affected by time limits will influence referrals and entries to WtW programs is uncertain. Therefore, clear answers to questions about program implementation and impacts will not come until later stages of this evaluation. However, the first grantee survey provides an early glimpse of grantees' own perceptions of how the overall WtW grants program responds to the challenge of moving those hardest to employ into jobs they will keep. We conclude this first report with a summary of those views, followed by a summary of future evaluation activities and the additional evidence they should provide.
The first evaluation survey provided a limited opportunity to gauge how program parameters and the economy in which grantees are operating might affect implementation and the overall importance of the program. A mail questionnaire sent to grantees starting a new program must avoid creating an unreasonable burden on them, so only a narrow set of questions could be explored. Moreover, their basis for responding to these questions is, in most cases, just a brief period of program operation and, in some cases, only a sense of what they will encounter when they actually begin their programs.
We therefore focused on exploring grantees' early views on four issues, identified from indepth discussions with some grantees as the survey instrument was being developed:
The survey explored grantee views on these issues by posing six statements and asking respondents to indicate whether their agreement with the statements was high, medium, or low (Table F.1). Four salient findings can be gleaned from their responses.
Funding is sufficient for defined program objectives but may not be enough for the larger challenge. Grantee responses to the first three statements, relating to funding adequacy, at first appear contradictory. Very few grantees (4.4 percent) agreed strongly that there were adequate resources in their area to help the target WtW population before the program was initiated, but more than threequarters (76.6 percent) agreed moderately or strongly that the level of funding they are receiving will suffice to provide WtW services. However, more than half of the respondents moderately or strongly agreed with the statement that there are more people in the target population than they can serve.
This apparent contradiction may arise from the difference between grantees' realistic program plans and their sense of the larger challenge of improving employment outcomes for the segment of the population with severe disadvantages. Discussions with a limited number of grantees, and the modest enrollment achieved to date, suggest that many grantees and their local partners are facing unexpected difficulties in identifying, referring, and actually enrolling the numbers of participants they had planned to serve. Although welfare rolls have shrunk dramatically, there remain in many grantees' service areas large numbers of TANF recipients who appear to fall within the defined target population. In many instances, WtW programs appear to be designed and funded at a level to serve just a portion of this target population, perhaps because of expectations that many will leave TANF on their own or through "work first" interventions that precede intake to WtW programs. Grantees may thus be convinced that overall needs for help in moving from welfare to work exceed the number that will be served in their WtW programs. At the same time, the slow pace of early enrollment may lead many grantees to believe that their grant budgets will be adequate, since they are struggling to find and enroll the numbers of people they projected serving.
|
TABLE F.1
GRANTEE VIEWS ON WtW IMPLEMENTATION ISSUES |
|||
|---|---|---|---|
| Level of Agreement | |||
| Statementa | High | Medium | Low |
| Resources were adequate without WtW funds. "Resources for services to groups identified as eligible in the federal WtW statute were adequate in our area even without WTW funds." | 4.4 | 21.7 | 73.9 |
| WtW funding is adequate. "It appears there will be adequate funding available to provide needed WtW services in our local service area." | 39.5 | 37.1 | 23.4 |
| Need exceeds WtW funds. "There are many more people in our defined target groups than we will be able to serve even with federal WtW funds." | 23.7 | 30.8 | 45.5 |
| WtW eligibility criteria are too restrictive. "The WtW eligibility rules sometimes exclude people who are truly among the hardtoemploy but who cannot meet all the required criteria specified in the WtW statute." | 66.4 | 23.6 | 10.0 |
| WtW funding is already having an effect. "Federal WtW funding is already having a substantial effect moving the hardtoemploy into employment." | 4.1 | 14.1 | 81.8 |
| Employer demand is strong. "There is strong demand among local employers for the people our WtW program will be placing in employment." | 14.9 | 49.9 | 35.2 |
|
Source: National Evaluation of the WelfaretoWork
Grants Program, First Grantee Survey (November 1998February 1999).
a The text of the statement as it appeared in the survey questionnaire is enclosed in quotation marks. The boldfont statement is added here to highlight in simple language the point that respondents confirmed or rejected. |
|||
Restrictive eligibility criteria are contributing to enrollment difficulties. A clear theme from the survey data and from more indepth contacts with WtW grantees is that many believe the specific combination of legislated eligibility criteria defining the population on which 70 percent of WtW funds must be spent is too restrictive. Ninety percent of grantees agreed moderately or strongly that these criteria exclude some people from their programs who truly fall within a group that has serious barriers to employment success. This issue elicited the most consistent views among the set of issues about implementation posed in the survey.
Discussions with grantees suggest that, for at least some, the restrictiveness of the eligibility criteria is contributing to the slow pace of early program enrollment. Grantees have reported, for example, that some prospective participants referred to them from the TANF caseload or other sources meet the broad description of "hard to employ" but fail to meet the specific criteria stated in the legislation. A commonly cited example is individuals who fail to meet the "education and skill deficit" criterion because they have low math or reading skills but had received a high school diploma or GED.1 Such individuals may have a poor work history, but would be eligible for WtW services only if they also have a substance abuse problem that requires treatment. The grantee or another organization might serve such individuals using other funding sources (such as JTPA or the TANF block grant), but those sources may not cover the particular package of services in the WtW program.2
WtW programs are too new to offer evidence of success. Twothirds of the survey respondents began delivering services under their WtW grants in the last quarter of 1998 or in 1999, and 87 percent began after July 1, 1998. At the time of the survey, therefore, grantees had been operating their WtW services for no more than four to six months, and many had just begun. It is thus not surprising that 82 percent of the grantees refrained from asserting that their programs had already had substantial effects on moving participants into employment.3
Employment opportunities are viewed as strong. Grantees' success in moving participants into jobs obviously depends on the readiness of employers in their local areas to hire, and specifically to hire the relatively lowskilled and inexperienced individuals who will be participating in WtW programs. Employer demand for WtW participants does indeed appear strong, as judged by the grantees; almost twothirds of grantees moderately or strongly agreed. This strong demand, however, may also be contributing to the slow pace of enrollment in WtW programs, to the extent that employers are as willing to hire members of the programs' target population directly as through program placement services.
2. Future Issues for the Evaluation
The findings presented in this report are derived from the earliest stages
of data collection and analysis in the National Evaluation of the
WelfaretoWork Grants Program. They therefore offer only
a preliminary and partial view of the ultimate shape of the program as
implemented at the local level and no sense at all of the difference it makes
in participants' employment outcomes. Both of these areas of
investigation — implementation experience and program outcomes
and impacts — will be explored further as the evaluation continues.
Implementation experiences will be the focus of several stages of the evaluation. During the first half of 1999, extensive contacts will be made with local WtW grantees to document the local context of their programs, the specific interventions they are attempting, their success in recruiting, and the factors that have shaped both their plans and their success in implementing them. The earliest of these contacts (some already made) are part of the process of recruiting sites for the indepth study component of the evaluation. By summer 1999, intensive site visits will be under way to selected sites. A process analysis report based on this field investigation will be prepared by fall 1999. Implementation experiences, services provided, and job placement outcomes for all WtW grantees will be examined again in a grantee survey in fall 1999 and reported on in early 2000. Comparison of that second survey and the first survey reported on here will provide a systematic measure of implementation progress.
The question of ultimate concern is which program approaches work best. Efforts are now under way to recruit grantees who will participate in the evaluation impact analysis. In an estimated 10 sites, random assignment of program applicants or referrals will be used to create a program group and a control group. This randomassignment process will begin in some sites as early as spring 1999 and in others as late as fall 1999; it will continue in each site for at least 12 months. Comparisons of employment outcomes for the program and control groups in each site (based on TANF and wage records and followup surveys) will be used to determine whether the special WtW program services helped participants achieve a level of employment success beyond that achieved by individuals who had access only to the standard services that would have been vailable in the absence of the local WtW programs that are the focus of the impact analysis. Such impact findings will be made available based on followup of the program group and control group over a minimum of 18 months and should be available in fall 2000.
1. Other factors are also contributing to slow enrollment. Some grantees note that, when they receive a list of longterm TANF recipients who are supposed to be potential WtW participants and attempt to contact them, many have already found jobs and left the TANF rolls. Others turn out to be exempt from TANF work requirements and thus may not face immediate pressure to participate, although they will still be affected by time limits and are eligible to receive WtW services.
2. The President's reauthorization proposal, if passed, would address some of these issues. The proposal retains the program focus on those most in need, while simplifying the eligibility criteria to avoid excluding individuals who are truly among the hardest to employ.
3. The survey question instructed respondents to answer "low" if "services are not yet being delivered or have just begun," to discourage grantees from answering based on their plans rather than on actual experience.
| [Section E] | [Table of Contents] | [Appendix] |