| Chapter 4 | Table of Contents | References |
The ultimate success of WtW programs remains to be determined. A few grantees received their awards only recently, and the majority of grantees have yet to reach the levels of enrollment they projected. Legislative changes to broaden the original eligibility criteria and expand the target population may increase the scale of program operations, but most grantees have not yet been allowed to implement these changes, under the congressionally defined phase-in of the new rules. How more liberal eligibility criteria and other changes aimed at strengthening interagency information-sharing and coordination will influence participation and outcomes in WtW programs is therefore still uncertain.(1)
The second grantee survey nevertheless provides a stronger foundation than the first survey for describing grantees' perceptions of their progress and of the WtW program overall. The survey questions included queries about respondents' views on how their program implementation is progressing and how the initiative as a whole responds to the challenge of helping the hardest to employ move into jobs, keep jobs, and move toward economic self-sufficiency. This report concludes with a summary of those views, followed by an overview of future evaluation activities and the additional evidence they should provide.
[ Go to Contents ]
The grantee survey posed a short list of questions on grantees' views of implementation progress, in keeping with the inevitable limitations of this form of information-gathering. The overall questionnaire had to avoid creating an unreasonable burden, and focused most heavily on actual program operations. Nevertheless, it was possible to include questions in both surveys that probe grantees' perceptions of the progress they are making and the problems they face.
Questions on grantees' views evolved slightly from the first survey to the second, as issues of concern became clearer. In the first grantee survey, we focused on four issues identified from in-depth discussions with some grantees as the survey instrument was being developed:
At the time of the first survey, most grantees had been operating their programs for just a brief period. To a large extent, their responses reflected expectations rather than actual experience. Therefore, it became important to revisit these same issues in the second survey — to gauge shifts in grantee perceptions, expectations, or both now that their programs are further along. In addition, interactions with grantees over the first year of program operations identified other issues that are included in the second survey:
The second grantee survey explored grantees' views on these issues by posing 10 statements and asking respondents to indicate whether their agreement with the statements was high, medium, or low (Table V.1). Four salient findings can be gleaned from their responses.
Restrictive eligibility rules are the main impediment to WtW program implementation. A clear theme from the survey data and from more in-depth contact with WtW grantees over the past year is that the BBA eligibility criteria defining the population on which WtW funds must be spent was too restrictive. In the second survey, as in the first, virtually all respondents (98 percent) agreed strongly or moderately that these criteria excluded people from their programs who truly fall within a group of individuals with serious barriers to employment. In both surveys, this issue elicited the most consistent views from respondents. Furthermore, grantees were almost evenly split in their opinions about whether the pace of enrollment into their programs had been improving. This finding suggests that, as the second survey was conducted, recruitment into WtW programs remained an important challenge for many grantees.(2)
Requirements for matching of federal formula funds by grantees could also potentially affect the scale of WtW programs, but survey responses suggest that such an effect is not widespread. In field visits and other interactions, some local grantees reported that their states had passed through the federal matching requirement to their substate formula grantees. In a few cases, local grantees reported that this delayed their WtW program implementation, because local funding commitments had to be obtained, or prevented them from accessing their full allocation of formula funds. The second grantee survey, therefore, included a statement on matching requirements to gauge the prevalence of this practice and the extent to which it might have constrained local WtW program efforts. Almost half of all respondents indicated low agreement with the survey's statement, while almost a quarter agreed strongly that matching requirements were limiting their program scale or scope. Thus, matching requirements have clearly been a factor in program implementation, though not nearly as widely as eligibility rules.
WtW funding is adequate to serve those who meet eligibility criteria. Respondents to the second grantee survey believe that WtW funds fill an important gap in the local service structure. Very few grantees (7.6 percent) agreed strongly that there were adequate resources in their area to help the target WtW population before the program was initiated. About 47 percent agreed strongly that the level of WtW funding they are receiving is adequate to provide needed WtW services to the participants they plan to recruit. This view, it must be assumed, reflects grantees' comparison of their WtW funding to the numbers they have stated they will serve, rather than a statement about the general population of hard-to-employ individuals in their communities.
Grantee views on how overall need for WtW services relates to the capacity of their programs are harder to establish, principally because of the timing of the second survey. The instrument queried grantees on whether they agreed that the numbers of people in their defined target groups exceeded those who could be served with federal WtW funding. Since most grantees had not begun implementing the recently approved WtW eligibility changes when they completed the survey, they could have interpreted the statement as referring to individuals meeting the BBA's eligibility criteria. Given the recruitment difficulties that most programs have encountered to date, it seems reasonable that 47 percent of grantees expressed low agreement; these grantees probably are noting in their responses that there is no indication to date that individuals who meet the original BBA criteria will overwhelm program capacity.
At the same time, some respondents perceive needs that exceed the capacity of their WtW programs. About 22 percent of the respondents agreed strongly that there are more people in their defined target population than they can serve. Beyond those they will serve in their grant-funded programs, these grantees see a larger population that needs help to improve their employment outcomes and overcome severe disadvantages. Some of these grantees, of course, may be thinking of needs among a population beyond those who technically qualify under the original eligibility rules.
Regardless of how grantees view the overall need for WtW services, more than two-thirds expressed low agreement with the statement that need for WtW services has declined. Thus, grantees appear not to have concluded from the drop in TANF caseloads or the slow enrollment in WtW programs that the number of people in their area who could benefit from their programs has fallen.
Many grantees are uncertain about the effect of WtW services to date. At the time of the second grantee survey, respondents had been operating their programs an average of about 11 months, and could therefore begin to form their own views on their success in moving participants into employment. A few grantees believe they are having such success; 18 percent of respondents to the second survey agreed strongly that WtW funding had already helped move the hard to employ into employment. More grantees appeared confident that their services are helping participants advance in their jobs; 26 percent of respondents indicated strong agreement that their programs are helping participants advance to higher wages and better jobs. To a large extent, these views could, of course, be regarded simply as an indication that the programs have reached a substantial level of operation and, as could be expected, at least some of the participants are being placed in employment. Grantees' belief in their success should not necessarily be interpreted as evidence that the programs are helping people achieve greater employment success than they would have without the program or with other kinds of help.
At the same time, a substantial number of grantees are not so convinced that their programs have actually begun to help the hard-to-employ move into employment. Almost 40 percent of respondents (188 grantees) expressed low agreement with that statement. Compared to other grantees that are moderately or strongly convinced of their success, these grantees (1) have been operating for less time (9.3, compared to 12.6 months); (2) have enrolled a lower proportion of their total projected WtW participants (16, compared to 35 percent); and (3) have made fewer placements in any type of work activity (39 placements, compared to 148). Moreover, 79 of these 188 grantees also expressed low agreement with the statement that the pace of enrollment has been improving, suggesting that many are organizations having severe and persistent recruitment difficulties.
Grantees are only moderately optimistic about employer demand for WtW participants. Obviously, grantees' success in moving WtW participants into jobs depends on the readiness of employers in their local areas to hire the relatively low-skilled and inexperienced individuals who participate in WtW programs. Some grantees see strong employer demand for WtW programs; about 17 percent agreed strongly that there is strong employer demand for the people whom they will attempt to place. However, a majority of grantees (52 percent) expressed some reservations by agreeing only moderately with that statement; their reservations about employer demand for WtW participants may offer further explanation for the programs' modest targets for WtW placements in unsubsidized employment. The remaining 30 percent of respondents that expressed low agreement with this statement tend to be in more rural and less densely populated areas. They serve areas where population density on average is less than half the density found in areas served by grantees that perceive strong employer demand.(3)
|
TABLE V.1 GRANTEE VIEWS ON WTW IMPLEMENTATION ISSUES (Percentage of Responding Grantees) |
|||
|---|---|---|---|
| Statementa | Level of Agreement | ||
| High | Medium | Low | |
| WtW eligibility criteria are too restrictive. "The WtW eligibility rules sometimes exclude people who are truly among the hard-to-employ but who cannot meet all the required criteria specified in the WtW statute." | 90.5 | 7.2 | 2.3 |
| WtW enrollment pace has been improving. "The number of people being enrolled each month in our WtW program has been increasing since the program began." | 33.8 | 31.7 | 34.5 |
| WtW match requirements are hampering program efforts. "Requirements for state or local funds to match federal formula funds are limiting the local scale or scope of the WtW program, or the number of participants who can be served." | 23.8 | 27.1 | 49.1 |
| Resources were adequate without WtW funds. "Resources for services to groups identified as eligible in the federal WtW statute were adequate in our area even without WtW funds." | 7.6 | 19.6 | 72.7 |
| WtW funding is adequate. "It appears there will be adequate funding available to provide needed WtW services in our local service area." | 47.3 | 37.5 | 15.2 |
| Need exceeds WtW funds. "There are many more people in our defined target groups than we will be able to serve even with federal WtW funds." | 22.4 | 30.9 | 46.7 |
| Need for WtW services has declined. "The need for services provided under federal WtW funding in our local area has declined since we got our grant." | 8.8 | 22.7 | 68.5 |
| WtW funding is already having an effect. "Federal WtW funding is already having a substantial effect moving the hard-to-employ into employment." | 17.5 | 43.7 | 38.8 |
| WtW services support career advancement. "Services we provide using federal WtW funding are helping participants not only to enter employment, but also to advance to higher wages and better jobs." | 26.0 | 49.9 | 24.1 |
| Employer demand is strong. "There is strong demand among local employers for the people our WtW program will be placing in employment." | 17.4 | 51.9 | 30.7 |
|
Source: National Evaluation of the Welfare-to-Work Grants
Program, Second Grantee Survey (November 1999 - February 2000).
a The text of the statement as it appeared in the survey questionnaire is enclosed in quotation marks. The bold-font statement is added here to highlight in simple language the point that respondents confirmed or rejected. |
|||
[ Go to Contents ]
Although the second grantee survey concludes the broad national data collection for the WtW evaluation, some of the issues raised in the two surveys will be pursued further in the evaluation in-depth study sites. Ultimately, a total of 12 to 15 sites will be selected; at the time of this report, 8 have been chosen.(4) With a combination of site visits and followup of participant samples using surveys and administrative data, the evaluation will explore such issues as:
Evaluation findings on these issues will be issued over the next several years. An interim report on program implementation issues will be issued in winter 2000, with a full report on program implementation in the in-depth study sites to be ready in summer 2001. Early results on outcomes for participants, from follow-up surveys and administrative data, will be issued in mid-2001. Later reports on more extensive followup and the complete evaluation sample will be available in 2002 and 2003.(6)
[ Go to Contents ]
1. Further information on participation levels and participant outcomes will be obtained later in the evaluation for in-depth study sites. For grantees as a whole across the nation, however, no further comprehensive surveys are planned.
2. With the new eligibility rules taking effect primarily in the middle of 2000, it is to be expected that enrollment difficulties related to the eligibility rules will continue to be reported.
3. The average population density for areas served by grantees that expressed low agreement with the statement on strong employer demand was 422 per square mile, compared to 806 in areas served by grantees expressing moderate agreement, and 942 in areas served by grantees expressing strong agreement.
4. The eight sites selected so far are in Boston, Fort Worth, Milwaukee, Nashville, Philadelphia, Phoenix, a large rural area in West Virginia, and Yakima. In all these sites, samples of participants are being enrolled in an evaluation sample and will be followed for two years.
5. In a few in-depth study sites, it may be possible to implement random assignment and estimate program impacts — the difference programs make in these employment outcomes.
6. The report on the evaluation component focusing on implementation of WtW programs by American Indian and Alaska Native grantees will be issued in fall 2000.
| Chapter 4 | Table of Contents | References |
Top of Page
Contents of This Chapter
Home Pages:
Human Services Policy
(HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)
Last updated June 26, 2000