Approaches to Evaluating Welfare Reform: Lessons from Five State Demonstrations. 2. State Approaches


The five evaluations studied differed in how much they examined entry effects. Two evaluations have devoted substantial attention to this issue. In Wisconsin's WNW evaluation, entry effects are being estimated using aggregate and disaggregate time-series modeling of application behavior before and after the implementation of welfare reform. Early evidence from process analyses suggests that entry effects may be responsible for a large portion of the caseload changes arising from the WNW package; these findings will need to be confirmed through the time-series analyses described above.

In the APDP/WPDP evaluation, entry effects were being estimated using both administrative data and data from the Current Population Survey (CPS).(3) A time-series model of the fraction of "at-risk" women starting a welfare spell is being estimated using data from the early 1970s to the early 1990s, combining CPS data on the number of women at risk of becoming welfare dependent with monthly caseload data on the number of new welfare spells. To investigate exit effects, another time-series model is being estimated of terminations from AFDC, with separate analyses for the AFDC-UP caseload. Approximately 240 observations are being analyzed. The models control for benefit levels, birth rates, real wages, minimum-wage changes, unemployment rates, and key milestones in welfare policy (such as the OBRA changes of the early 1980s, which substantially reduced earnings disregards). In general, policy changes such as those adopted under OBRA were associated with large entry effects. Using this model, caseloads for the period following the implementation of welfare reform are being forecasted and compared with actual caseloads.

In the other states, efforts to determine entry effects were more modest or nonexistent. In Michigan's TSMF evaluation, the evaluator proposed asking questions in the client survey about possible entry effects, but no time-series analyses of applications or terminations were planned. In Minnesota's MFIP evaluation, the importance of entry effects was recognized, but no attempt was made to estimate them: "It was decided that time-series analyses will yield little reliable data since none of the [demonstration] counties are saturating their caseload with MFIP.(4) In Colorado's evaluation, no efforts are being made to estimate entry effects.

None of the evaluations we reviewed are using analysis of exit rates of recipient cases to infer the direction of possible entry effects arising from welfare reform provisions.