Initial Synthesis Report of the Findings from ASPE's "Leavers" Grants. Appendix A: Methodology

01/04/2001

The studies summarized in this report use a variety of methods in obtaining their results. As part of the ASPE grant program, every attempt was made to encourage the use of comparable measures to enable comparison of results. However, many differences in methodology remain. This appendix attempts to give an overview of the differences. We do not attempt to connect differences in results to differences in methods. The primary areas of differences include the type of data being used, the cohort of exiters, and the specific types of cases included or excluded. The studies that have included surveys up to this point, have additional differences in response rates, timing of survey, and survey instrument. This appendix describes the extent and nature of these differences.

All of the studies focus on at least one cohort of families leaving the TANF/AFDC program. For the most part, the definition of who is a "welfare leaver" is comparable across studies. Most of the studies included as leavers those who received welfare in a specific month but did not receive any benefits for the following two months. The District of Columbia is an exception here, defining leavers as those who stopped receiving in a given month and did not receive at all in the next calendar month. Arizona reports results for both 1 month and 2 month leavers.

The broader difference in studies was the timing of the cohort(s) studied (table A1). Many include an early and late cohort, although some include multiple cohorts. Illinois includes six cohorts of information and a measure that combines information across all these cohorts. Six of the studies include a cohort from one of the quarters of 1996. Four of the studies include information from a cohort in 1998. Given the many changes occurring in welfare programs, caseloads, and the economy across these time periods, we expect that the particular timing of the cohort examined may make a difference in results. In our summary, we focus on the most recent complete cohort information available.

 

Appendix Table A1:
Methodology
State Type of Study Leaver Cohorts Definition of leaver Cases excluded Reported Subsets
Arizona 1998 Survey and Administrative 1Q98 One month, two months 2 Parent Units, Child only cases, cases under tribal jurisdiction Sanctioned, unsanctioned leavers
District of Columbia Administrative and Survey Two Cohorts: 4Q97, 4Q98 One month Child Only Continuous leavers
Florida1 Administrative 2Q97   Single Parent Continuous leavers
Georgia1 Administrative Four Cohorts: 1Q97, 2Q97, 3Q97, 4Q97 Two months 2 Parent Units, Child-only cases  
Illinois Survey and Administrative Six cohorts: 3Q97, 4Q97, 1Q98, 2Q98, 3Q98, 4Q98; Survey: Dec. 1998 Two months Child-only cases Single Parents, Two Parents; Cook County, Downstate
Missouri Administrative 4Q96 Two months Child only 2
Missouri Survey 4Q96 Two months Child only Kansas City, St. Louis
New York1 Administrative 1Q97 Two months Child only Single Parents, Two Parents
Washington Administrative Three cohorts: 4Q96, 4Q97, 4Q98 Two months   Leavers, on-TANF; continuous leavers
Washington Survey October 1998 Two months Child Only Single Parents, Two Parents; Leavers, on- TANF
Wisconsin Administrative July 1995-June 1996 Two months   Single Parents, Leavers, on-TANF; continuous leavers
Cuyahoga Co.1 Administrative 3rd Quarter 1996 Two months Child-only cases, single women without children  
Los Angeles Co.1 Administrative 3Q96 Two months    
San Mateo Co.1 Administrative Two Cohorts: 4Q96, 4Q97 Two months   Single Parents, Two Parents; Child Only; One or Two Parent Transitioning to Child Only
1 Jurisdictions with administrative data only will add survey data to their final reports (forthcoming).
2 Missouri will include a geographic subset in Section D of a forthcoming report.
Sources: See Appendix B for a complete listing of the leavers studies referenced.

Another methodological issue that may be important for interpreting results is the type of cases included and excluded in the study sample. Most of the studies exclude exiting child only cases completely or present results separately for non-child-only cases. In addition, a number of the studies exclude exiting two-parent cases or present results for single-parent cases separately. This allows comparisons on the subset of single-parent cases for most studies. The exceptions are District of Columbia and Missouri. However, the percentage of two-parent cases among the leaver population is less than 2 percent in the District of Columbia and less than 4 percent in Missouri.

Several of the studies present results on specific subgroups of leavers. Arizona presents results for sanctioned and non-sanctioned leavers separately. For their early cohort, Arizona presents separates urban and rural results. Illinois also presents a geographic break, reporting Cook County (Chicago) separately from downstate Illinois. Washington compares results for leavers with outcomes for current TANF recipients. Several studies report administrative program participation data for continuous leavers, that is, those who exit TANF and do not return in a specified time period (usually a year).