Prepared by
Staff of the
Office of the Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)
200 Independence Ave. SW, Room 404-E
Washington, DC 20201
(202) 690-6805
Please send questions and responses to:
julia.isaacs@hhs.gov
Contents
- Introduction
- Common Definition of "Leavers"
- Commonly Reported Administrative Data Outcomes for Leavers
- Definitions of Commonly Reported Administrative Data Outcome Measures
- Addendum: Additional Administrative Data Outcomes for Leavers
- Definitions of Optional Additional Administrative Data Outcomes Measures
In response to requests from state researchers and national analysts, ASPE staff have expanded the proposed set of Commonly Reported Administrative Data Outcomes for Leavers. As the following document demonstrates, the updated set of measures has two parts: the original core set of 9 measures (which were distributed through an ASPE list-serve in spring 1999 and distributed at various conferences in May and October 1999) and a new Addendum with about a dozen additional, secondary measures.
A Proposed Set of Commonly Reported Administrative Data Outcomes for Leavers Studies, revised to include ADDENDUM with Additional Measures
A number of different states are conducting "leavers studies," which track families after they leave TANF. Many of these studies focus on tracking leavers through linked administrative data. In order to facilitate cross-study comparisons, a small set of commonly reported outcomes has been developed by staff in the office of the Assistant Secretary for Planning and Evaluation (ASPE), HHS, in consultation with the 14 grantees receiving ASPE funding in 1998 to study welfare outcomes. The ASPE grantees have been encouraged to include these outcomes in their interim reports (in addition to the many other outcomes and analyses proposed in their various studies). Other researchers may wish to include these same measures in their reports, to further facilitate cross-study comparisons.
Introduction
Note that this small set of measures is limited to the most common measures available in administrative data. Further information on child and family well-being of TANF leavers will be included in the ASPE grantees’ final reports, which will draw upon survey as well as administrative data.
- Note: The last 2 pages of this document consist of an ADDENDUM of additional measures developed in Fall 1999. Though secondary in importance to the original set of measures, the additional measures provide further opportunities for cross-state comparisons among those states that choose to include them in their reports.
- Note: ASPE grantees who are studying various diverted populations, including people who initiate the application process, but do not enroll in TANF, are encouraged to report similar administrative data outcome measures. For such studies, outcomes should be reported by date since application for TANF, rather than date since exit from TANF.
Common Definition of "Leavers" to Include in Analysis
Please report each of the proposed measures for single-parent closed cases or leavers, defined as those cases that leave cash assistance and remain off cash assistance for a minimum of two months. The "two-month" definition was chosen to correspond to the definition used for the survey samples in the ASPE studies. It excludes closed cases that re-open within two months, because such cases are likely to be subject to "administrative churning." It includes, however, cases that close for two months but re-open subsequently.
Also note that in addition to reporting on single-parent cases closed for two months, many studies also will be reporting outcomes for the fuller sample of cases that may be included in their study (e.g., including closed two-parent or closed child-only cases, as well as closed cases that re-open within two months). To the extent sample sizes permit, some studies also will report measures for closed UP cases or closed child-only cases as separate sub-groups.
"Month of Exit"
Note that some studies define "month of exit" as the first month without any receipt of assistance, while other studies define "month of exit" as the month in which the case closing action is taken (and the following month is the first month without any receipt of assistance). Please document how "month of exit" is defined in your study.
Unit of Analysis
Studies have chosen different units of analyses. Many ASPE-funded studies are following the adult head of household, or the entire case. For some measures, however, it also is important to track children. One option is to track the children, but still report by the same case or adult unit of analysis (i.e., the study could report that x% of cases or adult leavers who have children receiving Medicaid 12 months after exit). Please document the unit of analysis used in your study and note in the measures below whether receipt of public assistance is tracked for heads of household only, all adults, all children, or any member of household.
Quarterly/Monthly
Except where noted otherwise, please report employment outcomes by calendar quarter, starting with the quarter of exit, and continuing for 4 quarters (or more, if you have it). (Data from quarters prior to exit are also useful). The definition of quarter of exit will be the quarter in which the month of exit (as defined by you) occurs.
For the measures of participation in TANF and other programs, please report under one of two alternatives: by months since exit, or by calendar quarter. If reporting by months since exit, you may either report every month, or report at key monthly intervals, including 3 months after exit, 6 months after exit, 9 months after exit, and 12 months after exit. Note: As of December 1999, 4 studies reported program participation by months since exit and 6 studies reported by quarter.
The measures and their definitions are shown in the following table. Please note that this set of proposed measures is very similar to the measures first proposed in March 1999, and revised in May 1999 and October 1999. For further information, please contact Julia Isaacs, at julia.isaacs@hhs.gov.
Commonly Reported Administrative Data Outcomes for Leavers
| Single-Parent Leavers (1) N = | Qtr before Exit | Quarter of Exit | 1st Qtr after Exit | 2nd Qtr after Exit | 3rd Qtr after Exit | 4th Qtr after Exit | (5) Ever empl. over 4 Qtrs |
|---|---|---|---|---|---|---|---|
| Employment Outcomes: | |||||||
| (2) Employment Rate (% with any earnings over quarter) (%) | • | • | • | • | • | • | • |
| (3) Mean Quarterly Earnings (across those with earnings) | • | • | • | • | • | • | (*) |
| (4) Median Quarterly Earnings (across those with earnings) | • | • | • | • | • | • | (*) |
| 3 mos (1Q) before Exit | Month (Q) of Exit | 3 mos (1Q) after Exit | 6 mos (2Q) after Exit | 9 mos (3Q) after Exit | 12 mos (4Q) after Exit | (7) Ever on over 4Qs | |
| Recidivism (reported by month or quarter) | |||||||
| (6) Receiving TANF(%) | • | • | • | • | • | • | • |
| Other Program Benefits (reported by month or quarter) | |||||||
| (8a) Participating in Medicaid (%) | • | • | • | • | • | • | • |
| (8b) Participating in Medicaid (Children) (%) | • | • | • | • | • | • | • |
| (9) Receiving Food Stamps (%) | • | • | • | • | • | • | • |
| * If available, report annual earnings (i.e., sum of earnings over 4 quarters) for those with any earnings in first 4 quarters. | |||||||
Definitions of Commonly Reported Administrative Data Outcome Measures
(Additions since October 1999 noted in italics)
(1) Total number of cases
Complete sample of closed cases for the identified cohort, regardless of current TANF or work status, provided the case was closed for two months or longer. This should be the denominator for all percentages in measures below, unless noted otherwise.
EMPLOYMENT OUTCOMES. Report by calendar quarter, including quarter before exit, of exit, and 4 quarters after exit:
(2) Employment rate
Percentage of total leavers with quarterly earnings reported in the Unemployment Insurance System. Assumed to be limited to UI-covered employment within state, unless documented differently. Note that many studies are reporting this for adult leavers (>18) only. Most ASPE-funded studies including any leavers with earnings >$1; at least one limiting to those with earnings >$100. Note: See optional additional measure of those with earnings >=$500 in Addendum.
(3) Mean earnings
Calculated across cases with earnings. If available, report mean annual earnings (sum of four quarters) as well as mean quarterly earnings. See addendum for additional measures.
(4) Median earnings
Calculated across cases with earnings. If available, report median annual earnings (sum of four quarters) as well as median quarterly earnings. See addendum for additional measures.
(5) Rate ever employed
Those who have been employed at any point within 4 quarters after exit (based on UI-reported earnings)
RECIDIVISM. Report for each month (or for each calendar quarter) after exit:
(6) Receiving TANF
Those receiving TANF in month (or quarter) as percentage of all leavers. May be reported by month, or at selected points (3 mos, 6 mos, 9 mos, 12 mos). Please report those receiving in each month (quarter), not those returning in each month (quarter). (The latter measure is included in Addendum). Document unit of analysis. (Most studies reporting on returns of adult head, or case.)
(7) Percentage Ever Returning to TANF Within 12 months
Those who have been received TANF at any point within first 12 months of exit (except excludes those whose cases were re-opened within 2 months of exit). Note that this is the sum of those returning to TANF in each of the first 12 months after exit (the optional measure discussed above).
OTHER PROGRAMS. Report for each month (or for each calendar quarter) after exit:
(8) Receiving Medicaid
Those families receiving Medicaid (regardless of TANF or Food Stamp status) as percentage of all leavers. That is, do not limit to "Medicaid only" or to "Medicaid but not TANF." Document unit of analysis, because this measure differs considerably whether analyzed for adults, children, or any member of household. Some studies are reporting separately for adults (8a) and for children (8b). Note that studies are encouraged to also report those receiving Medicaid at any point within 12 months of exit (last column in table). For additional Medicaid measures, see Addendum.
(9) Receiving Food Stamps
Those receiving Food Stamps (regardless of TANF or Medicaid status) as percentage of all leavers. That is, do not limit to "food stamp only" or to "Food stamps but not TANF." Though the differences for children and adults are not as large as in receipt of Medicaid, still important to note unit of analysis. Again, studies are encouraged to also report those receiving food stamps at any point within 12 months of exit (last column in table). For additional measures, see Addendum.
Please note that different studies also will be reporting the percentage participating in other programs (i.e., child care (see Addendum)), child support enforcement, child abuse and neglect, foster care, etc). To facilitate cross-state comparisons, please report these for closed single-parent cases off assistance for two months or longer, in addition to any other groups of interest.
ADDENDUM:
Additional Administrative Data Outcomes for Leavers
Please consider adding some, or all, of these optional measures.
| Single-Parent Leavers N= | Qtr before Exit | Quarter of Exit | 1st Qtr after Exit | 2nd Qtr after Exit | 3rd Qtr after Exit | 4th Qtr after Exit | Ever empl. over 4 Qtrs |
|---|---|---|---|---|---|---|---|
| Additional Employment Measures: | |||||||
| (A-1) Employed in All 4-Qtrs (%) | — | — | — | — | — | — | • |
| (A-2) Mean Earnings of Those Employed in All 4 Qtrs ($) | • | • | • | • | • | • | (*) |
| (A-3) Median Earnings of Those Employed in All 4 Qtrs ($) | • | • | • | • | • | • | (*) |
| (A-4) Leavers with Quarterly Earnings >= $500 (%) | • | • | • | • | • | • | • |
| 3 mos (1Q) before Exit | Month (Q) of Exit | 3 mos (1Q) after Exit | 6 mos (2Q) after Exit | 9 mos (3Q) after Exit | 12 mos (4Q) after Exit | (7) Ever on over 4Qs | |
| Recidivism (reported by month or quarter) | |||||||
| (A-5) Entering TANF, by month/quarter of entrance (%) (similar to original #6, but not cumulative entrants) | • | • | • | • | • | • | • |
| Other Program Benefits (reported by month or quarter) | |||||||
| (A-6) Continuous Leavers Participating in Medicaid (Adults) (%) | • | • | • | • | • | • | • |
| (A-6b) Continuous Leavers Participating in Medicaid (Kids) (%) | • | • | • | • | • | • | • |
| (A-7) Continuous Leavers Receiving Food Stamps (%) | • | • | • | • | • | • | • |
| (A-8) Not receiving TANF, Food Stamps or Medicaid (%) | • | • | • | • | • | • | • |
| (A-9) No earnings, no TANF, no Food Stamps or Medicaid (%) | • | • | • | • | • | • | • |
| (A-10) Receiving Child Care Subsidy (among all leavers) % | • | • | • | • | • | • | • |
| (A-11) Receiving Child Care Subsidy (employed, child 13) % | • | • | • | • | • | • | • |
| * If available, report annual earnings (i.e., sum of earnings over 4 quarters) for those with any earnings in first 4 quarters. | |||||||
Definitions of Optional Additional Administrative Data Outcomes Measures
Most of these additional measures were developed in consultation with the welfare outcomes grantees at the Fall 1999 Welfare Outcomes Grantee Meeting. Others (A-4, A-8 and A-11) were suggested by the National Academy of Sciences Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs. Though secondary to the first 9 measures, these additional measures provide further opportunities for cross-state comparisons across those states that choose to report them.
EMPLOYMENT OUTCOMES:
(A-1) Employed in All 4 Qtrs
Percentage of total leavers with quarterly earnings in all first 4 quarters after exi t. Same definitino of earnings as in Original Measure #2.
(A-2) Mean earnings of those Employed in 4 Qtrs
Calculated across cases with earnings in all 4 qtrs after exit (i.e., cases counted in A-1 above). In addition to reporting quarterly earnings, it is useful to report annual earnings (the sum of quarterly earnings across first 4 quarters after exit).
(A-3) Median earnings of those Employed in 4 Qtrs
As above, calculated across cases with earnings in all 4 qtrs after exit.
(A-4) Earnings > $500
Percentage of total leavers with quarterly earnings >$500 as reported in the UI system.
RECIDIVISM. Report for each month (or for each calendar quarter) after exit:
(A-5) Entering TANF
Those entering (e.g., returning to) TANF in month (or quarter) as percentage of all leavers. Most useful if reported by month, but can be reported by quarter. This differs from Original Measure 6, which looks at those receiving in each month (quarter), not those returning in each month (quarter).
OTHER PROGRAMS. Report for each month (or for each calendar quarter) after exit:
(A-6) Continuous Leavers Receiving Medicaid
Those families receiving Medicaid (regardless of TANF or Food Stamp status) as percentage of "continuous" leavers, that is, leavers who have NOT returned to TANF. Document the unit of analysis, because this differs considerably whether analyzed for adults, children, or any member of household. Some studies are reporting separately for adults (A-6) and for children (A-6ab). Also good to report those receiving Medicaid at any point within 12 months of exit (last column in table).
(A-7) Continuous Leavers Receiving Food Stamps
Those receiving Food Stamps (regardless of TANF or Medicaid status) as percentage of all “continuous” leavers, that is, leavers who have NOT returned to TANF. Again, good to report those receiving food stamps at any point within 12 months of exit (last column in table).
(A-8) No benefits
Percentage of total leavers who do not receive TANF, food stamps, or Medicaid.
(A-9) No earnings or benefits
Percentage of total leavers who do not have reported earnings and do not receive TANF, food stamps or Medicaid.
(A-10) Child Care Subsidy
Percentage of total leavers who receive a child care subsidy, by month or quarter.
(A-11) Child Care Subsidy
Percentage of employed leavers who receive a child care subsidy. If possible, limit the denominator to employed leavers with children.