Alternative Outcome Measures:
TANF Block Grant

Appendix D:
Selected Sources of Data for Outcome Measures

[ Main Page of Report ]

Contents

• Survey Data
• Administrative Data

References have been made throughout this report about the need for timely and reliable data that would permit an assessment of states’ performance in achieving the goals of TANF. Discussion of the various outcome-based performance measures has included brief references to various data sources that might or might not be suitable for that purpose. Below are described some of the major sources of data for potential outcome measures, including both survey and administrative sources. This is neither a comprehensive listing nor intended to be a comprehensive description of data sources, but rather a brief introduction to the types of information that are available through each of these sources. Internet addresses are provided for those seeking additional information about the surveys discussed.

Following the narrative description is a table of selected characteristics of each data source, which includes the agency that collects the data, the unit of analysis, the frequency with which data are collected, how soon after collection the data are available for analysis, the sample size and whether the sample is large enough to allow for state-level estimates.

Survey Data

The Decennial Census is a major source of detailed population information, and the benchmark for estimation from almost all other data sets. It is a comprehensive source of information on individuals’ economic and social characteristics in local areas across the country. The short form is conducted primarily as a mail-out, mail-back survey to every household in the United States and is the basis on which seats in Congress are apportioned. It asks only name, sex, age, race/Hispanic origin, relationship, and whether the household owns or rents its housing The long form is sent to a sample of the population (17 percent in 2000) and forms the basis for social and economic information published by the Census Bureau. The long form includes questions on marital status, education, ancestry, migration, employment, income, welfare receipt, and housing conditions. New in 2000 is a question required by PRWORA, on grandparents as caregivers for children. The census is conducted every 10 years.

Web site: http://www.census.gov/dmd/www/2khome.htm

The American Community Survey (ACS) is a new approach under development by the U.S. Census Bureau for collecting accurate, timely information needed for critical government functions. The ACS instrument is based on the Decennial Census long form. If fully implemented, this new data collection approach will provide data users with timely demographic, housing, social, and economic data updated every year that can be compared across states, communities, and population groups. In addition, the ACS is a flexible data collection method with the ability to adapt to changing data needs; for example, the potential exists for adding questions of national policy interest or specialized supplements in the future. With the American Community Survey, data will be available every year for all states, as well as for all cities, counties, metropolitan areas, and population groups of 65,000 people or more. For smaller areas, it will take two to five years to accumulate a sufficient sample to produce data for areas as small as census tracts. For example, for areas of 20,000 to 30,000, data can be averaged over three years. For rural areas and city neighborhoods or population groups of less than 15,000 people, it will take five years to accumulate an adequate sample size.

Web site: http://www.census.gov/acs/www/

The Current Population Survey (CPS) has been conducted monthly by the Bureau of the Census since 1942. Its main purpose is to provide estimates for employment, unemployment, and other characteristics of the labor force. The survey focuses on individuals aged 15 and older, but since 1979 limited demographic data have been collected on children in the sample. In addition to the core monthly survey, the CPS also collects annual data in the March Supplement on prior year work experience, education, income (including welfare receipt and program participation), and migration. Other supplements focus on such topics as school enrollment, child support and alimony, and fertility. The CPS is a probability based sample, with a total sample size of about 71,000 households per month (50,000 to 57,000 are actually interviewed). The sample is representative at both the state and the national level. However, the small sample size for many states restricts its usefulness as a source of annual data on state performance, as the standard errors of the state estimates for a single year are quite large. Various small area estimation techniques are currently used by the Census Bureau and others to produce reliable state-by-state estimates, including combining and averaging three or four years of data.

Web site: http://www.bls.census.gov/cps/cpsmain.htm

The Survey of Income and Program Participation (SIPP) is a continuous series of national panels begun in 1983 by the Census Bureau. The SIPP content is built around a core of labor force, program participation, and income questions designed to measure the economic situation of persons in the United States. Panel members are asked the core questions every four months, and are asked to recall their activities over the four previous months. In each wave of interviews, a set of modules on topics not covered in the core section are also asked. Topics covered by the modules include personal history, child care, wealth, program eligibility, child support, disability, school enrollment, and taxes. Until recently, the SIPP consisted of overlapping panels, with a new panel of 14,000 to 20,000 households introduced each February (through 1993) and interviewed for a total of 2 ½ years. Starting in 1996, the SIPP panels have been expanded in both size (to about 36,000 households) and duration (to 4 years in 1996, 3 years thereafter), but a new panel will only be drawn every 4 years. The redesigned SIPP includes enhanced questions about receipt of government program benefits, including reasons why receipt was begun or ended, and which household members were covered.

Web site: http://www.sipp.census.gov/sipp/sipphome.htm

The Survey of Program Dynamics (SPD) was created specifically for the Census Bureau to track the effects of PRWORA using a "pre-post" comparison. Starting with SIPP respondents first interviewed in 1992 and 1993, the SPD is a longitudinal survey with data from annual retrospective interviews conducted each year between 1997 and 2002. Combined with SIPP data collected from the 1992 and 1993 panels, the SPD will provide longitudinal panel data on approximately 18,500 households for 10 years. The survey primarily focuses on employment and earnings, income, and program participation, and also includes questions on child well-being and adolescent behaviors.

Web site: http://www.census.gov/apsd/www/spdmenu.html

The Panel Study of Income Dynamics (PSID) is designed to study the determinants of changes in the economic well-being of families and individuals across time and generations. The survey, conducted by the University of Michigan, is based on a probability sample of about 5,000 U.S. households first interviewed in 1968. The individuals in these households are interviewed through the years, regardless of whether they remain in the same household. For example, children are followed as they advance through childhood and into adulthood, forming family units of their own. Although the original design over-sampled lower income and minority households, the sample also included a complete representative sample of families at all income levels. Surveys were conducted annually through 1997, then switched to every other year for cost reasons. While the sample size is smaller than most other national data sets, the data collected are extremely rich. The survey emphasizes the dynamic aspects of economic and demographic behavior, but covers a broad range of topics including: employment, income, wealth, housing, food expenditures, transfer income, and marital and fertility behavior.

Web site: http://www.isr.umich.edu/src/psid/

The National Longitudinal Surveys (NLS) of Labor Market Experience are a collection of panel surveys sponsored by the Department of Labor. The primary focus in these surveys is on education and labor market transitions; however, all these surveys feature a comprehensive set of questions about family relationships, income, welfare receipt and numerous other subjects. Starting in the mid-1960s, four groups were surveyed: young men (aged 14-24 in 1966), older men (aged 45-59 in 1966), young women (aged 14-24 in 1968) and mature women (aged 30-44 in 1967). The first two groups were last interviewed in 1981 and 1990 respectively, while the other two groups are still being interviewed. Another youth (both young men and young women) survey (NLSY79) was begun in 1979 and, starting in 1986, supplemental information was collected on the children born to the young women in this panel. A new youth survey (NLSY97) was begun in 1997. A key feature of the NLSY is that all respondents were asked to take the Armed Services Vocational Aptitude Battery, a high-quality test of academic and non-academic knowledge and skills.

Web site: http://stats.bls.gov/nlshome.htm

Administrative Data

Under section 411 of the Social Security Act, states are required to collect and report TANF Administrative Data. The data are collected by states monthly, reported to HHS (Administration for Children and Families) quarterly, and consist of both disaggregated and aggregated data on TANF recipients and some others in TANF households. The states also report similar data on closed cases and on participants in separate state programs. The data are used for many purposes including the calculation of participation rates for one and two parent families, determining the number of families reaching the time limits for receipt of TANF, and compiling the characteristics of TANF recipients. States have the option of providing the data via a sample of the population or the entire population. This data source could be used for measuring:

• Recidivism rate;
• Food Stamp receipt among TANF recipients;
• Medicaid/SCHIP receipt among TANF recipients;
• Out-of-wedlock birth rates for TANF families;
• Level of participation in work and work-related activities;
• Percentage of caseload reaching time limits; and
• Caseload reduction.

Most states also collect additional data elements which are not required by the federal government for their own policy and program management functions. State administrative data systems frequently collect information on such areas as caseload demographics, caseload reduction, information related to the reasons clients left the TANF caseload, work participation rates, job placement data, caseload distribution by local office, and type of child care TANF clients use (APHSA, 2000). While these data elements are not collected consistently by all states, they could be the basis for optional performance measures.

States collect Food Stamp Quality Control (QC) Data as part of quality control reviews conducted in accordance with section 16 of the Food Stamp Act o f 1977, as amended, and Part 275, Subpart C of the Food Stamp Program regulations (7CFR275). Data are collected monthly from a sample of households selected for review as part of the Integrated Quality Control System (IQCS), an ongoing review of food stamp household circumstances. The IQCS is designed to determine (1) if households are eligible to participate or are receiving the correct benefit amount and (2) if household participation is correctly denied or terminated. The IQCS is based on a national probability sample of approximately 50,000 participating food stamp households, and on a somewhat smaller number of denials and terminations. The national sample of participating households collected in the IQCS is stratified by the 50 states, the District of Columbia, Guam and the Virgin Islands. Annual required state samples range from a minimum of 300 to 1200 reviews, depending on the size of the state’s caseload. State agencies select an independent sample each month that is generally proportionate to the size of the monthly participating caseload.

The Medicaid Management Information System (MMIS) is designed to collect, manage, analyze, and distribute information on eligibles, recipients, use and payment for services covered by State Medicaid programs. States provide the Health Care Financing Agency (HCFA) with quarterly files containing specified data elements for: (1) persons covered by Medicaid (Eligible File), and (2) adjudicated claims for medical services reimbursed with Title XIX funds (Claims File). These data are furnished on a Federal Fiscal Year quarterly schedule, which begins October 1 of each year. The Eligible File, which is used to tract enrollment on a quarterly basis, contains one record for each person who was eligible for Medicaid for at least one day during the reporting quarter. These files classify individuals by type of eligibility category and contain a flag for TANF receipt, even though Medicaid eligibility is no longer automatically linked to welfare receipt. The Claims File contains several types of records: all Current Claims for Medical Services, Adjustments to Previously Paid Claims, Premium Payments, Lump Sum Adjustments, and Dummy Claims. Dummy Claims simulate claims that would have been generated for Managed Care patients if they were billed on a fee-for-service basis. The Claims Files are submitted quarterly based on the date of payment, not on the date of service.

The State Children’s Health Insurance Program (SCHIP) requires States to provide quarterly expenditure reports to support claims for federal matching funds and financial/statistical data for purposes of program monitoring and evaluation. Regarding expenditure reports, in order to receive Federal matching funds, States report quarterly expenditures made during any quarter of the State program's operation. Such expenditure reports should be submitted within 30 days of the end of the quarter for use in calculating federal funding requirements. For enrollment data, States report enrollment figures on a quarterly basis in four age categories for children under age 19 (under 1, 1-5, 6-12, and 13-18) and family income categories related to Federal poverty levels and State cost sharing requirements. States report on the total unduplicated number of children served in the SCHIP program (both separate SCHIP and Medicaid expansion) as well as the non-SCHIP-related children in the regular Medicaid program, and enrollees and disenrollees for each quarter. These reports also provide for reporting enrollment status data for each type of service delivery system that the individual is enrolled in (i.e., fee-for-service, managed care, or primary care case management) by age and income categories.

As under TANF, states frequently collect additional information regarding their Food Stamp program, Medicaid, and SCHIP that is not required by the federal government. These data may be stored in separate or unified data systems. Some states record monthly data in such a manner that longitudinal event histories for cases my be constructed, while others store only the most current information.

State employment security agencies collect Unemployment Insurance (UI) Quarterly Earnings Records for the purposes of determining eligibility for unemployment insurance. UI records are available on a relatively timely basis – within six months of the end of a quarter. UI reports total quarterly earnings, but does not provide information on wage rates or hours of work. UI records do not provide the start or end date of employment, so it may be difficult to determine whether an individual left assistance before or after they started a job. UI does not capture all employment – by law, certain types of employment are not "covered" by unemployment insurance, meaning the system does not capture the employment of individuals working in specific types of jobs. Employment that is not covered by the UI system includes independent contractors, federal (including military) and foreign government employees, student employees at an educational institution, domestic employees (below a specified earnings level), certain agricultural workers and elected officials. Moreover, there is reason to believe that much casual or irregular employment is never reported to any government agency, and is therefore missed by the UI data. While there is not believed to be significant variation among states in terms of how coverage is defined, states with higher percentages of non-covered employment may be disadvantaged by use of UI data to measure employment rates. Finally, because UI records are maintained at the state level, they do not record circumstances when individuals find employment in another state. This would affect areas where major employment centers cross state lines.

UI data can be linked to a list of social security numbers in order to calculate employment rates and earnings levels for a given population. While this linking task is not trivial, it is being widely performed by states for research purposes as well as to calculate the performance measures for the TANF High Performance Bonus. The main advantage of using UI records is that all of the data collection is occurring anyway, and therefore the additional burden and cost is minimal.

The National Directory of New Hires (NDNH) is part of the expanded Federal Parent Locator Service provided by the Office of Child Support Enforcement, Administration for Children and Families, U.S. Department of Health and Human Services. The primary purpose of the NDNH is the establishment and enforcement of child support obligations by helping states locate non-custodial parents and identify their employers. Other purposes of the database include support for the administration of IV-A programs (TANF) and research. The NDNH contains information obtained from States’ New Hire Directories, State Employment Service Agencies (quarterly earnings and Unemployment Insurance benefits) and Federal Agency personnel offices (new hire and quarterly wage data). The data are maintained for a period of two years unless there is an active child support enforcement case.

The NDNH shares many of the strengths and weaknesses of the underlying UI data on which it is based. One key difference is that the NDNH includes federal civilian and military workers, who are not covered under states’ UI systems. In addition, because the NDNH is a national dataset, it captures information on employment even if a recipient is not working in the same state where she receives welfare. Finally, because the federal government could perform the match between the NDNH and the list of TANF recipients, the data could be more consistently measured across states and the burden on state agencies could be reduced.

Where to?

Top of page
Contents

Main page of report
Contents of report

Home Pages:
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation (ASPE)
Administration for Children and Families (ACF)
U.S. Department of Health and Human Services (HHS)

Updated: 02/12/01