All 14 of the studies funded by the Department of Health and Human Services, Office of Assistant Secretary for Planning and Evaluation (ASPE), are using a combination of linked administrative data sets and surveys of former recipients to monitor the economic status and general wellbeing of families leaving welfare. At this point (July 1999), most grantees are about half way through their projects; and six have released interim reports based on administrative analyses of an early cohort of leavers. Most grantees define this early cohort as those that ceased receiving cash assistance in late 1996 or early 1997, as states were beginning to implement the Personal Responsibility and Work Opportunities Act (PRWORA). Grantees are still working on their administrative analyses of later cohorts of leavers (e.g., those leaving in late 1997, 1998 or early 1999), and on collecting and analyzing survey data for these later cohorts. Although a number of states have released reports based on statefunded surveys of leavers, most of the HHS-funded surveys are still in the field. Final reports, which will combine analyses of linked administrative data with findings from surveys of former recipients, are scheduled for completion at various times over the next twelve months, depending on the schedule of the individual grantees.
As stated above, each grantee is fielding a survey to gather information about the overall wellbeing of families leaving welfare. Although research proposals ranged across grantees, most grantees proposed surveying a sample of between 600 and 1200 former recipients, at a point between 4 to 12 months after exit from cash assistance.(3) With one exception, grantees are conducting mixed mode surveys, consisting of telephone interviews for the majority of households, supplemented by in-person follow-up for those who cannot be reached by telephone. Most surveys are expected to take 20-40 minutes. In this short time period, grantees are planning to ask questions across a broad array of topics, including employment and earnings, program participation, health insurance, child wellbeing, barriers to selfsufficiency, etc.
One key challenge has been the design of a survey instrument to gather information across this broad array of topics, within the constraints of a timelimited telephone interview. One potential solution to this challenge would have been to design a single survey for use across multiple sites. Such an approach would have had the advantage of providing common measures for crossstate comparisons and the development of a national picture of welfare leavers.(4) It was rejected early in the design of the project, however, because of its disadvantages, particularly in the current era of devolution of welfare policy making to the state and local levels.
One disadvantage of a nationally designed survey is that it might not address questions of most policy relevance to a particular state. For example, a state with a fullfamily sanction might want to ask different questions than a state with a partial sanction. Second, a national study would not be as effective in building state and local capacity for ongoing TANFrelated research to support state and local policy development. Finally, the consensus process for working with multiple states to develop a single instrument would likely be quite lengthy, with additional time required for official clearance of a national survey instrument by the Office of Management and Budget (OMB). A lengthy research process, however, conflicted with the desire of both Congress and state and local policy makers to know as soon as possible what was happening to families leaving welfare.
In sum, the Department of Health and Human Services decided to award grants that provided each research team with considerable flexibility in designing its own research project, including its own survey instrument. At the same time, Federal staff worked to encourage some degree of comparability of outcomes across states, within the constraints of a tight time schedule and limited resources. Steps taken to facilitate comparability included: holding an initial planning meeting in November 1998; establishing of an electronic “listserve” for discussion among researchers; developing consensus on a common definition of the “leavers” study population;(5) developing guidance on a proposed set of nine commonly reported administrative outcomes for leaver studies;(6) and, of most relevance for this paper, disseminating resources on surveys, including examples of survey items.
As shown in Table I, grantees were provided with paper copies of, or website addresses for, selected components of the Census Bureau’s Survey on Income and Program Participation (SIPP) and Survey on Program Dynamics (SPD), the USDA’s “Food Security/Hunger Core Module,” the Urban Institute’s “National Survey of America’s Families (NSAF), and other selected survey items. Some of these items were distributed in a resource book provided to all participants at the first grantee meeting, others were the subject of presentations at that same meeting, mailed to all grantees, or discussed over the listserve, primarily in December 1998 and January 1999.
Table I. Dissemination of Survey Items
|Survey||Topic Area(s) Disseminated||Resource Book||Presentation||ListServe||Mailed|
|Census Bureau’s Survey of Program Dynamics (SPD)||Employment and earnings||X||X|
|Census Bureau’s Survey of Income and Program Participation (SIPP)||Training||X||X|
|USDA’s Food Insecurity Module in CPS||Food Security||X||X||X|
|Urban Institute’s National Survey of America’s Families (NSAF)||Insecurity/Deprivation (6 items)||X|
|Child Well-Being (Sections C and N)||X|
|Health (Sections B, E, and F)||X|
|MDRC’s Urban Change||All||X|
|State and Local Area Integrated Telephone Survey (SLAITS)||Health||X|
|Abt’s Family Transition Program||Employment and earnings||X||X|
|South Carolina Survey||Insecurity/Deprivation||X||X|
|Draft items developed by Mary Ellen Colten at U. Mass, Boston||11 items on Domestic Violence||X|
- Resource Books with photocopies of surveys and other technical assistance items were distributed to all 60 participants at November grantee meeting.
- Presentations were made at the November grantee meeting, on topics such as measuring employment and earnings, measuring insecurity/deprivation, and measuring child well-being.
- The List-serve was used to disseminate web-site addresses and text of surveys, as well as to facilitate discussion among researchers.
- MDRC mailed copies of the Urban Change survey to each of the 14 grantees.
Given the nature of the grants, and the limited amounts of technical assistance provided on surveys, one might expect that each grantee would develop quite different surveys. On the other hand, there was sufficient sharing of questionnaires and discussions on the listserve that one might expect some areas of similarity. Most of the rest of this paper reviews specific items to get some sense of the similarity and dissimilarity in the grantees’ survey instruments. Before doing so, however, I provide a brief overview of the types of research topics included in the grantees’ study designs.