Monitoring Outcomes for Former Welfare Recipients: A Review of 11 Survey Instruments. Food Insecurity

08/11/1999

Food insecurity differs from the other four outcome measures in this review, because a “core module” of questions on food security/hunger was distributed to all grantees.  This module was developed by a collaboration of Federal agencies and academic researchers, fielded in the April 1995 Current Population Survey (CPS) Supplement, and analyzed in a series of USDA reports released in September 1997.(10)  Researchers from the U.S. Department of Agriculture encouraged the incorporation of this new set of questions in the leavers’ surveys, disseminating copies to all grantees and discussing it at the grantee meeting and through the list­serve.

From the beginning, grantees were uncertain that they could incorporate the full set of 16 questions, four of which have multiple parts.  USDA researchers, on the other hand, urged grantees to include all questions in their leavers’ surveys, because the full set of questions can be combined into a single overall measure of food security, drawing on 18 items from the 16 questions.(11)  They were reluctant to promote a reduced form of the 18-item scale, arguing that the module was shorter than it seemed, since many questions are skipped if the household responds “no” to the initial screening questions.  Inclusion of the full 18-item scale would also allow comparison to other surveys incorporating these items, such as the Survey on Program Dynamics (SPD).  Grantees were concerned, however, that the questions seemed repetitive.

In the end, none of the grantees included the full core module with its 21 items (the 18-item scale plus a 3-item introductory screener).  As shown in Table IV, the highest number of items included in a leavers’ study instrument was seven items, in the Cuyahoga/L.A. instrument.  Three grantees — Arizona, South Carolina, and Wisconsin — ask only question (although they ask it about two time periods, while on welfare and since exit).  The average grantee included 3.3 items.  Although most (86 percent) of these items were drawn directly or with slight modifications from the USDA Food Security/Hunger Core Module, they were selected across the breadth of the survey instrument.

Table IV:  Food Insecurity

Questions Other
#8.   Cut size of or skip/meals (*)
#8a.  How often (*)
X
X
X X   X
X
  X         O N
O N
5
2
#1.  Food adequacy;
#1a.  Why not enough food
#1b.  Why not kinds of foods wanted
X
 
  X
X
X
X
 
        X
 
    O S
 
4
1
1
#3.  Food did not last (*) X     X X   X         O N 4
No way to buy food (not from USDA)           X   X   X X   4
#4.  Not afford balanced meals (*) X X   X               O 3
#2.  Worried about food running out       X X             N 2
#12.  Adults not eat full day
#12a.  How often
  X X                   2
0
#14.  Children ever skip meals;
#14a. How often
  X       X
X
            2
1
#13.  Ever cut size of children's meals   X                     1
#16.  Any children not eat food for day   X                     1
#10.  Eat less than felt should (*) X                     O 1
#9.  Ever hungry (*) X                     O 1
Satisfaction with food [not from USDA]               X         1
#5.  Relied on low-cost foods                         0
#6.  Not afford balanced meals (children)                         0
#7.  Children not eating enough                         0
#11.  Lost weight                         0
#15.  Children ever hungry                         0
Total Number of Questions 7 6 5 4 4 3 2 2 1 1 1    

Notes:
(*) Indicates on USDA short (6-item) scale.

Other:  N - National Survey of America's Families;  O - Child Outcomes Project;  S - SIPP.

Note:  Arizona, Illinois, South Carolina, and Wisconsin ask each question twice (period on welfare, period off welfare)

 

The most common question, asked by five grantees, is #8 in the USDA instrument (shown in Appendix IV-A), concerning whether adults in the household had cut the size of meals, or skipped meals because there was not enough money for food.  Two grantees ask a follow-up question (#8a) about frequency of cutting the size of or skipping meals.

Another two grantees (Washington and Missouri) ask questions about meals being cut or skipped by children (#13, #14, #14a).  Both previous research and analysis of the 18-item scale indicate that households tend to reduce food intake by adults before they reduce food intake by children and so reductions in meal sizes or meals by children is a greater sign of food insecurity and hunger.  In fact, the items used by Washington and Missouri are used in the 18-item scale to distinguish households with “severe” hunger from households with “moderate” hunger.(12)

The questions about adults cutting the size of or skipping meals, #8 and #8a, are two of the six items included in a short (6-item) version of the USDA scale.  This scale was included in the grantees’ resource books, but was not discussed much at the grantee meeting or the list­serve, where the discussion focused on the full scale.  This was perhaps unfortunate, because it turns out that the short scale is being used by four of the five states involved in the HHS­funded “Project on State­Level Child Outcomes,” a project that adds child outcome measures to existing welfare reform evaluations.(13)  Moreover, three of the six items in the short scale also are included in the Urban Institute’s National Survey of America’s Families (NSAF), which reports outcomes related to welfare reform in 13 states as well as on a national basis.(14)  If grantees had known of its widespread use, they might have focused more attention on it.  Instead, some dismissed it, because all six questions ask about adult rather than child experiences of reduced food intake and hunger, giving it the appearance of being ill­suited for studies of families with children.  In fact, the six questions about adult experiences with hunger can be used in households with children, because hunger in such households is likely to first show up as reduced food intake among adults, as discussed above.  Another possible concern about the 6-item scale is that it cannot distinguish between households with “moderate” hunger and those with such “severe” hunger that children are affected.  The latter group is relatively rare, however, and so may be difficult to identify accurately without use of the full 18-item scale.

Another common question, included in four leavers’ surveys, is #1, which asks about food insufficiency, that is, whether the household has enough food to eat and enough of the kinds wanted.  A follow­up question probing for reasons for food insufficiency is asked in one survey (San Mateo).  This question and its follow­ups are not included in the short 6-item version, nor are they strictly part of the 18-item scale.  The question is, however, is included in the Survey of Income and Program Participation, the Survey of Program Dynamics, and other national surveys, and has been used to measure food insufficiency over a 20-year period.

Two other common questions are #3, which asks whether the family found that the food that they bought just didn’t last, and #4, which asks whether the family found it could not afford to serve balanced meals.  Four grantees, the Child Outcomes Project, and the NSAF are fielding #3, and three grantees and the Child Outcomes project are fielding #4.  Both these questions are in the short (6-item) versions of the food security scale.

Other questions drawn from the Core Module were asked by one or two grantees, including:

  • Whether the respondent worries about food running out (#2, asked by two grantees);
  • Whether adults in the household skipped food for a day (#12, asked by two grantees);
  • Whether the adult experienced hunger (#9, asked by one grantee and the Child Outcomes project and part of the short scale);
  • Whether the respondent felt there was not enough food (#10, asked by one grantee and the Child Outcomes project and part of the short scale); and
  • Whether children in household skip food for a day (#15, asked by one grantee).
  •  

Six items in the Core Module (#5, 6, 7, 11, 12a, and 15) were not asked by any grantee.

Finally, four grantees asked a question that was not directly from the USDA scale, namely, “was there a time when you had no way to buy food?”  First used in the South Carolina survey, this question has been picked up by Wisconsin, Missouri, and Florida.  If respondents answer yes, South Carolina and Wisconsin ask the respondent to state whether this occurred while the family was on welfare, or since exit.  Florida also asks a question not included on other surveys concerning satisfaction with foods eaten by the household.  The wording of these items is shown in Appendix IV-B.

Note that even when four or five grantees do ask the same general question about food insecurity, there may be variations which make it harder to make cross­state comparisons.  The most important of these concerns time period.  Although some grantees ask about food insecurity “during the past twelve months,” as in the Core Module, others ask over a 6-month time period, a 30-day time period, or the varying period since exit from TANF.  Four grantees (Arizona, Illinois, South Carolina, and Wisconsin) ask the respondent to compare food insecurity in the 6-month or longer period since exit from welfare with food insecurity while on welfare.  While this question gets directly at an issue of great interest to policy makers, it is not clear if respondents can accurately recall hunger experiences across two different 6-month time periods.

Only five food insecurity questions were asked by three or more grantees.  These questions include three questions from the short version of the USDA scale (#8, #3, #4 ), one other question from the full core (#1), and a question not found on the USDA module (the South Carolina question).  Four items were asked by two grantees, eight items by only grantee (including the Florida item not from the USDA module), and six items were not asked by any grantee.  Overall, items on the short scale were asked by more grantees than other items in the full module:  the 6 items on the short scale were asked by an average of 2.7 grantees, and the 15 other items from the Food Security/Hunger Module scale were asked by an average of 1.4 grantees.

In sum, comparisons of food insecurity across leavers’ studies will be limited by wording variations, and even more by the fact that grantees chose food insecurity items from across the broad array of 21 different items provided in the full USDA Food Security/Hunger Module.  Opportunities will exist, however, for comparisons of food insecurity among leavers to food insecurity among more general national and state populations, given the inclusion of the short or full versions of the USDA module in such surveys as the SPD, NSAF, and welfare reform evaluations involved in the Project on State­Level Child Outcomes.