Including:
To Leavers Studies Colleagues:
In response to your requests, staff at Child Trends have recommended a set of child outcome items that can be used in leavers surveys. In forming our recommendations, we sought to include items from three domains of development (covering health, socioemotional/behavioral, and cognitive/school-related child outcomes) while still limiting the timing of our recommended set of items to about five to seven minutes.
Focus on child outcomes rather than intervening mechanisms.
We focused our recommendations on child outcomes only (i.e. measures of the child's well-being and development), and not on intervening mechanisms (i.e. factors that may shape child outcomes). By doing this, we are in no way suggesting that intervening mechanisms are less important to children. Potential intervening mechanisms such as health insurance, use of health care, food insecurity, and child care are very important to children's development. We focused our task on recommending child outcomes because of the limited time available in the surveys, and because we know that other parts of the surveys will cover many of the important candidate intervening mechanisms.
Priority placed on potential for benchmarking with national surveys.
In selecting items for inclusion, we made it a priority to select items primarily from national surveys. We have turned most often to the National Survey of America's Families (NSAF), as this is a dataset that is nationally representative (though with the important resource of an oversample of low income families), with a wave of data collected recently (1997). It is available on the Urban Institute's web site to other researchers to use for secondary data analysis. It was administered by telephone, so we are certain that individual items work well in this mode of administration.
The use of items from national surveys will allow you as researchers to relate findings on children from the leavers studies to those in nationally representative datasets ("benchmarking"). We assumed that this would be especially helpful given that the leavers studies are not experimental in design, and do not have built-in comparison groups. The national data do not provide the equivalent of a comparison group in an experimental evaluation, but do provide a context within which to view the results (e.g., to relate findings to those on the same measure for all children in the same age range nationally; children in poverty; children receiving public assistance). Benchmarking is most appropriate where the item is identical to that used in a national survey, when the item uses the same time frame, and is asked about a child in the same age range as in the national survey. Many of the existing leavers surveys already include items similar to those we recommend from national surveys. With these items, depending on the specific item and its closeness to the national survey item, benchmarking can still be done but may be more approximate.
In several instances, we selected items from previous welfare studies rather than from national surveys. We did this for two reasons: (1) when existing national surveys did not have items available on issues we thought to be particularly important for the leavers studies, and (2) when asking about low frequency (but important) events that we thought would be better to document for "any child in the family" rather than only for a particular focal child (see below for discussion of focal children. National surveys tend to address their questions to be about particular focal children). While on these items it will not be possible to relate findings to data from nationally representative surveys, findings can be related to previous research on welfare families. In addition, it is helpful to use the wording of items that have a track record of "working" with families of similar backgrounds in other surveys.
For benchmarking purposes, items should ideally use the same age groups, time frames, and focal child methodology as recommended below and shown in the accompanying charts. In cases where this is not possible (e.g. where the time frame of interest is slightly different), researchers should be aware that this will make it more difficult to benchmark results.
Recommendation to choose a focal child.
Our recommendation for the study of child outcomes in the leavers studies is for the selection of one focal child age 6 to 17, as more of the recommended items are appropriate for this range. (This is because several items, such as school-related items, are appropriate only for older children, and also because survey items addressing certain issues may only become reliable at older child ages, e.g. behavior problems). Where time and resources permit, we recommend selecting a second focal child in the age range of 0 to 5 (just as in the NSAF) because it will add important information on the health of younger children in the context of their families leaving welfare. Selecting a focal child in the range of 6 to 17 and one in the range of 0 to 5 will assure that data can be related to findings in the NSAF. Changing the age ranges of focal children will make it more difficult to benchmark.
Using a focal child is an effective way to obtain information about how a particular child is doing within a family. The information is more precise than when asking about all of the children in the family. For this reason, it is the method used to obtain information about child outcomes in many national and welfare studies. As we have noted, in the NSAF, two focal children are selected, one age 0 to 5 and one age 6 to 17. Each is randomly selected from among the children in the family in these age ranges. The studies of welfare and children have usually started with more constrained age ranges. For example, the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies started with children ages 3 to 5 at baseline, and randomly selected one child within the family in this age range. The Project on State Level Child Outcomes (5 state project) focuses specifically on school-age children ages 5 to 12.
In addition to questions designed to be asked of focal children, our recommended core set of items includes two questions designed to be asked about "any child." These questions are designed to get information about more infrequent events that have a higher probability of being observed if examined across all children in the household. If you are unable to choose focal children in your study, you may still be able to use these recommended items.
Because there are fewer items in national surveys and welfare studies designed to be asked about "any child in the family," it would be difficult to obtain a breadth of information on children and to allow the possibility of benchmarking without selecting focal children. With each of our recommended items to be asked of focal children, it is possible to benchmark with NSAF (and sometimes with other national datasets). With the two further recommended items to be asked about any child, the reference point is previous welfare studies.
Core set of recommended items and additional items to consider.
In the charts that follow, you will find our set of recommended items. The first two charts (IA and IB) comprise our core set of recommended items. Section IA contains items to ask of the focal children, and Section IB contains items to ask of any children in the family. We define "any child" as any child in the household for whom the respondent is responsible. This core set includes 17 items (13 for a focal child in the age range of 6 to 17, two for a focal child in the age range of 0 to 5, and two for any child in the family) and should take approximately five to seven minutes.
The second two charts (IIA and IIB) contain additional child outcome items that we recommend if you are interested in expanding your child outcomes module. Section IIA provides additional items to ask about focal children in the two age ranges. Section IIB provides items to ask about any child in the family. Three of these are modifications of focal child items as an alternative if you are unable to choose a focal child and one item is entirely new (item re: accident or injury). While Section I is recommended over Section II, items are not prioritized within sections.
In summary, we recommend that you select at least one focal child, and that you ask all items in Section IA, all items in Section IB, as much of Section IIA as there is time/interest, and as much of Section IIB as is not duplicative of Section IA. If your survey does not select a focal child and you are unable to change the survey to select one, we recommend that you ask items in Section IB and Section IIB. It will be difficult, however, to obtain the same breadth of information and possibility of benchmarking without selecting focal children.
We look forward to hearing your reactions to our recommendations. Please let us know if you have any questions. We will be watching the list-serve for your questions/reactions.
Potential Constructs and Items for a Child Well-Being Module: Core Set of Recommended Items (These charts are in PDF format)