Impacts on Young Children and Their Families Two Years After Enrollment: Summary Report. Footnotes


1.  An unfavorable impact of Riverside's labor force attachment program on the percent of focal children rated by their mothers to be in very good or excellent health.

2.  All statistically significant impacts on measures of children's development warrant continued monitoring. However, impacts that meet the criterion for policy relevance are relatively larger impacts and, thus, may have greater implications for children's development. Consequently, policy makers may want to take special note of these impacts in their policy decisions.

3.  The only exceptions relate to unfavorable health impacts of Riverside's human capital development program, a single unfavorable health impact of Atlanta's labor force attachment program, and a single unfavorable behavioral impact of each of Atlanta's JOBS programs.

4.  Researchers in the behavioral sciences often rely on Cohen's (1988) characterization of effect sizes (in standard deviation units) of .20 as "small," .50 as "medium," and .80 as "large."

5.  The report of the Descriptive Study ("How Well Are They Faring? AFDC Families with Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation") asked whether, overall, this group of children was at risk for poor developmental outcomes, and which factors were most closely associated with the development and well-being of the children (Moore, Zaslow, Coiro, Miller, and Magenheim, 1995). This study identified risk factors for poor developmental outcomes in the children, but also protective factors associated with more positive development.

6.  Findings will also be reported in the future from a special in-depth study of parenting behavior (the JOBS Observational Study; see Zaslow et al., 2000). This study, carried out within a subset of the Child Outcomes Study sample, involved direct observation of mother-child interaction soon after baseline and again four and a half years after baseline. Fine-grained observational measures of parenting behavior are used to ask whether mother-child interaction was affected by assignment to one of the JOBS programs (the Atlanta human capital development program). The JOBS Observational Study was funded by the Foundation for Child Development, the William T. Grant Foundation, the George Gund Foundation, and an anonymous funder, with additional funds for pretesting of middle childhood observational measures provided by the U.S. Department of Health and Human Services.

7.  Individuals in need of basic education, and those not in need of basic education were then assigned to different random assignment processes in the Riverside site. Those who were considered to be in need of basic education were randomly assigned to any one of the three research groups. However, those considered not in need of basic education could be randomly assigned only to the labor force attachment or control groups (see Hamilton et al., 1997). As a result, when contrasts of research groups are carried out in the Riverside site, those in the human capital development group are compared to control group members who are likewise considered in need of basic education, whereas members of the labor force attachment group (who could be in need or not in need) are compared to all control group members. By contrast, Atlanta and Grand Rapids did not take mothers' basic education needs into account when randomly assigning them to a program or control group; thus, mothers in these sites' human capital development programs were not as disadvantaged (with respect to literacy and/or educational attainment) as the mothers assigned to Riverside's human capital development program. Accordingly, in considering the patterning of findings for human capital development programs across all three sites, the reader should keep in mind, not only how the sites differ in terms of population and economic conditions (see Chapter 3), but also differences in the experimental designs.

8.  Mothers were asked to rate the focal child's overall health in the following question: "Would you say that (CHILD's) health in general is: excellent, very good, good, fair, or poor?"

9.  Chapter 2 of the report provides details about the specific "focal child" and "any child" measures. Chapter 6 presents impact results for specific child outcome measures.

10. Some would argue that each statistically significant finding is valid irrespective of the number of analyses conducted. Others would argue that the greater the number of analyses conducted, the greater the likelihood of chance findings and, thus, one needs to take the number of analyses into account. In order to minimize the likelihood of reporting chance findings, we chose to calculate the proportion of statistically significant impacts across all child outcome measures and across all six programs. Given that the experiment-wise Type I error rate was set at .10, we might expect to find significant results 10 percent of the time due to chance alone, translating into just over 13 statistically significant impacts across the 22 child outcome measures and the six programs (22 x 6 = 132, 132 x .10 = 13.2). Some argue that a more stringent standard is needed, requiring that the number of significant impacts within each program must exceed chance levels, or that the number of significant impacts within each domain of child development must exceed chance levels. Because there is a lack of consensus on this issue among statisticians, and given that a goal of the Child Outcomes Study was to provide a thorough examination of program impacts, we did not adhere to this more stringent standard. Moreover, our examination of impacts at the subgroup level (see Chapter 7) and of pathways through which particular impacts on children appear to have come about (see Chapter10) provides further evidence that even rare impacts are not necessarily chance findings.

11.  The full report discusses the more subtle ways in which subgroup impact findings differed across the approaches to defining risk (sibling constellation risk, educational risk, work risk, maternal psychological well-being risk, and cumulative risk; with an exploratory examination of reservations about working as well).

12.  The proportion of findings to reach statistical significance relative to the number of impacts examined, exceeded what might be expected on the basis of chance for all six approaches to defining risk. The general pattern of findings that we discuss for higher- and lower-risk subgroups held across the different approaches to defining risk.

13.  Includes all earnings, AFDC, food stamps, WIC, SSI, social security, any unemployment insurance or workers' compensation, refugees assistance, foster care payments, family or friends outside the household, estimated earned income tax credit, and is net of estimated child care expenses.

14.  The 1994 poverty level for a single mother with two children was $11,940 and for a single mother with three children, $15,081. This poverty line for the appropriate household size was divided by 12.

15.  Notably, moving from experimental to non-experimental analyses creates some potential difficulties with respect to selection bias. For instance, because JOBS did not randomly assign mothers to various types of child care, mothers' child care choices reflect, to some degree, their preferences, motivations, and other characteristics. As a hypothetical illustration, if mothers with problem behavior children are disproportionately more likely than mothers without problem behavior children to seek quality child care in hopes of curbing the problem behavior  and assuming that quality child care truly leads, causally, to better behavioral outcomes in children  then the observed statistical "effect" of child care on children's behavioral outcomes will underestimate the true positive effect of child care. The mediational analyses reported on here and in the full report (as well as the experimental impact analyses that preceded them) did not control for children's prior developmental and behavioral outcomes; however, we did control for numerous other variables representing prior characteristics of the child, mother, and family at baseline that may serve as selection factors. (For a complete list, see the third footnote of Chapter 10 of the full report.) The availability of this number and range of baseline variables is a great asset of this dataset, and previous research using these data indicate that many of these baseline variables do, to some extent, capture selection into employment (Zaslow, McGroder, Cave, and Mariner, 1999), selection into child care (Zaslow, Oldham, Magenheim, and Moore, 1998), and selection into parenting pattern (McGroder, 2000). Thus, we are likely to have controlled for many, though not all, possible selection effects.

16.  Future work may attempt to examine in greater detail how such variables are linked.

17.  The analyses for this particular child outcome indicate a statistically significant role for both AFDC receipt and mother's work hours in models considering these variables separately (along with covariates), but not in a combined model. This appears to be due to multicollinearity of the mediating variables. For all other results, the findings summarized held in a final combined model as well as in models considering each mediating variable separately.

18.  As we have noted, researchers in the behavioral sciences often rely on Cohen's (1988) characterization of effect sizes (in standard deviation units) of .20 as "small," .50 as "medium," and .80 as "large." Cohen (1988) acknowledges that this characterization is somewhat arbitrary and "is recommended for use only when no better basis for estimating the effect size index is available" (p. 25). For example, when the accumulation of empirical research demonstrates that a given effect size is predictive of a "meaningful" difference in an outcome generally agreed to be "important," this effect size may well be considered "large" regardless of its absolute size.

19.  The only exceptions relate to unfavorable health impacts of Riverside's human capital development program, a single unfavorable health impact of Atlanta's labor force attachment program, and a single unfavorable behavioral impact of each of Atlanta's JOBS programs.