Impact on Young Children and Their Families 2-Years After Enrollment: Methods: How Did We Study Impacts on Children? . Child Outcome Measures Included in the Two-Year Follow-Up

09/19/2001

The two-year follow-up included measures of children's development in three broad aspects, or "domains," of development:

  • cognitive development and academic achievement
  • emotional and behavioral adjustment, including both behavior problems and positive social behavior; and
  • physical health and safety.

We note that some measures of development are available for the focal child in each family only, and some are available pertaining to any child in the family. Below, we briefly describe the measures used to assess development in each of these domains. A number of the measures pertaining to any child in the family come from the core instrument, and thus were asked of all families in full NEWWS sample participating in the survey component.(7) We note that other measures central to our analysis of child outcomes are described in detail in other chapters. In particular, Chapter 8 lists and describes the particular measures we use in assessing how child impacts come about (i.e., asking whether variables such as total family income, maternal depression, and participation in child care changed in response to JOBS, and whether changes on these variables help explain impacts on child outcomes).

I. Measures of Cognitive Development and Academic Achievement

  • The Bracken Basic Concept Scale/School Readiness Composite (BBCS/SRC) (focal child only): This is a direct assessment of school readiness administered to the focal child during the two-year follow-up survey. While the full Bracken Basic Concept Scale consists of 11 subtests, in the Child Outcomes Study we use only the 5 subtests comprising the School Readiness Composite (for which administration averaged about 15 minutes in the present study). These 5 subtests assess the child's knowledge of colors, letters, numbers/counting, comparisons, and shapes. Previous research provides evidence of reliability as well as validity for the Bracken Basic Concept Scale. The New Chance Evaluation, a study examining the development of the young children of teenage mothers on welfare, found the School Readiness Composite to be highly correlated with the full Bracken scale, and also reported significant correlations between the School Readiness Composite and teacher reports of the children's academic progress (Polit, 1996). Within the NEWWS Child Outcomes Study sample, internal consistency reliability(8) for the School Readiness Composite was high (.97). When describing the cognitive school readiness of focal children in the control groups of the Child Outcomes Study and comparing these children to the national standardization sample (see Chapter 5), we will report on BBCS/SRC standardized scores. When presenting program impacts, we will report on focal children's raw BBCS/SRC scores, indicating the number of concepts (out of a possible 61) that the child answered correctly. We used raw scores in impact analyses in order to describe experimental-control group differences in meaningful units (i.e., the number of concepts answered correctly). Because raw scores are not adjusted for the age of the focal child (and since older children tend to answer more concepts correctly than younger children), focal child's age was controlled in all impact analyses involving children's raw BBCS/SRC scores.

    In this report, we report on mean BBCS/SRC scores, as well as on the proportions of children scoring at the "low" and "high" ends of the distribution of raw scores. Specifically, we determined that the bottom quartile of the national standardization sample had an age-standardized score of 7.8 or lower. Thus, children in the Child Outcomes Study sample with age-standardized scores of 7.8 or lower were categorized as scoring "low" on the BBCS/SRC. In a parallel fashion, we determined that the top quartile of the national standardization sample had an age-standardized score of 12.2 or higher. Thus, children in the Child Outcomes Study sample with age-standardized scores of at least 12.2 were categorized as scoring "high" on the BBCS/SRC.

  • Academic Problems in School Rating (focal child/any child in the family): This is a dichotomous (yes/no) measure, calculated separately for the focal child and for any child in the family. A "yes" on this measure indicates the presence of either one or both of the following academic problems in school: (1) the focal child/any of the children in the family has repeated any grade for any reason; (2) the focal child/any of the children in the family goes to a special class or special school or gets special help in school for learning problems. A score of zero indicates that neither problem has occurred, while a score of one indicates that at least one has occurred.

    In previous research, it has proven useful to have both measures of children's cognitive development obtained via direct assessment, and measures of children's academic progress/difficulties. For example, evaluations of early childhood interventions sometimes show enduring impacts on measures like those included here (retention in grade and use of special services for help with learning problems) even in instances where initial positive program impacts on assessments of cognitive development have not been shown to endure over time (Barnett, 1995; Entwistle, 1995). Because of previous evidence that programs can have differential impacts on assessments of cognitive development and measures of school progress, both kinds of measures are included here.

II. Measures of Behavioral and Emotional Adjustment

  • Behavior Problems Index (focal child only): In the Behavior Problems Index the mother is asked to indicate whether statements are not true, sometimes true, or often true about her child. These describe such behaviors as: the child is high strung, tense or nervous; the child cheats or tells lies; the child has trouble getting along with other children. Previous work with the Behavior Problems Index indicates high internal consistency reliability. Further, this measure discriminates between children who have and have not received clinical treatment (Zill, 1985).

    In the present analyses, we use a total score and subscale scores for externalizing behavior problems (such as bullying, lying, and cheating) and internalizing behavior problems (such as feeling unhappy, sad, or depressed, and feelings of worthlessness). The choice of these particular subscales was made in light of factor analyses with the Behavior Problems Index items in the present sample, and also in light of previous work by Peterson and Zill (1986). In the present sample, internal consistency reliability coefficients were .89 for the total scale; .61 for the externalizing subscale; and .64 for the internalizing subscale.

    In this report, we present means on these three measures, as well as the proportions of children scoring at the "low" and "high" ends of the distributions of these three measures, based on a national sample of five- to seven-year-old children. Specifically, we determined that the 25th percentile of total behavior problem scores for five- to seven-year-old children in the full National Longitudinal Survey of Youth-Child Supplement (NLSY79-CS) sample was 0.18, and we categorized Child Outcomes Study children as having "infrequent" total behavior problems if they scored this low or lower on the BPI. Similarly, we determined that the 75th percentile of total behavior problem scores for five- to seven-year-old children in the full NLSY79-CS sample was 0.54, and we categorize Child Outcomes Study children as having "frequent" total behavior problems if they scored 0.54 or higher.

    With respect to externalizing behavior problems, we determined that the 25th percentile of scores for five- to seven-year-old children in the full NLSY79-CS sample was 0.20, and we categorized Child Outcomes Study children as having "infrequent" externalizing behavior problems if they scored this low or lower. Similarly, we determined that the 75th percentile of externalizing behavior problem scores for five- to seven-year-old children in the full NLSY79-CS sample was 0.60, and we categorized Child Outcomes Study children as having "frequent" externalizing behavior problems if they scored 0.60 or higher.

    Finally, for internalizing behavior problems, we determined that 55 percent of five- to seven-year-old children in the full NLSY79-CS sample were reported to have no internalizing behavior problems; we similarly categorized Child Outcomes Study children at the "low" end if they had no internalizing behavior problems. We determined that the 75th percentile of internalizing behavior problem scores for five- to seven-year-old children in the full NLSY79-CS sample was 0.20, and we categorized COS children as having "frequent" internalizing behavior problems if they scored 0.20 or higher.

  • Positive Child Behavior Scale, Social Competence Subscale (PCBS/SCS) (focal child only): The Positive Child Behavior Scale was used in the present evaluation, as in other evaluation studies, to assure that program effects on positive social behaviors (and not only effects on problem behaviors) could be assessed. The Positive Child Behavior Scale was developed by Polit for the New Chance Evaluation (Polit, 1996), using modifications of items from existing scales so as to be appropriate for a sample of disadvantaged mothers. As for the Behavior Problems Index, the mother is asked to indicate whether behaviors are not true, somewhat true, or often true of her child. Seven items from the Social Competence Subscale developed by Polit were included in the Child Outcomes Study. Examples of the behavioral descriptions in these items are: the child is helpful and cooperative; shows concern for other people's feelings; is admired and well-liked by other children. Internal consistency reliability of this 7-item version of the subscale within the present sample is strong (coefficient alpha of .77). In addition, as would be expected, scores on this subscale are significantly negatively correlated with the total score on the Behavior Problems Index, though the magnitude of the correlation is moderate rather than high, suggesting that the two measures are related but not highly overlapping (r=-.29, p<.001).

    In this report, we present means on this measure, as well as the proportions of children scoring at the "low" and "high" ends of the distributions of the PCBS/SCS, based on estimated scores of a national sample of five- to seven-year-old children. Because national data on the PCBS/SCS are not available, we calculated z-scores on the Behavior Problems Index, identified the corresponding raw score on the PCBS/SCS, and categorized Child Outcome Study children as "low" or "high" in terms of positive behaviors if their mean scores fell below the low or above the high cutoffs. Specifically, Child Outcomes Study children with a mean PCBS/SCS score of 1.28 or lower were categorized as having "infrequent" positive behaviors, and COS children with a mean PCBS/SCS score of 1.85 or higher were considered to have "frequent" positive behaviors.

  • Emotional Problems Rating (focal child/any child in family): This is a dichotomous (yes/no) measure. A "yes" indicates the presence of any one or more of the following, according to maternal report: (1) the focal child/any child in the family is currently getting help for any emotional, mental or behavioral problem; (2) since random assignment in the evaluation, the mother feels or someone has suggested that the focal child/any child in the family needs help for any emotional, mental or behavioral problem; (3) the focal child/any child in the family goes to a special class or special school or gets special help for behavioral or emotional problems. A score of 1 indicates that the focal child/any child has had at least one of these problems, while a score of 0 indicates that none of these is true. As an indication of convergent validity for this measure, the Emotional Problems Rating for the focal child was significantly and positively correlated with the total score on the Behavior Problems Index (r=+.34, p<.001).
  • Suspended/Expelled from School (focal child/any child in family): This measure is based on a single maternal response item, recorded separately for the focal child and regarding all the children in the family: "Have any of your children ever been suspended, excluded or expelled from school? Was that [the focal child] or another child?" A score of 0 indicates that the focal child/any child in the family has not been suspended, excluded or expelled from school, while a score of 1 indicates that this has occurred. As an indication of convergent validity for this measure, for focal children there was a significant positive correlation between Suspended/Expelled and the Emotional Problems Rating (r=+.15, p<.001).

III. Measures of Child Health and Safety

  • Child Health Rating (focal child only): The mother provided a rating of her child's overall health in response to the single interview question: "Would you say that your child's health in general is excellent, very good, good, fair or poor?" This measure has been widely used(9). Validation work indicates that this health rating primarily reflects physical health problems (Krause and Jay, 1994).

    In the present report, we analyze this measure in two ways: (1) we compute mean scores on the full 1-5 rating; and (2) we dichotomize the measure, distinguishing between focal children with a rating of excellent and very good (to which we assigned a score of 1), and those with a rating of good, fair or poor (to which we assigned a score of 0). In analyzing data from the National Health Interview Survey, Coiro and Zill (1994) found it useful to consider the percentage of children given a "favorable" health rating by their mothers (rated as in excellent health and with no limiting condition). For example, in this national dataset, the percentage of children given a favorable health rating was found to differ by family income, parental education, metropolitan vs. rural residence, and race/ethnicity. Children from poor families were much less likely to receive a favorable health rating. In keeping with these findings, in the present analyses we will consider not only mean scores on the child health rating completed by the mothers, but also the proportion of children receiving favorable scores, defined here as ratings of either "excellent" or "very good" health.

  • Accident or Injury (focal child/any child in family). This measure was based on a single maternal report item, answered separately for the focal child and for all children in the family: "Since [the date of the respondent's random assignment in the evaluation], has the focal child/any of your children had an accident, injury, or poisoning requiring a visit to a hospital emergency room or clinic?" A score of 0 indicates that the focal child/any child in the family did not have such an accident or injury, while a score of 1 indicates that such an accident or injury did occur since random assignment. There is a low but significant negative correlation between the occurrence of an accident and/or injury for the focal child, and the focal child's overall health rating (r=-.06, p<.001), such that children who had had an accident and/or injury had lower overall health ratings.