How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Academic Functioning

12/01/2001

Table 12.3 shows the impacts on child outcomes related to academic functioning. As mentioned, these outcomes were reported by mothers, teachers, and children, as well as obtained from a standardized assessment, and fell into the following subdomains: behavioral adjustment to school, academic achievement, academic progress and placement, and attendance.

Table 12.3
Impacts on Academic Functioning

Site and Program

Sample Size Program Group Control Group Difference (Impact) Effect Size

School engagement (range of 0 to 21) (child report)

Atlanta Labor Force Attachment 552 16.6 16.8 -0.2 -0.07
Atlanta Human Capital Development 617 16.7 16.8 -0.1 -0.05
Grand Rapids Labor Force Attachment 377 16.3 17.2 -0.9** -0.30
Grand Rapids Human Capital Development 358 16.5 17.1 -0.6* -0.21
Riverside Labor Force Attachment 459 16.2 16.3 -0.1 -0.02
Lacked high school diploma or basic skills 290 16.2 16.0 0.2 0.05
Riverside Human Capital Development 379 16.4 16.1 0.3 0.09

School engagement (range of 0 to 33) (teacher report)

Atlanta Labor Force Attachment 364 19.8 19.1 0.7 0.11
Atlanta Human Capital Development 414 20.6 19.2 1.4* 0.21
Grand Rapids Labor Force Attachment 286 18.9 19.2 -0.3 -0.04
Grand Rapids Human Capital Development 262 18.2 19.1 -0.9 -0.14
Riverside Labor Force Attachment 324 18.7 19.5 -0.9 -0.13
Lacked high school diploma or basic skills 205 19.0 18.5 0.5 0.08
Riverside Human Capital Development 274 19.0 18.7 0.4 0.06

Suspended or expelled since last interview a (%) (mother report)

Atlanta Labor Force Attachment 599 8.5 6.5 2.0 0.10
Atlanta Human Capital Development 676 7.0 6.1 1.0 0.05
Grand Rapids Labor Force Attachment 428 5.7 9.0 -3.3 -0.13
Grand Rapids Human Capital Development 410 7.0 8.3 -1.3 -0.05
Riverside Labor Force Attachment 533 2.8 6.8 -4.0* -0.17
Lacked high school diploma or basic skills 338 3.3 4.6 -1.3 -0.07
Riverside Human Capital Development 438 6.0 4.5 1.4 0.08

Had disciplinary action taken weekly (%) (teacher report)

Atlanta Labor Force Attachment 369 30.8 37.3 -6.5 -0.17
Atlanta Human Capital Development 412 32.1 36.6 -4.5 -0.12
Grand Rapids Labor Force Attachment 285 34.3 31.6 2.7 0.06
Grand Rapids Human Capital Development 261 37.8 31.2 6.6 0.15
Riverside Labor Force Attachment 325 39.4 29.5 9.9* 0.24
Lacked high school diploma or basic skills 206 32.4 30.6 1.8 0.05
Riverside Human Capital Development 274 33.9 30.9 3.0 0.08

Had a discipline problem that resulted in parent(s) being notified this school year (%) (teacher report)

Atlanta Labor Force Attachment 365 46.4 55.3 -8.9 -0.22
Atlanta Human Capital Development 407 42.4 55.8 -13.4*** -0.34
Grand Rapids Labor Force Attachment 284 33.4 32.7 0.7 0.02
Grand Rapids Human Capital Development 259 47.1 31.4 15.7** 0.36
Riverside Labor Force Attachment 325 35.7 29.2 6.4 0.15
Lacked high school diploma or basic skills 205 24.6 31.9 -7.3 -0.19
Riverside Human Capital Development 274 35.0 33.2 1.8 0.05

Broad Math Score on the Woodcock-Johnson Tests of Achievement-Revised

Atlanta Labor Force Attachment 557 99.7 99.5 0.2 0.01
Atlanta Human Capital Development 627 101.1 99.4 1.7 0.11
Grand Rapids Labor Force Attachment 377 100.0 99.2 0.8 0.05
Grand Rapids Human Capital Development 363 97.8 99.0 -1.2 -0.08
Riverside Labor Force Attachment 463 98.9 97.8 1.1 0.07
Lacked high school diploma or basic skills 292 97.1 94.6 2.5 0.19
Riverside Human Capital Development 383 95.6 95.4 0.2 0.01

Had above-average Broad Math Score on the Woodcock-Johnson Tests of Achievement-Revised (%)

Atlanta Labor Force Attachment 557 30.2 27.4 2.8 0.08
Atlanta Human Capital Development 627 31.3 27.8 3.5 0.10
Grand Rapids Labor Force Attachment 377 31.4 23.6 7.8* 0.20
Grand Rapids Human Capital Development 363 28.0 24.2 3.8 0.10
Riverside Labor Force Attachment 463 30.8 24.2 6.6 0.16
Lacked high school diploma or basic skills 292 30.7 15.9 14.8*** 0.49
Riverside Human Capital Development 383 18.6 17.9 0.7 0.02

Had below-average Broad Math Score on the Woodcock-Johnson Tests of Achievement-Revised (%)

Atlanta Labor Force Attachment 557 23.5 25.8 -2.3 -0.06
Atlanta Human Capital Development 627 22.2 26.3 -4.1 -0.11
Grand Rapids Labor Force Attachment 377 24.5 21.4 3.1 0.08
Grand Rapids Human Capital Development 363 28.5 21.8 6.8 0.18
Riverside Labor Force Attachment 463 25.2 27.0 -1.9 -0.05
Lacked high school diploma or basic skills 292 27.9 31.7 -3.8 -0.10
Riverside Human Capital Development 383 34.9 30.1 4.9 0.13

Broad Reading Score on the Woodcock-Johnson Tests of Achievement-Revised

Atlanta Labor Force Attachment 557 94.7 95.0 -0.2 -0.02
Atlanta Human Capital Development 627 95.5 95.0 0.5 0.04
Grand Rapids Labor Force Attachment 377 100.2 98.8 1.4 0.09
Grand Rapids Human Capital Development 363 97.7 98.7 -1.0 -0.06
Riverside Labor Force Attachment 463 95.4 95.2 0.3 0.02
Lacked high school diploma or basic skills 292 93.5 92.6 0.9 0.06
Riverside Human Capital Development 383 93.7 93.2 0.5 0.03

Had above-average Broad Reading Score on the Woodcock-Johnson Tests of Achievement-Revised (%)

Atlanta Labor Force Attachment 557 15.9 17.5 -1.6 -0.05
Atlanta Human Capital Development 627 17.5 17.6 -0.1 -0.00
Grand Rapids Labor Force Attachment 377 31.5 25.4 6.1 0.15
Grand Rapids Human Capital Development 363 27.1 26.1 1.0 0.02
Riverside Labor Force Attachment 463 18.9 19.6 -0.7 -0.02
Lacked high school diploma or basic skills 292 17.4 16.9 0.5 0.02
Riverside Human Capital Development 383 15.2 17.9 -2.7 -0.09

Had below-average Broad Reading Score on the Woodcock-Johnson Tests of Achievement-Revised (%)

Atlanta Labor Force Attachment 557 33.3 36.5 -3.2 -0.08
Atlanta Human Capital Development 627 32.1 36.3 -4.3 -0.11
Grand Rapids Labor Force Attachment 377 24.4 27.9 -3.5 -0.08
Grand Rapids Human Capital Development 363 30.2 28.7 1.5 0.04
Riverside Labor Force Attachment 463 36.0 33.7 2.2 0.05
Lacked high school diploma or basic skills 292 43.2 38.7 4.5 0.11
Riverside Human Capital Development 383 40.1 37.1 3.0 0.08

Repeated a grade since last interview a (%) (mother report)

Atlanta Labor Force Attachment 599 12.8 13.0 -0.1 -0.00
Atlanta Human Capital Development 676 10.9 12.9 -2.0 -0.07
Grand Rapids Labor Force Attachment 428 16.4 11.8 4.6 0.15
Grand Rapids Human Capital Development 410 16.1 12.2 3.8 0.13
Riverside Labor Force Attachment 533 3.2 9.4 -6.1** -0.23
Lacked high school diploma or basic skills 338 4.8 10.2 -5.4* -0.22
Riverside Human Capital Development 438 6.1 9.9 -3.8 -0.16

Ever repeated a grade (%) (teacher report)

Atlanta Labor Force Attachment 336 15.0 11.8 3.2 0.13
Atlanta Human Capital Development 381 11.8 11.9 -0.1 -0.00
Grand Rapids Labor Force Attachment 248 8.9 13.2 -4.3 -0.14
Grand Rapids Human Capital Development 238 15.7 12.0 3.6 0.12
Riverside Labor Force Attachment 288 3.7 5.3 -1.6 -0.08
Lacked high school diploma or basic skills 181 5.6 7.0 -1.4 -0.07
Riverside Human Capital Development 242 3.7 6.8 -3.1 -0.16

Below grade level in math (%) (teacher report)

Atlanta Labor Force Attachment 362 35.9 35.7 0.2 0.01
Atlanta Human Capital Development 401 32.1 35.0 -2.9 -0.08
Grand Rapids Labor Force Attachment 283 38.9 38.4 0.5 0.01
Grand Rapids Human Capital Development 257 42.7 38.2 4.4 0.10
Riverside Labor Force Attachment 321 47.4 39.7 7.8 0.18
Lacked high school diploma or basic skills 204 46.6 47.1 -0.5 -0.01
Riverside Human Capital Development 269 43.0 45.0 -2.0 -0.05

Above grade level in math (%) (teacher report)

Atlanta Labor Force Attachment 362 9.8 8.0 1.8 0.08
Atlanta Human Capital Development 401 7.7 8.3 -0.6 -0.03
Grand Rapids Labor Force Attachment 283 8.1 11.7 -3.6 -0.12
Grand Rapids Human Capital Development 257 9.2 11.9 -2.7 -0.09
Riverside Labor Force Attachment 321 6.7 8.5 -1.8 -0.06
Lacked high school diploma or basic skills 204 4.3 4.8 -0.5 -0.03
Riverside Human Capital Development 269 5.2 5.3 -0.1 -0.00

Below grade level in reading (%) (teacher report)

Atlanta Labor Force Attachment 369 38.8 42.7 -3.9 -0.10
Atlanta Human Capital Development 412 39.3 42.8 -3.5 -0.09
Grand Rapids Labor Force Attachment 285 50.3 44.7 5.6 0.12
Grand Rapids Human Capital Development 259 50.9 45.6 5.4 0.12
Riverside Labor Force Attachment 326 49.5 42.4 7.1 0.16
Lacked high school diploma or basic skills 207 56.0 54.1 2.0 0.05
Riverside Human Capital Development 274 50.1 53.1 -3.0 -0.07

Above grade level in reading (%) (teacher report)

Atlanta Labor Force Attachment 369 12.6 6.9 5.6* 0.27
Atlanta Human Capital Development 412 11.4 6.2 5.2* 0.25
Grand Rapids Labor Force Attachment 285 9.6 12.9 -3.2 -0.11
Grand Rapids Human Capital Development 259 14.8 13.1 1.7 0.05
Riverside Labor Force Attachment 326 9.6 15.6 -6.0 -0.17
Lacked high school diploma or basic skills 207 7.5 9.8 -2.3 -0.09
Riverside Human Capital Development 274 9.4 10.9 -1.5 -0.06

In remedial math group (%) (teacher report)

Atlanta Labor Force Attachment 368 27.7 33.9 -6.2 -0.17
Atlanta Human Capital Development 411 31.1 32.5 -1.4 -0.04
Grand Rapids Labor Force Attachment 287 30.7 28.1 2.7 0.06
Grand Rapids Human Capital Development 262 24.7 28.4 -3.7 -0.09
Riverside Labor Force Attachment 326 27.5 27.5 -0.0 -0.00
Lacked high school diploma or basic skills 207 28.7 33.5 -4.7 -0.13
Riverside Human Capital Development 272 33.2 32.0 1.2 0.03

In remedial reading group (%) (teacher report)

Atlanta Labor Force Attachment 368 36.7 36.7 -0.0 -0.00
Atlanta Human Capital Development 413 35.6 36.8 -1.2 -0.03
Grand Rapids Labor Force Attachment 285 34.1 39.3 -5.3 -0.12
Grand Rapids Human Capital Development 262 41.8 40.6 1.2 0.03
Riverside Labor Force Attachment 326 44.5 34.8 9.7* 0.23
Lacked high school diploma or basic skills 207 54.3 42.9 11.4 0.28
Riverside Human Capital Development 274 42.8 43.8 -0.9 -0.02

Goes to a special class or school, or gets special help in school, for a physical, emotional, or mental condition (%) (mother report)

Atlanta Labor Force Attachment 600 8.2 6.7 1.5 0.07
Atlanta Human Capital Development 678 5.9 6.7 -0.8 -0.04
Grand Rapids Labor Force Attachment 428 21.5 20.0 1.4 0.04
Grand Rapids Human Capital Development 410 23.9 19.7 4.2 0.11
Riverside Labor Force Attachment 533 14.5 12.2 2.3 0.08
Lacked high school diploma or basic skills 338 17.3 14.1 3.2 0.11
Riverside Human Capital Development 438 12.0 14.2 -2.2 -0.08

Identified as needing and receiving special services (%) (teacher report)

Atlanta Labor Force Attachment 377 33.5 42.9 -9.4* -0.24
Atlanta Human Capital Development 419 43.0 42.5 0.5 0.01
Grand Rapids Labor Force Attachment 288 44.4 45.5 -1.1 -0.02
Grand Rapids Human Capital Development 264 48.4 45.8 2.5 0.05
Riverside Labor Force Attachment 330 46.7 44.4 2.2 0.05
Lacked high school diploma or basic skills 209 52.3 54.1 -1.8 -0.04
Riverside Human Capital Development 279 50.1 54.9 -4.8 -0.12

Identified as needing and not receiving special services (%) (teacher report)

Atlanta Labor Force Attachment 377 40.7 38.2 2.6 0.07
Atlanta Human Capital Development 419 32.9 38.3 -5.4 -0.14
Grand Rapids Labor Force Attachment 288 33.0 29.3 3.7 0.09
Grand Rapids Human Capital Development 264 31.9 28.3 3.6 0.09
Riverside Labor Force Attachment 330 41.2 32.0 9.2 0.22
Lacked high school diploma or basic skills 209 43.7 34.2 9.5 0.24
Riverside Human Capital Development 279 29.1 33.2 -4.1 -0.11

Days absent during current school year (%) (teacher report)

Atlanta Labor Force Attachment 318 5.3 4.2 1.1 0.23
Atlanta Human Capital Development 359 5.9 3.8 2.1** 0.43
Grand Rapids Labor Force Attachment 254 7.2 6.6 0.6 0.07
Grand Rapids Human Capital Development 243 6.1 6.8 -0.7 -0.09
Riverside Labor Force Attachment 303 6.2 5.6 0.6 0.09
Lacked high school diploma or basic skills 191 6.4 6.1 0.3 0.06
Riverside Human Capital Development 256 8.8 6.3 2.5** 0.48

Days tardy during current school year (%) (teacher report)

Atlanta Labor Force Attachment 316 4.0 2.3 1.7* 0.44
Atlanta Human Capital Development 360 2.7 2.2 0.5 0.13
Grand Rapids Labor Force Attachment 253 3.0 3.9 -0.9 -0.13
Grand Rapids Human Capital Development 242 3.9 3.9 -0.0 -0.00
Riverside Labor Force Attachment 301 4.0 3.4 0.6 0.09
Lacked high school diploma or basic skills 190 3.5 3.2 0.3 0.05
Riverside Human Capital Development 253 3.4 3.3 0.1 0.01

SOURCE:  Child Trends calculations from the Five-Year Child Outcomes Study survey (mother, teacher, and child reports).
NOTES:  See Appendix A.2.
Woodcock-Johnson Broad Math and Broad Reading scores are age-standardized, with a mean of 100 and a standard deviation of 15.
a Measures of events "since the last interview" apply to years 3 to 5 for 2,163 COS sample members (who responded to the survey at 2 years) and years 1 to 5 for 169 sample members (who were interviewed at baseline but not at 2 years).

In general, the impacts on academic functioning were less numerous and their patterns less clear than those found within the social skills and behavior domain. Yet in five of the six programs there were impacts on measures relating to behavioral adjustment to school that, though few, were consistent with the impacts on social skills and behavior noted above. This suggests that impacts on more global ratings of children's social skills and behavior were accompanied by simultaneous changes on measures pertaining specifically to children's behavioral adjustment to school  namely, disciplinary problems and engagement in school.

The pattern of impacts on disciplinary problems and school engagement generally follows that found in the social skills and behavior domain. Both Atlanta programs decreased the likelihood that focal children had a discipline problem requiring parental notification, and the Grand Rapids HCD program increased this likelihood. The Riverside LFA program also increased problems in the full sample, increasing the likelihood of having disciplinary action taken at least weekly with the focal child in the few months prior to the survey. These four programs generally altered the likelihood of discipline problems by between 9 and 16 percentage points, decreasing levels from about 56 to 42 percent in both Atlanta programs and increasing levels from 31 to 47 percent in the Grand Rapids HCD program and from 30 to 39 percent in the Riverside LFA program. In addition, following the pattern of social skills and behavior impacts, the Atlanta HCD program also increased teacher-reported levels of children's engagement in school, whereas both Grand Rapids programs decreased child-reported levels of school engagement. The effect sizes of these impacts ranged between .20 and .30. Hence, the general pattern of findings in the social skills and behavior domain was replicated and extended for measures of academic functioning relating to children's behavioral adjustment to school. The singular exception to this pattern is a decrease in suspensions and expulsions (as reported by mothers) in the Riverside LFA program.

Impacts in the other subdomains of academic functioning were less prevalent across the six programs. Impacts on standardized tests of academic achievement were rare, with only two impacts found across the six programs and across the six measures. There were no aggregate impacts on mean age-standardized achievement test scores, with mean scores for program and control group children across the six programs falling between 93 and 100 on the reading assessment (representing the 32nd and 50th percentiles, respectively) and between 95 and 101 on the math assessment (representing the 37th and 51st percentiles, respectively).(18)(19) There were also no impacts on the prevalence of "below-average" scores  that is, mean age-standardized scores less than 90. The two impacts that were found were both favorable, increasing the prevalence of "above-average" scores (scores above 110) on the math assessment in the Grand Rapids LFA program and in the Riverside LFA program for the subgroup of children whose mothers had limited education at baseline.

Within the subdomain of academic progress and placement, impacts were also few, with no impacts on five of the eight measures. There were some impacts on measures of whether the focal child had repeated a grade (as reported by the mother), was in a remedial reading group, and performed above grade level in reading. The Riverside LFA program decreased the likelihood of having repeated a grade, dropping to about 4 percent from a control group level of about 10 percent for both the full sample and the subgroup of children whose mothers lacked a high school diploma or GED at baseline. The Riverside HCD program also decreased grade repetition by about 4 percentage points, though this difference was just above the cutoff for statistical significance. Yet the Riverside LFA program simultaneously increased the number of children who were in a remedial reading group by about 10 percentage points for both the full sample and those with limited education (from levels of about 35 and 43 percent, respectively), although the difference for the subgroup with limited education was just beyond the cutoff for statistical significance.(20) The Riverside LFA program also increased the percentage of children needing but not receiving services (an unfavorable finding), a difference just beyond the cutoff for statistical significance.

Finally, both Atlanta programs increased the proportion of children reported by their teachers as being above grade level in reading, increasing the levels from about 6 to 12 percent in both programs. In addition, the Atlanta LFA program increased the proportion of children needing and receiving services (an impact that may be favorable or unfavorable depending on whether it reflects more children needing services or more of those who need services receiving them).

Impacts on measures of attendance were also relatively scarce. Generally, control group children were absent for between 4 and 7 percent of days and were tardy between 2 and 4 percent of days in the current school year. Three of the programs (Atlanta LFA and HCD and Riverside HCD) increased absences, although the difference for the Atlanta LFA program was just beyond the cutoff for statistical significance.(21) The Atlanta LFA program also increased the percentage of days that children were tardy. All four of these differences were in the range of 1 to 3 percentage points. As speculated in the next section, the impacts may be related to unfavorable health impacts that were found in these three programs.