How Effective Are Different Welfare-to-Work Approaches? Five-Year Adult and Child Impacts for Eleven Programs. Analysis Samples


Table 2.2 shows the dates of random assignment and sizes of the samples used in this report, by site and research group. All the analysis samples used in the report are listed and described below.

Full impact sample. The full impact sample includes 41,715 program and control group members from all seven sites, for whom five years of administrative records data were collected (Figure 2.3, box A).(12), (13)

Figure 2.3
Sample Size and Data Sources

Figure 2.3 Sample Size and Data Sources

Five-Year Client Survey sample. Additional data on outcomes for adults and children were collected by interviewing sample members around two years after their date of random assignment and, in four of the seven sites, around their five-year anniversary. This report focuses on outcomes from the Five-Year Client Survey (detailed findings from the Two-Year Client Survey can be found in the two-year report from the NEWWS Evaluation).(14) The Five-Year Client Survey sample (Figure 2.3, box B) includes 5,408 members of the full impact sample in Atlanta, Grand Rapids, Riverside, and Portland. In each site, survey selection took place during some, but not all, months of random assignment. The survey sample was drawn from members of the full impact sample who had earlier been selected to be interviewed at two years, whether or not they actually responded to the two-year survey (4,974 sample members answered both surveys, and 434 responded to only the five-year survey; see Table 2.2).(15) Those selected to be interviewed at two years were a stratified random sample of the full impact sample members who were randomly assigned during the months when the survey sample was selected.

Certain subgroups were intentionally oversampled to produce large enough samples for special analyses such as the Child Outcomes Study (COS) and an intensive examination of adult education.(16) Results from all programs in this report have been weighted to reflect the overall demographic characteristics of the larger sample. The survey response rates exceed 70 percent for all programs and research groups (and 80 percent in Atlanta and Grand Rapids).(17)

Child Outcomes Study sample. The COS, which is part of the NEWWS Evaluation, includes the families of 2,332 sample members who responded to the Five-Year Client Survey in Atlanta, Grand Rapids, and Riverside (Figure 2.3, box C).(18) Individuals in these three sites who were selected for the Two-Year Client Survey and who had at least one child aged 3 to 5 at random assignment were randomly selected to be part of the COS. Each family included a child aged 3 to 5 at random assignment who was randomly selected to serve as the focal child, that is, the child about whom the most extensive information was collected. Response rates to the COS survey in Atlanta and Grand Rapids exceeded 75 percent; response rates in Riverside were between 63 and 67 percent for different research groups.(19)

Teacher survey sample. In a study of children's school progress, which is also part of the evaluation, COS mothers who were interviewed at the five-year follow-up point were asked for permission to mail a survey to the focal child's current elementary school teacher that asked about outcomes such as academic performance. (For details, see Section III.) The teacher survey sample includes responses from 1,472 teachers of focal children (Figure 2.3, box D).(20) Response rates were lower for the teacher survey than for the other surveys, ranging from 37 to 57 percent.(21)