Welfare Reform/Child Well-Being Administrative Data Linking. Limitations

06/01/1999

  1. Analysis of variance with repeated measures is subject to some statistical assumptions involving the sample distributions. Repeated measures ANOVA assumes normal distributions with homogeneous variance in the pre- and post- groups. Unlike two-way and r-way ANOVA and least squares regression, repeated measures ANOVA is not particularly robust when these assumptions are violated. Violations tend to result in inflated F-ratios and greater likelihood of falsely rejecting the null hypotheses. In the case of this study, plots did not uncover dramatic departures from normality and the distributions will converge towards normality as sample size increases (i.e. they are asymmetrically normal). Plots also indicate rough homogeneity of variance though formal tests were not performed given the weakness of the substantive findings.
  2. The primary limitation of this design is the lack of a control group. Since all subjects in the study had at least one person in the household who became employed, no comparison can be made that can evaluate whether any potential differences in Medicaid utilization are due to changes in employment of persons in the household. In addition, finally, there may be factors other than employment, which explain any potential differences observed in Medicaid utilization. Examples of potential confounding factors in this study could be age or obtaining private medical insurance with employment.