Across a series of alternative models that address different potential sources of specification error and bias (Table IV.4), we consistently find a positive estimated ELE effect, supporting the findings from the main model. In all of the alternative models in Table IV.4, the ELE coefficient remains positive with a central tendency that is close to what we find in the main model; we find that the magnitude associated with the ELE variable in the total Medicaid/CHIP alternative models ranges from 2.4 to 4.8 percent and in the Medicaid-only alternative models ranges from 4.0 to 7.3 percent. For all other models in which the results are not shown, we find that the ELE effect is also close to what we find in the main model.
Although remaining consistently positive, however, we find that the statistical significance of the estimated ELE effect varies across the model specifications. The estimated ELE coefficient in the basic unadjusted difference-in-difference model (alternative 1) is still similar in magnitude to the main fully adjusted model result, but is not statistically significant (p-value = .11 in the Medicaid model and .20 in the Medicaid/CHIP model). Alternatives 2 and 4 show that controlling for differential policy changes among ELE states and the comparison group strengthens the precision of the estimated effect, but that the inclusion or exclusion of the policy variables do not drive the magnitude and direction of the ELE variable in the main model.
We also find that the ELE effect is slightly smaller in magnitude and statistically insignificant (p-value = .12 in the Medicaid model and .14 in the Medicaid/CHIP model) when we use all 41 non-ELE states as the comparison group, as opposed to using states with similar pre-ELE enrollment trends (alternative 5). However, the estimates of the ELE effect from this model could be biased downward because they include comparison states with quarterly enrollment levels trending upward relative to ELE states during the pre-implementation period. We also find that the ELE effect in the Medicaid model is statistically significant in all of the remaining comparison group sensitivity models (alternatives 6 through 11), whereas the ELE coefficient in the Medicaid/CHIP model is statistically significant in only two of these six alternatives.
Appendix C contains a more detailed discussion of each alternative and describes how these results raise confidence in the direction of the ELE effects found in the main Medicaid model, but introduces some uncertainty about the underlying impact of ELE on separate CHIP enrollment based on the findings in the combined Medicaid/CHIP model.