Using 2007 to 2011 quarterly SEDS data, we estimate separate regression models for total Medicaid/CHIP enrollment and for Medicaid enrollment only, where the dependent variable is the log transformation of children’s enrollment in each state and quarter. We estimate two-way fixed effect difference-in-difference equations with balanced panels as our main models for this analysis, where the eight ELE states constitute the treatment group (with the intervention occurring at different points in time) and matched non-ELE states with similar pre-2009 enrollment trends comprise the comparison group. The main estimation equations are the following:
(1) Log(McaidCHIP)i,t = ∝ + β1ELEi,t+ β2OTHERPOLICYi,t + β3COVARIATESi,t + υi + δt + ∈i.t
(2) Log(Medicaid)i,t = ∝ + β1ELEi,t+ β2OTHERPOLICYi,t + β3COVARIATESi,t + υi + δt + ∈i.t
where ∝ is the intercept term, i is an index for state, t is an index for unique quarter, υ is a set of state dummy variables (state fixed effects), δt is a set of quarter-specific dummy variables (quarter fixed effects), and isin;i,t is a random error term. The dependent variable, Log(McaidCHIP)i,t, is the log of the number of children ever enrolled in Medicaid or CHIP in state i during quarter t, and corresponds Log(McaidCHIP)i,t to the number of children ever enrolled in Medicaid. We log transform enrollment so that the dependent variable has a normal distribution; otherwise the distribution of the untransformed variable is heavily skewed. We report robust standard errors clustered at the state level to correct for possible heteroskedasticity and autocorrelation (White 1980; Bertrand et al. 2004).
The key independent variable of interest is ELTi,t, which is set to one when the observation is an ELE state and the quarter either contains the month when ELE was implemented or is after ELE implementation. This variable measures the effects of ELE on Medicaid/CHIP or on Medicaid-only enrollment, depending on the model. With a log-transformed dependent variable, the estimated ELE coefficient reflects the percentage change in total enrollment associated with ELE implementation. We anticipate that ELE will have a positive impact on Medicaid/CHIP enrollment—that is, β1 is greater than zero.
Compared with the simple descriptive comparisons, findings from this model offer more rigorous evidence of the effects of ELE because they control for many sources of potential confounding factors. The state fixed effects ϒi help control time-invariant differences across states that could be correlated with the ELE variable, such as inherent differences between ELE and non-ELE states, for example, potential differences in reporting accuracy of the SEDS data. The quarter fixed effects βt control for factors common to all states that vary from quarter to quarter.
By including indicators for other state policy changes and time-varying covariates, we control for other factors that change over time, which could also contribute to differences in aggregate Medicaid and CHIP enrollment numbers. OTHERPOLICY is a series of state policy variables and COVARIATES is a series of other state-level controls that vary over time and that could influence Medicaid/CHIP enrollment. In the combined Medicaid/CHIP model—Equation (1)—OTHERPOLICY includes the simulated Medicaid/CHIP eligibility threshold for children;53 the simulated Medicaid eligibility threshold for parents; and dummy indicators for the presence of a separate CHIP program, joint applications for Medicaid and CHIP, presumptive eligibility for Medicaid, administrative verification of income for Medicaid, no in-person interview for Medicaid, continuous eligibility for Medicaid, presumptive eligibility for CHIP, administrative verification of income for CHIP, no in-person interview for CHIP, elimination of asset test for CHIP, and continuous eligibility for CHIP. In the Medicaid-only model—Equation (2)—we use the simulated child Medicaid eligibility threshold and do not include the CHIP-specific policy dummy variables. In the main specification, COVARIATES includes the state quarter-specific unemployment rate and year-state child population estimates that are log transformed.
53 The simulated CHIP eligibility threshold is used for states with separate CHIP programs and the simulated child Medicaid eligibility threshold is used for all other states. In sensitivity models in which we focus on separate CHIP only, COVARIATES includes the CHIP eligibility threshold and CHIP-specific administrative simplification dummy variables.
a. Choosing Comparison States
Difference-in-difference models provide consistent estimates of the treatment effect only if, in the absence of the policy intervention, the time path in the outcome is the same for both the treatment and comparison states (Meyer 1995). For example, if Medicaid enrollment trends upward (downward) at a faster rate within the comparison group relative to the ELE states, the difference-in-difference model will understate (overstate) the benefits of ELE implementation. Given the widespread variation in Medicaid/CHIP participation, enrollment, and policies across states, we anticipate that some non-ELE states will have similar trends in enrollment compared with ELE states, whereas others will have dissimilar trends.
Using a method similar to that employed by Lien and Evans (2005), we chose comparison states that had similar pre-ELE trends in Medicaid and Medicaid/CHIP enrollment as the ELE states. Because the first ELE program was implemented in 2009, we focus on trends in the 2007 and 2008 quarters before adoption of ELE. To select the comparison states, we estimate models similar to Equations (1) and (2) that include a time trend interacted with an ELE state indicator. We include one non-ELE state at a time and test if the average trend among ELE states differs from the trend for that non-ELE state. If we reject the hypothesis at the 5 percent level that the coefficient associated with the interaction term equals zero, we exclude the non-ELE state from the sample, thus increasing the likelihood of choosing comparison states that possess a similar trend in Medicaid or Medicaid/CHIP enrollment as the average treatment state before ELE implementation.
The final Medicaid model includes 33 comparison states and the final Medicaid/CHIP model includes 25 comparison states. In the Medicaid model, we exclude Colorado, and Wyoming from the comparison group. In the combined Medicaid/CHIP model, we exclude California, Connecticut, Florida, Indiana, Kentucky, Missouri, North Dakota, Ohio, Tennessee, and Texas. We excluded Arizona, Illinois, Maine, Montana, Nevada, New Mexico, Virginia, and Washington from both models.
b. Sensitivity Tests
We conduct a series of robustness checks to explore the consistency of the ELE parameter estimates. To the extent that these estimates display consistency, it strengthens the evidence provided by the original model specification and, thereby, the conclusions that can be drawn from the analysis. These robustness checks include reestimating the main model with the following variants:
- Alternative specifications of the control variables to determine the source of the ELE effect:
- To start, we remove the policy variables, unemployment rate, and child population from the main model specification (that is, this model includes only state and quarter fixed effects). This simple unadjusted difference-in-difference model removes all time-varying covariates and approximates the average ELE treatment effect from the descriptive data, relative to the chosen set of comparison states (alternative 1).
- We then add the policy variables to the simple model (all at once and each individually) to determine if their inclusion alters the magnitude and significance of the ELE variable (alternative 2).
- We also add the unemployment rate and child population variables to the simple model to determine if their inclusion alters the magnitude and significance of the coefficient on the ELE variable (alternative 3).
- We replace all of the administrative simplification dummy variables with a policy index, ranging from 0 to 5 in the Medicaid model and 0 to 10 in the Medicaid/CHIP model (alternative 4).
- Alternative specifications with respect to how the comparison group is defined, excluding non-ELE states in a systematic manner to determine if specific control states drive the main results. These tests are important because the non-ELE states control for what the baseline trend in Medicaid/CHIP enrollment would have been in the absence of ELE.
- We include all 41 non-ELE states as the comparison group in the Medicaid/CHIP and Medicaid models (alternative 5).
- We use the same methodology from the main model to select comparison states, but exclude non-ELE states in which the time trend interaction term is statistically significant at the 10 percent level (alternative 6) and at the 1 percent level (alternative 7). In the Medicaid/CHIP model, there are 22 comparison states in alternative 6 and 35 in alternative 7, compared with 25 in the main model. In the Medicaid-only model, there are 30 and 36 comparison states in alternatives 6 and 7, respectively, compared with 33 in the main model.
- We use a similar but more restrictive method to select comparison states (alternative 8). Instead of interacting the ELE indicator with the time trend, we interact each quarter dummy with the ELE variable and exclude non-ELE states in which we reject the null hypothesis that the joint interaction terms are zero at the 5 percent level. This method increases the likelihood of choosing comparison states that have the same quarter-to-quarter pattern in enrollment before 2009 and excludes more comparisons states relative to the main model scenario. Under this alternative, there are 15 comparison states in the Medicaid/CHIP model and 19 comparison states in the Medicaid model.
- We exclude non-ELE states that are statistical outliers and might not serve as ideal comparison states. For this exercise, we remove 8 non-ELE states from the Medicaid/CHIP model and 9 non-ELE states from the Medicaid-only model that had observations with studentized residuals greater than 2.5 and less than –2.5 in the main model specification (alternative 9).
- Similarly, we reestimate the simple unadjusted Medicaid/CHIP and Medicaid-only difference-in-difference models, including one non-ELE state at time to determine which comparison states have the strongest influence on the ELE coefficient magnitude. We then rank the states based on the estimated ELE coefficient when they are included in the model and reestimate the main model, excluding the comparison states that resulted in the 5 highest and the 5 lowest ELE effects, respectively (alternative 10). We also estimate a variant that excludes comparison states with the 10 highest and 10 lowest ELE effects (alternative 11).
We also estimate several other alternative models to support the robustness of the ELE variable, but the results are not included here.54 For instance, we include a control for whether the state expanded coverage to children who have lawfully resided in the United States for fewer than five years under the new CHIPRA option.55 We also add controls for the receipt of Cycle I (awarded September 2009) or Cycle II (awarded August 2011) CHIPRA outreach grants.
54 These results are discussed in the final report to ASPE (Blavin et al 2012) and are available upon request.These results are discussed in the final report to ASPE (Blavin et al 2012) and are available upon request.
55 We also created different ELE policy variables—“ELE through SNAP” and “ELE through tax returns”—to explore whether there appeared to be a differential effect based on the type of ELE program implemented, but the limited experience with ELE to date constrains our ability to make such an assessment.