Readers should be aware of several limitations of this study. First, in many states, filings were missing in some study years. Moreover, important data elements were sometimes absent from existing filings, or of questionable quality. Lack of consistent data on medical loss ratios (MLRs) required us to drop that analysis. Sometimes filings included information on either the requested increase or the approved increase, but not both. Thus, in calculating approval rates or modifications, we had to base our estimates on a sub-population of the sample. Enrollment data on products were generally available, but sometimes of questionable validity. For example, some carriers reported equal enrollment for HMO, PPO and indemnity plans and these figures were equal to total enrollment.
Because of missing or low quality data, it was necessary to impute enrollment in plans for some observations. (Carrier enrollment was based on the NAIC data so there was no imputation for carrier enrollment.) We attempted to use best available data such as MEPs-IC in the state’s small employer market to estimate enrollment in a carrier’s HMO, PPO, and indemnity plan if enrollment data were missing or of inadequate quality.
Because of missing data, we have conducted extensive sensitivity testing around our weighting strategy and criteria for data to be included in the sample. For four different scenarios, point estimates were statistically different for premium increases in eight of 30 estimates. However, these point estimates when statistically significant changed by approximately one percentage point. The major conclusion of the study – that premium increases declined substantially in both the individual and small group markets in 2011 – is not changed with different weighting methods.
Another limitation concerns the sample. Although the original sample design was a stratified random sample, we replaced states purposively when it became clear it was not possible to collect data from some states. Within states, we were to select carriers using a random stratified sample design. Later in the study, to increase the total number of filings, we added as many carriers as possible within the sample states.
Lastly, although the study tracks trends in premiums before the passage of the Affordable Care Act and for the first year after rate review went into effect, the study’s methods do not enable us to assert that the ACA is responsible for the decline in premium increases from 2010-2011.