Review of Existing Data Sources to Assess the Impacts of the Affordable Care Act Medicaid Expansions: Working Paper. Impacts on Individuals


For purposes of evaluating the impacts of Affordable Care Act on Medicaid-eligible populations, the data sources included in our review were the ACS, CPS, NHIS, MEPS, SIPP, BRFSS, NHANES, and NSDUH. The appendix to this report provides detailed information about each of these surveys.

A key issue in assessing the usefulness of the different surveys for evaluating the Affordable Care Act's impacts on Medicaid-eligible populations is sample size for the various population groups of interest. As shown in Table 4, the surveys vary quite a bit in this regard. The ACS has the largest sample size, with nearly 335,000 non-elderly adults at or below 138% of poverty included in the 2010 public use file. At the other end of the spectrum, the NHANES public use file for calendar years 2009 and 2010 combined includes only 1,770 observations for this group, and 380 observations at or below 50% of poverty.6 Many of the surveys have fairly limited sample sizes for the extreme poverty group. Although a common way of addressing this limitation is to combine data from multiple years, using this strategy would have negative impacts on the timeliness of the evaluation.

Only one data source, the NSDUH, has data that are specific and comprehensive enough on substance abuse and mental illness to be useful for purposes of evaluating the Affordable Care Act's impacts on this population. As shown in Table 4, however, sample sizes for the low-income population with substance abuse or dependence are limited (1,146 in 2010), as are sample sizes for mental illness and low income (1,113 in 2010).7

The surveys also vary substantially in their inclusion of questions that allow for identification of the outcomes of interest described earlier. Tables 5-11 illustrate which of the outcome indicators of interest are available from each of the population surveys included in our review. ACS and CPS are among the most commonly used sources of information on health insurance coverage (Table 5), particularly for state estimates. The measures of health insurance coverage differ between the two surveys--ACS is a point in time measure, and the CPS question is about full-year insurance status (although many analysts interpret the CPS measure to be more similar to a point in time measure8). Both ACS and CPS include only limited questions related to the other outcome indicators of interest. The Census Bureau has recently added questions about financial burden of health care to the CPS (Table 8) and is considering changes to the CPS health insurance questions to collect information about point in time coverage status and number of months of coverage by type during the year. Both ACS and CPS include information about participation in other government programs (Table 10) and many of the demographic and control variables of interest (Table 11). CPS also has a general health status question (Table 9).

CPS interviews the same individuals at multiple points in time: in 4 consecutive months, then 8 months without an interview, and again for 4 consecutive months. As a result, around half the sample in the CPS Annual Social and Economic Supplement (ASEC--the supplement to the CPS that collects health insurance information) each year was also interviewed in the previous year. This feature of the CPS ASEC presents both opportunities and challenges with regard to evaluating the Affordable Care Act's impacts. A potential advantage is the ability to follow individuals over a 2year period; however, the overlapping samples also complicate the analysis of changes over time. For state-level estimates, it is typically recommended to use 2year or 3year averages from CPS to ensure adequate sample size; for purposes of the evaluation, this means that a state-level analysis using CPS could not be available in as timely a way as an analysis from a survey with larger state sample (such as ACS).

Because they are specifically designed as in-depth health surveys, NHIS and MEPS contain a wealth of information on the indicators of interest that ACS and CPS do not; however, NHIS and MEPS have much more limited sample sizes and both allow access to state identifiers only through a Research Data Center. Each includes substantial detail and depth on health insurance coverage and type (Table 5), health care access and use (Table 6), and the financial burden of health care (Table 8). MEPS is more detailed than NHIS in terms of patient experience (Table 7) and health outcomes (Table 9), whereas NHIS has more detail than MEPS on participation in other government programs (Table 10). Both surveys include most of the demographic and control variables of interest (Table 11).

As a panel survey, MEPS has the advantage of potentially being used to understand the impacts of the Affordable Care Act on specific individuals over time; however, its usefulness for this purpose will be limited by its sample size for the income groups of interest (Table 4). SIPP is also a panel survey, and it has a substantially larger sample size for the income groups of interest for this project (Table 4). Whereas MEPS enrolls a new panel each year, the SIPP enrolls a new panel only once about every 4 years--the 2008 panel is currently scheduled to finish at the end of 2013. The Census Bureau is planning to change the data collection frequency for SIPP to an annual data collection, with the first interview of the 2014 panel collecting data from the beginning of 2013 through the interview month.9 Thus, the 2014 SIPP panel could potentially be used to track the impacts of the Affordable Care Act on Medicaid-eligible populations for the first few years after implementation. With the exception of the health insurance measures, however, other current SIPP content related to the outcomes of interest for this project is fairly limited--it includes some measures of health care access and use, as well as unmet need (Table 6) and some of the measures related to financial burden (Table 8).

A key strength of the BRFSS is the fact that it produces state-level estimates; however, its measure of health insurance coverage indicates only whether a person is insured at the time of the survey, with no specific information about the type of health insurance coverage.10 This is a significant weakness in terms of this project's goal of identifying the impacts of changes in Medicaid coverage. In addition, the income measure in the BRFSS is relatively imprecise, and BRFSS includes only a limited number of the outcomes of interest--some measures of access/use and unmet need (Table 6), as well as key health outcome measures (Table 9). Furthermore, the content of the core survey that is conducted by all states varies from year to year; other topics are included in optional modules that are fielded by a subset of states, and some states also add their own questions to the survey.11 Although some of the indicators of interest, such as health insurance status and unmet need because of cost, are collected as part of the core survey each year, other indicators such as mammograms and Pap smears are not (these are typically collected every other year in the core survey). Finally, although substantial efforts are made to ensure that the data are comparable across states, cross-state comparisons using the BRFSS may be complicated by the fact that there is variation in survey fielding across states.

NHANES is a very in-depth health survey, but its sample size is much more limited than any of the other national population surveys. A key strength of NHANES is its focus on clinical measures and its ability to identify undiagnosed and untreated conditions (Table 9). Relative to the other health surveys (NHIS and MEPS), however, it has much less depth in terms of outcomes of interest for evaluating the Affordable Care Act's impact on Medicaid-eligible populations--particularly health insurance coverage (Table 5), health care access/use (Table 6), and financial burden (Table 8).

Finally, as noted earlier, the NSDUH is the only survey with specific and comprehensive measures of substance abuse and mental illness. The NSDUH's questions about substance use and mental illness are based on specific diagnostic criteria for substance abuse or dependence and mental disorders, in contrast to the more general sets of questions on these issues in other population surveys. NSDUH also includes measures of health insurance coverage (Table 5), some measures of health care access and use, unmet need for substance abuse/mental health services (Table 6), some financial burden measures (Table 8), and some measures of health outcomes (Table 9), as well as information about participation in other public programs (Table 10) and demographic information (Table 11).

View full report


"ACAImpact.pdf" (pdf, 1.37Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®