ASPE ISSUE BRIEF

Understanding Estimates of the Uninsured:
Putting the Differences in Context

U.S. Department of Health and Human Services
Office of the Assistant Secretary for Planning and Evaluation

Updated September 2005

This Issue Brief is available on the Internet at:
http://aspe.hhs.gov/health/reports/05/uninsured-understanding-ib/index.htm

Printer Friendly (PDF) Version

Contents

Endnotes
Additional Information

Acknowledgments

ASPE would like to thank those reviewers at the U.S. Census Bureau, Agency for Healthcare Research and Quality (AHRQ), and the National Center for Health Statistics (NCHS), whose helpful comments and expertise contributed greatly to this document.

Introduction

The number of uninsured individuals in the United States has been an important policy concern for many years. The four federal surveys that are major sources of data on the uninsured have played an important role in informing policy on this issue. These surveys: the Current Population Survey (CPS), the Medical Expenditure Panel Survey (MEPS), the National Health Interview Survey (NHIS), and the Survey of Income and Program Participation (SIPP) can each provide useful estimates of the number of uninsured individuals during a particular period of time, and in some cases, at a point in time. Both MEPS and SIPP also provide information on changes in the insurance status of individuals over time.

Why are there so many surveys which measure health insurance? Actually, each of the surveys measures far more than insurance status. Surveys which measure insurance status actually address multiple topics and each has different strengths. The CPS, primarily a survey related to employment, has the best source of income data and is large enough to yield state data. The NHIS, the broadest survey of health, is useful in analysis of disease prevalence and health risks in the population. The MEPS provides the greatest depth regarding expenditures and utilization. The SIPP can measure monthly changes in income and employment status. Because health insurance is so important, it is collected on each of these surveys so that it can be related to other content that is unique to those surveys. What follows is a more complete description of each survey.

[ Go to Contents ]

General Survey Descriptions

CPS:  Conducted by the U.S. Census Bureau, the CPS is a monthly survey designed to collect data on the population’s employment status. The Annual Social and Economic Supplement to the CPS asks additional questions about health insurance coverage. The CPS data are widely used because the CPS has the largest household survey sample size, is designed to produce credible state-level estimates (less populous states may require 3 years of CPS data to produce such estimates), is available on a timely basis, and provides information on coverage rates for socio-demographic subgroups of the population. However, as with most surveys, the CPS has been subject to questions regarding over- and under-reporting. Specifically, it is thought that the CPS over-counts the number of individuals who have been uninsured for an entire year, possibly because respondents answer based on current, rather than previous, coverage status. In addition, Medicaid coverage status is likely under-reported.(1)

MEPS:  Administered by the Agency for Healthcare Research and Quality (AHRQ), MEPS is a two-year panel survey that collects information about health care use and expenditures, and the dynamics of health insurance and delivery systems. MEPS data on an individual’s health insurance status are collected several times a year and have the capacity to produce a variety of estimates of health insurance coverage, including point-in-time, monthly, and annual estimates. As a longitudinal survey, MEPS is able to examine health insurance dynamics, including changes in coverage and spells without coverage. MEPS data can be used to analyze the relationship between insurance status and a variety of individual and household characteristics, including use of and expenditures for health care services. Data on sources of payment for health care are collected and can be used to add greater accuracy and precision to the insurance status estimates. The MEPS sample is smaller than the CPS and NHIS samples and does not contain a representative sample from each state, and thus cannot be used to make state-level estimates.

NHIS:  Administered by the National Center for Health Statistics (NCHS), the NHIS is a continuous household interview survey of the civilian noninstitutionalized population. The NHIS collects detailed information on insurance status, type of coverage, measures of health status, health care utilization, and socio-demographic characteristics of members of sampled households. It provides estimates of the number of persons uninsured at a point-in-time, uninsured for a full-year, and ever uninsured during the year. The NHIS also collects data on illnesses, injuries, activity limitations, chronic conditions, health behaviors, and other health topics, which can be linked to characteristics such as insurance status. Starting in 2001, NCHS began publishing quarterly early reports that provide insurance data only six months after a given quarter, which is the most up-to-date data of any survey. Like MEPS and SIPP, the NHIS cannot provide reliable state estimates, except in the largest states.

SIPP:  Also administered by the Census Bureau, SIPP is a 36- to 48-month longitudinal panel survey that collects information on health insurance status, income, labor force activity and federal program participation. As a longitudinal survey, SIPP data provide the capacity to examine the dynamics of health insurance. It measures the duration of spells without health insurance and provides health insurance estimates for various time periods, such as point-in-time, monthly, annual, or over the full panel. As part of the core data collected in the SIPP, health insurance data can also be linked to other sections of the survey, such as utilization of health care services, child well-being, and disability. The SIPP sample is smaller than the CPS and NHIS samples and does not contain a representative sample from each state, and thus cannot be used to make state-level estimates.(2)

[ Go to Contents ]

The Estimates

The following table presents the uninsured estimates from the major surveys discussed above. As the table shows, there is significant variability in the number of uninsured from survey to survey.

Table 1.
Uninsured Estimates from Various National Surveys
Survey Most Recent Year Method of Estimate
Uninsured For Full Year Point in Time Uninsured Ever During the Year
Current Population Survey (CPS) 2004 45.8 million
15.7%
N/A N/A
Medical Expenditure Panel Survey (MEPS)* 2003 33.7 million
13.6%
48.1 million
19.0% (2004)
62.9 million
25.4%
Survey of Income and Program Participation (SIPP) 2001 18.9 million
6.8%
38.7 million**
14.0%
66.5 million
24.0%
National Health Interview Survey (NHIS) 2004 29.2 million
10.1%
42.1 million
14.6%
51.6 million
17.9%
Notes:
N/A = Survey does not capture this dimension.
MEPS point in time refers to the period from Jan. 1 through the interview date, which is 5 months on average.
* non-elderly
** Estimate for March 2001.

[ Go to Contents ]

Some Reasons for Differences Among Estimates

As is clear from the above table, the number of uninsured can vary widely from survey to survey. There are many variables that contribute to variance in survey estimates, such as general survey design and data collection methods. Below is a discussion of the major reasons that most likely explain the variance in uninsured numbers.(3)  (See Table 2 at the end of this paper for a summary of key survey differences)

1. Survey Design Differences

2. Differences in Timeliness of Data

Because of different lag times between data gathering and data availability, it is often the case that results cited at the same time actually reflect data gathered in different years. For example, the most recent data available from the CPS were collected in March of 2005 and reflect 2004 coverage rates, the most recent data available from the NHIS were collected in 2004 and reflect 2004 coverage rates, and the most recent MEPS data on full year insurance coverage were collected throughout 2003 and into 2004, reflecting 2003 coverage rates. MEPS data for point in time insurance coverage is collected in early 2004 and released in June of 2005. Therefore, when comparing estimates of the uninsured, it is important to note which year the survey covers.

3. Data Reporting

Surveys often make available public data files which contain the raw survey data, allowing organizations to publish their own uninsured estimates. Various researchers and organizations have adopted different methods to adjust the raw survey data to meet their specific needs. For example, some will adjust the CPS to account for Medicaid under-reporting while others will adjust income measures to better simulate eligibility for government programs. These researcher-specific adjustments can result in different estimates derived from the same survey, particularly for population subgroups, such as the number of uninsured below poverty or the number of uninsured children eligible for Medicaid. Therefore, the uninsured estimates from CPS published by the U.S. Census Bureau have at times differed from those published by other organizations because of differences in the way the data are compiled.(5)

Table 2.
Selected Differences Between Surveys: Uninsured Estimates
Survey Type of Estimate Produced Respondent Recall Period Sample Size Most Recent Data From Source of Data on Health Insurance Dynamics? Source of State Estimates?
CPS Full Year Uninsured Prior 13 to 15 months 210,648
(98,664 households)
2004 no yes
SIPP Full Year Uninsured Point in Time Ever Uninsured During the Year Prior 4 months 60,000
(26,000 households)
2001 yes no
NHIS Full Year Uninsured Point in Time Ever Uninsured During the Year Time of interview and prior 12 months 94,460
(36,579 households)
2004 no only for large states
MEPS Full Year Uninsured Point in Time Ever Uninsured During the Year Time of interview and prior 3-5 months 34,215
(12,860 households)
2003 yes no

[ Go to Contents ]

Conclusion

It is clear that the estimates of the uninsured may vary depending upon the data source and data adjustments. The decision of which survey to use may depend on the purpose of the analysis. For credible state-level estimates, the CPS is the only source for all 50 states. Larger sample sizes enable CPS and NHIS to produce more reliable estimates for subgroups of the population (i.e. children, low-income workers, etc.). MEPS and SIPP are the best sources for examining changes in individuals’ insurance status over time and NHIS, MEPS, and SIPP can provide point-in-time estimates of the uninsured.

Despite the differences that distinguish these surveys, the estimates derived from the surveys in total paint a relatively consistent picture of health coverage rates in the United States. While the overall number of uninsured varies from survey to survey, rising and falling trends in the estimates are consistent between the surveys. In addition, uninsured estimates among population subgroups also tend to be consistent. For example, all surveys show that the uninsured are concentrated among low-income individuals, and all surveys show that Hispanics have the highest uninsured rate among the racial and ethnic groups. Therefore, the surveys can still help policy makers tackle critical policy concerns related to health insurance coverage.

[ Go to Contents ]

Endnotes

1. Previous version of this memo contained a section called Medicaid Adjustments, which explained how some surveys, most notably the CPS, assign Medicaid coverage to individuals who also receive welfare assistance even if the survey respondents do not report Medicaid coverage. Historically, this assignment of Medicaid, called an imputation, was done because a significant percentage of respondents who indicated they had AFDC (the pre-welfare reform cash assistance program) failed to report Medicaid, even though people who received AFDC automatically received Medicaid by law. The Census Bureau continues to assign Medicaid to those who indicate they receive cash assistance through the new TANF program, though Medicaid and welfare eligibility have been delinked. Census continues to make this imputation because it appears States still provide Medicaid to the vast majority of TANF cash recipients. It was decided to drop this section from the memo because some surveys which do not assign Medicaid to welfare recipients find a higher number of Medicaid enrollees than the CPS. Therefore, it is no longer clear that the Medicaid imputation is a major factor in explaining why survey estimates vary. Nonetheless, Medicaid imputation is an important survey design difference that analysts should be aware of.

2. In 1996, SIPP started asking about coverage during the month of the interview, in addition to asking about coverage during the months prior to the interview. In 1994, it only asked about coverage during the months prior to the interview.

3. Also affecting estimates are factors such as the target population (e.g. U.S. civilian noninstitutionalized), the survey mode (face-to-face interview vs. telephone), survey response rates, and nonresponse rates to health insurance coverage questions, which are not discussed in this paper but should be noted.

4. Lewis, Kimball, Marilyn Ellwood and John Czajka. Counting the Uninsured: A Review of the Literature. Assessing the New Federalism: Occasional Paper Number 8. July 1998. Pg 15. http://newfederalism.urban.org/html/occ8.htm

5. CPS switched to a computer-assisted personal interview (CAPI) system in 1994. Also, in 1995, changes were made to the health section of the CPS March supplement such as reordering the questions and changing some questions about private insurance. In 1997, major changes were made to the NHIS, such as no longer imputing Medicaid to AFDC recipients, switching to a CAPI system and redesigning the survey to include more specific health insurance questions. Survey instrument adjustments need to be taken into account when looking at a historical time series because new survey designs can produce changes in estimates that may not reflect actual changes in the insurance coverage of the population.

Additional Information

Websites

Agency for Healthcare Quality and Research, Medical Expenditure Panel Survey:
http://www.meps.ahrq.gov

Center for Studying Health System Change, Community Tracking Study:
http://www.hschange.com/tracking.html

Economic Research Initiative on the Uninsured, University of Michigan:
http://www.umich.edu/eriu

National Center for Health Statistics, National Health Interview Survey:
http://www.cdc.gov/nchs/nhis.htm

Urban Institute, National Survey of American Families:
http://newfederalism.urban.org/nsaf/index.htm

U.S. Census Bureau. Health Insurance Data, Current Population Survey (CPS) and Survey of Income and Program Participation (SIPP):
http://www.census.gov/hhes/www/hlthins.html

U.S. Census Bureau, Survey of Income and Program Participation Main Page:
http://www.sipp.census.gov/sipp

U.S. Census Bureau, Current Population Survey Main Page:
http://www.bls.census.gov/cps/cpsmain.htm

Articles

Bennefield, Robert. A Comparative Analysis of Health Insurance Coverage Estimates: Data From CPS and SIPP. Presentation at the 1996 Joint Statistical Meetings of the American Statistical Association, 1996c.

Bennefield, Robert. Who Loses Coverage and for How Long? Current Population Reports, P70-54. Washington, DC: Census Bureau, May 1996b.

Bhandari, Shailesh, 2004, "People with Health Insurance: A Comparison of Estimates from two Surveys", Survey of Income and Program Participation Working Paper 243, U.S. Census Bureau. http://www.sipp.census.gov/sipp/workpapr/wp243.pdf

Center for Studying Health System Change and Mathematica Policy Research, Inc. Estimates of Health Insurance Coverage in the Community Tracking Study and the Current Population Survey. Technical Publication 16, November 1998. http://www.hschange.org/CONTENT/172/

Kenney, Genevieve et al. The National Survey of America’s Families: An Overview of the Health Policy Component. Inquiry. Fall 1999. Vol. 36, pg. 353-362.

Lewis, Kimball, Marilyn Ellwood and John Czajka. Counting the Uninsured: A Review of the Literature. Assessing the New Federalism: Occasional Paper Number 8. July 1998. Pg 15. http://newfederalism.urban.org/html/occ8.htm

Monheit, Alan. Underinsured Americans: A Review. Ann. Rev. Public Health. 1994. Vol. 15, pg. 461-85.

Rajan, Shruti, Stephen Zuckerman, and Niall Brennan. Verifying Insurance Coverage: Impact on Measuring the Uninsured with NSAF. Assessing the New Federalism, The Urban Institute. August 1999.

Short, Pamela. Counting and Characterizing the Uninsured, Economic Research Initiative on the Uninsured, ERIU Working Paper 2, Dec. 4, 2001. http://www.umich.edu/~eriu/pdf/wp2.pdf


Where to?

Top of Page | Contents

Home Pages:
Office of Health Policy (HP)
Assistant Secretary for Planning and Evaluation ASPE)
U.S. Department of Health and Human Services (HHS)

Last revised: 10/19/05