UNDERSTANDING ESTIMATES OF UNINSURED CHILDREN: PUTTING THE DIFFERENCES IN CONTEXT

Acknowledgments: ASPE would like to thank those reviewers at the Agency for Health Care Policy and Research (AHCPR), the National Center for Health Statistics (NCHS), and the Census Bureau whose helpful comments and expertise contributed greatly to this document.

Introduction

The number of uninsured children in the United States has been an important policy concern for several years, as reflected in initiatives such as the State Children's Health Insurance Program. The four Federal surveys that are major sources of data on uninsured children have played an important role in informing this policy debate. These surveys the National Health Interview Survey (NHIS), the March supplement to the Current Population Survey (CPS), the Survey of Income and Program Participation (SIPP), and the Medical Expenditure Panel Survey (MEPS) can each provide useful estimates of the number of uninsured children during a particular period of time, and in some cases, at a point in time.1 Both MEPS and SIPP also provide information on changes in the insurance status of individuals over time. This note will explain some of the major reasons why estimates of uninsured children from these surveys differ and explore the strengths and weaknesses of each survey.

Some Reasons for Differences Among Estimates

1. Survey Design Differences
   • The Length of Time Uninsurance is Measured. Uninsurance estimates from different surveys may vary because the surveys measure a lack of insurance over different lengths of time.
     • The CPS identifies individuals as uninsured if they have lacked coverage for the entire previous calendar year (although, as discussed below, many analysts believe that respondents provide information about their current insurance status). Based on this definition, 10.7 million or 15.0% of all children under age 18 were uninsured in 1997 and 10.6 million or 14.8% of all children were uninsured in 1996.
     • The SIPP can identify individuals who are uninsured for each month of a 36-month panel, a calender year, or the entire 36-month panel. Generally, the SIPP cross-sectional average monthly uninsured estimates are consistent with the CPS annual uninsured estimates. In contrast, the SIPP longitudinal data show annual uninsured estimates which are about half as large as the CPS uninsured estimates.
     • The NHIS identifies individuals as uninsured if they lacked coverage in the month prior to the survey. Because the month individuals are interviewed varies, the survey produces an average monthly uninsurance estimate. Thus, an NHIS estimate for a given year is for an "average" month during that year. Although both short-term and long-term uninsured would be included in this definition, the 1996 NHIS uninsured count (9.5 million children or 13.4% of all children under 18) is slightly lower than that of the 1996 CPS (10.6 million or 14.8%).
     • The current 1996 MEPS data are from the first round of a two-year panel survey and therefore will count as uninsured those without coverage for the entire interview round (an average of 3-5 months). Because of this time frame difference, MEPS estimates for each round can be expected to be somewhat higher than the 12-month estimate from the CPS (but many analysts believe that CPS does not provide an accurate 12-month uninsurance estimate). For example, the MEPS 1996 round one estimate (just under 11 million children under age 18) is somewhat higher than the 1996 CPS estimate (10.6 million children under 18).2 Once complete 1996 data are available, MEPS will also have the capacity to produce point-in-time, monthly, and annual estimates of health insurance coverage.
   • Point in time estimates. As discussed above, different surveys provide uninsurance estimates which cover different lengths of time. At times it may also be useful to know the number who are uninsured at a given point in time (e.g., the date of the survey interview). One would expect point-in-time estimates to be larger than estimates which count only those uninsured over an entire period of time (e.g., an entire year). 1996 MEPS data can provide point-in-time estimates (as of the Round 1 and Round 2 interview dates and as of December 31, 1996). The NHIS estimates can be considered close to being point-in-time as they indicate lack of coverage in an average month in that year. In addition, many analysts also believe that a number of respondents to the CPS provide point-in-time information, i.e., information about their status at the time they are participating in the survey (March) or about their status at the end of the previous calendar year, despite the fact that the survey questions ask about the entire previous calendar year. That CPS annual estimates are similar to point-in-time estimates generated from the SIPP and the NHIS has been cited as evidence of this occurrence. How many respondents misinterpret the relevant questions in this manner is unknown, but this uncertainty may make it more difficult to interpret and compare uninsurance data.
   • Recall periods. Each survey uses different respondent recall periods. The respondent is required to remember the previous 14 months for the CPS since the health supplement is conducted in March following the calendar year to which the questions pertain. Length of recall is two months for the NHIS, three to five months for MEPS, and four months for the SIPP. Recall about insurance status wanes over time, affecting both accuracy and comparability.
   • How "Insurance" is Defined. MEPS defines private insurance as coverage for hospital and physician services, thereby eliminating single service, serious and dread disease, workers' compensation, accident, and disability policies from counting as "coverage." NHIS excludes single service plans, except for those that cover hospital care, from the definition of private insurance.3 CPS and SIPP instruct interviewers to not count single service plans (such as dental plans) as private insurance, but some single service coverage may get misreported as comprehensive coverage. All four surveys count military and veterans health plans as health insurance. However, while CPS, SIPP, and NHIS include health services received directly from the Department of Veterans Affairs as coverage, MEPS does not.

   • How respondents are asked about insurance. In the NHIS, respondents are directly asked whether they lack insurance coverage. In the MEPS, SIPP, and CPS, however, respondents are not explicitly asked this question; those who deny any type of insurance coverage are assumed to be uninsured.

2. Differences in Data Handling.
   • Data adjustments. Analysts often adjust the raw survey data to reflect assumed under- or overreporting or to account for non-response. Since these adjustments can differ depending on the data analyst, estimates of the same population derived from the same survey can differ from one another. Analysts from different organizations tend to employ different decision algorithms for determining how to count various responses, each of which may be equally valid. For example, the uninsured estimates from CPS published by the Employee Benefit Research Institute have at times differed from those published by the Administration because of differences in the way the data are compiled.
   • Medicaid adjustments. The Census Bureau adjusts Medicaid coverage data on the CPS raw data file by assigning Medicaid coverage to individuals whose families receive Aid to Families with Dependent Children (AFDC) and some individuals who report receipt of Supplemental Security Income (SSI), even if they do not report Medicaid coverage. This imputation may affect the counts and characteristics of uninsured children obtained from the CPS. Nevertheless, CPS estimates of Medicaid enrollment have historically been lower than administrative data on Medicaid enrollment. NCHS uses a similar adjustment for estimates based on the NHIS.

   • The age-range used to define "children." By convention, the term "children" is often defined to include only those individuals under age 18. However, analysts sometimes find it appropriate to include all individuals under age 19 in the definition of "children." The latter definition is more appropriate for Medicaid estimates because it is consistent with the program's eligibility rules; many states have opted to expand Medicaid eligibility to children up to age 19 meeting certain income and age criteria. If individuals under 19 are included as "children," the CPS estimate increases from 10.7 million to 11.6 million uninsured in 1997.

3. Differences in Timeliness of Data. Because of different lag-times between data gathering and data availability, it is often the case that results cited at the same time actually reflect data gathered in different years. For example, the most recent data available from the NHIS were collected in 1996 and reflect 1996 coverage rates. The most recent data available from the CPS were collected in March of 1998 and reflect 1997 coverage rates. The most recent MEPS data on insurance coverage were collected in 1996 and reflect coverage rates for the first half of 1996.

Strengths and Weaknesses of Each Survey

• CPS Data. The CPS data are widely used because the CPS is based on a very large sample, is designed to produce credible state-level estimates (less populous states may require 2 or 3 years of CPS data to produce such estimates), is available on a timely basis, and provides information on coverage rates for socio-demographic subgroups of the population. However, as with most surveys, the CPS has been subject to questions regarding over- and under-reporting. Specifically, it is thought that the CPS over-counts the number of individuals who have been uninsured for an entire year, possibly because respondents answer based on current, rather than previous, coverage status. In addition, Medicaid coverage status is likely under-reported.
• MEPS Data. MEPS data on an individual's health insurance status is collected several times a year during a two-year panel survey and will be collected continuously. These data have the capacity to produce a variety of estimates of health insurance coverage, including point-in-time, monthly, and annual estimates. In addition, since these data will be collected over a two-year period, MEPS data will enable analysts to examine health insurance dynamics, including changes in coverage and spells without coverage. MEPS data will provide highly reliable estimates of the population's health insurance status which can be linked to a variety of individual and household characteristics, including use of and expenditures for health care services. Once complete 1996 MEPS data are available in 1999, analysts can examine insurance status in conjunction with data on sources of payment for health care to add greater accuracy and precision to the insurance status estimates. The MEPS sample is smaller than the CPS and NHIS samples, does not contain a representative sample from each state, and thus cannot be used to make state-level estimates.
• NHIS Data. NHIS data are gathered continuously, are highly reliable, and provide detailed information on insurance status, including type of coverage. The survey also provides information on several measures of health status, health care utilization, and socio-demographic characteristics of survey respondents. Relative to the CPS, there is a longer lag-time between data gathering and data availability, which may cause estimates from the two surveys to differ even when they are released simultaneously. NCHS is taking steps to address this situation and anticipates shorter turn-around times in the future with the implementation of CAPI (Computer Assisted Personal Interviewing). Like MEPS and SIPP, the NHIS does not contain a representative sample from each state and is not designed to make state-level estimates.

• SIPP Data. As a longitudinal survey, the SIPP data provide the capacity to examine the dynamics of health insurance. It is possible to estimate the duration of spells without health insurance. These data are also capable of producing health insurance estimates for various time periods, such as point-in-time, monthly, annual, or over the full panel. As part of the core data collected in the SIPP, health insurance data can also be linked to other sections of the survey, such as utilization of health care services, child well-being, and disability. The SIPP sample is smaller than the CPS and NHIS samples, does not contain a representative sample from each state, and thus can not be used to make state-level estimates.

Conclusion

The bottom line is that the estimated rate of uninsurance among children may vary depending upon the data source and data adjustments. The decision of which survey to use for uninsurance estimates may depend on the purpose of the analysis. The CPS is the only source of state-level uninsurance estimates. MEPS and SIPP are the best sources for examining changes in individuals' insurance status over time. NHIS, MEPS, and SIPP can provide point-in-time estimates of uninsurance rates. Despite the differences and the strengths and weaknesses that distinguish these surveys, the estimates derived from each paint a relatively consistent picture of health coverage rates in the United States. Critical policy concerns such as the disparity in coverage rates across income groups and the number of children that lack coverage are clearly apparent in the empirical data from all four surveys.

Notes

1. The NHIS is administered by the National Center for Health Statistics, which is part of the Department of Health and Human Services (HHS). The CPS and SIPP are administered by the Census Bureau. The MEPS is administered by the Agency for Health Care Policy and Research, also part of HHS.

2. According to MEPS 1996 round one data, 15.4% of all children under 18 are uninsured. The 95% confidence interval around this estimate (14.6%-16.2%) overlaps the CPS estimate of uninsured children in 1996 (14.8%), which means that there is no statistically significant difference between the two estimates.

3. In 1997, the National Center for Health Statistics (NCHS) made some minor changes in how the uninsured are defined based on the NHIS. These changes have tended to decrease the percent of uninsured slightly. Those with public assistance coverage or AFDC (but without a report of Medicaid) are now counted as insured. In addition, fewer persons are deleted from calculations due to missing data. In 1998, NCHS revised estimates to count those with only Indian Health Service coverage as uninsured. This change has had little effect on national estimates. NCHS has produced a revised series of estimates that appear in Health, United States, 1998

Authors:

• Ariel Winter, Office of Health Policy.
• M. Eugene Moyer, Office of Health Policy.

ASPE Research Notes is circulated periodically to the Department of Health and Human Services by the Office of the Assistant Secretary for Planning and Evaluation. For further information on health insurance estimates, contact the authors at 202-690-6870.