Using National Survey Data to Analyze Children’s Health Insurance Coverage: An Assessment of Issues. 5. Conclusion


The estimates of the incidence of uninsurance among children presented in Table 1 beg the question: Which estimate is the most correct? The short answer is that we do not know. There is no agreement in the research community. Clearly, the CPS estimate has been the most widely cited, but, as we explained, its timeliness and consistency account for this more than the presumption that it is the most accurate. When the estimate from the CTS was first announced, it was greeted with skepticism. Now that the NSAF, using similar survey methods, has produced a nearly identical estimate, the CTS’ credibility has been enhanced, and the CTS number, in turn, has paved the way for broader acceptance of the NSAF estimate. Yet neither survey has addressed what was felt to be the biggest source of overestimation of the uninsured in the federal surveys: namely, the apparent, substantial underreporting of Medicaid enrollment, which we discuss in Section C.(25)

Much attention has focused on the impact of the verification questions in the CTS and NSAF. Indeed, Urban Institute researchers have indicated that without the verification question the estimate of uninsured children in the NSAF would be as high as it is in the CPS. Yet analysis of CTS data has shown that the CTS estimate would have been 2 percentage points below the CPS estimate even without the verification question (Rosenbach and Lewis 1998). With the addition of a verification question to the NHIS in 1997 and an experimental application to the SIPP questions under way, we will soon know if the presumably better reporting of coverage elicited by a verification question really does account for the lower estimates obtained by the CTS and NSAF. Our suspicion is that the verification question will not account for all or even most of the difference, which leads us to consider the differences in survey methodology detailed above. From a survey design perspective, the selection of a sample based on telephones is a very different exercise from the drawing of a household sample from a list frame (CPS and SIPP) or an area frame (NHIS). Both the CTS and NSAF include nontelephone households in their samples as well as additional corrections for potential bias. Kenney et al. (1999) have documented that the NSAF matches the income distribution and other characteristics of the population as reported by the CPS, so there are no clear differences that we can point to as evidence that the CTS and NSAF samples include too few households with uninsured children. Both surveys also sample children within households rather than collecting data on all children, but there is no evidence as yet that the sampling or subsequent weighting of these children was biased. Nevertheless, it would seem that more detailed evaluation of the potential impact of sample design on the differences between the CTS and NSAF, on the one hand, and the federal surveys, on the other, is warranted--and, indeed, necessary if we are to understand the differences that we see in Table 1.