Using National Survey Data to Analyze Children’s Health Insurance Coverage: An Assessment of Issues. 3. Representativeness of Panel Samples


Panel samples lose respondents at each interview--a phenomenon described as attrition. For analyses that span the duration of a panel or focus on the later interviews, the sample available for analysis is affected by the cumulative attrition. Because it tends to be nonrandom, attrition affects the representativeness of a panel sample. There is evidence from the SIPP, for example, that poor people are more likely to leave than people at higher income levels. Major changes in life circumstances may also contribute to attrition. When this occurs, the changes that prompted the attrition may go unobserved, leaving no evidence to connect them to attrition. When differentials in attrition probabilities are observed, as with poverty, the sample weights of the respondents who remain can be adjusted to improve the overall representativeness of the sample. When factors that contribute to attrition are not observed, however, then compensating adjustments cannot be made, and the representativeness of the sample is reduced.

A second way in which the representativeness of panel samples changes over time is aging. The 9-year-olds in the first year of the panel become the 10-year-olds of year two and the 11-year-olds of year three. This aging of the sample carries a number of implications, but we consider two of them. Each year an entire birth cohort “ages out” of the population of children while a new birth cohort enters at the bottom. This raises issues about how to define the study population. Can an uninsured child simply age out of the population of uninsured children? Indeed, that is what happens in the real population, just as infants are born into uninsurance. With its aging, a panel sample merely replicates real life. Nevertheless, the analyst must determine whether analytical objectives may dictate retaining such children in the study population past the point where they would otherwise exit. For example, in measuring the duration of spells of uninsurance among older children, it may be desirable to follow these spells past the point where the child leaves childhood.

A second aspect of aging in a panel database is that chance differences in sample sizes by age group will be carried forward along with real differences, which can affect the measured impact of age-sensitive phenomena. Thus if the weighted number of children age 10 is 20 percent larger than the weighted number of children age 9, then one year later this difference will be manifested in the relative numbers of 11- and 10-year-olds. Two places where there seems to exist a potential for systematic bias are among infants and older teens. There is evidence that the SIPP underestimates births by as much as 25 percent, so that by the end of a three-year panel the number of children under age three is understated by that amount. Older teens seem to be underrepresented as well. Over time, the proportions of children who are estimated to be covered by Medicaid or to have no insurance at all are affected by these distortions in the age distribution, as the frequency of both Medicaid coverage and uninsurance vary by age. Adjusting the sample weights to match independent estimates of population size by single year of age can correct this problem, but it may alter the composition of the weighted sample on other dimensions.(34)

A third way in which the representativeness of panel samples may change over time is through their exclusion of new additions to the population. People who immigrate to the United States after the start of a SIPP panel are not represented in the panel sample. The same is true of people who leave institutions or the military and return to the household population. Over time, then, an ongoing SIPP panel represents a decreasing fraction of the universe that would be eligible for selection into a new sample. The magnitude of the decline in representativeness is not nearly as large as that due to the underrepresentation of births. We estimated that at the end of three years the 1992 SIPP panel underrepresented the number of children in the population by about 4 million (out of 74 million), but less than 2 million of this could be attributed to the exclusion of children who entered the population by a means other than birth (Czajka 1999).