The data on which this report is based are from the 1992 panel of the SIPP. The SIPP is a longitudinal survey whose respondents are interviewed every four months about their activity during the preceding four months. The questions include a lengthy series of “core” items, included in every interview, and periodic “topical modules” that collect data more infrequently on specialized areas. One quarter of the sample, constituting a “rotation group,” is interviewed in every month, so that the data for a given calendar month are based on a roughly equal distribution of respondents answering questions about activities one month ago, two months ago, three months ago, and four months ago. The staggered interviewing is intended to ensure that no calendar month of data is affected unduly by recall bias or other error associated with distance from the interview.
The Census Bureau collected nine waves of data--that is, nine interviews--from the entire 1992 SIPP panel sample and a tenth wave of data from three of the four rotation groups (that is, three- quarters of the sample). These data provide a common reference period covering three full calendar years--1992, 1993, and 1994--although, as we will explain, the Census Bureau is not releasing all of the data collected for the final three months of 1994.
1. Strengths and Limitations of SIPP
Several features of the SIPP make these data especially appealing for the analysis of children’s health insurance coverage. The SIPP provides a detailed measure of health insurance coverage for every month of the two to three year duration of a panel. Because of the SIPP rotation group design, the estimates for a given calendar month are based on a median two-and-a-half-month recall, with one quarter being only one month and one quarter being four months. Measures of the duration of new spells of particular types of coverage or lack of coverage can be constructed by aggregating the reports from successive interview waves, so that no matter how long the measured duration of a particular spell, no part of a reported spell relies on respondent recall beyond four months. In addition to providing measures of health insurance coverage, SIPP also provides very detailed measures of demographic and economic characteristics--again on a monthly basis. This affords us the opportunity to construct contemporaneous measures of circumstances that may affect eligibility for and enrollment in particular types of coverage.
Despite the strengths of the SIPP design, however, there are some notable limitations. Monthly reports of a number of characteristics--including Medicaid participation and uninsurance--show evidence of a pronounced “seam effect.” That is, monthly transitions (for example, in reported Medicaid coverage or health insurance coverage generally) are reported as occurring disproportionately between the four-month reference periods of interview waves rather than within these reference periods. Whereas we would expect only one quarter of such transitions to occur between reference periods, we find evidence that 75 to 90 percent of the transitions in certain statuses occur between interview waves--as if respondents were reporting their coverage (or interviewers recording them) in four-month chunks rather than month-by-month. As we demonstrate in another report [no citation yet], this has a profound impact on the reported distribution of spells of Medicaid coverage and uninsurance, and it may affect the point-in-time estimates as well (unless respondents are equally likely to “round up” as “round down” their reported months of coverage in a reference period). Like other surveys, SIPP shows evidence of underreporting of program participation and many sources of income, although there is evidence to suggest that SIPP does better in this regard than surveys with annual reference periods and less detailed measurement of these characteristics.
2. Representativeness of the SIPP Panel
It is important to recognize the implications of the SIPP’s longitudinal design for the representativeness of the information that it collects. The SIPP sample is selected from the resident population of the United States, excluding persons living in military barracks or institutions. The sample is designed to be representative of this population at the time that it is drawn, and the initial respondents are weighted to Census Bureau estimates of the size of this population by age, sex, race, and Hispanic origin. The SIPP sample is dynamic, however. Over the life of a SIPP panel some respondents leave the sample and others are added. As a result, the sample size and its representativeness change over time.
Attrition. Respondents who refuse to continue participating in the survey, move to an unknown address, or move more than 100 miles from a SIPP primary sampling unit and cannot be interviewed by telephone are lost from the panel. Because they continue to belong to the population that the SIPP panel was selected to represent, the sample weights of other sample members must be adjusted to compensate for their loss. The 1992 panel had a 9.3 percent nonresponse rate to the initial interview and a cumulative nonresponse rate of 26.2 percent through the ninth interview.
Exits from the Population. Persons who die, move outside the country, enter institutions, or move into military barracks leave the population as well as the sample. Because these losses affect the population as well as the sample, they are not treated as attrition. There is no adjustment to the weights of other panel members to compensate for their loss.
Additions to the Sample. Persons who move into the households of panel members (including those who are born to panel members) become sample members and remain so for as long as they continue to reside with original panel members. Likewise, persons into whose household an adult SIPP panel member moves become sample members as well--again, for as along as the panel member continues to reside with them.
Births clearly represent additions to the population as well as the sample. Other persons added to the sample after the initial interview may or may not represent additions to the population that the SIPP sample represents. Persons who move into or return to the country, leave institutions, or move out of military barracks constitute additions to the population. If they move into SIPP households they become SIPP sample members, and through their addition to the sample the SIPP can be said to represent all persons who joined the population and moved into households that were included in the initial population. Persons who move into the population but form their own households cannot join the SIPP sample. Strictly speaking, then, the SIPP sample does not represent these additions to the population over time. But the SIPP sample weights, as we shall explain, take account of these additions, and so they are represented in number if not actual sample members.
SIPP Weights. To enable inferences from the SIPP sample to the total population the Census Bureau constructs both cross-sectional and longitudinal weights. The cross-sectional weights are created for each calendar month. Weights for a given month are assigned to all persons for whom data were collected in that month, and they are constructed so that they sum to an estimate of the total population by age, sex, race, and Hispanic origin in that month. These weights account for sample attrition (see below) as well as net additions to the population.
The Census Bureau assigns longitudinal weights to all initial sample members who remain through the final interview or leave the survey universe, providing that they miss no more than one consecutive interview.(1) For the 1992 panel, these persons constituted about 74 percent of the initial sample (where the latter includes first wave nonrespondents). These longitudinal weights are adjusted to compensate for persons who attrited through nonresponse (or were never interviewed), and at the outset they sum to the Census Bureau’s estimate of the SIPP population in March 1992. Because of panel members who exit the population as well as the sample, the weighted sample total declines over time. At any point after the first interview, the longitudinally-weighted SIPP panel represents the survivors of the population that the panel represented fully at the start.
The Census Bureau does not assign longitudinal weights to children born after the first interview. These children cannot be weighted with the same scheme that is used for sample members who were actually present for the initial interview, and the Bureau has elected not to apply an alternative weighting scheme.(2) After the first year, then, the weighted longitudinal sample contains no infants. A year later it contains no children under age two, and a year after that it contains no children under age three. For many research purposes--including ours-- this is not acceptable. Therefore, we have followed what has become a commonly used practice of assigning newborns the weights of their mothers.(3)
Adjustments for Nonresponse. Both the cross-sectional calendar month weights and the longitudinal weights take into account characteristics of the panel members who attrited, but the limitations of this adjustment must be recognized. The nonresponse adjustments cannot fully account for the ways in which the attriters may differ from panel members who remain in the sample because some of these differences cannot be known. For example, some of the attrition may be influenced by important changes in circumstances--loss of employment, divorce, birth of a child-- that occurred after the attriter’s last interview. In addition, attriters may simply be different in ways that are not observed but which affect their behavior post-attrition.
3. Representation of Children Over Time
Table 1 presents comparative estimates of the population of children represented by the SIPP panel sample and the population of children represented by the individual calendar month samples. The latter estimates were obtained by summing the calendar month weights by single year of age for selected months. These population totals represent, approximately, the Census Bureau’s estimates of the population that would have been eligible for selection into the SIPP sample in each of the individual months. Estimates of the populations represented by the panel and cross-section samples are compared at four points in time: January 1992, October 1992, September 1993, and September 1994. January 1992 is the common reference month for the four rotation groups in the first wave of the 1992 panel. The next three months represent the beginning, middle, and end of the two-year period defined by FY93 and FY94, or the period on which our analysis is focused.
The first thing to note in this table is that the population of children to which the SIPP panel “weights up” in January 1992 actually exceeds the size of the population implied by the calendar month weights--by about 1.1 million children. We have no explanation for a difference of this magnitude. While the SIPP panel is weighted to estimates of the relevant population in March rather than January and, therefore, would not be expected to reproduce the January 1992 population totals, neither would we expect it to exceed the January 1992 population counts, much less by such a large margin. With this discrepancy in 1992, and the opposite trends in the two series, the two estimates of children under 19 cross between January and October, 1992.
From January 1992 through September 1994, the SIPP panel estimate of the population of children declines by about 2.2 million while the population implied by the calendar month weights rises by 2.9 million. The decline in the SIPP estimates can be attributed to the SIPP panel’s underrepresentation of births, which propagates through the younger ages. In January 1992 the SIPP panel sample represents an estimated 4.5 million infants. By October of that year the number of infants has dropped by nearly 1.4 million to 3.1 million. The number rises some by September 1993 but then drops by half a million by September 1994. Because the children born into the panel in 1992 become the panel’s one-year-olds in 1993 and two-year-olds in 1994, the effect of the underrepresentation of births is compounded. By September 1994 the SIPP panel represents 3.5 million fewer children under the age of three than it does in January 1992. The Census Bureau’s population estimates reflected in the calendar month weights indicate that the size of this population did decline over this period, but by only 150,000.
The net difference of 3.35 million between the two samples accounts for most of the 4.0 million children that would be eligible for SIPP in September 1994 but are not represented by the 1992 panel. The remaining .65 million is spread over the ages 3 through 18. The population estimates show this population growing by 3.1 million between January 1992 and September 1994 whereas the SIPP panel shows growth of 1.3 million. The population of children 3 through 18 grows in SIPP because the number of children who move into this age group from younger ages exceeds the number who “age out” at the upper end or leave the population through death, migration, or institutionalization. The SIPP panel estimates of children 3 to 5 grow by more than the Census Bureau’s population estimates because the SIPP panel overrepresents infants in January 1992. In the 6 to 10 age group, the SIPP panel declines slightly over time while the Census Bureau’s population estimates grow by nearly 650,000. In the 11 to 15 age group, the SIPP panel estimate increases by 376,000 while the population estimate rises by 1.1 million. Finally, in the 16 to 18 age group, the population represented by the SIPP panel grows by 156,000 compared to 797,000 for the cross-section sample..
That the differences between the growth trajectories of the SIPP panel and the total population (of SIPP-eligible children) are relatively small over most of the age range suggests that we can generalize from the SIPP panel to the full population fairly readily. It is only at the lower end of the age distribution that we need to be conscious of major differences between the sample and the population. Indeed, with the SIPP panel representing just over three-quarters of the estimated number of children under three in the population, we should be aware of the potential impact on the distribution of characteristics that differ substantially between very young children and older children.