Using National Survey Data to Analyze Children’s Health Insurance Coverage: An Assessment of Issues. 2. Measuring the Characteristics of a Population over Time


Adding a time dimension to data introduces complications along with the benefits. We provide two illustrations with respect to what might appear to be very simple problems: (1) estimating the number of children who were ever without insurance over the course of a year and (2) evaluating survey data on children who were ever enrolled in Medicaid over a year. Then we examine what longitudinal data may or may not tell us about causal sequences.

a. First Illustration: How Many Children Are Uninsured in a Year?

Typically, analysts define children as all people who are under a particular age. For example, children may be defined as all people under 19, or all people under 18. However we choose to define the population of children, when we add a time dimension by looking at behavior over the course of a year (or some other period), we must recognize that the set of people who are defined as children changes over this period. Over the course of a 12-month period, a new cohort of children is born while the oldest cohort “ages out” of the population of children--by turning 19, for example, if children are defined as people under 19. While this “cohort succession” may not affect appreciably the number who are defined as children at any one time, if there is no net growth, it does affect the number who would ever meet the definition of children over the year. In the United States currently, the size of a birth cohort is about 4 million. Over the course of a year, then, the number of people who were ever under 19 is about 4 million or more than 5 percent larger than the number who are under 19 at any one time during the year.

In counting the number of children who were ever uninsured in a year, we can elect to include all of those who were ever defined as children during that span or we can limit the count to those who were children at a specific point in time--say, the beginning of the year or the end of the year. No one way of defining a population over time should be regarded as the “correct” way. What is important to recognize is that there are different ways to define the population of children over time, and this awareness must carry over to how we count the uninsured and how we compare different measures of the prevalence of uninsurance among children.

Table 6 contrasts two different ways of counting the number of children who were ever uninsured during a year. The first approach, shown in the upper half of the table, tabulates the number of children ever uninsured in FY93 for a fixed population of children--specifically, those who were under 19 on September 30, 1993. This population numbers 70,868,000 children, and the number who were ever uninsured during the year is 15,360,000. The second approach, shown in the lower half of the table, tabulates the number of children ever uninsured in FY93 for a dynamic population of children. This population, consisting of those who were ever under 19 during FY93, numbers 74,691,000, and the number of these children who were ever uninsured is 16,089,000. The two alternative populations of children show essentially identical proportions of children who were


Description of Population and Estimate Estimate
Fixed Population  
Number of Children under 19 on September 30, 1993 70,868,000
Number Uninsured in September 1993 9,271,000
Proportion Uninsured in September 1993 13.1%
Number Ever Uninsured in FY93 15,360,000
Proportion Ever Uninsured in FY93 21.7%
Dynamic Population  
Number of Children Ever under 19 in Year Ending September 30, 1993 74,691,000
Number Uninsured in September 1993 9,271,000
Proportion Uninsured in September 1993 NA
Number Ever Uninsured in FY93 16,089,000
Proportion Ever Uninsured in FY93 21.5%
SOURCE: Survey of Income and Program Participation, 1992 Panel.  

ever uninsured during the year, but the dynamic population includes about 3.8 million more children than the fixed population and about 700,000 more children who were ever uninsured.(32) Either approach provides a correct use of the data. A comparison of the two sets of tabulations illustrates that when we examine the incidence of uninsurance among children over time, part of why we may observe more children to have been uninsured than at a point in time is that, depending on how we define children for this purpose, more children may have been exposed to the risk of being uninsured.

b. Second Illustration: Age Distribution of Medicaid Enrollees?

The Medicaid enrollment data that are reported by HCFA(now known as CMS) represent the number of individuals who were ever enrolled over the course of a year. There are, in fact, a number of issues that arise in trying to compare HCFA(now known as CMS) statistics with survey estimates of Medicaid enrollment, but one of them illustrates the complexity of dealing with the characteristics of a population over time. The issue is how we classify children by age.

HCFA(now known as CMS) includes among its published tabulations a table that presents enrollment by age, and in evaluating the completeness of reporting of Medicaid coverage in survey data, an analyst could use this table to obtain an administrative count of all people under a given age who were ever enrolled in Medicaid during the year. With the published data the analyst has limited flexibility in defining the upper-age boundary for children because HCFA(now known as CMS) reports ages in groups rather than single years. In particular, people 15 to 20 are reported in a single group. With the survey data the analyst has considerably more flexibility, of course. One approach that the analyst can take is to match the survey data to the published age categories in order to measure the completeness of reporting for those categories and then, if desired, extrapolate the findings to children at somewhat higher ages.

Over the course of a year every child experiences a birthday. For a year in which Medicaid enrollment is observed, therefore, each child for whom such coverage is reported can be assigned to either of two ages. Which is the more appropriate? More generally, is there a preferred strategy for assigning age to children who may have been enrolled in Medicaid at any time over the course of a year? What, exactly, does HCFA(now known as CMS) do?

It turns out that states employ at least two different conventions for reporting age to HCFA(now known as CMS). States that participate in the Medicaid Statistical Information System (MSIS) and submit electronic case records to HCFA(now known as CMS) assign age as of the end of the fiscal year (HCFA(now known as CMS) 1994). States that submit 2082 reports instead of MSIS electronic case records are instructed to assign age as of the middle of the fiscal year. This latter system classifies about 50 percent more children as infants and, generally, shifts the coverage of each reported age group by one-half year. Thus in the non-MSIS states the reported population under 15 will include children who turned 15 in the second half of the fiscal year.

Many states, both MSIS and non-MSIS, take a few months to process the enrollment of newborn infants. In these states, infants may not appear on state Medicaid files until their second or third month of life. As a result, infants who are born in the final months of the fiscal year may not be counted in the 2082 data as enrolled in Medicaid for that year (Lewis and Ellwood 1998). This produces a net undercount of infants--and therefore all enrollees. The fact that these infants show up in the next year’s enrollment statistics does not compensate for their omission, as they would have appeared in the next year’s statistics anyway.

The impact of these differences in the reporting of enrollment counts by age can be seen in Table 7, which presents the reported FY93 enrollment counts for children under age 1 and children 1 to 5. States are sorted by the ratio of the infant enrollment count to the count of children 1 to 5. This ratio, expressed as a percentage, appears in the final column. There is a clear break between the MSIS states and most of the non-MSIS states. No MSIS state has a ratio as high as 30 percent whereas most of the non-MSIS states (all but seven) lie above this value, between 31.7 and 43.4 percent. The average ratio for the MSIS states is about 21 percent while the average ratio for the non-MSIS states, excluding the seven, is about 38 percent. The difference between these mean values is consistent with the non-MSIS states counting about 50 percent more children as infants, but it also suggests that MSIS states may be more likely to undercount their infants. The seven non-MSIS states that fall into or below the range of ratios exhibited by the MSIS states may very well be assigning ages with the same convention as the MSIS states--that is, defining age as of the end of the fiscal year rather than the middle.


  Children Ever Enrolled Children Under Age 1 As a Percent of Children
State Age under 1 Ages 1 to 5 1 to 5
NEW MEXICO 10,613 69,356 15.3
MICHIGAN 46,357 293,558 15.8
1/ NEW HAMPSHIRE 3,301 18,238 18.1
1/ VERMONT 2,969 15,395 19.3
1/ NEW JERSEY 33,901 170,763 19.9
1/ MAINE 6,261 30,499 20.5
1/ DELAWARE 3,842 18,579 20.7
1/ GEORGIA 50,511 237,872 21.2
1/ NORTH DAKOTA 2,718 12,711 21.4
1/ INDIANA 32,258 150,761 21.4
1/ MONTANA 4,570 21,309 21.4
1/ PENNSYLVANIA 64,890 301,037 21.6
MINNESOTA 25,298 116,721 21.7
1/ CALIFORNIA 284,859 1,305,012 21.8
1/ MISSOURI 35,089 158,060 22.2
MASSACHUSETTS 33,018 145,515 22.7
1/ WYOMING 3,085 13,439 23.0
ILLINOIS 88,193 380,031 23.2
1/ KENTUCKY 29,345 124,180 23.6
1/ KANSAS 13,690 57,754 23.7
ARKANSAS 18,395 76,968 23.9
1/ IOWA 14,582 60,824 24.0
1/ WASHINGTON 36,052 148,723 24.2
1/ ALABAMA 37,138 150,949 24.6
1/ WISCONSIN 33,728 131,079 25.7
IDAHO 8,406 31,634 26.6
1/ NEVADA 7,647 28,679 26.7
1/ MISSISSIPPI 34,096 121,933 28.0
1/ HAWAII 7,560 26,324 28.7
1/ UTAH 13,833 47,492 29.1
TENNESSEE 58,205 183,875 31.7
NEBRASKA 11,459 35,999 31.8
FLORIDA 159,288 485,074 32.8
CONNECTICUT 20,827 62,491 33.3
OHIO 109,671 325,504 33.7
DISTRICT OF COLUMBIA 10,581 29,761 35.6
SOUTH DAKOTA 6,859 18,610 36.9
VIRGINIA 53,910 142,476 37.8
ALASKA 6,791 17,731 38.3
NEW YORK 213,538 551,463 38.7
LOUISIANA 62,083 158,470 39.2
OREGON 31,288 79,034 39.6
ARIZONA 64,111 160,246 40.0
NORTH CAROLINA 79,726 196,770 40.5
COLORADO 31,820 77,946 40.8
MARYLAND 46,848 114,684 40.8
TEXAS 271,434 656,465 41.3
OKLAHOMA 35,772 83,428 42.9
SOUTH CAROLINA 45,359 104,589 43.4
WEST VIRGINIA 35,026 80,648 43.4
SOURCE: HCFA(now known as CMS) (1994).
NOTES: "1/" designates a state that submits electronic case record data in lieu of 2082 tabulations. Rhode Island submitted no 2082 data for FY93 and is excluded.

The implications of these different conventions, as we said, are that in one set of states the reported enrollees under age 15 will include children who turned 15 in the second half of the year whereas in the other set of states the reported enrollees under age 15 will include no children who turned 15 during the fiscal year. If we wish to compare survey and administrative counts, therefore, we need to emulate these conventions in our survey tabulations or make some adjustment for the fact that the age categories used in the administrative records will line up with those in the survey data in about half the states but be somewhat more inclusive in the remaining states.

c. Causal Sequences

Longitudinal data with frequent observations allow us to examine not only changes in family circumstances but the sequence of changes. With that information it may be possible to infer causality. For example, we may observe that a child loses employer-sponsored coverage from one time period to the next. Why does that occur? Does a parent lose the job that provided coverage? Does the parent leave the household? Or does the parent remain in the job, suggesting that the parent or the employer simply dropped the coverage? In either case, does the parent lose coverage as well? These are the types of questions that longitudinal data may be able to answer.

In using longitudinal data to examine the sequence of events with an eye to inferring causality, analysts must be cognizant of the possibility of measurement error in the reporting timing of events. Changes in children’s family circumstances and, in particular, their health insurance coverage may not get reported exactly as they occur. They may be reported late or even early. This weakens the observed relationships between changes in economic circumstances and changes in children’s health insurance coverage, and analysts need to recognize this. With the SIPP, given the four-month frequency of interviews, this means that a change reported within a four-month reference period could very well have occurred anywhere within that period while changes reported at the boundary between reference periods almost certainly occurred earlier or later.(33)

Another aspect of the analysis of longitudinal data is that repeated measures of a characteristic over time may show inconsistencies or changes that are rapidly reversed. Some or even many of these inconsistencies may be due to reporting errors. With cross-sectional data, where there is but a single measure, such error is not evident. This does not mean that the error is not present. Rather, the same error may be present, but without the benefit of repeated measures we cannot detect it.