Like all studies, this one has a number of limitations related to generalizability and reliability of results. First, we relied on data from three states. Although this is an improvement over previous studies that used data from only a single state, it still nevertheless cannot be generalized to all states or to the nation as a whole. The value of multiple states, however, is that it demonstrates the extent of variation across the country, and hopefully, can provide useful comparisons to other states. Second, the data are from the early-to mid-1990s (1994-95 for California and Florida and 1993-94 for Pennsylvania). These were the most recent data available through SMRF. Clearly, more recent data would be desirable to ascertain whether patterns of enrollment, diagnosed conditions, utilization, and expenditures have changed.
Third, the analyses of diagnosed conditions, expenditures, and utilization exclude children enrolled in managed care, since SMRF did not gather encounter data for capitated services (the one exception was for dental services provided through prepaid dental plans in California). Therefore, to the extent that there are systematic differences in the utilization patterns among children in foster care who are enrolled in managed care, these will not be captured in the analysis. Ideally, future studies would include encounter data from managed care organizations to allow a comparison of utilization patterns in managed care versus fee-for-service.
Fourth, the analyses rely on Medicaid claims data submitted by providers. The reliability of the CDPS classification hinges on the reliability of diagnostic coding by providers. Likewise, the analysis of Medicaid expenditures and utilization patterns relies on the accuracy of procedure codes. To the extent that there are errors in coding, our results will be less than precise.
We also encountered a number of limitations in using the SMRF data, which other researchers should be aware of. First, there is no indicator of provider specialty on the SMRF files, which precluded us from looking at continuity of care or specialty referral patterns. Second, not all states report basic data such as diagnoses, which precluded us from conducting analyses of diagnosed conditions in Florida.(2) To our knowledge, however, there is no central database that indicates which SMRF files contain which data elements and to what degree of completeness. Third, states often use state-specific procedure codes for such specialized services as case management, EPSDT, and mental health. We were fortunate to have access to state-specific procedure codes, but these are not uniformly available to researchers. Fourth, we found significant inconsistencies in the way states classified services by type of service. This was especially problematic for mental health services, where the three states each used different type-of-service categories. Again, we were fortunate to have access to state-specific procedure codes that enabled us to sort out these inconsistencies. Fifth, the SMRF file contains a single eligibility code for each month, which means that children with dual SSI and foster care eligibility during a given month would only be classified in a single category. This hampered our ability to identify foster care children with SSI coverage, and conversely, to identify SSI children in an out-of-home placement. Our solution was to identify children with any foster care eligibility during the year (our analytic group), and then screen for SSI eligibility during the full 24-month study period. This yielded relatively few children, but was the best we could do under the circumstances. A more desirable solution would be to obtain eligibility information from both child welfare offices and the Social Security Administration and match these records to Medicaid data to more reliably identify the populations of analytic interest. Sixth, it is unclear whether the date of initial foster care placement is reliably identified in Medicaid eligibility files. Anecdotal evidence suggests there are significant lags in obtaining Medicaid coverage for those who are not covered or in switching the reason for eligibility among those with prior Medicaid coverage (Rawlings-Sekunda 1999; Schneider and Fennel 1999). That might explain why we see such low levels of utilization immediately following foster care placement, but we cannot be sure.