The success of any effort to obtain an unbiased, representative, sample of facilities and their residents is dependent on comprehensive identification of all (or as close to all as possible) facilities in the universe of facilities of the type being studied. Because most sample studies attempt to make population estimates by weighting sample members by the reciprocal of their proportion of the universe, the extent to which the sample frame includes all facilities of the type being studied (and thereby includes their residents), determines the success of efforts to estimate the population. In addition, the extent to which exclusions or omissions from the sample frame tends to be disproportionately distributed across different subpopulations within the universe affects not only the population estimates, but also the proportional representation of certain groups which may be of interest. Of course, problems in acquiring the universe of all facilities from which an unbiased sample can be drawn are found to some extent in all sample surveys of long-term care settings and their residents to some extent. Given the range of different types of facilities (from foster care to large institutions), the different levels and types of agencies licensing and monitoring the different settings (from local to state), and the variability across states and among agencies within the same states, it is practically impossible to develop a sample frame containing absolutely all long-term care settings of interest. The challenge is to establish one which contains as much of the universe of programs of interest as is possible.
The NMES Sample Frame
The sample frame of Mental Retardation Facilities in the Institutional Population Component of the National Medical Expenditure Survey was the Inventory of Long-Term Care Places. Like all sample frames it has its limitations. The most notable of these were: (1) it did not include the full universe of facilities, and (2) it disproportionately excluded certain types of facilities and, thereby, certain subpopulations of residents. The limitations evident in the sample frame of the Institutional Population Component probably can be expected to have had two important effects on the outcomes of the study. First, because the sample frame appears not to have included large numbers of facilities and residents who were in the universe for which information was desired, the samples selected are not weighted so as to provide precise estimates of the population of all mental retardation facilities. Specifically, because sample members (facilities and residents) are weighted by the proportion of the sample frame they represent, the fact that the true universe of interest (all long-term care facilities) is considerably larger than the sample frame results In an underestimation of total mental retardation facilities and residents. Second, because the sample frame underidentifies specific types of facilities within the universe of interest, NMES contains underrepresentation of specific subpopulations of facilities and residents within its sample. It is relatively easy to identify the subpopulations of facilities and residents that are underrepresented. Unfortunately, it is difficult to say with confidence how adjustments might be made to correct for these limitations, although simple considerations of how this might be done are provided in Part 3 of this report. Underrepresentation of certain types of facilities has a direct effect on estimations of the size and characteristics of their resident populations, but there are indirect population estimation problems that may be just as significant. For example, children tend to reside in higher proportions in small facilities than do adults. Because small facilities are considerably less comprehensively included in the sample frame than were large facilities, children make up a disproportionately small part of the sample and of the estimated population than is the case in reality. In addition, the overall depiction of the residential status of children in mental retardation facilities is probably skewed toward larger, institutional settings.
Definition and Identification of Facilities
The 1986 Inventory of Long-Term Care Places, which was conducted specifically to provide a sample frame for NMES. Specific findings on mental retardation facilities from the ILTCP have been published by the National Center on Health Statistics (Sirrocco, 1989). For the purposes of establishing the sample frame for NMES, the ILTCP served to identify facilities primarily serving persons with mental retardation, verify eligibility as a "mental retardation facility", and to provide statistics on population and administrative characteristics of facilities on which the sample stratification and eventual weighting could be based.
For the purposes of this study the universe of all mental retardation facilities of interest was defined as: state licensed, contracted or operated living quarters which provided 24-hour, 7-days-a-week responsibility for room, board and supervision of mentally retarded persons. This definition excluded households providing services to relatives and residential service and support programs in which staff did not provide continuous supervision.
Construction of the registry. Prior to the actual "inventory" portion of the ILTCP, a list of facilities potentially meeting the definition of a mental retardation facility was constructed using the 1982 National Census of Residential Facilities for persons with mental retardation of the Center for Residential and Community Services, University of Minnesota. To that registry of 15,633 facilities were added facilities reported by states and "relevant associations" in the latter half of 1985, which did not appear on the CRCS registry. No known documentation is available on the number of facilities added to the original NCRF-based registry as part of this process, or how those facilities were distributed by type, size or state.
Surveying the registry (the ILTCP). The Inventory of Long-Term Care Places was a simultaneous survey of mental retardation facilities identified as described above and nursing and related care homes identified in a similar manner using the 1982 National Master Facility Inventory as the base list of nursing and related care facilities. To complete the ILTCP, the Bureau of the Census surveyed 56,728 total facilities using a 4-page questionnaire that was identical for all facilities, irrespective of the registry from which they were originally identified. Of these 56,728 facilities, statistics reported by staff of the National Center for Health Services Research (NCHSR) (Potter, Cohen & Mueller, 1987) indicate that 5,808 could not be surveyed because of insufficient address or telephone information, inability to locate or contact individual names, and the eventual dropping of individual nonrespondents. There were 174 direct refusals to participate. Another 5,500 places on the registry were not operating as residential facilities at the time of the survey, or residential services were not being provided at the specific address, for example, in the case of home offices for groups of residential facilities.
The ILTCP survey outcomes were used by NCHSR to evaluate all 56,728 facilities in the registry for their status as a mental retardation facility. This was done according to a set of hierarchical decision rules. The process eliminated from the sample frame facilities that were nursing or related care homes, duplicate addresses or otherwise out-of-scope. For example, these rules led to exclusion of 233 facilities not providing full-time supervision and another 434 for having no residents with mental retardation at the time of the Inventory.
Originally, the Institutional Population Component's sample frame was intended to include all types and sizes of mental retardation facility meeting the operational definition. However, during the sampling process, it became clear that the sample frame included substantially fewer small facilities than were identified in the 1982 National Census of Residential Facilities for persons with mental retardation of the University of Minnesota. As noted in a NCHSR staff paper on the NMES sample frame development (Potter, Cohen & Mueller, 1987).
A final comparison of the 1986 ILTCP MR universe to the 1982 NCRF universe (Hauber, et al. 1984) suggested undercoverage of one and two bed MR facilities by the ILTCP. A likely explanation is that the very small MR facilities are more likely to close or move than large facilities (Hauber, et al., 1984). This jeopardized completeness of the frame, so one and two bed MR's were deleted at the end of the eligibility determination process (p. 9)
A separate analyses of the ILTCP by NCHS (Sirrocco, 1987) noted procedural differences in the surveys that may have accentuated the difference noted above:
In creating the mailing list for the MR portion of the ILTCP, NCHS started with a file produced in 1982 by the University of Minnesota's Center for Residential and Community Services (CRCS). The 15,000 MR facilities on the file were matched against current state and local directories obtained by NCHS. Due to time constraints Imposed on the ILTCP, NCHS was unable to contact all local sources identified by CRCS in its study. It is believed that most of all places missed would be small MR facilities (fewer than 16 beds).
Comparison of NCHSR and NCHS identification of mental retardation facilities. Discrepancies existed between NCHSR and NCHS determinations of what constituted a mental retardation facility in the ILTCP. This must be expected when confronted with thousands of "generic" residential facilities operating across the country with more than one categorical disability evident among the people living there (i.e., people who are mentally retarded, elderly/disabled, and/or mentally ill). To exemplify the difficulty in determining facility types, NCHSR determined that the ILTCP included 17,265 mental retardation facilities, 1914 of which had 1 or 2 residents. NCHS on the other hand, determined that there were 14,639 mental retardation facilities, 1350 of which had 1 or 2 residents. NCHSR determined that the ILTCP contained 12,914 mental retardation facilities with 3-15 residents, NCHS identified 11,353. In short, even once physically located, the determination that a place is a mental retardation facility as opposed to some other type of residential setting is often not easily nor reliably accomplished.
Comparison of sample frame with state reports. The Center for Residential and Community Services at the University of Minnesota conducts annual surveys of state mental retardation/developmental disabilities agencies to obtain aggregated statistics on persons with mental retardation in residential facilities that are licensed, contracted or operated by states to provide residential services for persons with mental retardation and related conditions. The reports gathered for June 30, 1986 provide a useful point of comparison for the ILTCP sample frame, which was gathered in the first half of 1986. According to states on June 30, 1986, they had a total of 251,908 persons with mental retardation residing in 29,285 separate mental retardation "facilities". They identified 2,080 facilities of 16 or more residents (147,719 total residents with mental retardation) and 27,205 facilities of 15 or fewer residents (104,189 total residents with mental retardation). The NCHS analysis of the ILTCP indicated 1,936 facilities of 16 or more residents and 12,703 facilities with 15 or fewer residents.
Comparison of multiple sources. Table 1 briefly summarizes comparative statistics related to the completeness of the ILTCP coverage and NMES population estimates. Available analyses of the ILTCP (Sirrocco, 1987, 1989) have included only total residents (both with and without mental retardation). However, assuming that the proportion of mentally retarded to total residents in the ILTCP is similar to that found in the 1982 NCRF (which, as noted earlier, was the basis for the ILTCP registry of mental retardation facilities), the 14,639 facilities in the NCHS analyses with a total resident population of 250,472 would be estimated to house 217,164 individuals with mental retardation (the 1982 NCRF found 86.7% of the residents of mental retardation facilities were persons with mental retardation). Again, using data from the 1982 NCRF, the estimated number of mentally retarded residents in small facilities (15 or fewer residents) in the ILTCP would be 89.3% of the total 73,493 residents, or 65,627 residents with mental retardation. Using the same procedure, residents with mental retardation in large facilities would be estimated to be 85.8% of total residents of mental retardation facilities or 151,881 individuals. Table 1 contains four comparative statistics: (1) the findings of the 1982 NCRF; (2) the estimates of total number of residents of mental retardation facilities from the NCHS analysis of the 1986 ILTCP, with estimates of the proportion of total residents with mental retardation based on the findings of the 1982 NCRF: (3) state reports of total residents with mental retardation as of June 30, 1986; and (4) the population estimates from the 1987 National Medical Expenditure Survey.
Statistics presented in Table 1 show the National Medical Expenditure Survey to provide population estimates of persons with mental retardation in mental retardation facilities of 16 or more residents which appear close to what would be expected given other sources of information. But, with respect to smaller facilities substantial differences exist. The total number of persons estimated to be in small facilities in the NMES is very similar to the number obtained in the 1982 NCRF. But much is known to have changed in residential services between 1982 and 1987. In their annual reports to the Center for Residential and Community Services, states indicated that their small residential facilities housed 104,189 people with mental retardation on June 30, 1986. This represents a large increase from 1982, but one which is corroborated by the same state statistics showing a large decrease in residents of facilities with 16 or more residents. The statistics obtained in the state reports, the ILTCP, and even the NMES population estimates, all indicate large mental retardation facility populations of around 150,000 or about 30,000 less than 1982. Most of this population decrease took place because of people being moved to small facilities. Underidentification of persons in small facilities was further reflected in the differences between ILTCP and NMES estimates of the total population of persons with mental retardation in residential care (about 218,000) and the total number identified in the 1982 NCRF (244,000) and the 1986 state reports (252,000). Even including the 1 and 2 person facilities identified in the ILTCP, the estimated population of people with mental retardation and related conditions in mental retardation facilities in the NMES would have been only about 220,000-221,000 persons. This is about 25,000-30,000 fewer than the other available comparative statistics. (See the note at the foot of Table 1 for additional comments on this disparity.)
In summary then, it is clear that the ILTCP and as a result, the National Medical Expenditure Survey substantially undercounted persons with mental retardation. It also seems clear that this undercount is confined to facilities of 15 or fewer residents. The magnitude of the undercount appears to be over 30,000 small facility residents (including facilities with 1 or 2 residents), or in the neighborhood of one-third of all small facility residents.
Analyzing whether there are any particular subpopulations of small facilities and residents that were systematically undercounted in NMES could be accomplished by state-by-state analyses using state reports and state-by-state breakdowns of facilities on the ILTCP. It would also be possible to hypothesize about effects of the general undercounting and the associated elimination of all facilities with 2 or fewer residents from the sample frame. For example, specialized (mental retardation/developmental disabilities) foster care settings are underrepresented because they are more likely to have 1 or 2 residents. Therefore, children and youth are probably underrepresented because they are somewhat more likely to live in specialized foster homes. Still despite its limitations in the representation of small facility populations, it is important to stress that the NMES provides much useful data on both small and large facilities and their residents. In Part 2 of this report the first available sets of these data are presented and briefly discussed. In Part 3 some consideration is given to the possibility of ways to use the NMES data to adjust the population estimates to make them more reflective of the known universe of mental retardation facilities and people living in them.
TABLE 1. Comparison of the Population Estimates of the National Medical Expenditure Survey with Related Studies
|Study||MR Facilities||Residents w /MR||Total Residents|
|15- res.||16+ res.||Total||15- res.||16+ res.||Total||15- res.||16+ res.||Total|
|*NOTE: Since issuance of the data tapes used in these analyses, the staff of the Agency for Health Care Policy and Research (formerly the National Center for Health Services Research) has undertaken extensive efforts to identify and correct for facility duplication in the Inventory of Long-Term Care Places (ILTCP). A relative large amount of duplication was noted In the ILTCP, which served as the sample frame for NMES. Because these duplications Increased the probability of certain facilities and thereby certain residents being sampled, sample weights were recomputed to adjust for the increased probability of selection due to duplication of facilities in the sample frame. The resulting adjustments have reduced the estimated number of mental retardation facilities by 7.8% and the total number of people living in those facilities by about 12%. Presumably the resulting reduction in estimates of residents with mental retardation and related conditions is somewhat less. These reweightings increase further the disparity between estimates obtained in the NMES and those obtained from other sources. Statistics followed by an "(e)" are estimates based on the proportion of residents with mental retardation to total residents obtained In the 1982 NCRF.|
|1986 State Reports||27,205||2,080||29,285||104,189||147,719||251,906||116,782(e)||172,211(e)||288,993(e)|
|1987 NMES Est.*||11,054||2,276||13,330||64,936||153,697||218,633||69,481||170,137||239,619|