U.S. Department of Health and Human Services
The Evaluation of the National Long Term Care Demonstration: Final Report
Peter Kemper, Randall S. Brown, George J. Carcagno, Robert A. Applebaum, Jon B. Christianson, Walter Corson, Shari Miller Dunstan, Thomas Grannemann, Margaret Harrigan, Nancy Holden, Barbara Phillips, Jennifer Schore, Craig Thornton, Judith Wooldridge, and Felicity Skidmore
Mathematica Policy Research, Inc.
This report was prepared under contract #HHS-100-80-0157 between the U.S. Department of Health and Human Services (HHS), Office of Social Services Policy (now the Office of Disability, Aging and Long-Term Care Policy) and Mathematica Policy Research, Inc. For additional information about the study, you may visit the DALTCP home page at http://aspe.hhs.gov/daltcp/home.htm or contact the office at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, SW, Washington, DC 20201. The e-mail address is: webmaster.DALTCP@hhs.gov. The DALTCP Project Officer was Robert Clark.
In September 1980 the National Long Term Care Demonstration--known as channeling--was initiated by three units of the United States Department of Health and Human Services--the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Administration on Aging (AOA), and the Health Care Financing Administration (HCFA). It was to be a rigorous test of comprehensive case management of community care as a way to contain the rapidly increasing costs of long term care for the impaired elderly while providing adequate care to those in need.
A. THE INTERVENTION
Channeling was designed to use comprehensive case management to allocate community services appropriately to the frail elderly in need of long term care. The specific goal was to enable elderly persons, whenever appropriate, to stay in their own homes rather than entering nursing homes. Channeling financed direct community services, to a lesser or greater degree according to the channeling model, but always as part of a comprehensive plan for care in the community. It had no direct control over medical or nursing home expenditures.
Channeling was implemented to work through local channeling projects. The core of the intervention--i.e., case management--consisted of seven features:
Outreach to identify and attract potential clients who were at high risk of entering a long term care institution.
Standardized eligibility screening to determine whether an applicant met the following preestablished criteria: (a) Age--had to be 65 years or older. (b) Functional Disability--had to have two moderate disabilities in performing activities of daily living (ADL), or three severe impairments in ability to perform instrumental activities of daily living (IADL), or two severe IADL impairments and one severe ADL disability. Cognitive or behavioral difficulties affecting ability to perform ADL could count as one of the severe IADL impairments. (c) Unmet Needs: had to have an unmet need (expected to last for at least six months) for two or more services or an informal support system in danger of collapse. (d) Residence: had to be living in the community or (if institutionalized) certified as likely to be discharged within three months.
Comprehensive inperson assessment to identify individual client problems, resources, and service needs in preparation for developing a care plan.
Initial care planning to specify the types and amounts of care required to meet the identified needs of clients.
Service arrangement to implement the care plan through provision of both formal and informal in-home and community services.
Ongoing monitoring to assure that services were appropriately delivered and continued to meet client needs.
Periodic reassessment to adjust care plans to changing client needs.
Two models of channeling were tested. The basic case management model relied primarily on the core features. The channeling project assumed responsibility for helping clients gain access to needed services and for coordinating the services of multiple providers. This model provided a small amount of additional funding to purchase direct services to fill in gaps in existing programs. But it relied primarily on what was already available in each community, thus testing the premise that the major difficulties in the current system were problems of information and coordination which could be largely solved by client-centered case management.
The financial control model differed from the basic model in several ways:
It expanded service coverage to include a broad range of community services.
It established a funds pool to ensure that services could be allocated on the basis of need and appropriateness rather than on the eligibility requirements of specific categorical programs.
It empowered case managers to authorize the amount, duration, and scope of services paid out of the funds pool, making them accountable for the full package of community services.
It imposed two limits on expenditures from the funds pool. First, for the entire caseload average estimated expenditures under care plans could not exceed 60 percent of the average nursing home rate in the area. Second, for an individual client estimated care plan expenditures could not exceed 85 percent of that rate without special approval.
It required clients to share in the cost of services if their income exceeded 200 percent of the state's Supplemental Security Income (SSI) eligibility level plus the food stamp bonus amount.
B. THE DEMONSTRATION AND EVALUATION
In September of 1980, the participating states, a technical assistance contractor, and a national evaluation contractor were awarded contracts and began planning channeling. Among the criteria used for selection among states that competed to be part of channeling were demonstrated interest and commitment at the state level; capacity to perform the basic case management functions; whether channeling would represent a change from the existing system; and general quality of the proposal. A local project in each state was then selected. The host agencies of these projects were well established as departments within existing human service organizations (typically area agencies on aging or private nonprofit service providers).
Initial plans had called for four different models of channeling to be tested in 23 sites, selected through two procurements. Federal cutbacks reduced the models to two and the number of sites to 10 making it necessary to select from among 10 already chosen 5 that would implement the financial control model. Selection of financial control sites was based on perceived capacity of the projects to implement the more complex financial model, combined with judgments about the existence in the remaining sites of real differences between the basic model and the existing service system. Both considerations led to assigning the financial control model to the richer service environments.
The 10 sites participating in the demonstration and their model designations were:
|Basic Case Management||Financial Control Model|
|Baltimore, Maryland||Cleveland, Ohio|
|Eastern Kentucky||Greater Lynn, Massachusetts|
|Houston, Texas||Miami, Florida|
|Middlesex County, New Jersey||Philadelphia, Pennsylvania|
|Southern Maine||Rensselaer County, New York|
The 10 local projects opened their doors to clients between February and June of 1982, and were fully operational through June of 1984. The projects were phased out of the federal program in March of 1985, although most continued to operated under state or other agencies.
The goal of the evaluation, in addition to documenting the implementation of channeling, was to identify its effect on:
Use of formal health and long term care services, particularly hospital, nursing home, and community services.
Public and private expenditures for health services and long term care.
Individual outcomes including mortality, physical functioning, unmet service need, and social/psychological well-being.
Caregiving by family and friends, including the amount of care provided, the amount of financial support provided, and caregiver stress, satisfaction, and well-being.
To compare channeling's outcomes with what would have happened in the absence of channeling, the evaluation relied on an experimental design. Elderly persons referred to each channeling project were interviewed (most by telephone) to determine their eligibility for channeling. If found eligible, they were randomly assigned either to a treatment group whose members had the opportunity to participate in channeling or to a control group whose members did not receive demonstration services but continued to rely on whatever services were otherwise available in their community. Over the life of the demonstration (which included the period after the end of randomization for the research) 11,769 applicants were screened, 9,890 of whom were determined eligible. In all 6,341 persons were randomly assigned. Given the substantial death rate among this population as well as interview noncompletion, this yielded research samples of 3,372 to 6,326 elderly persons, depending on the analysis.
Several data sources were used. In addition to the telephone screening interviews, an extensive in-person survey was administered to the elderly members of the research sample (both treatment and control groups) at baseline and 6, 12, and (for half the sample) 18 months thereafter. Another survey was administered (usually by telephone) to the primary informal caregivers of a subset of the sample members at baseline, and 6 and 12 months thereafter. Service use and cost data were collected from Medicare, Medicaid, and channeling records, and from providers directly; participant tracking data and project cost records were collected from the channeling projects; official death records were obtained from state agencies. Finally, federal, state, local, and project staff were interviewed about the implementation and operation of the demonstration.
The basic methodology was to measure differences between treatment and control groups in the average levels of the variables for which effects were expected. Multiple regression was used to estimate the averages because it controls for different distributions of treatment and control groups across sites and to some extent for different patterns of attrition. It also takes account of variation due to factors other than channeling, thus yielding more precise estimates. In addition to analyzing the effects by model, we examine effects disaggregated by site and by subgroups of the sample. There were few instances of significant differences in effects across models, sites, or subgroups.
C. FINDINGS ON IMPLEMENTATION
Channeling's selection criteria did identify an extremely frail group. Consistent with the eligibility criteria, clients reported major limitations in functioning--with over 22 percent unable to perform any of five common activities of daily living (ADL) (eating, transfer, toileting, dressing, bathing), 53 percent incontinent, and 81 percent restricted in their mobility. There was also overwhelming dependence in instrumental activities of daily living (IADL), for example, meal preparation (88 percent), shopping (96 percent), housekeeping (97 percent), and a high number of unmet needs (averaging almost 4 out of 8 possible needs). Mental functioning was also limited; channeling sample members missed on average 3 to 4 items of a 10-item mental status questionnaire. Fifty-two percent reported incomes below $500 per month (which includes spouse income where applicable). Over one-third of the sample lived alone, although more than 90 percent reported receiving some informal care. Most (87 percent) had experienced a major stressful life event during the previous year. The average age of the channeling sample at baseline was 80 years. Nearly half reported a hospital admission in the 2-month period prior to channeling, and about three-fifths were already receiving in-home care.
The program elements were implemented largely as designed. Case managers successfully coordinated delivery of a broad range of services to those in the community. An in-person structured assessment, taking 75 minutes to complete, served the important clinical function of providing the basis for care planning as well as the research function of providing baseline data for the evaluation. Assessments were completed on all clients. Then a formalized care plan, which included both informal and formal services, was completed for each client and reviewed by a supervisor.
Case managers under the financial model reported being able to purchase services under the funds pool in all the specified service categories generally without constraint, although supply shortages limited the ordering of some services (e.g., homemaker services) in some sites. Case managers under the basic model, consistent with design, relied primarily on a brokering approach to arrange services. They reported having great freedom to use the gap filling funds as needed. These funds were (as intended) a small fraction of the funds available to case managers under the financial model.
The cost controls of the financial control model were also implemented according to plan. They did not prove to be as binding a constraint as had been expected. Estimated care plan costs in the five projects ranged from 30 to 47 percent of the cost of a nursing home in the site--well below the average expenditure cap of 60 percent. However, case managers did report that the requirement of calculating costs and comparing them to the limit, and the ability to trade off expenditures among clients did increase their cost-consciousness.
Because the incomes of the vast majority of clients fell below the cost-sharing level and because key services were exempt from cost-sharing, only about 5 percent of clients shared in the costs of their care. Even so, case managers under both models felt that cost-sharing contributions increased both client and family interest in the care and their willingness to notify the case managers in instances of inadequate care.
Ongoing case management, including regular monitoring and formalized reassessment and care plan adjustment, was implemented successfully. Telephone contacts to monitor changes in clients' situations occurred in a majority of cases very frequently, in-person visits less frequently. Reassessments and care plan revisions occurred at 6-month intervals for the majority of clients. The initial requirement that the first reassessment occur at 3 months was relaxed, partly because of high work loads but partly because case managers were in frequent contact with clients during that period in any case.
Although implementation across sites was remarkably uniform, implementation differed across models in ways which, though the differences were not large, could potentially influence the effects of the case management component. Total staff resources were approximately the same for the two models, but case managers under the basic model were able to spend a greater proportion of their time on direct client functions. This was probably due to the extra responsibilities under the financial model of ordering direct services and associated paperwork. Taken together the differences suggest that the basic model case managers may have provided more reassurance and personal support for clients and their informal caregivers than their counterparts under the financial model.
The technical evaluation design was implemented successfully. The large sample sizes made it unlikely that important channeling effects either went undetected or were seriously overestimated. The data collected on sample members provided measures of all the central outcomes of interest. In any evaluation, and particularly in one of channeling's scale and complexity, qualifications and uncertainties inevitably surround some of the results. Extensive methodological research, however, substantially reduces the risk that the basic conclusions of the channeling evaluation are subject to uncertainty due to sample attrition, estimation methodology, data noncomparability, or other technical matters.
The demonstration did not evaluate the effects of community care per se. Rather, it evaluated the effects of adding comprehensive case management and expanded community services to a system that already provided a substantial amount of community care. Not only was service-based case management already available in the channeling sites; a limited amount of case management like channeling's in its comprehensiveness was also available. Ten to twenty percent of the control group received such comprehensive case management, more in financial than in basic sites. Receipt of direct community services was substantial also; 60-69 percent of controls received in-home care visits in the week six months after randomization, with the proportion receiving and the number of visits received being substantially greater in financial sites.
D. CHANNELING'S EFFECTS ON SERVICE USE AND COST
Service use and cost results are summarized in Table 1.
Channeling increased formal community service use. Community service use increased, not because of the substitution of community for nursing home care, but because of increased use among those in the community. The bulk of these services was in-home care from visiting service providers. Personal care and homemaker services--reported by practitioners to be the most difficult types of services to obtain in sufficient quantity under the existing system--increased the most. Community service increases were modest under the basic case management model (about half a visit a week over a control group average of 2.2 visits). They were substantial under the financial control model (more than 2 visits a week over a control group average of 2.8 visits). This difference is consistent with the models' different capacities to pay for community services.
Neither model had a major effect on informal caregiving, although the financial control model led to small reductions in some areas. Most of the informal care received was from caregivers who lived with the sample member. The proportion of those caregivers giving care was not affected by either channeling model. The basic case management model did not affect the amount of care given by visiting family and friends either. Both treatment and control group members were receiving about three visits a week from visiting caregivers at the end of the first year. Under the financial control model treatment group members were receiving about two and a half visits a week at the end of the first year, compared to about three for control group members. Although this difference in visits was not statistically significant, the reduction in the proportion receiving such visits was significant under the financial control model. In particular, the proportion receiving help from friends and neighbors was significantly reduced under the financial model. The areas where small reductions were observed were the proportions receiving help with housework/laundry/ shopping, help with meal preparation, delivery of prepared meals, and transportation.
Despite success in targeting an extremely frail population, channeling did not identify a population at high risk of nursing home placement, and did not substantially reduce nursing home use. At 12 months, 13 percent of control group members in the basic sites and 14 percent in the financial sites were in a nursing home. This was much lower than expected, given the channeling eligibility criteria. Even by 18 months (not shown) only 19 percent of surviving control group members were in a nursing home. Nursing home use was lower among the treatment than the control group under both models at 12 months but the differences were small and not significant.
The channeling population was frequently hospitalized and made heavy use of physicians and other medical services. Channeling did not affect these types of service use. Use of hospitals was considerable--45 to 46 percent of the control group had a hospital admission during the first six months after enrollment. During the first year of channeling the control group in the basic sites spent 19.8 days in the hospital and 26.8 days in the financial sites. Hospital use by the treatment group was virtually the same--19.2 days and 25.6 days, respectively--and the differences were not significant. Other medical service use (not shown) was also high. In the basic sites, 71 percent of members of the control group visited a physician during months 7-12 and in the financial sites 81 percent did so. Use of outpatient, x-ray, and laboratory services among control group members was also high--60-65 percent per 6-month period in the basic sites, 73-77 percent per 6-month period in the financial sites. There was no evidence that channeling had an effect on physician or other medical service use.
The costs of expanding case management and community services were not offset by reductions in nursing home or other costs. Channeling increased the costs of case management and direct service use by design. Since it had little effect on nursing home use and none on hospital, physician, or other medical service use, the cost increases were not offset by cost decreases in other areas. In the basic sites, control group costs (including all service and room and board costs) averaged about $1,330 per month alive. Channeling resulted in a net increase in these costs of $83 (6 percent). In the financial sites, control group costs averaged about $1,592 per month alive. Channeling resulted in a net increase in these costs of $287 (18 percent). The cost burden was redistributed by channeling. Government costs increased by about 14 percent under the basic model, 28 percent under the financial model. Costs to clients and their families were reduced by about 7 percent under both models.
E. CHANNELING'S EFFECTS ON WELL-BEING, FUNCTIONING, AND MORTALITY
Channeling effects on well-being, functioning, and mortality are summarized in Table 2.
Channeling reduced unmet needs, increased clients' confidence in receipt of care, and increased their satisfaction with life. At the end of the first year the control group averaged one unmet need (out of a maximum of four). Both models of channeling reduced the number of unmet needs by 0.2 (equivalent to removing an unmet need for one out of five sample members). Both models of channeling increased the percentage expressing confidence that they would get needed care (increases of 8-9 percentage points over a control group average of just over 70 percent) and reported satisfaction with service arrangements. Both models also increased satisfaction with life generally, with the financial control model having the stronger effect (5.5 percentage points over a control group average of 56.3 percent). Channeling did not affect a number of other measures of quality of life for clients (including morale, social interactions, self-perceived health, and contentment).
Channeling increased informal caregivers' satisfaction with service arrangements and satisfaction with life. The majority of primary caregivers of the channeling client population expressed positive feelings about care arrangements and about their own life satisfaction. In the basic sites, for example, 76.8 percent of primary caregivers of control group members said they were confident about care arrangements; in the financial sites 71.8 percent said they were. Channeling increased these high percentages to 83.2 and 91.1 percent, respectively, with the financial control difference being statistically significant. With respect to primary caregivers' satisfaction with life, in the basic sites 75.3 percent of primary caregivers of control group members expressed satisfaction, in the financial sites 59.0 percent. Here again channeling increased these percentages (to 79.2 and 67.8 percent, respectively) with the financial control difference being statistically significant. Channeling did not affect perceived emotional, physical, and financial strain due to caregiving, employment, or limitations on employment or personal activities.
Channeling did not affect measures of client functioning, with the possible exception of physical functioning under the financial model. The basic model did not affect ADL or any other measure of functioning. The financial model did not affect the number of days restricted to bed or the ability to perform IADL. However, the treatment group reported performing fewer personal care (ADL) tasks than the control group (2.3 tasks out of 5 versus 2.5)--a small, but statistically significant difference. Significantly lower levels of functioning were also reported on some individual ADL items. This could reflect a real change in functioning. But it could also be an artifact of measurement; perhaps treatment group members reported doing less simply because of the high level of assistance provided. These possibilities cannot be disentangled with the available data.
The channeling population was at high risk of dying. Channeling did not affect mortality. At the end of the first year, 29.7 percent of the control group members in the basic sites had died, 27.4 percent in the financial sites. By the end of the demonstration these rates had risen to 39 percent and 33 percent, respectively (not shown). Channeling did not significantly affect mortality.
F. CONFIDENCE IN THE RESULTS OF CHANNELING AS FIELDED
Inevitable uncertainty surrounds some results in any evaluation of this kind. However, there is, in our judgment, little doubt about the basic conclusions concerning the channeling demonstration as fielded. Three pieces of evidence increase our confidence in the results.
First, the results were generally consistent across the sites in which each model was tested, making it unlikely that effects in one or two sites dominated the results, or that there were significant offsetting results in different sites.
Second, changes of any plausible magnitude in the channeling results would not alter the basic conclusions about costs. A rough comparison or the costs of community and institutional care illustrates the point. Because the channeling population's risk of institutionalization was so low, the trade-off between cost in a nursing home and cost in the community plus channeling services indicates that the basic model would have had to reduce average nursing home use to less than half actual control group use just to break even. The financial control model, given its larger increases in community care, could not have broken even at all, because the required reduction in nursing home use would have exceeded total control group use.
Third, the channeling results are consistent with those of other community care demonstrations, which generally found (with one important exception discussed below) relatively low risk of nursing home use among the populations served, and insufficient nursing home cost savings to offset the increased costs of expanded case management and community services.
The findings and conclusions reported here are, of course, for channeling as fielded in the ten demonstration sites in 1982-1984. Determining whether the results are generalizable to other interventions, populations, or environments is difficult for any demonstration, and channeling is no exception. Assessment of these issues to the extent possible will, however, assist users of the research in making judgments about its applicability to their particular situation.
The intervention. Success of the demonstration makes it clear that the channeling intervention itself could be successfully replicated in other settings as a permanent program. Indeed, the demonstration's documented experience in case management, provider relations, and cost controls is a useful guide for practice in replication of channeling or in other case management programs.
The demonstration tested two models of a particular approach to long term care--comprehensive case management combined (in the financial control model) with expanded community services and cost controls. Thus, the demonstration cannot speak to the effectiveness of case management within other approaches (such as a social/health maintenance organization, mandatory preadmission screening, or vouchers).
The population served. Channeling was tested with the particular population who applied to channeling. Because of the voluntary nature of application, the population may have been a selected subset of the eligible population who had more needs related to an acute care episode and were more likely to be connected with the existing community care system. The channeling population turned out to have relatively low risk of institutionalization despite state of the art screening criteria and assessment techniques. Since channeling was designed there has been no new research suggesting alternative screening instruments for community care populations that appear substantially better able to separate those who will go into nursing homes from those who will stay in the community.
The one evaluation that used a randomized design and came to a different conclusion about the substitution of community for institutional care is of special interest in this regard. The South Carolina Long Term Care Project served a slightly more disabled population with high nursing home use among the control group (48 percent of the controls were institutionalized after 12 months). The reduction in nursing home use was substantial (40 days during the first year after enrollment--a 31 percent reduction). The South Carolina project differed from channeling and most of the other community care demonstrations in that it was integrated with the state's nursing home preadmission screen from which it received all its clients. Whether because of this or some other reason, it was able to reduce nursing home use and break even on (but not reduce) costs.
Environment. Whether the demonstration sites were similar to the nation with respect to the difficulty of admission to a nursing home and the availability of community services is particularly important to interpreting the results. If nursing home beds were in shorter supply and community services more available in the demonstration sites than in the nation, channeling would have been less able to affect institutionalization rates than if it had been fielded elsewhere. Available evidence suggests that nursing home beds may have been somewhat less available than in the nation as a whole, but that severe shortages were probably not a major factor affecting channeling's outcomes for a majority of clients.
Data on the availability of community care are even more limited. Channeling sites were similar to the nation with respect to the proportion of states covering optional services under Medicaid and to home health expenditures under Medicare and Medicaid. No data on community care under other programs such as state home care programs are available. Given that the demonstration projects applied to participate in the demonstration through a competitive process, however, the case management and community care systems in the selected sites may have been more developed than in sites that did not apply or were not selected.
Conclusions. It is clear that channeling tested the effect of adding comprehensive case management and expanded community care to service systems that already provided such services to some of the frail elderly. It was not an evaluation of community care compared to its total absence. Its population, which voluntarily applied to the demonstration, was extremely frail and had unmet service needs but turned out to be not at high risk of nursing home placement. Substantial reductions in nursing home use were not possible given that only a relatively small portion of the population would have used nursing homes even without channeling.
The channeling evidence indicates that expansion of case management and community services beyond what already exists does not lead to overall cost savings. But it does yield benefits in the form of increased in-home care, reduced unmet need, and improved satisfaction with life for clients and the informal caregivers who bear most of the care burden. Whether these benefits are commensurate with its costs is a decision for society to make.
|TABLE 1: CHANNELING EFFECTS ON SERVICE USE AND COST DURING OR AT THE END OF THE FIRST YEAR|
|Formal In-Home Services (visits per week)|
|Basic Case Management Model||2.73||2.17||0.56*|
|Financial Control Model||4.93||2.75||2.18*|
|Informal Care (visits per week)|
|Basic Case Management Model||3.0||2.9||0.1|
|Financial Control Model||2.6||3.1||-0.5|
|Nursing Home Use (percent in nursing home)|
|Basic Case Management Model||11.6||13.0||-1.4|
|Financial Control Model||11.4||14.0||-2.6|
|Hospital Use (days per year)|
|Basic Case Management Model||19.2||19.8||-0.6|
|Financial Control Model||25.6||26.8||-1.2|
|Costs (dollars per month alive) a|
|Basic Case Management Model||1,413||1,330||83b|
|Financial Control Model||1,879||1,592||287b|
* Statistically significant at the 1 percent level.
|TABLE 2: CHANNELING EFFECTS ON WELL-BEING, FUNCTIONING, AND MORTALITY AT THE END OF THE FIRST YEAR|
|Unmet Needs (4 maximum)|
|Basic Case Management Model||0.8||1.0||-0.2**|
|Financial Control Model||0.8||1.0||-0.2**|
|Confidence About Receiving Care: Elderly (percent)|
|Basic Case Management Model||80.0||72.1||7.8**|
|Financial Control Model||80.0||71.0||9.0**|
|Satisfaction with Life: Elderly (percent)|
|Basic Case Management Model||65.0||62.8||2.2|
|Financial Control Model||61.8||56.3||5.5*|
|Satisfaction with Care Arrangements: Caregivers (percent)|
|Basic Case Management Model||83.2||76.8||6.4|
|Financial Control Model||91.1||71.8||19.3**|
|Satisfaction with Life: Caregivers (percent)|
|Basic Case Management Model||79.2||75.3||3.9|
|Financial Control Model||67.8||59.0||8.8*|
|Disabilities in ADL (5 maximum)|
|Basic Case Management Model||2.3||2.2||0.1|
|Financial Control Model||2.5||2.3||0.2**|
|Mortality Rate (percent after one year)|
|Basic Case Management Model||27.3||29.7||-2.4|
|Financial Control Model||27.5||27.4||0.1|
| * Statistically significant at the 5 percent level.
** Statistically significant at the 1 percent level.