Evaluation of the District of Columbia's Demonstration Program, "Managed Care System for Disabled and Special Needs Children": Final Report Summary

Abt Associates Inc.

June 2000

---

This report was prepared under contract #500-96-0003 between the U.S. Department of Health and Human Services (HHS), Health Care Financing Administration (HCFA) and Abt Associates Inc. In addition to HCFA, other support for the study, Evaluation of the District of Columbia's 1115 Waiver for Children with Special Health Care Needs, has been provided by HHS's Office of the Assistant Secretary for Planning and Evaluation, Office of Disability, Aging and Long-Term Care Policy. For additional information about the study, you may visit the DALTCP home page at http://aspe.hhs.gov/daltcp/home.htm or contact the ASPE Project Officer, Gavin Kennedy, at HHS/ASPE/DALTCP, Room 424E, H.H Humphrey Building, 200 Independence Avenue, SW, Washington, DC 20201. His e-mail address is: Gavin.Kennedy@osaspe.dhhs.gov.

---

 

BACKGROUND: The Demonstration was a unique and ambitious experiment launched in a difficult environment.

After over two years of preparation, the Demonstration began providing services in February 1996.

In February 1994, the Hospital for Sick Children (HSC) in the District of Columbia (DC) incorporated Health Services for Children with Special Needs, Inc. (HSCSN). HSCSN proposed to demonstrate the effectiveness of a managed care organization that would (1) operate under a fixed capitation payment system, (2) serve only children with special health care needs who are eligible for Supplemental Security Income (SSI), and (3) integrate and manage care actively across the full spectrum of services, in a way that would be sensitive to the particular needs of each child. In March 1994, HSCSN submitted a formal application for a waiver to the Health Care Financing Administration (HCFA). The waiver was approved in October 1995. HSCSN began providing services to participants in February 1996. After two extensions, the Demonstration is scheduled to end in November 2000. DC Medicaid is in discussion with HCFA over the terms of a new waiver.

The Demonstration enrolls SSI-eligible children, a relatively small part of the population of children with special health care needs.

Children who receive assistance under SSI are a minority of children with special health care needs, defined by Newacheck and his colleagues (1998) as "those who have or are at risk for a chronic physical, developmental, behavioral, or emotional condition and who also require health and related services of a type or amount beyond that required by children generally."¹ Based on this definition, and using data from the National Health Interview Survey, Newacheck estimates that 12.6 million children, 18 percent of all children younger than 18 years, suffered from a chronic disabling condition in 1994. In 1997, the Social Security Administration reported that SSI provided assistance to nearly 900,00 children.

This Demonstration broke new ground in the Medicaid program. In expanding state Medicaid managed care programs, few attempts have been made to address the concerns of children with special needs.

Over the past several years, state Medicaid programs have encouraged or mandated the enrollment of Medicaid recipients in managed care organizations. Although children who receive SSI are generally eligible for Medicaid, most states enroll special needs children in mainline managed care organizations and carve out at least some benefits, particularly behavioral health services. Care management designed to meet the needs of these children is not always provided.

Michigan is an exception to this rule, and is most similar to the DC Medicaid experiment in the sense that it is a separate, capitated, voluntary program for special needs children. Fee-forservice alternatives to the Medicaid managed care model include Minnesota and Florida.

The Demonstration was planned and implemented in a difficult environment.

HSCSN faced several challenges in moving forward with the Demonstration.

• DC Medicaid experienced severe program cost problems in the early 1990s, and during waiver negotiations the District's financial problems deteriorated further.

• Available data suggested that the target population was poorly served by the District's health care system, so that children enrolling were expected to enter with many unmet needs for services.

• Key participants in discussions of the waiver, including the HSC, DC Medicaid, and HCFA's Office of Research and Demonstrations, had not previously worked together.

• Negotiations took 19 months from waiver submission to waiver approval, requiring HSCSN to furlough some staff and take other measures to keep the initiative alive.

In addition, opponents questioned both the need for the Demonstration and many of its key features. Project opponents organized themselves as "Responsible Parents Against DC Medicaid," a group that included day treatment centers and other organizations that serve the target population. This group challenged (1) awarding only one contract, (2) the proposal for mandatory enrollment, (3) lack of community involvement in planning the Demonstration, and (4) the need for such a Demonstration. In the end, the waiver was approved as a three-year, non-competitive, voluntary program.

Evaluating the DC Demonstration is complicated by the design of the program and by unreliable data.

HCFA's evaluation contractor has faced several challenges in evaluating the Demonstration. First, when implemented, the Demonstration was a unique program in a unique urban environment. The option of comparing processes or outcomes to similar models in other urban areas has not been available. Second, by the time the evaluation was ready to collect data, well over 1,500 children had enrolled. Thus it was impossible to collect "baseline" information that could have shown how health status, utilization and other outcomes changed for participants from before to after enrollment. Third, participation in the Demonstration is voluntary. This means that, through selfselection, children who enroll are likely to be different from children who do not enroll. Bias due to self-selection makes it more difficult to use outcomes compared between participants and non-participants to estimate Demonstration impacts, since some of the differences between groups may be due to factors other than the Demonstration. Fourth, due to problems of duplicate billing, it is impossible to use claims and eligibility data maintained by DC Medicaid and by HSCSN to estimate impacts of the Demonstration on expenditures. Faced with these challenges, the evaluator used many complementary analyses based on different data sources to address key research questions.

 

DEMONSTRATION MANAGEMENT: HSCSN experienced and attempted to address several management challenges to the Demonstration.

Initially, HSCSN treated the Demonstration as part of a larger commercial venture.

Rather than use "off-the-shelf" software to monitor care management, HSCSN elected to work with a software developer to create a customized package. HSCSN invested heavily in this venture. By the Fall of 1997, software development had cost $2 million of an estimated $5 million in startup costs. Early in the Demonstration, HSCSN set up a for-profit subsidiary to market the managed care capabilities developed under the contract. According to the current HSCSN management, HSCSN had engaged in this venture to make it a cutting-edge enterprise that could market its capabilities far beyond DC.

Mid-way through the Demonstration, the organization underwent a fundamental restructuring, accompanied by revisions to HSCSN's goals and objectives.

Originally, HSCSN operated as a separate entity, with a for-profit entity for marketing capabilities developed under the Demonstration. After the reorganization, several things changed:

• There was a major change in senior management at HSC and HSCSN. Although several Vice Presidents in functional areas remained in their positions, the CEOs of both organizations and the COO of HSCSN were replaced.

• New management wanted to change the culture at HSC/HSCSN, from a philosophy of social service orientation to one of resource-conserving cost consciousness. Management dissolved the for-profit subsidiary, an implicit admission that the marketing goals of HSCSN were to be abandoned.

• New management also moved to integrate administrative support services and certain expensive non-medical services, notably transportation.

Throughout the Demonstration, HSCSN experienced problems coordinating with other DC agencies and providers.

From the planning stage of this Demonstration, HSCSN faced administrative barriers and outright opposition to collaboration from other agencies and stakeholders.

• The DC public school system. As a Medicaid provider, the DC public school system has extensive responsibility for the health care of children with special needs. HSCSN officials report that, from the beginning, the school system has resisted all efforts at collaboration. By 1999, no formal agreement on collaboration had been reached, though school personnel were said to be somewhat more responsive.

• Residential treatment facilities. Coordination with residential treatment facilities (RTFs) is difficult because many agencies (schools, courts, other DC government agencies) can refer children to RTFs and because there are no RTFs that operate in the District. To improve coordination among all agencies, DC Medicaid organized a Residential Placement Unit Committee. Views on how successful this Committee has been vary, but its formation is seen as recognition that a problem exists. In addition, HSCSN assigned a care manger to monitor care of Demonstration participants placed in RTFs.

• Foster care. Adding children in foster care to the Demonstration should lower HSCSN's risk exposure, by increasing the size and reducing the average acuity of the target population. However, issues of accountability to many other agencies and geographic dispersion frustrate attempts to include these children in the program. The foster care program is managed by the Commission on Social Services, and many agencies, in particular the court system, are involved in foster care placement and monitoring. Most DC children in foster care are placed outside the District, in Maryland and Virginia. So far, attempts to include foster children in the re-competition of the waiver have failed.

• Public health clinics. Although initially interested in the Demonstration, public health clinics withdrew because they objected to credentialing requirements and to the need to participate in developing and maintaining plans of treatment.

• DC hospitals. Although relations with DC General and Children's Hospital have been strained, there have been signs of improvement in response to proactive efforts from DC Medicaid and HSCSN.

• The community. HSCSN has engaged the community, by creating a Community Advisory Board, expanding the provider network to include advocacy groups that offer services, and recruiting outreach workers from the community. Over time, community opposition to the Demonstration has gradually abated.

It is difficult to mesh any managed care program with all of the different systems of care and coverage outside its scope. What is notable in this case is how difficult progress has proved to be to date. From the current vantage point, integration of a special-needs HMO with the broader service system appears to be the work of many years, not a task readily accomplished during a brief demonstration.

HSCSN experienced financial losses during most years of the Demonstration.

Start-up costs of the Demonstration were nearly $5 million, much higher than initially predicted. As mentioned, about $2 million was spent to develop customized care management software, substantially more than the estimated $0.1 million off-the-shelf software. Also, the costs of finding and enrolling eligible children proved to be higher than anticipated.

Even excluding overruns in start-up costs, HSCSN experienced substantial losses throughout the Demonstration, cushioned in part by a risk-sharing arrangement with DC Medicaid. DC Medicaid agreed to pay HSCSN a flat capitated rate for each enrolled child, set at $997.71 in 1996 and trended forward at an annual inflation rate of 4.6 percent. DC Medicaid also shared risks with HSCSN, to guard against excessive gains or losses relative to a target ratio of costs to revenues (the "medical claims ratio") of 85 percent. HSCSN paid Medicaid a small amount in the first two program years but, in the face of losses of $0.2 million and (expected) $1.1 million in 1998 and 1999, DC Medicaid made payments that management expected would reduce losses by about 50 percent.

HSCSN's losses increased from 1998 to 1999, in part because the costs of several important service categories grew substantially. Inpatient hospital expenditures increased by 16 percent, driven largely by utilization patterns of a few particularly expensive cases. Expenditure increases of 20 percent in day treatment and 13 percent in home health in part reflect efforts by HSCSN to substitute home and communitybased care for institutional forms of care. Drug expenditure increases (17 percent) were driven largely by price inflation.

HSCSN management concludes that the model tested in this Demonstration is not viable.

Reflecting on its experience working within the system in DC and on the deteriorating financial conditions of the Demonstration, HSCSN argues that the current model would not be viable in the long run, for several reasons:

• At roughly 2,000 participants, enrollment levels are too low for full capitation without substantial risk sharing. With so few participants, a handful of very expensive cases can produce major financial losses.

• The risk sharing arrangement in place during the Demonstration can moderate potentially damaging losses, but it is not a substitute for a more explicit stop-loss arrangement.

• The "any willing provider" model adopted for the Demonstration produces a large network that is expensive for HSCSN to maintain, and it complicates efforts to engage participating providers in the goals and objectives of the project.

• Because eligible children are difficult to identify and contact, the enrollment process is expensive and should be separately reimbursed.

• Poor coordination of the Demonstration with other agencies reduces the efficiency and comprehensiveness of care management for participating children.

A proposed new waiver addresses some of HSCSN's concerns.

The original waiver for this Demonstration, to be re-competed as the Child and Adolescent SSI Program (CASSIP), was scheduled to end in November 1998. It has been extended twice, and is now expected to end in November 2000. HSCSN has announced that it will not participate in the competition for CASSIP. Currently, the model proposed for CASSIP would have the following features:

• mandatory enrollment of all SSI-eligible children, replacing voluntary enrollment in the current program;

• multiple contracts, awarded through open competition, in which eligible bidders are licensed HMOs, replacing the sole contract held by an entity (HSCSN) that is not a licensed HMO;

• an "adequate" provider network, replacing "any willing provider" provisions in the current program;

• a single capitation rate plus risk sharing between contractors and DC Medicaid, with stop loss protection against the risks of outlier inpatient, drug and home health costs, replacing the single rate and risk-sharing arrangement under the current program.

 

DEMONSTRATION ENROLLMENT: HSCSN enrolled a substantial proportion of eligible children. Children who enrolled were quite different from those who did not enroll.

At a given capitation rate, the volume and composition of enrollment contributes substantially to the financial viability, or non-viability, of a health plan. Initially, HSCSN designed the Demonstration on the assumption that enrollment of the target population would be mandatory, ensuring that the program would serve both the largest and most balanced risk pool. In fact, as negotiations with community groups unfolded, mandated enrollment was dropped in favor of a wholly voluntary program. As a consequence of this decision, HSCSN faced unanticipated enrollment challenges that contributed both to its financial difficulties and to HSCSN management's conclusion that the Demonstration model is not viable.

HSCSN's proactive enrollment process succeeded in enrolling eligible children early in the Demonstration, but at a substantial cost.

Over a four-year period, HSCSN enrolled nearly half of the eligible children in DC. Because lists of eligible children provided by DC Medicaid were often incomplete and inaccurate, HSCSN experienced serious difficulties (and unforeseen costs) in identifying and contacting potential participants throughout the Demonstration. Initially, HSCSN relied heavily on mailings to families of potentially eligible children, followed by direct marketing efforts by outreach workers to families that did not respond positively to the mailings within 45 days (later reduced to ten days). Mailings were eliminated in 1998, following management changes at HSCSN.

After an initial period of accelerated growth from start-up in February 1996 into the Fall of 1996, enrollment grew at a slower rate. The largest subsequent increases in enrollment were associated with two large HSCSN mailings, in January 1997 and August 1997. Whether or not additional mailings could have revived what was clearly a declining growth rate is unclear. What does seem clear is that proactive outreach paid off in enrollment growth, although there was no evidence that HSCSN ever assessed the costs and benefits of intensive outreach -- management simply perceived that enrollment costs were excessive.

Disenrollments from the program averaged 8 percent of total enrollments. From start-up in 1996 through mid-1999, 583 children disenrolled from the Demonstration. There is no evidence that disenrollment was associated with serious dissatisfaction. Over half had either moved out of the District or "aged out" of the Demonstration. Seventy participants lost SSI eligibility. For 98 children, caregivers had "changed their minds" about participation.

The Demonstration is voluntary. Families and children who enroll are different from those who do not enroll in several ways. This has affected HSCSN performance and financial viability, and has also complicated the task of assessing impacts of the Demonstration.

As noted, the Demonstration was planned as a mandatory program for all SSI- eligible children with special needs in the District. In response to stakeholder pressure, HSCSN and DC Medicaid implemented a voluntary Demonstration. As reported in Chapter 2, management perceives the limited number of participants as contributing to financial difficulties faced by HSCSN in most years of the Demonstration. This decision also affected the evaluation design. First, it makes available an internal comparison group of SSI-eligible children who elected not to participate. Second, by introducing a potential for bias due to self-selection in enrollment, it complicates the evaluator's job of assessing program impacts through comparisons of participants with non-participants.

To assess the nature of selection bias in enrollment, the evaluator compared several measures derived from Medicaid claims and eligibility data between Demonstration participants and non-participants. These comparisons show that:

• Age. Older children are less likely to enroll than younger children. The estimates imply that a 1 percent increase in the percent of eligible children 18 years and older is associated with a decrease in the enrollment rate of about 10 percentage points.

• Gender. When adjusted for other influences, gender has no effect on enrollment, though a comparison of averages shows that girls are less likely to enroll than boys.

• Race. African American children are substantial majorities of both participant and non-participant samples, but being referrals. Unlike the prescribed services mentioned above, there were very few "unknown delivery" for these referred services, making these percentages more stable.

• African American still significantly increases the probability of enrollment (by a little more than 39 percentage points above other racial groups).

• Diagnosis. Children with diagnoses in a broad category of "mental retardation, psychotic and neurotic disorder, and developmental impairments" are more likely to enroll than children with other diagnoses related to their SSI-eligibility or no diagnoses at all.

• Prior utilization: probability of use before enrollment. Children who use any Medicaid-reimbursed services before enrollment are more likely to enroll than children who use none.

• Prior utilization: amount of use before enrollment. For those who use any services, total Medicaid expenditure has no effect on enrollment, but total expenditures on both inpatient and long-term care/residential services are negatively related to the probability of enrollment.

Over time, characteristics of the population of children enrolled in the Demonstration will also be changed through disenrollment, if the children who disenroll are different from those who remain. Analyses of children who disenroll shows the following:

• Age. As older children are less likely to enroll, they are also more likely to disenroll than younger children. A 1 percent increase in the proportion of participants 18 years and older is associated with a nine percentage point increase in the probability of disenrollment.

• Gender. Being female increases the probability that a child will disenroll, by about 4.4 percentage points for every 1 percent increase in the proportion of female participants.

• Race. When adjusted for other influences, race does not significantly affect enrollment, though African Americans seem to be slightly less likely to disenroll than others.

• Diagnosis. Children with diagnoses in the group "mental retardation, psychotic and neurotic disorder, and developmental impairments" are less likely to disenroll than children with other diagnoses.

• Prior utilization. As expected, since most disenrollments happen a few months into the Demonstration, utilization before enrollment has no consistent impact on the probability of disenrollment.

There is no evidence that HSCSN has enrolled selectively from the least-impaired among children with special needs.

Participants seem to be heavier users of Medicaid-reimbursed services before enrollment than non-participants, over a similar time period. This suggests that the Demonstration has experienced adverse selection in enrollment -- Demonstration participants are "sicker" than non-participants. However, the facts do not necessarily support only this interpretation. Although participants use more services, data show that those non-participants who so use services incur higher expenditures than participants. In addition, the higher frequency of diagnoses for participants might reflect greater acuity, but might also reflect relatively extensive use: the more contacts a child has with the medical system, the more likely is that child to have a relevant diagnosis recorded. Therefore, instead of (or in addition to) the acuity interpretation of adverse selection, these findings might support a "sophisticated consumer" hypothesis. Regardless of need, participants may be more likely than non-participants to belong to families that know the local health care system. Through relatively extensive contacts with medical and social providers, these families may have gained the knowledge needed to take advantage of the available services, whether in fee-for-service or under managed care. Findings from the survey reported below tend to suggest that differential selection based both on acuity and consumer sophistication may be at work.

If mandatory enrollment (or voluntary enrollment with a substantial outreach effort), combined with more restricted provider choice are features that are necessary to make a program like HSCSN viable, then the assumption that this model of managed care will work for highly vulnerable populations deserves reexamination.

Under the best of circumstances, children with special needs are a population with highly diverse needs. A small number of utilization and expenditure outliers in a risk pool of 2,000 individuals can have devastating effects on program viability.

Options to expand the risk pool (to include presumably healthier foster children or to require enrollment of all SSI-eligible children), to restrict provider choice, and protect against damaging financial losses through stop-loss protection have been discussed in negotiations over a new waiver for the Demonstration, as described in more detail in Chapter 2. However, based on the most recent waiver request for proposal, foster children will not be eligible to participate. Mandatory enrollment is not a certain option, though it has been proposed. Restrictions on provider choice, in addition to hindering voluntary enrollment from a population that values regular provider attachments (see Chapter 5), also runs afoul of the Americans with Disabilities Act. Stop-loss provisions offer some financial protection, but managing a modified capitation system increases Medicaid and plan administrative costs. In addition, the perceived need for stop-loss protection complicates the payment model and raises questions about the wisdom of capitating programs for vulnerable populations.

 

CARE MANAGEMENT: Despite many problems, most major stakeholders value the care management system of the Demonstration.

The Demonstration implemented a unique care management model to meet the needs of its participants.

Care management goals include gaining greater understanding of individual participants' needs, in order to (1) reduce the frequency of inappropriate use and duplication of services and (2) maximize the benefits of medical care by providing supportive services and resolving non-medical issues.

To accomplish these goals, care managers assist in:

• locating, coordinating and evaluating health services,

• monitoring the delivery of services to participants,

• assuring that services are appropriate to participants' needs,

• assuring the attainment and maintenance of every child's optimum level of health,

• facilitating access to preventive services, and

• facilitating health education and family involvement in community activities that will promote family self-reliance.

HSCSN staffs four care management teams that are responsible for coordinating the care of all participants in the program, in collaboration with each child's primary care provider (PCP). Each team includes one team leader, three to four care managers, and four to six care manager associates. Fewer, and more complex, cases go to the team leader, while care manager associates carry larger numbers of less complex cases. Teams meet regularly. If the complexity of a participant's case changes, responsibility can be passed up or down the team hierarchy, as appropriate.

Problems in communications, between providers and care managers, between care managers and organizations outside HSCSN, and between care managers and families limit the effectiveness of care coordination and integration.

In general, providers and care managers express satisfaction with their respective roles and with levels of communication. However, some providers observe that high turnover of care managers and heavy caseloads may keep care managers from forming effective relations with participants and families.

Incomplete documentation is a source and symptom of poor communication, among members of the care management team and between care managers and providers. Case Record Reviews show that less than one-half include clear treatment goals in the plan of treatment (POT). PCP signatures are on all POTs in only five of 100 records. In about three-quarters of records maintained by PCPs, no POTs are included. There is documentary evidence that the results of less than one-half of all referrals are conveyed to PCPs. As amply documented in Chapter 2, the Demonstration has never successfully coordinated with other DC agencies that have medical or social service responsibilities for SSI-eligible children. Care managers express frustration with barriers that they face in trying to coordinate with agencies outside of HSCSN, many of which, like the school system, have never accepted the idea that care needs to be coordinated.

Care managers see themselves as having generally good communication with families, but express several specific concerns. These include:

• excessively dependent caregivers, who monopolize a care manager's time,

• caregiver tendencies to be reactive rather than proactive, forcing care managers to spend excessive amounts of time in crisis interventions,

• lack of consistent procedures for effecting a smooth transition when participants age out of the program, and

• the extra burden of handling "hidden caseloads," the social needs of other family members which, if not addressed, may impede the achievement of POT goals for the child.

Based on information gathered in focus groups, family perceptions of care management have changed somewhat as the Demonstration has matured. Early in the Demonstration, having been frustrated with the barriers they faced operating, largely unsupported, in a fee-for-service environment, parents and other caregivers were grateful for the "caring attitude" of HSCSN care managers. Over time, families have expressed more dissatisfaction with care manager communications and actions, including limited contact, inadequate care manager follow-up on referrals, care manager contact limited to crisis interventions, and specific problems of obtaining services or equipment.

In spite of these problems, families generally agree that care managers have eased their burdens by assuming many of the logistical functions of coordinating care. Although initial enthusiasm for the Demonstration has dissipated somewhat as parents experience day-to-day difficulties with the care management system, most continue to recognize that their children are better off in the Demonstration than they were before.

In response to concerns about overburdened care managers, a factor cited in complaints about inadequate communications, HSCSN has made organizational and technological changes in the Demonstration.

Most notably, in 1998, HSCSN management:

• added staff to facilitate outreach and conduct utilization review, under a new Director of Care Management,

• installed a new software system for tracking care provision and referrals,

• added a fourth care management team, and

• recruited a care manager who focuses exclusively on residential care.

With the new system in place, team leader caseloads have dropped from 65 to between 6 and 21 cases per manager; care manager caseloads dropped from 100 to between 58 and 68 cases; and care manager associates' caseloads dropped from 150 to 110 cases.

Despite organizational problems at the highest levels of HSCSN, the care management system in the Demonstration is much valued by providers, families and care managers.

As these stakeholders note, however, there is still room for improvement, through further increases in staff to reduce caseloads, additional technological tools to support effective care management, and improved documentation of treatment needs and services. Had the Demonstration not been as troubled at the top, and had management and funds not been diverted to developing what ended as a failed care management software tool (described in greater detail in Chapter 2), progress on these improvements might have been more substantial.

The HSCSN care management system is a work in progress, but one with an uncertain future.

In planning the Demonstration, HSCSN had no well-tested template available for designing an efficient and effective care management program. Management learned on the job, recognizing and beginning to address barriers that reduce the effectiveness of care managers and providers. However, since HSCSN appears reluctant to enter the competition for a contract under a new waiver, it is uncertain if the improvements initiated during the last years of the Demonstration will be extended and strengthened.

Clearly a longer and more stable experiment in several sites would provide a more reliable test of the value of this model. Nonetheless, most of the problems in communication and coordination encountered in this Demonstration are not unique to DC. Therefore, HSCSN's experience offers useful lessons for other programs that serve children with special health care needs.

 

OUTCOMES FOR CHILDREN, CAREGIVERS AND FAMILIES: Evidence for positive Demonstration impacts is persuasive for caregiver satisfaction and for children's use of preventive services but inconclusive for other measures.

Based on analyses of survey data, the Demonstration appears to have had a positive impact on caregiver satisfaction. However, with the exception of measures that show participants to be more likely to use preventive services than non-participants, findings on other potential impacts (health status, functional outcomes, costs, utilization) are inconclusive.

To estimate impacts, outcomes are compared between children in the Demonstration and a comparison group of eligible children who did not enroll, based on data collected in a Caregiver Survey. Some utilization estimates are constructed from HSCSN and Medicaid claims and eligibility data.

Although many estimates show differences between participants and non- participants that are apparently in the "right direction," (e.g., lower relative burden for participants' caregivers), most do not achieve accepted levels of statistical significance. Problems with achieving statistical significance are partly a function of the relatively small size of the Caregiver Survey, which produced completed interviews of the caregivers of 740 participants and 323 non-participants. In addition, however, fundamental weaknesses in data quality and in the research design place limits on the evaluator's ability to detect effects. To summarize, these weaknesses include:

• Voluntary participation in the Demonstration leads to potential selection bias which can never be wholly eliminated from impact estimates.

• The survey lacks of true baseline observations. The evaluator was able to field the survey only after 1,500 children had already been enrolled for several months.

• Concerns about possible double-billing, based on documentation of this problem for a small set of claims, raise questions about the reliability of DC Medicaid and HSCSN claims which, in turn, limit the value of claims-based cost and utilization measures.

With these limitations in mind, the evaluator compared participants to nonparticipants over a wide range of outcome domains.

According to most measures, Demonstration participants enjoy equal or better access to services, compared to non-participants. Participants also generally express greater satisfaction with access to services.

Participants are more likely than non-participants to have usual places and professionals that provide medical and preventive services, and less likely to report unmet needs. For some, but not all, of these measures of access, participant/nonparticipant differences are statistically significant. Based on size and significance of the differences, findings for satisfaction are more persuasive:

• Having a usual provider. Most children (96.6 percent) have a usual place for medical care, and most (85.2 percent) have a usual physician. Participants are more likely than non-participants to have a usual physician (87.8 percent, compared to 79.3 percent). They are also more likely to have a usual place for care, though the difference is smaller and less significant. Participants are more likely than non-participants to report that their usual place/provider addresses all their needs for preventive services (98.0 percent, compared to 95.9 percent).

• Time as a cost of access to services. About 13 percent of children enjoy minimum access time, defined as time required to make appointments, travel to appointments and wait for service at the place where care is to be received. There are no significant differences between participants and non-participants in time costs.

• Unmet needs. The frequency with which caregivers report unmet needs varies from 1.1 percent for hospital care to 13.4 percent for dental services, and 8.9 percent report more than one unmet need. Participants are generally less likely to report unmet needs, though the differences are rarely significant. However, participants are significantly less likely to report multiple unmet needs (7.8 percent, compared to 11.5 percent). Longer-stay participants are generally less likely to articulate unmet needs than shorter-stay participants, a finding that supports the hypothesis that the Demonstration helps families reduce unmet needs.

• Satisfaction. Over 80 percent of caregivers report good to excellent access to specific services and products, though satisfaction with access to home care and social services falls below 70 percent. Of seven specific categories of services and products, participants are more satisfied than non-participants in five. Participants are also more satisfied overall, with 91.6 percent expressing good to excellent satisfaction, compared to 85.8 percent for non-participants. Generally, the inference that satisfaction is associated with participation in the Demonstration is strengthened by the fact that for most service-specific satisfaction measures and for overall satisfaction, longer-stay participants are more satisfied than shorter-stay participants.

These findings are not surprising. Family focus groups show that, despite some specific concerns, families of participating children generally view the Demonstration as an improvement for them and for their children.

Despite positive perceptions of the Demonstration expressed by caregivers, there is little consistent evidence that the Demonstration has reduced caregiver burden.

Caregivers report use of formal and informal care in the home as well as "problems" that they have experienced, both in the workplace and at home caring for the child.

• Formal and informal care utilization. Very few children use any formal care at home -- 5.9 percent use formal skilled care, and 1.8 percent use formal unskilled care. Participants use more than non-participants, but the differences are small. Nearly 70 percent of all children receive nothing but informal care at home. Participants are somewhat more likely than non-participants to use a mix of formal and informal care.

• Caregiver burden. One-third of all caregivers have not held a job since learning of the child's condition -- this was more frequently true for participants' caregivers (37.5 percent, compared to 28.3 percent). Specific job-related problems ranged in frequency from 14.0 percent of caregivers who changed jobs because of their child's condition to 31.5 percent who have reduced their hours at work. Of participant caregivers who do work, generally fewer experience other job-related problems than non-participant caregivers. Problems met in caring for the child ranged in frequency from negative health effects on the caregiver (7.8 percent) to limits on the caregivers social life and free time (46.8 percent) -- there are no differences between participant and non-participant caregivers in these measures.

While there is some evidence that caregivers for children participating in the Demonstration experience fewer job-related problems than non-participants' caregivers, there are no statistically significant differences between groups in the frequencies of other problems. It seems clear that most caregivers of both participating and nonparticipating children are heavily invested in their roles and, for whatever reason, are reluctant to request help from formal service providers.

Participant caregivers report heavier use of medical services by their children than non-participant caregivers. However, the most salient difference is in preventive services. Participants are more timely in their use of preventive services than non-participants.

These analyses test three hypotheses:

• Participants should report more timely use of preventive services (routine physical exams/immunizations, routine dental visits, eye exams) than nonparticipants.

• Participants and non-participants should not differ in use of mental health services.

• Participants should be more likely to use high-tech medical services.

Findings include the following:

• Preventive and mental health services. Few children have gone for over a year without a preventive service (3.9 percent for routine/immunization visits, 6.1 percent for dental visits, 10.8 percent for eye exams), and fewer participants than non-participants are not timely in the three categories (2.6 percent, compared to 6.8 percent for routine/immunization visits, for example).

• "High-tech" medical treatments and therapies. Few children use high-tech treatments. Except for physical therapy (27.7 percent of all children) and inhalation therapy (11.9 percent), fewer than 3 percent of children use any one of several treatments. Participants use more of all treatments (except for oxygen), both overall and in a home setting. Multiple treatments are also more common among participants than non-participants.

Other data analyzed in the survey support these findings on use of preventive services. Claims based analyses, which show participants to be more frequent users of most services than non-participants, show this to be particularly true for preventive services. Also, record reviews show that participants are much above the national averages in timeliness of EPSDT exams, with percentages ranging from 60 percent timely for dental exams to 73 percent for hearing exams (compared to 1996 national EPSDT averages of 21 percent and 13 percent respectively).²

In general, Demonstration participants appear to be more physically impaired than non-participants, but other measures of mental, emotional and developmental impairments provide no clear pattern. Despite higher reported satisfaction with access, participants' caregivers are not more positive about the child's current or future health status.

Lacking baseline observations, there is no satisfactory way to estimate the extent to which the Demonstration does or does not improve participants' health status and functional levels. Claims data suggest that participants might be more impaired than non-participants. This finding should be confirmed by survey data. However, there are no consistent patterns of differences between groups in measures of cognitive, emotional development and behavioral problems. Also, for several measures of physical function, impairments are somewhat more frequently reported for participants than non-participants, though the differences are not statistically significant.

• Cognitive and emotional development and behavioral problems. Over one-third of all children are reported to have frequent problems or delays in learning or mental development, while slightly more than one-quarter have frequent problems managing their emotions. Nearly half have one or both of these problems. Participants seem to be less likely than non-participants to have cognitive problems but more likely to have emotional problems. However, differences between participants and non-participants in these measures are small and not significant. Among children age five and older, slightly under one-quarter have frequent difficulties getting along with other children their age, while a small minority (2 percent) display frequent violent behavior. Although participants and non-participants do not differ on general measures of behavior, participants are significantly less likely to exhibit violent behavior.

• Physical and overall functional capacity. A substantial proportion of all children have mobility problems (25.1 percent) and difficulty breathing (31.8 percent). About half use equipment to assist with mobility and breathing. Over several measures of functional capacity, impairments are somewhat more frequent among participants than non-participants, though the differences are not significant. This is true for the percentages with multiple impairments as well. Under 5 percent of all children have multiple physical and/or cognitive impairments, with participants slightly more impaired than non-participants. Measures for children age five and older add to the impression that participants have more impairments than non-participants. Slightly over one-third of all children are limited in strenuous activities most of the time, more so for participants (39.0 percent) than non-participants (35.1 percent). Communication with persons outside the family is an exception to the pattern. Participants appear to be somewhat less limited in their ability to communicate than nonparticipants.

• School and work. Most school-age children attend school (89.0 percent), though school attendance is slightly less frequent among participants (88.8 percent) than non-participants (92.1). However, participants who do attend school are less likely than non-participants to be limited in attendance by their conditions. About 20 percent of children age 14 and above work. As with schooling, fewer participants of working age have jobs, but of those who do, the percent with work limitations is lower (33.3 percent for participants, compared to 52.0 percent for non-participants).

On measures of present and future health status, given the positive attitudes toward the Demonstration expressed by caregivers in other contexts, it seems reasonable to expect participants' caregivers to be more positive about present health status and more optimistic about improvements to future health status than non-participants.

• Health status. Most children (78.9 percent) are reported by caregivers to be in good to excellent health at the time of the interview. Compared to six months ago, a higher percentage (95.8 percent) see no change or an improvement in the child's health, and 94.1 percent expect no change or an improvement in the child's health six months in the future. Participants appear to be somewhat more likely to foresee stability or improvement in the future (94.5 percent, compared to 93.2 percent) even though they also seem to be less positive about their child's current health status than non-participants (78.8 percent, compared to 79.3 percent). Neither difference is statistically significant.

• Disability and bed days. In a typical month, 40.9 percent of children miss one or more days from play, school or work due to their conditions. A smaller fraction (24.1 percent) spend at least half a day a month in bed. Participants seem more likely than non-participants to have disability days, but less likely to have bed days.

In the absence of baseline data, more favorable outcomes for longer-stay participants, compared to short-stay participants, would provide some evidence, however incomplete, for Demonstration impacts. No statistically significant differences related to length of enrollment were detected.

Participants use more of most ambulatory and institutional services than nonparticipants, both before and after the Demonstration. Although it is possible to compute how much the Medicaid program spent on the Demonstration, it is impossible to compare expenditures between participants and non-participants, due to claims data problems.

Before enrollment, participants tend to be heavier users of almost all institutional and ambulatory services than non-participants observed during a similar period. Overall, 87.6 percent of participants use "any" service, compared to 58.1 percent of non-participants. Once enrolled, participant utilization increases in all service categories except outpatient hospital, and nearly all participants (99.3 percent) use some Medicaid- or HSCSN-reimbursed service. Age is associated with utilization patterns. For example, younger children are more likely to use EPSDT services, and older children are more likely to use emergency room services. However, regardless of age, participants use more services than non-participants.

DC Medicaid spent more per child per month for participants after enrollment (an average of $1,402, including capitation payments) than before ($1,151). The average non-participant incurred $1,056 in Medicaid-reimbursed services per month during a similar time period, in part a reflection of relatively lower utilization rates. However, both DC Medicaid and HSCSN reimbursed providers for similar services during the Demonstration. Clearly, total expenditure estimates may include services that represent double billing, making it impossible to use actual claims data to analyze expenditure impacts of the Demonstration.

The reasons for higher utilization among Demonstration participants are unclear. Participants may be perceived to use more services because:

• they have better access to appropriate services within HSCSN,

• they simply use more services, appropriate or not,

• claims data systematically inflate utilization rates of participants, or

• some combination of these factors.

Some patterns derived from claims, for example, the large increase in EPSDT use among participants after enrolling, are supported by findings from the Caregiver Survey and from the Case Record Review. Nonetheless, the uncertainty surrounding payments recorded by HSCSN and Medicaid for Demonstration services suggests caution in interpreting estimates derived from claims.

Given weaknesses in the Demonstration design and data, the question of whether or not this program was "cost-effective" for Medicaid, for participants or for society cannot be answered. It is clear that government expenditure on the Demonstration far exceeded the capitation payments specified in the initial waiver agreement. Moving beyond this straightforward assertion is difficult. If the claims that are justified and appropriate given the care objectives of the POTs could be identified and all instances of double-billing corrected, then one could gain some insight into relative costs and savings of the Demonstration. Unfortunately, there is no way to accomplish this task without extraordinary expenditure of time and resources, and in the end, no amount of effort will resolve all uncertainties.

 

CONCLUSIONS AND IMPLICATIONS FOR POLICY: Experience under the Demonstration affirms the proposition that care management provides added value for children with special health care needs. It does not show that the model tested is the best way to deliver care management to these children.

For nearly four years, HSCSN has provided managed care services to a substantial number of SSI-eligible children in DC. When first implemented, this Demonstration was unique. Though other, similar programs now operate elsewhere in the country, findings from this experiment are still the first evidence available on whether or not a specialty managed care model for children with special needs will succeed. As one might expect, the evidence is mixed and inconclusive.

Enrolled children differ from eligible children not enrolled in demographic characteristics and, most notably, in more frequent use of medical and social services.

It is no surprise that, through enrollment and disenrollment, there has been selfselection into and out of this voluntary Demonstration. However, the nature of this selection is unexpected. Participating children are heavier users of services than nonparticipants. Though definitive measures of children's baseline health are unavailable, measures of utilization before enrollment for participants, compared to similar time periods for non-participants, show participants to be heavier users in total and in almost all institutional and ambulatory service categories. Also, though few measures of impairment from the Caregiver Survey show statistically significant differences between participants and non-participants, participants' caregivers tend to report higher levels of physical impairments. It is always risky to conclude that more frequent use means greater health needs. However, it does seem clear that selection effects in the Demonstration have been neutral to adverse, not favorable.

The Demonstration does appear to "add value" for many caregivers.

Focus group participants and Caregiver Survey respondents have generally been positive about the Demonstration. Caregivers of participants are more satisfied than non-participants' caregivers with their ability to gain access to needed services, and report that they have achieved some stability in relating to a usual physician to handle most of their child's needs. There is slight evidence that caregivers of participating children are more likely than non-participant caregivers to mitigate the burden of their responsibilities, by engaging formal skilled and unskilled assistance in the home. Of course, because no true baseline measures are available, it is impossible to be confident that these differences represent impacts of the Demonstration. Nonetheless, one can conclude that a positive impact on caregiver satisfaction is highly probable, based on the size, statistical significance, and consistency among multiple measures.

HSCSN achievements at the caregiver/child level have been gained despite turmoil and apparent inefficiency at the top of the organization.

Over the course of the Demonstration, HSCSN suffered from misguided investment decisions (to develop original care management software) and management instability (including nearly complete turnover in top management). In three out of four years of operation, HSCSN apparently suffered sizable financial losses. Demonstration officials believe that the Demonstration model is not sustainable, because the participant base is too small to spread risks in a way that makes capitation feasible. HSCSN does not now plan to compete for a contract under the new waiver.

Given problems with claims data, there is no way to measure the relative costs incurred to serve children in the Demonstration, rather than in fee-for-service environments.

Enrolling children in the Demonstration has probably been more costly for Medicaid than keeping them in fee-for-service. Claims-based data show that the average participant uses more of most services after as well as before enrollment. Nonetheless, concerns about the quality of claims data make it impossible to estimate unduplicated Medicaid expenditures, in order to compare participants to non-participants.

Although the managed care model tested in this Demonstration may not be viable, care management can be a valuable service for children with special needs.

Throughout the Demonstration, DC Medicaid paid HSCSN a capitation rate per enrolled child per month, beginning at $997.71 and increased annually by 4.6 percent. HSCSN and DC Medicaid agreed to share financial risks, making balancing payments to assure achievement of a target medical claims ratio (actual claims expenses as a percent of total capitation payments). HSCSN's financial losses during the Demonstration led to two outcomes, both of which raise questions about the capitation model:

• DC Medicaid restructured the risk-sharing model for the Demonstration recompetition. The new model will include stop-loss provisions for participating plans, to protect them against losses caused by children with excessive inpatient, pharmaceutical or home care costs.

• Based on its experience in the current Demonstration, HSCSN has announced that it will not compete for the next version of the Demonstration.

Even if a full capitation model cannot reasonably be expected to succeed with a population of small size and complex needs, some method of encouraging more comprehensive care management for SSI-eligible children with special needs should be considered. Current plans in DC to open competition for these contracts to main-line managed care organizations may not achieve the desired objectives. While HSCSN may have avoided the expected pattern of favorable selection, main-line managed care organizations that contract to serve DC Medicaid-eligible children may not be able to avoid the temptation to enroll children with relatively low expected resource needs. If special needs children become part of the larger population of covered lives, paid under a common capitation rate, limited research suggests that barriers to access may actually increase. Special needs populations, or some subset of services, might be "carved out" and paid on a fee-for-service or partial capitation basis. However, it is not clear why, except for administrative convenience, managed care organizations should be chosen to coordinate services under the carve-outs.

Children in this Demonstration appear to have benefitted from care management by professionals who understand their needs. The fact that this system operates under capitation may or may not affect provider, family and plan behavior, but the assumed efficiency-enhancing effects of capitation are not at all obvious. Further analysis of special managed care initiatives in other sites, and of programs that main-line managed care organizations have implemented to address the needs of these children, could provide better information on the relative importance of financial and other incentives on behavior and clinical outcomes.

In the meantime, policy makers might consider alternative models of financing care management. Primary care case management (PCCM) programs that pay a child's principal physician directly to provide coordination and integration have been used by state Medicaid programs in rural areas not served by managed care organizations. There is currently no information on how effective these programs have proved to be. Alternatively, Medicaid might contract with other health care professionals, for example groups organized by nurses and social workers with specialized knowledge of children with special needs, to provide care management services directly. Neither of these proposals avoids thorny questions of how to price and pay for care management services. There should still be incentives, perhaps based on linking payments to measures of performance, for physicians or care management professionals to achieve appropriate objectives of cost-effective care coordination and integration.

 

NOTES

1. Newacheck, PW et al. An epidemiological profile of children with special health care needs. Pediatrics. 1998:98(5): 952-958.

2. Olson, K., Perkins, J., and Pate, T. Executive Summary. Children's Health Under Medicaid. A National Review of Early Periodic Screening Diagnosis and Treatment. National Health Law Program. August 1996.