Evaluating PACE: A Review of the Literature

Publication Date

Dec 31, 2013

Arkadipta Ghosh, Cara Orfield and Robert Schmitz

Mathematica Policy Research

January 2014

Printer Friendly Version in PDF Format: http://aspe.hhs.gov/daltcp/reports/2014/PACELitRev.pdf (77 PDF pages)Free PDF Reader: http://get.adobe.com/reader/

This report was prepared under contract #HHSP23320095642WC between the U.S. Department of Health and Human Services (HHS), Office of Disability, Aging and Long-Term Care Policy (DALTCP) and Mathematica Policy Research. For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/office_specific/daltcp.cfm or contact the ASPE Project Officers, Hakan Aykan and Jhamirah Howard, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. Their e-mail addresses are: Hakan.Aykan@hhs.gov and Jhamirah.Howard@hhs.gov.

The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.

ACRONYMS

EXECUTIVE SUMMARY

I. INTRODUCTION

II. CHALLENGES IN EVALUATING PACE

III. SCREENING CRITERIA AND STUDY RATING SCHEME

IV. FINDINGS FROM THE REVIEW OF PRIOR EVALUATIONS OF PACE

A. Costs

B. Nursing Home and/or Hospital Utilization

C. Quality of Care

D. Participant Satisfaction and Quality of Life

V. CONCLUSIONS

REFERENCES

NOTES

APPENDICES

APPENDIX A: Summary of Prior PACE Evaluations and Their Strength of Evidence

LIST OF TABLES

TABLE 1: Criteria for Rating Strength of Study

TABLE 2: Evidence Summary from Studies Examining the Impact of PACE on Medicare and Medicaid Costs

TABLE 3: Evidence Summary from Studies Examining the Impact of PACE on Hospital and Nursing Home Utilization

TABLE 4: Evidence Summary from Studies Examining the Impact of PACE on Quality of Care

TABLE 5: Evidence Summary from Studies Examining the Impact of PACE on Participant Satisfaction and Quality of Life

ACRONYMS

The following acronyms are mentioned in this report and/or appendix.

ADL	Activity of Daily Living

CMS	Centers for Medicare and Medicaid Services

ED	Emergency Department
ER	Emergency Room

FFS	Fee-For-Service
FY	Fiscal Year

HCBS	Home and Community-Based Services

IADL	Instrumental Activity of Daily Living

MA	Medicare Advantage
MCBS	Medicare Current Beneficiary Survey
MMLTC	Medicaid-Only Managed Long-Term Care

NH	Nursing Home

OASIS	Osteogenesis Imperfect

PACE	Program of All-Inclusive Care for the Elderly
PBPM	Per Beneficiary Per Month
PIP-DCG	Principal Inpatient-Diagnostic Cost Group
PPI	PACE Prognostic Index

SNF	Skilled Nursing Facility

UPL	Upper Payment Limit

VA	U.S. Department of Veterans Affairs

WPP	Wisconsin Partnership Program

The Program of All-Inclusive Care for the Elderly (PACE) provides coordinated acute and long-term care services to nursing home (NH) eligible seniors in the community. PACE is a Medicare managed care program and a Medicaid state plan option. Individuals who are 55 or older, certified by their state of residence as being eligible for NH level of care, and live in the service area of a PACE program are eligible to enroll in PACE. The underlying premise for Medicare and Medicaid financing of PACE is that these programs enable some frail elderly enrollees to remain in the community, increase enrollees' satisfaction with health care services and quality of care, and save money for both the Medicare and Medicaid programs.

Based on a comprehensive review of existing evaluations of PACE, this paper brings together available evidence on the effect of PACE on several key outcomes of interest--Medicare and Medicaid costs; hospital and NH utilization; quality of care, satisfaction and quality of life; and mortality. We summarize findings from past studies and assess their methodological approach. We include both published articles as well as research reports in this review and identify key themes that emerge from past findings when viewed in the light of their underlying strength of evidence. This review improves upon an earlier literature review (Galantowicz 2011) by utilizing stricter inclusion criteria and conducting a more detailed review of the studies, as well as a more rigorous assessment of the quality of evidence presented in each study.

Several key findings emerge from this literature review regarding the design and methodological approaches of prior PACE evaluations as well as on the effectiveness of PACE in controlling spending, reducing hospitalizations and NH use, and improving quality of care and satisfaction. These can be summarized as follows.

There are significant challenges in evaluating PACE, given the characteristics of the program and its beneficiaries, the most significant of such challenges being the identification of an appropriate comparison group.
With no studies using an experimental design for evaluation, none offers strong evidence on the effectiveness of PACE.
Most quasi-experimental studies of PACE fail to meet the standards of a rigorous evaluation. Only four of the 22 studies included in this review met our standards for offering "moderate to strong" evidence on the effects of PACE, and seven other studies could only be rated as offering "moderate" evidence, given their inability to establish baseline equivalence between the treatment and comparison groups. Half (11) of the reviewed studies received either "moderate to weak" (five studies) or "weak" (six studies) ratings.
The evidence from studies with the strongest design show that PACE has no significant effect on Medicare costs, but it is associated with significantly higher Medicaid costs, with the Medicaid spending gap between PACE and matched comparison enrollees decreasing over time. Therefore, based on currently available evidence in the literature, we conclude that PACE does not save costs for either program, and it raises overall cost through an increase in Medicaid expenditures. Prior findings on Medicare and Medicaid costs need to be updated, given changes to the Medicare capitation payment approach as well as variation in the Medicaid capitation rate calculations across states.
Evidence on the effect of PACE on the utilization of expensive acute and long-term care services is mixed--studies with the strongest design find PACE enrollees have fewer inpatient hospitalizations than their fee-for-service counterparts, but they appear to have higher rates of NH admission. None of the studies, however, differentiate between post-acute and long-term NH stays. The findings concerning hospitalizations are expected, given the program's emphasis on care coordination, but the NH findings are counterintuitive. It is possible that the higher rates of NH admission under PACE are a consequence of the substitution of short-term NH use for hospitalizations, although the literature provides no direct evidence on this front. Furthermore, if short-term nursing facility stays are being substituted for some hospitalizations under PACE, the reduction in hospitalizations is somewhat of an overstatement of the program's success in reducing the number of enrollees experiencing exacerbations of their health problems.
There is some evidence that PACE improves certain aspects of care quality--for example, those related to management of specific health issues such as pain--and that PACE enrollees have a lower mortality rate.
Although PACE participants are satisfied with their medical and personal care, there is insufficient evidence as to whether their satisfaction and quality of life are greater than what they would have experienced if not in PACE.

Overall, the only outcome for which we found strong evidence of favorable effects under PACE is inpatient hospitalizations. At the same time, evidence from prior studies suggests that PACE was associated with higher NH utilization and greater costs to Medicaid, which are clearly causes for concern. Given the possibility of the PACE plans substituting short-stay NH use for hospitalizations, investigating the differential effect of PACE on short-stay versus long-term NH utilization is a promising avenue for future research, especially given the lack of such distinction in PACE's effects on institutionalization in the existing literature.

Several studies included in this review have significant limitations in terms of external validity or the generalizability of their findings and in their continued relevance for assessing the current PACE model. These limitations arise due to study setting, small sample sizes, and study timing. Given the variation in program implementation and site characteristics across PACE sites, studies which make comparisons across a few PACE programs may have limited generalizability to the broader PACE population. Also, even if credible, the findings from prior evaluations looking at potential cost savings under PACE are likely to be outdated, given changes to the PACE financing structure over time, especially for Medicare capitation payments. Findings from studies included in this review are unlikely to be useful in assessing possible PACE expansion efforts or new care coordination and integration models being proposed for dually eligible beneficiaries, since the effect of a different program structure or organization cannot be predicted based on these results that apply to the PACE model existing at the time of each study. Also, the literature to date offers no evidence on whether a more flexible variant of PACE that allows enrollees to maintain ongoing relationships with their existing primary care providers would have the same effects on enrollees as the existing PACE model.

I. INTRODUCTION

The Program of All-Inclusive Care for the Elderly (PACE), established as a permanent Medicare and Medicaid benefit by the Balanced Budget Act of 1997, attempts to help nursing home (NH) eligible seniors avoid institutional care by providing them with a rich mix of coordinated acute and long-term care services in the community. PACE is a Medicare managed care program and a Medicaid state plan option. Therefore, PACE organizations receive two capitation payments per month for their dually eligible enrollees, and assume full financial risk for all the health care services that participants use.

Individuals who are 55 or older, certified by their state of residence as being eligible for NH level of care, and live in the service area of a PACE program are eligible to enroll in PACE. Designed for the frail elderly or people with disabilities, PACE programs are centered around: (1) the adult day health center where participants receive medical and social services; and (2) an interdisciplinary team comprising physicians, nurse practitioners, social workers, nutritionists, therapists, personal care attendant, and drivers. The typical PACE enrollee is dually eligible for both Medicare and Medicaid, over 75 years old, White, and female with multiple chronic conditions as well as more than one activity of daily living (ADL) limitation (MedPAC 2012; Hirth, Baskins, and Dever-Bumba 2009). As of 2012, there were 84 PACE sites across 29 states serving approximately 21,000 enrollees (National PACE Association 2012).

On Lok Senior Health Services in San Francisco, established more than 40 years ago in 1971, is the precursor to current PACE programs. On Lok, meaning "peaceful, happy abode" in Cantonese sought to integrate acute and long-term care services for frail and chronically ill elders in San Francisco's Chinese community to enable these elders to remain at home and avoid institutionalization. It offered adult day care along with a comprehensive package of medical, rehabilitation, and social services. After more than a decade of operation, On Lok obtained Medicare and Medicaid waivers in 1983 that allowed it to receive capitated payments for integrated care. Based on On Lok's success, federal legislation enacted in 1986 led to the expansion of the On Lok model under the PACE demonstration program to multiple sites across the country. PACE was finally recognized as a permanent Medicare and Medicaid provider in the Balanced Budget Act of 1997 (Bloom, Sulick, and Hansen 2011; Hirth, Baskins, and Dever-Bumba 2009; Eng 2002; Bodenheimer 1999). Currently, PACE is the only statutory program that integrates services covered by Medicare and Medicaid through a system of capitation payments from both programs.¹

Naturally, integral to PACE's cost saving potential is the question of whether these capitation payments are set at the right level, that is, whether the capitation payments are lower than what Medicare or Medicaid would have paid for PACE enrollees under the traditional fee-for-service (FFS) arrangement. Although several studies included in this review have addressed this question with mixed findings, these findings are unlikely to be as relevant today as they were during the time period covered by each study (White 1998; White, Abel, and Kidder 2000; Mancuso, Yamashiro, and Felver 2005; Foster, Schmitz, and Kemper 2007; Wieland et al. 2012). This is because the methodology underlying the calculation of Medicare's capitation payment to PACE plans has evolved over time.² Also, the Medicaid monthly capitation rate is negotiated between the PACE organization and the state Medicaid agency and is contractually specified, with the state rate-setting methods for the Medicaid capitation payment varying across states and over time.³ Also, states vary in their approaches for adjusting rates for trends and for frailty and health status. Given changes to both Medicare and Medicaid rates over time, prior findings on cost savings in PACE need to be updated by new studies that examine Medicare and Medicaid costs for PACE under the current capitation system relative to the prevailing FFS model.

Although PACE has been hailed by some as a success in improving health outcomes for participants, saving costs, and improving end-of-life care, questions remain on the future of PACE, especially with regard to barriers to its expansion. These barriers include the considerable costs of opening up and running a day care center, staff-model organization with salaried clinicians that require enrollees to leave their existing primary care physicians, shortage of qualified staff, lack of sufficient numbers of eligible enrollees in a market, inadequate financing for long-term care for nondual Medicare beneficiaries, and budget and resource constraints faced by state Medicaid agencies (MedPAC 2012; Bloom, Sulick, and Hansen 2011; Hirth, Baskins, and Dever-Bumba 2009; Bodenheimer 1999). The underlying premise for establishing PACE as a permanent Medicare provider and a state option under Medicaid was that by providing integrated acute and long-term care as well as social support services, PACE programs would: (1) enable some frail elderly enrollees to remain in the community, that is, prevent or delay the need for long-term institutionalization of such enrollees (PACE plans cover long-term NH stays, once an enrollee is admitted to such an institution); (2) increase enrollees' satisfaction with their health care services and enhance their quality of care as well as quality of life; and (3) save money for both the Medicare and Medicaid programs. The future of PACE, including possible innovations in its design and its expansion beyond populations currently served (for example, extending PACE coverage to disabled individuals below age 55 or nonMedicaid populations) is likely to depend, at least in part, on how well PACE programs have met each of the goals mentioned above.

An earlier literature review assessing the existing evidence base for PACE identified 10 studies examining the effects of PACE (Galantowicz 2011). The review--in the form of a memorandum to CMS--included original studies that investigated the impact of PACE on cost, utilization, and/or other outcomes for Medicare-Medicaid enrollees. To be included in the memo, impacts had to be measured either relative to a comparison group or over time. Our report improves upon the earlier literature review in several key ways. We conduct a more thorough review of each study, discuss each paper in greater detail, and also compare studies within the same outcome domain against each other. Finally, we perform a rigorous assessment of the quality of evidence presented in each study, giving each study a rating based on the strength of the evidence presented. Studies with higher ratings were given considerably more weight in our discussion and summary of evidence.

Based on our systematic review of existing evaluations of PACE, this paper brings together available evidence on the effect of PACE on several key outcomes of interest--Medicare and Medicaid costs; hospital and NH utilization; quality of care, satisfaction and quality of life; and mortality. We summarize findings from past studies and assess their methodological approach. We include both published articles as well as research reports in this review and identify key themes that emerge from past findings when viewed in the light of their underlying strength of evidence. The rest of this paper is organized as follows. Section II offers an overview of the methodological challenges in evaluating PACE and how past studies have usually addressed such challenges. In Section III, we describe the screening criteria used to identify prior evaluations for this review, and also describe our approach towards rating the quality of evidence from each study. Section IV describes the findings from the review along with an assessment of the strength of evidence or the methodological soundness of reviewed studies, and Section V concludes.

II. CHALLENGES IN EVALUATING PACE

The challenges in evaluating PACE mainly arise from the structure and characteristics of PACE as a program and also from the characteristics of beneficiaries it serves, who self-select into the program. In this section, we provide an overview of these challenges along with a discussion of how past studies have addressed such challenges.

No opportunity to conduct an experimental study. Obtaining the right counterfactual, that is, finding an answer to the question--what would have happened to PACE enrollees in the absence of the intervention (PACE)--is a daunting challenge for quasi-experimental studies. Since PACE is a permanent component of the Medicare and Medicaid programs, frail, elderly individuals who are NH certifiable and therefore eligible for PACE, cannot be denied entry into the program.⁴ As such, a randomized evaluation of PACE, where eligible beneficiaries are randomly assigned to a PACE plan or to a nonPACE alternative is impossible to implement without changes to federal regulation. Therefore, prior evaluations have relied on quasi-experimental research designs with alternative comparison group strategies to evaluate PACE. Comparison group designs, however, are susceptible to selection bias, if beneficiaries in the treatment group (PACE) differ from those in the comparison group along both measured and unmeasured characteristics that are correlated with outcomes. Furthermore, the comparison group must receive care that mirrors what the treatment group would have received in the absence of PACE. Ensuring that comparability can be hard when comparison groups are drawn from geographic areas that differ in the mix and intensity of medical care and community supports available.

Difficulties in defining the "right" comparison group. Defining the right counterfactual to PACE--that is, the care alternatives that PACE enrollees would have faced were PACE not an option--poses a significant conceptual problem in evaluating PACE programs. Identifying a comparison group of beneficiaries who are similar to PACE enrollees in their pre-enrollment characteristics, not enrolled in PACE, and reside in a health service environment comparable to that for PACE enrollees is a serious challenge for evaluations of PACE. The difficulties in defining an appropriate comparison group are exacerbated by the characteristics of the beneficiaries typically served by PACE--frail, elderly individuals who are eligible for both Medicare and Medicaid, are NH certifiable, and volunteered to enroll in PACE. These characteristics that drive enrollment decisions are particularly problematic because they are not generally observable in the data available for evaluations. For example, because PACE operates as a capitated, managed care program, beneficiaries entering PACE have to stop seeing their usual providers, including their primary care physicians, and instead rely on providers who are a part of the PACE plan. Beneficiaries willing to make this change may be systematically more open than others to new ways to maintain their health or less likely to have established strong bonds to their providers. In some cases, beneficiaries may be unable to make decisions related to their own care, resulting in an informal caregiver taking over the responsibility of choosing the best care model for the beneficiary. The dynamics of the decision making process for enrolling in PACE are likely to be different in these circumstances, and may reflect the caregiver burden as well as the severity of the older person's condition. In either case, if there are such systematic differences in attitudes and behaviors between PACE enrollees and other dual eligibles, and those differences are correlated with outcomes, the estimate of PACE's impacts on those outcomes would be biased unless a comparison group strategy can be devised that accounts adequately for these unmeasured factors.

Studies have responded to these challenges in different ways, that is, through the adoption of different comparison group strategies. These can be broadly categorized into three alternative comparison groups: (1) community-dwelling, frail elderly who are not enrolled in PACE but who require long-term support services, for example, participants in HCBS waiver programs; (2) NH residents; and (3) other groups of nonparticipants, for example, individuals who expressed interest in PACE and had a home visit to confirm eligibility, but did not eventually enroll in the program.

Problems with the comparison group strategies. Although each of these comparison group strategies are potentially valid, each has its own share of problems as well. For instance, although HCBS waiver participants are eligible for a NH level of care, similar to those served in PACE, the wider array of services available under PACE raises concerns that PACE enrollees have unmeasured characteristics that put them at greater risk of needing expensive services in the future. Also, the concern remains that PACE enrollees' willingness to give up their existing health care providers in order to enroll in PACE suggests a possible bias. Moreover, while PACE programs offer a standard, comprehensive package of services, HCBS waiver programs vary in their coverage of specific services. For cross-state studies, this heterogeneity or variability in HCBS waiver programs across states leads to a lack of standardization in the counterfactual against which PACE is being compared. To conduct a deeper investigation into the nature of each and every HCBS waiver program and their covered services against which PACE plans are being compared would possibly lead to excessive study costs, and are therefore, not feasible to propose.

The use of NH entrants as a comparison group for PACE enrollees is valid under the assumption that most beneficiaries enrolling in PACE would have entered a NH in the absence of the program. However, once again, enrollment is likely to be tied to a beneficiary's perception of both current and future need for services. NH entrants are likely to be sicker than those enrolling in PACE at the time of enrollment, or they may have a weaker informal support system than enrollees in PACE.⁵ Furthermore, even if PACE enrollees would have eventually entered a NH, had PACE not been an alternative, they may have remained in the community for several months or years before doing so. Hence, with a comparison group of new NH entrants, the effects of PACE on mortality or health care costs could potentially be overstated due to the expected difference in the mortality trajectory between the two groups--with NH entrants dying far more quickly--and the increase in health care expenditures towards the end of life.

Finally, the third strategy of using a group of eligible nonenrollees who had originally expressed interest in PACE does partially address selection concerns by equating the two groups in their avowed willingness to enroll in PACE, but does not quite overcome the problem of eventual enrollment being linked to underlying health status and anticipated need for health services. Mindful of these concerns, some studies have additionally employed propensity score matching as a technique to achieve baseline equivalence between the treatment and matched comparison groups, while some others have relied on using two different comparison groups of HCBS enrollees and NH residents for a more robust analysis. Given the variety of comparison group strategies used in the literature, it is important to consider the counterfactual when trying to generalize findings from a specific study to the broader PACE population.

Difficulties in uncovering program features associated with success. Regardless of the specific comparison group strategy employed by a study, none have been able to attribute the success or failure of PACE to specific components or program characteristics. This inability is a direct consequence of the complex and diverse ways in which PACE is implemented, and the small size of most PACE programs, limiting the ability of studies to estimate site-specific effects with adequate precision. Given that outcomes could be affected by one or more of the many factors that vary across PACE sites, uncovering the probable reasons for a PACE plan's success or the lack of it is extremely difficult. Hence, strictly from an evaluation perspective, PACE has remained a black box of varying program features whose association with success or failure cannot be determined.

Difficulties in constructing claims-based health care utilization measures. Since PACE plans receive capitation payments from both Medicare and Medicaid, detailed information on health care utilization of PACE enrollees is not available in claims data, unlike enrollees in traditional FFS Medicare or Medicaid. Hence, claims-based measures of health care utilization, including claims-based quality of care measures, are typically not available for PACE enrollees. These include measures such as hospitalizations or emergency room (ER) visits, as well as process of care quality measures for beneficiaries with specific chronic conditions, for example, quality measures for beneficiaries with diabetes or heart disease.⁶

With this background on the challenges involved in evaluating PACE, we now move on to reviewing prior evaluations of PACE, starting with the screening criteria used to identify studies for this review and our approach towards rating the quality of evidence from each study.

III. SCREENING CRITERIA AND STUDY RATING SCHEME

This literature review aims to gain insight into the effects of PACE on several key outcomes of interest: costs, NH and/or hospital admission, quality of care, mortality, and participant satisfaction. To identify published literature addressing research questions relevant to these outcomes of interest and to capture relevant studies and important findings from less formal sources such as government reports, we systematically searched the following databases: Ovid, PubMed, EBSCOhost, Google Scholar, and the Mathematica Integrated Library. In our search for citations, we used keywords relevant to PACE, limiting the search to publications between 1997 and 2013. Through this process, we identified 58 potentially relevant papers. We then screened abstracts from the 58 papers using the criteria identified below. When we could not apply the inclusion criteria with certainty, we obtained the full text of the article for further evaluation. All 58 studies, with the ones we eventually included in the review being separately identified, are listed in Section IV of the Appendix.

Three general criteria were used to evaluate studies for inclusion in this literature review:

Was the study published in the relevant timeframe? The PACE model was established as a permanent provider type under both the Medicare and Medicaid programs in 1997. We were therefore interested in studies published from 1997 onwards, including some studies that evaluated PACE demonstrations that existed prior to 1997, but were published in 1997 or later.⁷
Does the study present information about the impact of the PACE program on one or more of the key cost, utilization, or quality outcomes identified for this study? Papers that did not present any impact analysis or comparisons for outcomes relevant for this study were excluded from the review. Articles that did not carry out an examination of PACE outcomes but were useful for background information are included as references but are not considered part of the review. Studies of PACE programs with outcomes clearly outside the scope of our areas of interest (costs, NH and/or hospital admission, quality of care, mortality, and participant satisfaction) were also excluded from the review. Examples of outcomes outside the scope of our review include PACE care management practices, factors determining PACE enrollment, and predictors of disenrollment.
Does the study have a comparison group design? Because PACE is a permanent part of Medicare and Medicaid, individuals who are eligible for PACE cannot be denied entry into the program. Random assignment evaluations--the gold standard in social policy research--are thus impossible to conduct. As such, this literature review is limited to papers that used quasi-experimental or nonexperimental research designs, mainly focusing on studies that used a comparison group strategy to evaluate PACE. Studies that compared PACE to nonPACE populations, or studies that compared outcomes across different PACE sites were included in the review. Studies that did not involve any comparison between PACE and nonPACE alternatives or across PACE plans were excluded from the review.

In spite of the challenges involved in evaluating PACE, after applying these criteria, we identified a total of 22 studies for inclusion in our review. The presence of 22 credible studies highlights both the importance of PACE as a program and the high level of interest in understanding its impacts.

Each paper included in the review was thoroughly read, assessed, and summarized by the authors of this report, comprising two Ph.D. researchers and one research analyst. One team member conducted the online searches to gather articles, and two team members were involved in reading, assessing, and summarizing the articles. The entire team discussed the inclusion criteria and contributed to determining which papers were included in the final review. Detailed descriptions of each study, including the timeframe, evaluation design, sample, data source(s), outcomes, methods of impact evaluation, and key findings, are listed in Appendix A. In addition to summarizing information included in each paper, we performed a thorough assessment of the research design, evaluated the potential threats to validity of the findings, and rated each study based on the strength of its research evidence. A summary of our assessment and study rating are also included in Appendix A for each study. No paper garnered a "strong" evidence rating, since none were based on a random assignment design. The applicable study ratings--moderate to strong, moderate, moderate to weak, and weak--were assigned based on the evaluation design, comparison group strategy, the quality of the data, the soundness of the statistical analyses, and the adequacy of the authors' attempts to address potential threats to internal validity. The rating scheme is explained in Table 1.

TABLE 1. Criteria for Rating Strength of Study
Rating	Design	Number of Studies
Strong	Randomized controlled trial.	0
Moderate to Strong	Quasi-experimental design with comparison group, regression adjustment, and established baseline equivalence.	4
Moderate	Quasi-experimental design with comparison group and regression adjustment, but without established baseline equivalence.	7
Moderate to Weak	Quasi-experimental design with comparison group, but without regression adjustment or baseline equivalence.	6
Weak	Comparisons across PACE sites without a nonPACE comparison group.	5

Where applicable, we also comment below on two other important study aspects that are likely to affect the findings. These additional aspects are: (a) a study's sample size or its ability to detect meaningful impacts; and (b) the duration of a study's followup period, that is, the length of the period over which impacts are examined. An underpowered study is unlikely to detect impacts even if there are no threats to the study's validity. Similarly, a study with a limited followup period will be unable to uncover longer term impacts. This is especially true for some outcomes, such as NH entry or mortality, that may take longer to manifest themselves than effects on hospitalizations or other services. However, the desirability for a longer followup period over which outcomes are observed needs to be balanced against the limited life expectancy of many PACE enrollees due to their health conditions and frailty, as well as changes in program rules and capitation payments over time that could potentially confound impact estimates over a longer followup duration.

IV. FINDINGS FROM THE REVIEW OF PRIOR EVALUATIONS OF PACE

We categorized studies selected for the review into four groups, based on the outcomes measured: costs, NH and/or hospital admission, quality of care, and participant satisfaction and quality of life. Many studies included outcomes from multiple categories and are thus discussed in the context of more than one outcome category. This section is devoted to discussing studies under each of the four outcome categories. Each subsection includes a brief definition of the outcome, followed by a detailed description of the studies reporting on that outcome. Study descriptions include methods, findings, and a brief discussion of study limitations. Subsections conclude with an overall assessment of the strength of evidence and findings surrounding the outcome of interest. Unless otherwise mentioned, the findings discussed below were reported to be statistically significant in the study itself. As mentioned above, Appendix A summarizes each study, including an assessment of the strength of evidence from the study.

A. Costs

1. Evidence Summary

Based on evidence from studies with the strongest design, it appears that PACE has no significant effect on Medicare costs, but is associated with significantly higher Medicaid costs. Although the findings from PACE cost assessments are mixed, studies that are methodologically stronger tend to report some common findings (Table 2). Two of the eight studies included in this subsection garnered a moderate to strong rating due to their use of a quasi-experimental design with regression adjustment and baseline equivalence (Foster, Schmitz, and Kemper 2007; Mancuso, Yamashiro, and Felver 2005). Another three studies received a moderate rating, meaning they were quasi-experimental studies that used regression adjustment to control for differences across groups, but failed to show baseline equivalence (White, Abel, and Kidder 2000; Wieland et al. 2012; White 1998). The remaining study received a moderate to weak rating due to its study design. Among the more rigorous studies, one found Medicare costs to be similar under PACE and HCBS (Foster, Schmitz, and Kemper 2007), and two studies ranked moderate (albeit with the same sample) found Medicare costs to be lower than they would have been had the individuals remained in FFS (White, Abel, and Kidder 2000; White 1998). Both studies with favorable Medicare cost savings findings in PACE, however, suffered from data and analysis limitations as discussed below. Findings regarding the effects of PACE on Medicaid costs were more consistent, with both studies with moderate to strong rating showing increased Medicaid costs under PACE, and all but one of the studies with moderate rating also pointing to increased Medicaid costs under PACE. The only study showing Medicaid costs to be lower under PACE received a moderate rating and was limited to a single state and a single point-in-time and was therefore less reliable than the other studies.

TABLE 2. Evidence Summary from Studies Examining the Impact of PACE on Medicare and Medicaid Costs
Study	Summary of Findings	EvidenceRating
Both Medicare and Medicaid Costs
Foster, Schmitz, & Kemper (2007)	Higher Medicaid costs under PACE, but no significant difference in Medicare costs between PACE and matched HCBS enrollees.	Moderate to strong
White, Abel, & Kidder (2000)	Higher Medicaid costs but lower Medicare costs under PACE compared to individuals who expressed interest in PACE but did not enroll.	Moderate
Medicaid Costs Only
Mancuso, Yamashiro, and Felver (2005)	Medicaid costs for PACE enrollees higher than that for HCBS participants, but similar to that of NH residents.	Moderate to strong
Wieland et al. (2012)	Medicaid costs under PACE were lower than that for the comparison group comprised of waiver participants and NH entrants.	Moderate
Medicare Costs Only
White (1998)	Medicare costs under PACE were lower than that for individuals who expressed interest in PACE but did not enroll.	Moderate
MedPAC (2012)	Medicare spending on PACE enrollees exceeded spending on comparable FFS enrollees.	Moderate to weak
NOTE: For each set of outcomes, studies have been listed in descending order of their evidence rating--starting with the strongest possible study.

In sum, PACE appears to not yield savings for either of the two programs, and it raises cost overall through an increase in Medicaid expenditures. However, as mentioned above, these findings from prior evaluations looking at potential cost savings under PACE are likely to be outdated, given changes to the PACE financing structure over time. Moreover, the finding in at least two studies (Foster, Schmitz, and Kemper 2007; Mancuso, Yamashiro, and Felver 2005) of a decrease in the Medicaid spending gap between PACE and matched HCBS enrollees over time suggests that the limited followup used by many of the studies examining the effect of PACE on Medicaid expenditures are too short. At least half the studies followed patients for only one year.

2. Detailed Discussion of Studies Examining Cost Outcomes

Six of the 22 studies included in the review assessed PACE from a cost perspective. Two of these six studies compared both Medicare and Medicaid costs in PACE with costs for comparable HCBS populations or with FFS costs incurred prior to enrollment in PACE (Foster, Schmitz, and Kemper 2007; White, Abel, and Kidder 2000). Two studies conducted Medicaid-only cost comparisons with HCBS and NH clients (Mancuso, Yamashiro, and Felver 2005; Wieland et al. 2012), and two studies conducted Medicare-only cost comparisons--either against individuals who had expressed interest in PACE but who did not enroll or against projected estimates for FFS spending (White 1998; Medicare Payment Advisory Commission 2012). The discussion below is organized according to the specific cost outcomes examined by each study--starting with studies examining both Medicare and Medicaid costs, and moving on to those examining Medicaid and Medicare costs only. Within each of these subsections, we start by describing the studies with the strongest design, before moving on to the relatively weaker analyses.

a. Medicare and Medicaid Costs

Both studies reviewed in this subsection found Medicaid costs to be higher under PACE, but the findings for Medicare costs differ across the two studies. Foster, Schmitz, and Kemper (2007) compared mean per beneficiary per month (PBPM) Medicare and Medicaid expenditures for a matched HCBS comparison population to actual (capitated) expenditures for PACE enrollees. The study focused on 17 PACE organizations in nine states from 1999 to 2004. In comparing unadjusted group means between these groups, monthly Medicare expenditures were found to be similar over a five year followup, whereas monthly Medicaid expenditures for the PACE group significantly exceeded those for the matched comparison group over all four six-month intervals from the time of enrollment, that is, over a somewhat limited two-year followup (the difference decreased from $926 during months 1-6 to $536 during months 19-24). Differences between actual and predicted expenditures were consistent with these unadjusted group mean findings. The authors note that the claims data used in their analysis did not necessarily reflect all service costs because some Medicaid providers may not have billed for small reimbursements to which they were entitled. Thus, comparison group Medicaid expenditures may be slightly underestimated. Also, as acknowledged by the authors, in spite of propensity score matching, unmeasured differences between the groups could potentially bias impact estimates. For instance, the two groups could still differ on one or more unmeasured characteristics, such as cognitive or functional limitations, not included in the matching model.

White, Abel, and Kidder (2000) used multivariate regression models to estimate and compare predicted costs to PACE capitation payments in the initial year of participation, using individuals who expressed interest in PACE and had a home visit but who did not ultimately enroll as the comparison group. Comparison group individuals had to meet the PACE eligibility criteria, including the state skilled level of care requirements, even if they were not enrolled in Medicaid, and also reside in the catchment area of a PACE site. Eleven PACE programs operating in eight states participated in this evaluation from 1995 through 1997. The authors found that the total or combined Medicare and Medicaid capitation payments per month were about 10 percent higher than projected total costs (statistical significance was not reported for this difference). However, the findings varied by program: the Medicare portion of the capitation payment was 42 percent less than projected Medicare costs although the difference was not statistically significant, while the Medicaid portion of the capitation payment was 86 percent higher than projected Medicaid costs and significant at the 5 percent level.⁸ The findings, apart from being more than a decade old, are seriously limited by the lack of baseline equivalence between the two groups, small sample sizes leading to high variability in the data, and a limited one-year followup. The response rate for the baseline interview in this study was low (44 percent). To deal with potential nonresponse bias, the authors estimated a more limited model for a sample of nonrespondents based on available Medicare claims and enrollment data and found similar results. Furthermore, individuals whose claims records could not be linked across Medicare and Medicaid were excluded from the analysis and thus the study sample was somewhat restricted. The sample included 1,367 PACE enrollees and only 671 comparison group members across the 11 sites; however, Medicaid data were available for only 381 comparison group members, meaning that for 43 percent of the original comparison group sample, Medicare and Medicaid claims could not be linked. This is a sizeable loss in the analysis sample for Medicaid and total costs, with potential for introducing serious bias, as well as imprecision, in those findings.

b. Medicaid Costs Only

The two studies looking only at Medicaid costs reached different conclusions with the stronger study finding higher Medicaid costs under PACE, consistent with the findings above. Mancuso, Yamashiro, and Felver (2005) estimated Medicaid-only cost differences across PACE, HCBS participants, and NH residents in the State of Washington from 1998-2003 using propensity score matching and regression adjustment. They selected both an HCBS comparison group and a NH comparison group for PACE enrollees, matching the treatment and comparison group members on several baseline characteristics, such as age, gender, race, functional status, level of care, eligibility for a state program that allows clients at the risk of institutionalization to remain in the community, months of medical assistance eligibility, and medical cost risk score. The same variables were used as regression controls. In this analysis with a four year followup, PACE enrollees were found to be more expensive than HCBS participants--with the gap in Medicaid expenditures declining from $1,442 in the first year to around $1,018 in the fourth year, and about as costly as the NH comparison group. In spite of using a detailed list of controls, the study did not match the treatment and comparison groups on prior service use patterns, which is a limitation. Also, because this study did not factor in Medicare costs, it presents a limited picture of true program costs. Further, the small sample sizes (227 PACE enrollees, 1,891 HCBS enrollees, and 399 NH clients) limit the reliability of its findings. Nonetheless, the lack of savings even when PACE enrollees are compared solely to NH residents suggests that the capitation rate was not set at a level that generates savings.

Wieland et al. (2012) used state admission records in South Carolina to model and compare Medicaid costs of PACE participants to FFS costs for HCBS and NH participants. They found that the PBPM Medicaid capitation rate in PACE was 22-26 percent below the predicted FFS payments.⁹ This study failed to achieve baseline equivalence across the three groups and had a limited followup of only one year for Medicaid expenditures. In addition, the study's sample of beneficiaries were enrolled over an 11-year period (1994-2005) which leaves open the possibility of changes in program rules and requirements as well as payments over time, which the authors do not seem to account for in their analysis. Finally, the period of study preceded the implementation of Medicare Part D, which transferred drug benefit coverage for dual eligibles from Medicaid to Medicare; this systemic change would be expected to lower the cost differences between PACE and Medicaid FFS for comparable populations.

c. Medicare Costs Only

White (1998) used regression models to calculate projected Medicare FFS reimbursement levels for PACE enrollees and compared the projected rates to the actual capitation rates to find that Medicare costs under PACE were lower. Similar to the White, Abel, and Kidder (2000) study, the comparison group included individuals who expressed interest in PACE and had a home visit but who did not enroll in the program. As mentioned above, comparison group individuals had to meet the PACE eligibility criteria, including the state skilled level of care requirements, even if they were not enrolled in Medicaid, and reside in the catchment area of a PACE site. The findings suggest that costs to Medicare under PACE were considerably lower (38 percent lower in months 1-6, and 16 percent lower in months 7-12) than costs would have been under a FFS arrangement (statistical significance not reported). However, given the reliance on the comparison group to obtain projected cost estimates for PACE enrollees had they not been in PACE, concerns remain as to whether the regressions models were able to adequately account for all important differences between the two groups in Medicare reimbursement. Nonparticipating applicants may have differed from participants on unobserved characteristics that are associated with Medicare costs, leading to selection bias in the estimates of the predicted cost model (e.g., they may have been in greater immediate need than PACE enrollees for expensive health care services, leading to overestimation of what PACE enrollees' Medicare costs would have been). This concern is exacerbated by the fact that the study was also unable to establish baseline equivalence between the two groups (PACE enrollees and the comparison group) on even the observed variables, although the use of multivariate regression models helped to control for these differences. As previously described, the authors attempted to adjust for nonresponse bias by estimating more limited models for a sample of nonrespondents. Finally, the study did not factor in Medicaid payments, so it was unable to compare total capitation rates.

In their analysis of the Medicare payment method for PACE sites, the Medicare Payment Advisory Commission (2012) found that Medicare capitation payments for PACE enrollees exceeded FFS payments for comparable Medicare beneficiaries by 17 percent, on average. As noted by the authors, the Affordable Care Act revised the county benchmarks for the MA payment system to try to ensure that Medicare payments were more closely aligned with FFS spending. PACE providers, however, are still paid on the pre-Affordable Care Act benchmarks, which are on average 17 percent higher than FFS in the counties where PACE providers operate (statistical significance not reported). Although the authors noted that the inclusion of dementia in the risk adjustment model for PACE enrollees and the use of a frailty adjuster for PACE payments could improve the accuracy of the Medicare capitation calculations, additional analyses of Medicare payments revealed that the risk adjustment model together with the frailty adjustor leads to an over-prediction of costs for certain PACE enrollees by more than 17 percent and for some others by less than 17 percent. As such, the authors concluded that 17 percent was a reasonable estimate by which aggregate Medicare spending on PACE enrollees exceeded spending on comparable FFS beneficiaries. In its recommendations to the Congress, the Commission suggested improving the MA risk adjustment models to more accurately predict risk for enrollees and paying PACE providers based on the MA benchmarks and quality bonuses. The findings of this study are limited by the study's inability to establish baseline equivalence and due to the focus on spending comparisons at a single point-in-time that do not take into account length of enrollment in PACE or FFS Medicare.

B. Nursing Home and/or Hospital Utilization

1. Evidence Summary

Evidence on the effect of PACE on hospital and NH utilization is mixed--studies with the strongest design find PACE enrollees have fewer inpatient hospitalizations than their FFS counterparts, but higher rates of NH admission. Also, there is some evidence that program maturity is correlated with greater success in reducing hospitalization rates.

In general, findings regarding the impact of PACE on utilization vary, especially the findings for NH utilization (Table 3). Among the five studies with either moderate to strong or moderate study rating, two included NH utilization outcomes, with divergent findings. Higher rates of NH admission were found among the PACE population than among matched HCBS participants in the moderate to strong study (Beauchamp et al. 2008), while the study with moderate rating reported lower rates of NH admission as well as fewer days in the NH when compared to individuals who expressed interest but did not enroll in PACE (Chatterji et al. 1998). The comparison group in the latter study (Chatterji 1998) consisted of PACE decliners, who could potentially be sicker than the HCBS comparison group used by Beauchamp et al. (2008), and thus more likely to enter a NH. It is plausible that PACE decliners were sicker than PACE enrollees, and that concerns about the adequacy of services under PACE or the difficulty of getting to the PACE site could have prompted them or their caregivers to decline PACE enrollment. Any such difference in baseline sickness levels could be an additional source of bias in the study findings. In sum, based on evidence from the study with the strongest design, it appears that PACE was associated with higher rates of NH admission. However, what remains unclear is whether the effect of PACE differs across short NH stays versus permanent or long-term institutionalization. While Beauchamp et al. (2008) attributed their finding of greater NH use among PACE enrollees to the possibly greater use of NHs for short-stay purposes under PACE, they did not provide any direct evidence for such a phenomenon. Interestingly, Nadash (2004) obtained similar findings of higher NH utilization for PACE enrollees, with the median length of stay being shorter under PACE, although these findings were based on a somewhat weak study design.

TABLE 3. Evidence Summary from Studies Examining the Impact of PACE on Hospital and Nursing Home Utilization
Study	Summary of Findings	EvidenceRating
Both Hospital and Nursing Home Use
Beauchamp et al. (2008)	PACE enrollees had lower rates of hospitalizations but higher rates of NH admissions compared to matched HCBS waiver participants.	Moderate to strong
Chatterji et al. (1998)	PACE enrollees had lower rates of hospitalizations and NH admissions, as well as fewer hospital and NH days compared to PACE decliners.	Moderate
Nadash (2004)	Lower rates of hospital utilization and shorter median length of stay, and higher rates of NH admissions but shorter median length of stay under PACE than the comparison group of MMLTC plan enrollees.	Moderate to weak
Weaver et al. (2008)	Lower rates of inpatient admissions and fewer inpatient days, and higher rates of NH admissions and higher NH days under VA-community partnership with PACE and VA as care manager with care provided by PACE, compared to VA as sole care provider.	Moderate to weak
Mukamel, Bajorska, & Temkin-Greener (2002)	Increased utilization of services by enrollees across 10 PACE sites towards the end of life, with the increase attributable to hospital use, NH use, use of home care, primary physician care, and social services, but dominated by increased hospital use.	Weak
Temkin-Greener, Bajorska, & Mukamel (2008)	Significant association between change in functional status of enrollees and hospital admissions across 29 PACE programs. Sites providing more day center care and more therapy had significantly fewer hospital admissions.	Weak
Hospital Use Only
Kane et al. (2006b)	WPP enrollees from waiver waiting lists had fewer hospital days and fewer preventable hospital admissions relative to the FFS comparison group.	Moderate to strong
Kane et al. (2006a)	Compared to WPP, there were fewer hospital admissions, hospital days, and ED visits under PACE.	Moderate
Meret-Hanke (2011)	PACE enrollees had fewer hospital days compared to frail and community-dwelling elderly not in PACE.	Moderate
Wieland et al. (2000)	Hospital days were comparable for PACE and the general Medicare population, with newer PACE sites showing substantially higher hospital utilization than more mature sites.	Moderate to weak
Division of Health Care Finance & Policy (2005)	PACE enrollees had similar inpatient discharge rates as NH residents, but lower discharge rates than waiver participants; also, fewer ED visits under PACE relative to both comparison groups.	Moderate to weak
NOTE: For each set of outcomes, studies have been listed in descending order of their evidence rating--starting with the strongest possible study.

None of the studies distinguished between the effect of PACE on short-term versus long-term NH stays, which is a significant weakness of the literature in general. Hence, investigating the differential effect of PACE on short-stay versus long-term NH utilization is a promising avenue for future research.

Findings on hospital utilization were less divergent among the same five studies with moderate to strong or moderate ratings; four studies--including one with a moderate to strong design--found significant decreases in hospital utilization for PACE enrollees than in their respective comparison groups. The fifth study did not directly evaluate PACE, since it used Wisconsin Partnership Program (WPP) as a proxy for PACE, but observed comparable inpatient admission rates between WPP enrollees and the comparison group members. Overall, therefore, the evidence from studies with the strongest design suggests that PACE is successful in reducing hospital utilization, and this is corroborated by findings from relatively weaker studies as well. Interestingly, several studies with cross-site comparison of utilization noted that newer PACE sites had substantially higher hospital utilization rates than the more mature sites; thus, it is possible that hospital utilization decreases with PACE program maturity.

2. Detailed Discussion of Studies Examining Hospital and Nursing Home Utilization

Eleven of the 22 papers reviewed examined health care utilization outcomes, specifically the utilization of expensive health care services such as hospitals or NHs. Utilization was typically measured in terms of admission to a NH or hospital and the length of stay. Six studies included both NH and hospital utilization outcomes (Beauchamp et al. 2008; Chatterji et al. 1998; Nadash 2004; Weaver et al. 2008; Mukamel, Bajorska, and Temkin-Greener 2002; Temkin-Greener, Bajorska, and Mukamel 2008), and the remaining five studies looked at hospital utilization only (Meret-Hanke 2011; Wieland et al. 2000; Kane et al. 2006a; Kane et al. 2006b; Division of Health Care Finance and Policy 2005).

It is important to bear in mind that reductions in the use of these expensive services under PACE do not by themselves yield savings to Medicare or Medicaid programs; such savings are determined only by where the capitation rate is set. Favorable effects on these services do, however, provide some indication of whether PACE programs may be able to generate enough savings for themselves to cover the cost of the extensive care coordination benefits they provide. Our discussion of these studies is organized according to the specific utilization outcomes examined by each study--starting with studies examining both NH and hospital utilization, and then studies examining hospital utilization only. Within each subsection, we start by describing the studies with the strongest design, before moving on to the relatively weaker analyses.

a. Nursing Home and Hospital Utilization

The studies examining both NH and hospital utilization outcomes used a variety of comparison group strategies. Two of the six studies received either a moderate or moderate to strong rating on evidence, while four others had moderate to weak or weak evidence, as discussed below.

(i) Studies with Moderate to Strong or Moderate Evidence Rating

Beauchamp et al. (2008) found that PACE enrollees were less likely to have a hospital admission but more likely to use a NH, compared to matched HCBS enrollees. This study used propensity scores to construct a matched comparison group of HCBS waiver enrollees similar to those enrolled in PACE at baseline. The authors interviewed PACE and HCBS enrollees in eight states in 2005, about 1.5-5 years after program enrollment, and for a second time in 2006, approximately one year after the first survey round, comparing unadjusted group mean scores on questions related to a number of different outcomes, including service utilization. There was strong evidence that PACE participants were 25-30 percent less likely to have had a hospital admission than their HCBS counterparts. However, PACE participants were found to have a significantly higher likelihood of NH use that exceeded the likelihood for their HCBS counterparts by 50-60 percent. The authors attributed this finding to the greater use of NHs for short-stay purposes among PACE enrollees, since unlike beneficiaries in Medicare FFS, who require a three-day hospital stay prior to being eligible for Medicare coverage of a skilled nursing facility (SNF) stay, the PACE plan has the flexibility to directly pay for the SNF stay of a PACE enrollee even without a prior hospital stay, especially if the substitution of hospitalization by a SNF stay saves money. Overall survey completion rates were similar across the groups (77 percent and 78 percent on the first survey for the PACE and HCBS respondents; 88 percent for those in PACE, and 85 percent for those in HCBS for the second survey). However, the inability to locate sample members was higher among the HCBS sample in the first interview than it was for the PACE sample. If these reasons for attrition are correlated with outcomes affected by PACE, the study results could be biased. More significant concerns, however, are the inability of propensity score models to account for unobserved baseline differences between the two groups, the short followup period for the sample of new PACE enrollees, and the long gap (1.5-5 years) between program enrollment and the first round of surveys that make the findings representative of only those with longer survival.

Chatterji et al. (1998) found PACE enrollees to have a significantly lower rate of hospitalization and also of NH use. They used multivariate regression to compare PACE participants' service utilization and health status to individuals who expressed interest in PACE and had a home visit but did not enroll in the program. Data was gathered from 11 PACE sites from 1995 through 1997. The authors found those enrolled in PACE to have significantly lower NH utilization and inpatient hospitalization rates than their nonPACE counterparts, with the magnitude of the impact diminishing over time. The benefits of PACE appeared to be higher for participants with high levels of physical impairment--enrollees with the most ADL limitations experienced the largest decreases in service utilization. The lack of baseline equivalence between the two groups on several characteristics (e.g., gender, living arrangements, supportive care arrangements, and health care utilization) suggests that the impact estimates could be confounded by unmeasured differences between the two groups. Further, the comparison group sample size was limited to only 374 individuals, and limited sample sizes were available for the 18-month and 24-month impact estimates for both the treatment and comparison groups, due to mortality and survey nonresponse. For example, the treatment group sample size diminished from 790 to 210 between the 6-month and 24-month followup surveys, while the comparison group sample size diminished from 308 to 86 over the same period. The small sample sizes are particularly troubling for the study findings for subgroups of enrollees defined by impairment level. The authors also noted the possibility of nonrandom attrition; for example, if PACE reduced the likelihood of death for the frailest elders, the PACE sample eventually could include a higher proportion of very frail elders. Although a sensitivity analysis found the estimated PACE impacts to be robust to varying methods of accounting for mortality in the analysis, the very small sample size for the fourth followup limits the study's ability to estimate impacts after two years of enrollment. Further, there were significant differences in response rates across the 11 sites, making the findings nonrepresentative of all sites at large.

(ii) Studies with Moderate to Weak or Weak Evidence Rating

Nadash (2004) found lower hospital utilization under PACE in comparing the health, functional status, and service utilization outcomes of PACE participants with individuals enrolled in a Medicaid-only managed long-term care (MMLTC) plan in the New York City area. The study used administrative data from the plan, data from the outcomes and assessment information set (OASIS) as well as the PACE public use dataset (DataPACE). During the first year of enrollment, total hospital utilization was significantly lower among PACE enrollees--both in terms of a slightly lower likelihood of hospital admission as well as a shorter length of stay. NH utilization was significantly higher for PACE enrollees, although the median length of stay was shorter under PACE. The study, however, suffers from serious limitations. The treatment and comparison populations differed significantly in their sociodemographic characteristics, functional status, and the incidence of chronic conditions by a wider margin than the HCBS and NH comparison groups used in a number of other studies. Also, the authors note significant measurement discrepancies across the two datasets, namely the plan administrative data and DataPACE, stating they may underestimate diagnoses for the MMLTC sample. More importantly, this raises serious concerns about the comparability of the health care utilization outcomes across these two datasets. Furthermore, data was collected from the two groups over two different time periods (individuals in the PACE plans were enrolled between 1996 and 1997; the MMLTC plan sample included individuals enrolled in 2000). Finally, the two samples differ in their geographical location, with the PACE enrollee sample drawn from 12 different sites across the country and the MMLTC plan sample restricted to the New York City area. This raises further questions about the comparability of the service environments faced by the treatment and comparison groups, and therefore, about the validity of the study's findings.

Weaver et al. (2008) studied PACE outcomes within three different U.S. Department of Veterans Affairs (VA) system models, including VA as the sole care provider, a VA-community partnership with PACE, and VA as care manager with care provided by PACE. However, the study sample included only 368 veterans enrolled during 2001-2003, and followed for a maximum of 36 months. The authors reported descriptive statistics highlighting differences in patterns of service use across the three models, both before and after enrollment, with the patterns reflecting the different approaches taken by each site. For example, as expected, participants in the two PACE-related groups used significantly more adult day health care services. NH admissions and days increased after enrollment in all three care models. Hospital utilization, both admissions and number of days, was higher under VA-community partnership with PACE and VA as care manager with care provided by PACE, compared to VA as sole care provider (statistical significance not reported), while the findings were exactly opposite for NH use. Approximately 30 percent of the veterans in the study sample died by the end of evaluation, with around half the deaths occurring in a NH. Among survivors, a large majority resided in the community. Limitations to the study include lack of baseline equivalence including major demographic differences observed across the three groups and differences in disenrollment rates. The authors also reported using incomplete data on Medicare use, which may have resulted in an undercount of health care utilization. The very small sample sizes in each of three groups--ranging from 85 to 181--for a total of 368 veterans in the study sample, along with baseline differences and the lack of regression adjustment limit the validity of this study.

Two studies reported on similar utilization outcomes but did not utilize a nonPACE comparison group. Instead, these studies relied on comparison across PACE sites, and therefore provide somewhat weak evidence on the effect of PACE on health care utilization. Mukamel, Bajorska, and Temkin-Greener (2002) identified individuals in ten PACE programs who died before January 2000. The timeframe of the study varied depending on the site; program data was analyzed from the inception of the program through 2000. Using PACE administrative data (DataPACE), the authors conducted multivariate regression analyses to identify time trends in utilization as well as the relative importance of individual risk factors versus site effects in determining utilization in the last three years before death. They found that utilization of health services among PACE participants increases as early as seven months before death and can mostly be attributed to increased hospital use, NH use, use of home care, primary physician care, and social services. The authors found that variation across program sites explained twice as much of the variation in service utilization towards the end of life as did variation in individual characteristics. However, they did not perform a detailed investigation of program features that could be associated with improved outcomes. Temkin-Greener, Bajorska, and Mukamel (2008) used administrative databases from 29 PACE programs from 2000 through 2002 to compare functional outcomes as they relate to service use within different PACE populations. They documented wide variations in functional status across sites, and they observed substantial differences across programs in risk-adjusted use of health care services. Sites providing more day center care and more therapy were found to have significantly fewer hospital admissions. Since both studies lacked a nonPACE comparison group, their findings, while interesting, provide weak evidence on PACE program effects. Also, the limited nature of the cross-site comparison of program features to identify those associated with improved outcomes reduces the usefulness of this evidence.

b. Hospital Utilization Only

Five studies reported on hospital utilization alone. As for the studies examining both NH and hospital utilization, we discuss these studies by splitting those into two broad groups--those with moderate or moderate to strong rating, and those that had moderate to weak or weak evidence.

(i) Studies with Moderate to Strong or Moderate Evidence Rating

Two studies by Kane et al. (2006a; 2006b) looked at outcomes under the PACE-like WPP. The WPP is a somewhat liberal variant of PACE that allows clients to continue using their regular primary care physicians and does not require use of the adult day center, but otherwise integrates funding from both Medicare and Medicaid to provide primary, acute, and long-term care services to elderly and disabled enrollees and employs a multidisciplinary case management team, similar to PACE.

In the stronger of the two studies, Kane et al. (2006b) used WPP enrollees as the treatment group, comparing them with two different frail elderly populations enrolled in traditional FFS and HCBS waiver programs: one group lived within the WPP service area and the other lived outside of the service area. The study sample was further split into two cohorts, based on whether enrollees in the treatment group (WPP) were from the waiver waiting lists (direct cohort) or transferred from the waiver program (transfer cohort). The sample included 652 individuals in the analyses for the direct cohort (210-220 enrollees in each group: WPP and the two control groups), and 210 enrollees in analyses involving the transfer cohort (70 enrollees in each group). Enrollees were allowed to move into the study sample during the course of the study (1999-2001) and outcomes were measured during the first 12 months after enrollment. Using their matched longitudinal cohort design and regression analyses, the authors found no significant differences in hospital admissions, emergency department (ED) visits, risk of entry into NHs, or in mortality between the treatment and matched comparison groups. Although the analyses involving the direct cohort found WPP enrollees to have fewer hospital days than the comparison group outside the WPP geographic area and fewer preventable hospital admissions than the comparison group within the same area, the small sample sizes used in the study could have posed a problem in the detection of significant impacts, in general.

In the other study, Kane et al. (2006a) used data from site billing records and used a cross-sectional time series approach to compare hospital and emergency service utilization for PACE and WPP enrollees over a two-year study period (1999-2001). PACE enrollees were found to have fewer hospital admissions, preventable hospital admissions, hospital days, ER visits, and preventable ER visits than WPP enrollees. No differences were found between the two groups in the length of their hospital stays. Significant demographic differences between the two groups at enrollment and the unverified quality and completeness of the outcomes data based on site billing records limit the study's findings.

The evidence from the second of these two studies by Kane et al. suggests that PACE reduces hospital utilization when compared to WPP, and evidence from the first study shows that hospital admissions in WPP and traditional FFS waiver programs are comparable. Taken together, therefore, these findings seem to suggest that PACE reduces utilization of inpatient hospital services compared to either WPP or FFS arrangements. However, both of these studies reported significant issues with the claims data used, such as: (1) unverified quality and completeness of the outcomes data based on site billing records (e.g., some of the claims data had to be abstracted from paper claims; due to cost and time constraints, only inpatient claims were abstracted); and (2) the inconsistency in outcome definitions across multiple data sources. Further, the small sample sizes, especially in the first study, could have led to potentially underpowered analyses. The use of WPP as a proxy for PACE in Kane et al. (2006b) is also problematic, since the WPP program intervention is markedly different from the traditional PACE program. For instance, WPP allows enrollees to retain their primary care physicians and does not rely on care provided through a day care center--a key feature of PACE.

Meret-Hanke (2011) compared hospital utilization of PACE participants with that of a propensity-score matched comparison group of frail, community-dwelling older adults from the Medicare Current Beneficiary Survey (MCBS) using multivariate regressions. PACE enrollees were found to have higher average hospital use in the six months before baseline (1.58 days per month alive vs. 1.04 days per month alive), but lower average hospital use during the two-year followup period (0.22 days per month alive vs. 0.80 days per month alive). Data was drawn from two different datasets: DataPACE for individuals newly enrolling in PACE during 1990-1998; and the MCBS during for those MCBS subjects who were first interviewed during 1991-1999. This raises the strong possibility that covariates included in the analysis are not comparably measured. More importantly, the PACE data may have less complete data than MCBS on outcomes, which would seriously confound impact estimates. The study also did not demonstrate baseline equivalence in individual and market characteristics across the two groups.

(ii) Studies with Moderate to Weak or Weak Evidence Rating

Wieland et al. (2000) looked at inpatient hospital utilization outcomes using data on beneficiaries enrolling in PACE from 1990 through 1997. Using a general Medicare aged and disabled beneficiary population as a national comparison group for short-term hospitalization outcomes, the authors found bed-days per 1,000 participants per year in PACE were comparable to the general Medicare population, despite the greater morbidity and disability of PACE participants. The authors found that the newest PACE site included in the study had substantially higher hospital utilization rates than the more mature sites. The authors noted the inherent difficulties associated with comparing the PACE population with the general Medicare population, which may be less similar to PACE participants than other potential comparison groups. As such, the study was unable to use an appropriate comparison group strategy or establish baseline equivalence.

Finally, the study by the Division of Health Care Finance and Policy (2005) used hospital discharge data from Massachusetts for fiscal year (FY) 2004 to compare PACE participants to waiver enrollees and NH residents in a descriptive analysis. PACE enrollees and NH residents were found to have similar inpatient discharge rates, but discharge rates for PACE enrollees were significantly lower than that for waiver participants for both preventable and nonpreventable conditions. PACE enrollees also had lower ED visit rates compared to both the NH and waiver comparison groups. The study was based on comparison of age-adjusted discharge rates, inpatient days, length of stay, and ED visit rates and it neither demonstrated baseline equivalence nor did it statistically control for observed differences across the three groups. Thus, the findings have somewhat limited validity.

C. Quality of Care

1. Evidence Summary

Based on evidence from studies with the strongest design, it appears that PACE improves certain aspects of care quality (e.g., those related to management of specific health issues such as pain) and reduces mortality risk among participants (Table 4). The evidence on lower mortality under PACE from studies with moderate to strong or moderate study designs was not corroborated by findings from studies with weaker designs. The methodology used in these studies was often less rigorous than in studies examining other outcomes, which could be a consequence of the more subjective nature of care quality measurement. Among studies examining health care management and unmet needs, changes in functional status, and program characteristics and quality of care, only one study received a moderate to strong rating (Beauchamp et al. 2008).

TABLE 4. Evidence Summary from Studies Examining the Impact of PACE on Quality of Care
Study	Summary of Findings	EvidenceRating
Health Care Management and Unmet Needs
Beauchamp et al. (2008)	PACE enrollees were more likely to have living wills and were less likely to experience pain that interferes with normal routine relative to the matched waiver participants.	Moderate to strong
Kane, Homyak, & Bershadsky (2002)	Compared to WPP enrollees, PACE enrollees had greater disabilities, greater use of day care services, outpatient rehabilitation, and speech therapy, and comparable levels of unmet needs.	Moderate to weak
Changes in Functional Status
Mukamel, Temkin-Greener, & Clark (1998)	Most significant changes in ADLs occurred within the first 6 months of enrollment, with disability levels remaining stable after that; significant differences across sites, with better outcomes among enrollees at mature sites such as On Lok.	Weak
Temkin-Greener, Bajorska, & Mukamel (2008)	Wide variation in functional status changes across 29 PACE sites, with greater use of hospital services significantly associated with worse functional status outcomes.	Weak
Quality of Care in Relation to Program Characteristics
Mukamel et al. (2006)	Performance of the PACE interdisciplinary care team was significantly associated with better functional status at 3 and 12 months after enrollment, and with better urinary incontinence outcomes.	Weak
Mukamel et al. (2007)	Several program characteristics (e.g., the medical director being a trained geriatrician, having more effective teams, program maturity and size were significantly associated with better functional status); fewer program characteristics were associated with self-assessed health.	Weak
Mortality as Primary Outcome
Wieland et al. (2010)	Longer median survival among PACE enrollees relative to enrollees in NHs or waiver programs.	Moderate
Studies with Survival or Mortality Rate Included in Findings
Mancuso, Yamashiro, & Felver (2005)	Significantly lower mortality rate for PACE enrollees compared to the HCBS comparison group.	Moderate to strong
Kane et al. (2006b)	No significant differences in mortality between WPP and matched FFS enrollees.	Moderate to strong
Chatterji et al. (1998)	PACE enrollees had lower mortality rate than PACE decliners.	Moderate
Mukamel et al. (2006)	No significant association between PACE team performance and mortality.	Weak
Mukamel, Temkin-Greener, & Clark (1998)	Significant predictors of mortality included dementia, urinary incontinence, and living with others versus living alone.	Weak
Studies Examining Mortality Rate in Conjunction with Location of Death
Wieland et al. (2000)	Over a third of PACE enrollees died by the end of study, with only a third of the decedents being hospitalized during the last 6 months before death and only 8 percentage of PACE enrollees dying in the hospital.	Moderate to weak
Weaver et al. (2008)	Across groups, around a third of the enrollees died by the end of the evaluation, with about half the deaths in NH.	Moderate to weak
NOTE: For each set of outcomes, studies have been listed in descending order of their evidence rating--starting with the strongest possible study

Among studies examining mortality, two studies received a moderate to strong rating. Kane et al. (2006b) found no significant differences in mortality between the WPP and other groups of frail elderly individuals; however, the use of WPP as a proxy for PACE is potentially problematic due to the different program requirements. The second study found significantly lower mortality rates among PACE enrollees compared to HCBS clients in Washington (Mancuso, Yamashiro, and Felver 2005). Two other studies receiving moderate ratings (Chatterji et al. 1998; Wieland et al. 2010) found PACE participants to have lower mortality than their comparison group counterparts. Although promising, these studies were limited in their ability to control for selection bias, and therefore, provide somewhat weak evidence of favorable impact of PACE on mortality.

2. Detailed Discussion of Studies Examining Quality of Care

Ten papers included in this review assessed aspects of the quality of care PACE enrollees receive, some of which measured multiple dimensions of care quality. This section is divided into four subsections, depending on the specific dimension of care quality that studies examined: health care management and unmet needs (Beauchamp et al. 2008; Kane, Homyak, and Bershadsky 2002); changes in functional status (Mukamel, Temkin-Greener, and Clark 1998; Temkin-Greener, Bajorska, and Mukamel 2008); the relationship between program characteristics and quality of care (Mukamel et al. 2006; Mukamel et al. 2007); and participant mortality (Wieland et al. 2010; Chatterji et al. 1998; Mukamel, Temkin-Greener, and Clark 1998; Mukamel et al. 2006; Weaver et al. 2008; Wieland et al. 2000). Within each subsection, studies with the strongest design are discussed first, before moving on to those with relatively weaker designs.

a. Health Care Management and Unmet Needs

Beauchamp et al. (2008) found PACE participants to have better overall outcomes than HCBS participants in areas reflecting superior health care management. For example, PACE participants were more likely to have living wills and were less likely to report pain interfering with normal routines. This study received a moderate to strong rating due to its strong study design, although, as discussed previously, the study had issues with attrition and, due to a longitudinal design, may not be representative of participants with shorter survival periods.

Kane, Homyak, and Bershadsky (2002) conducted qualitative interviews with 326 PACE and 303 WPP enrollees in two Wisconsin sites in 2000, finding PACE enrollees to be older and significantly more disabled with greater ADL and instrumental activity of daily living (IADL) dependencies and a higher rate of dementia than WPP enrollees. Given the differences in service arrangements between the two programs, PACE enrollees were significantly more likely to use adult day care and special transportation, but were less likely to receive nurse visits. The pattern of responses for unmet needs was comparable across PACE and WPP enrollees, resulting in no significant differences across the two programs. The use of advance directives was also similar (50 percent) for both groups and there were no significant differences in responses on beneficiary satisfaction items either. The lack of significant differences is perhaps not unexpected, given the limited sample sizes and the fact that the comparison group is a variation on PACE rather than a purely nonPACE alternative.¹⁰ The small geographic area included in this study as well as the lack of regression adjustment and baseline equivalence limits this study's findings.

b. Changes in Functional Status

Neither of the two studies that examined changes in functional status over time for PACE enrollees actually evaluated the impact of PACE on these measures. Mukamel, Temkin-Greener, and Clark (1998) looked at quality of care by comparing changes in functional status across 11 PACE programs from their inception to 1994. Over 50 percent of enrollees were found to remain at the same disability level 18 months after enrollment in PACE, and approximately 25 percent of enrollees improved or deteriorated (with a slightly greater percentage of enrollees improving than deteriorating). The analysis found that enrollees experienced significant changes in their ADLs shortly after admission, but, after the first six months of enrollment, the likelihood of either improvement or deterioration in functional status was very small. The authors found significant differences across PACE programs, however, with better outcomes observed among On Lok enrollees than at other sites, perhaps reflecting the improvements that come with program maturity. Further, the average disability level was found to be quite stable among the more mature PACE plans. This study lacked a nonPACE comparison group and had a limited followup of only 18 months for most sites included in the analysis. Although the study discussed differences in program characteristics that may have led to higher functional status in certain sites, these were somewhat speculative and not backed by methodologically sound analyses.

Using information from administrative databases, Temkin-Greener, Bajorska, and Mukamel (2008) also documented wide variation in functional status changes across 29 PACE sites, ranging from a decline of 0.4 ADL dependencies to an increase of 1.2 ADL dependencies per year. The authors found that greater use of hospital services was significantly associated with worse functional status outcomes and that sites providing greater day care and therapy services had fewer hospitalizations, but similar to Mukamel, Temkin-Greener, and Clark (1998), this analysis lacked a nonPACE comparison group. Further, the study was unable to rule out omitted variable bias in the estimated relationship between greater hospital use and deterioration in functional status.

c. Program Characteristics and Quality of Care

While none of the studies cited in this section provide evidence about the impact of PACE relative to alternative sources of care, they do provide important information about the variation in program implementation and outcomes among PACE sites.

The performance of the interdisciplinary PACE team has been found to be positively correlated with improvement in beneficiary health. Mukamel et al. (2006) used administrative data from 26 PACE programs on new enrollees during 1998-2001, and survey data from 1,209 PACE team members to assess the relationship between team performance and risk-adjusted health outcomes up to 12 months after enrollment across PACE sites. Higher overall self-reported team performance, based on the average score across all team members on survey responses recorded on a five-point Likert scale, was found to be a good predictor of improvement in risk-adjusted health outcomes such as functional status and urinary incontinence. For example, the measure was significantly associated with fewer ADL limitations at three and 12 months after enrollment. Mukamel et al. (2007) used information from a survey on team performance, conducted site visits to interview key staff at program sites, and looked at administrative data on new enrollees in PACE during 1997-2001 to study the relationship between program characteristics including team performance, and health outcomes up to 12 months after enrollment. The authors concluded that program characteristics mattered; for example, PACE enrollees in sites with a full-time medical director and in sites where the medical director was a trained geriatrician were more likely to experience better functional status outcomes, as were enrollees in larger and more mature sites as well as in sites in which the ethnic mix of PACE nonprofessional staff more closely matched that of the enrollees. The site-specific attributes that were correlated with better self-assessed health status included the total number of staff per 100 enrollees, more diverse staff, and program maturity. Both these studies utilized a similar enrollee sample, used a similar methodology, and thus shared similar limitations, including the lack of a nonPACE comparison group and only a limited 12-month followup of outcomes.

d. Mortality

Only one study included in this review used mortality as its primary outcome, although mortality or mortality risk were used as secondary outcomes in seven other studies. The single study using mortality as a primary outcome compared mortality risk and survival between PACE participants, waiver program participants, and NH entrants (Wieland et al. 2010). Five studies included survival or mortality rates in their findings (Chatterji et al. 1998; Kane et al. 2006b; Mancuso, Yamashiro, and Felver 2005; Mukamel, Temkin-Greener, and Clark 1998; Mukamel et al. 2006). Because a primary aim of PACE is to limit institutionalization, two studies discussed mortality rates in conjunction with location of death (Weaver et al. 2008; Wieland et al. 2000).

(i) Studies with Mortality as a Primary Outcome

Using DataPACE and state administrative records to compare long-term survival of PACE enrollees to that of waiver program participants and NH entrants during 1998-2003 in two South Carolina counties, Wieland et al (2010) found that PACE enrollees had a significant survival advantage over the two other groups. Participants were followed until death or up to a maximum of five years post-admission. Mortality risk at admission was assessed using the PACE Prognostic Index (PPI). PACE participants were found to be older and more cognitively impaired than waiver and NH participants; the authors determined the PACE admission mortality risk (72.6 percent; high-to-intermediate) was greater than that of waiver populations (58.8 percent) and similar to that of NH populations (71.6 percent). Despite the higher admission mortality risk of the PACE population, the median survival in PACE was 4.2 years, compared to 2.3 years for NH entrants and 3.5 years for waiver enrollees. After accounting for mortality risk at admission, PACE participants were still found to have a significant survival advantage over waiver enrollees in this study (4.7 years versus 3.4 years among moderate-risk enrollees, and 3 years versus 2 years among high-risk enrollees). In spite of the significant differences in baseline characteristics across the three groups of enrollees, no attempt was made to match participants across programs. However, the fact that PACE was found to have better survival rates than the younger and potentially less-disabled waiver population may mean that the study underestimates the positive effects of PACE on survival at these sites, since mortality risk at baseline may not adequately capture the differences between the two groups. For example, those enrolling in PACE are not homebound--a strong predictor of mortality risk--whereas a nontrivial proportion of HCBS recipients are likely to be homebound.

Among these studies that included survival or mortality rates in their findings, Mancuso, Yamashiro, and Felver (2005) found PACE clients to have a significantly lower risk of dying when compared to HCBS clients. After 12 months of PACE enrollment, 13 percent of PACE participants had died, compared to 19 percent of HCBS participants. The mortality gap grew over time: after three years of enrollment, 29 percent of PACE enrollees had died, compared to 45 percent of HCBS participants. Although the study also used a NH comparison group, the authors determined this group was not a plausible comparison for examining mortality due to systematic differences in mortality risk between PACE and NH clients. Limitations to this evaluation were described previously and include the study's small sample size, especially for the sample of PACE enrollees (N = 227), and the inability to match the treatment and comparison groups on prior service utilization.

However, Kane et al. (2006b) found no significant differences in mortality, relative risk of death, or monthly death rates between WPP enrollees and two other groups of frail elderly patients enrolled in traditional FFS programs or waiver programs, and living within or outside of the WPP service area. The small sample size, geographic limitations, and use of WPP as a proxy for PACE is potentially problematic, since the programs differ significantly in their requirements and care delivery process. Because WPP is a variant of PACE, the real effect of PACE on mortality is hard to judge from this particular study.

Chatterji et al. (1998) compared outcomes of those enrolled in PACE to those who expressed interest in PACE and had a home visit but did not ultimately enroll in the program to find that the average individual would have a median life expectancy of 5.2 years in PACE and 3.9 years in the comparison group. Mortality data was collected from Medicare enrollment files for 1,255 individuals. The sample used in the mortality analysis excluded participants for whom baseline survey data was not available, in order to account for initial differences between the two groups. Over the course of the 2.5-year observation period, 19 percent of PACE enrollees died, compared with 25 percent of comparison individuals. These results were statistically significant at the 5 percent level. As previously discussed, these findings may be affected by self-selection bias affecting who enters PACE as well as by the small comparison group sample of only 374 individuals.

Mukamel et al. (2006) found that self-assessed PACE team performance had no significant association with survival rates. The study used survey data and DataPACE to evaluate the relationship between PACE team performance and mortality, using a Cox proportional hazards model. Sites with more full-time nursing staff members appeared to have lower mortality rates, although not better functional outcomes. The authors discussed that the primary objective of PACE is to improve quality of life, and prolonging survival is not always compatible with this goal, or with patients' treatment preferences. However, they also noted that PACE sites with higher numbers of professional employees per 100 participants did have lower mortality rates, perhaps reflecting a more medically-focused model of care. Limitations to this study include lack of a nonPACE comparison group. A similar study without a nonPACE comparison group and primarily focusing on changes in functional status of enrollees across 11 PACE sites found that significant risk factors for death included dementia, urinary incontinence, and living with others (Mukamel, Temkin-Greener, and Clark 1998).

(ii) Studies Examining Mortality Rates in Conjunction with Location of Death

Wieland et al. (2000) found that only 8 percent of PACE deaths occurred in acute hospitals. This study examined the relationship of hospitalization to mortality across PACE sites for 5,478 beneficiaries enrolling in PACE during 1990-1997. The authors identified place of death and hospital utilization in the six-month interval before death. Of the entire PACE enrollment cohort, one-third died by the end of the followup in March 1997. Over 71 percent of those who died had been hospitalized at least once (ranging from 60 percent to over 90 percent for individual sites). However, as mentioned above, only 8 percent of PACE deaths occurred in acute hospitals; also, less than one-third of decedents spent any time in the hospital in the six-month interval before death. As observed by the authors, these rates were markedly lower than that for Medicare beneficiaries in general. For instance, in the general Medicare population, between 23 percent and 54 percent of deaths were found to occur in acute hospitals, varying across geographic location. The authors note that the comparatively low overall rate of death in hospitals in PACE suggests that the program may be successful in both eliciting and implementing directives for end-of-life care, although more research was needed on the role of various factors in determining end-of-life decisions in PACE. It is interesting to note that Beauchamp et al. (2008) did find that PACE participants were significantly more likely to have living wills, and Kane, Homyak, and Bershadsky (2002) found a high rate of use of advance directives (50 percent) among both PACE and WPP enrollees. Both of these findings may be the result of self-selection rather than a true program effect if individuals more likely to have an advanced directive or living will are also more likely to enroll in PACE or WPP.

Weaver et al. (2008) studied PACE outcomes across three different VA systems. At the end of the three-year followup, approximately 30 percent of the overall sample had died. Across all three groups, fewer than half of all deaths occurred in a NH, with the individual rates varying across the three groups: 46 percent of deaths using the VA as the sole care provider, 50 percent of those enrolled in a VA-community partnership with PACE, and 55 percent of those enrolled in a VA as care manager with care provided by PACE. Given the similar levels of frailty and long-term care needs of this population and the general PACE population, it is interesting to note that the percentage of deaths occurring in a NH, as reported by this study focusing on veterans, far exceeds the percentage of deaths in an acute hospital for PACE enrollees, as reported in the study by Wieland et al. (2000) and discussed above. The authors of the VA study, however, did not report statistical significance of their findings. Also, as previously described, the study suffered from a small sample size, lack of baseline equivalence, and lack of regression adjustment.

D. Participant Satisfaction and Quality of Life

1. Evidence Summary

Based on the evidence in the literature (reviewed below), it is difficult to arrive at any concrete conclusions regarding the impact of PACE on patient satisfaction and quality of life. It appears that PACE participants are generally satisfied with their medical and personal care, but not significantly more so than people receiving HCBS (Table 5). This is perhaps expected, since it merely points to the fact that beneficiaries receiving health care and support services--whether from PACE or through HCBS waiver programs--normally are satisfied with these services. Only one of the studies in this group was rated as moderate to strong (Beauchamp et al. 2008); however, this study found few significant differences between PACE and the HCBS comparison groups. Another study with a moderate study rating found positive effects of the PACE program on satisfaction with life and satisfaction with care (Chatterji et al. 1998). Interestingly, in this study, PACE enrollees were found to spend more days in the community than PACE decliners--a result that is in line with the positive findings for satisfaction, since spending time in institutional care is likely to reduce patient satisfaction and quality of life. Caregiver satisfaction and quality of life is not included as an outcome in any of these studies.

TABLE 5. Evidence Summary from Studies Examining the Impact of PACE on Participant Satisfaction and Quality of Life
Study	Summary of Findings	EvidenceRating
Health Care Management and Unmet Needs
Beauchamp et al. (2008)	Comparable levels of satisfaction with care received across PACE and matched HCBS enrollees.	Moderate to strong
Chatterji et al. (1998)	Greater satisfaction with life and with care arrangements among PACE enrollees versus PACE decliners.	Moderate
Kane, Homyak, & Bershadsky (2002)	Comparable levels of satisfaction with services received among PACE and WPP enrollees.	Moderate to weak
NOTE: Studies have been listed in descending order of their evidence rating--starting with the strongest possible study.

2. Detailed Discussion of Studies Examining Participant Satisfaction and Quality of Life

Three of the 24 studies included in this review surveyed participants' satisfaction and quality of life. These outcomes were measured differently across the studies, but, in general, participants or a proxy in cases where participants were too cognitively impaired to respond, were asked directly about their levels of satisfaction with different aspects of their medical and personal care, as well as questions about their overall quality of life. All three studies included used a primarily community-based comparison group, with one using a community-based sample eligible for PACE but not enrolled (Beauchamp et al. 2008; Chatterji et al. 1998; Kane, Homyak, and Bershadsky 2002).

Both PACE and HCBS participants have been found to report high levels of satisfaction with the care they received. Beauchamp et al. (2008) surveyed PACE and HCBS participants about their overall satisfaction with the quality of care they received, satisfaction with their medical care, and satisfaction with personal assistance. Within-group differences were found over time; however, across all time periods, there were very few measurable differences across the treatment and comparison groups. Both PACE and HCBS participants reported very high levels of satisfaction with the care they received, with limited improvement or decline over time across most of the measures. Of the few significant differences in these measures overall, some favored PACE participants and some favored HCBS participants. Limitations to the study include the previously-mentioned issue with nonrandom attrition in a longitudinal study setting.

Chatterji et al. (1998) found PACE to be associated with short-run improvements in quality of life and in satisfaction with care. They compared survey responses from PACE enrollees and individuals who expressed interest in PACE, received a home visit, but who did not enroll in the program. The comparison group was restricted to individuals who met the state skilled level of care criteria and were eligible for the PACE program, regardless of whether they were enrolled in Medicaid. Survey participants were asked about their satisfaction and quality of life, as measured by the probability of saying they were in good or excellent health, the probability of finding life to be satisfying, and the probability of being very satisfied with their overall care arrangements. Six months after baseline, 72 percent of PACE enrollees and 55 percent of nonPACE enrollees reported that their lives were at least "pretty satisfying," with the difference significant at the 1 percent level. The magnitude and significance of this difference was maintained at the 12-month followup survey. By the 18 month and 24 month surveys, PACE enrollees still had higher satisfaction with life, but the differences were no longer statistically significant. Also, six months after baseline, PACE enrollees were significantly more likely than PACE decliners to report being "very satisfied" with their overall care arrangements, and very few respondents in either group reported being dissatisfied with their care arrangements. Study limitations include the lack of baseline equivalence across the two groups, the small comparison group sample size, and the somewhat outdated nature of the findings.

Kane, Homyak, and Bershadsky (2002) assessed levels of satisfaction among both clients and family members of PACE and WPP enrollees. Satisfaction was measured in terms of receipt of services and the nature of the services received (for example, "Were services provided when needed?" "Did personnel communicate effectively?"). Levels of satisfaction among these two groups were essentially comparable, with none of the beneficiary satisfaction items showing a significant difference between WPP and PACE clients. Because WPP is more similar to PACE than many of the other comparison groups used across PACE studies, and the study was limited by small sample sizes in both the PACE (N = 326) and the WPP groups (N = 303), the lack of significant differences in this study's findings are perhaps not surprising.

V. CONCLUSIONS

There are significant challenges in evaluating PACE, given the characteristics of the program and its beneficiaries, the most significant of such challenges being the identification of an appropriate comparison group.
Without any randomized evaluation, no study offers strong evidence on the effectiveness of PACE.
Most quasi-experimental studies of PACE fail to meet the standards of a rigorous evaluation. Only four of the 22 studies included in this review met standards for offering "moderate to strong" evidence on the effects of PACE, and seven other studies could only be rated as offering "moderate" evidence, given their inability to establish baseline equivalence between the treatment and comparison groups. Half (11) the reviewed studies received either "moderate to weak" (six studies) or "weak" (five studies) ratings, bringing into question the credibility of much of the prior findings on PACE.
Based on evidence from studies with the strongest design, we found that PACE has no significant effect on Medicare costs, but it is associated with significantly higher Medicaid costs, with the Medicaid spending gap between PACE and matched comparison enrollees decreasing over time. Therefore, based on current evidence, we conclude that PACE does not save costs for either program, and it raises overall cost through an increase in Medicaid expenditures. However, prior findings on Medicare and Medicaid costs need to be updated, given changes to the Medicare capitation payment approach as well as variation in the Medicaid capitation rate calculations across states.
Evidence on the effect of PACE on the utilization of expensive acute and long-term care services is mixed--studies with the strongest design find PACE enrollees have fewer inpatient hospitalizations than their FFS counterparts, but they appear to have higher rates of NH admission. Also, there is some evidence that program maturity is correlated with greater success in reducing hospitalization rates. Although greater care coordination in PACE could reduce enrollees' need for hospitalizations, some of these comparisons may be distorted by the substitution of short-term nursing facility stays for hospitalizations under PACE--a neglected aspect of research on the effect of PACE on NH utilization.
There is some evidence that PACE improves certain aspects of care quality--for example, those related to management of specific health issues such as pain; also, based on a single study with a strong research design, it appears that PACE enrollees have lower mortality rates over the period 1-4 years after enrollment, a finding corroborated by results from two other studies with a relatively weak design.
Although PACE participants are satisfied with their medical and personal care, there is insufficient evidence as to whether satisfaction and quality of life actually improves under PACE or not, since the only study with a moderate to strong rating found few significant differences between PACE and the HCBS comparison group in patient satisfaction and quality of life.

Overall, the only outcome for which we found strong evidence of favorable effects under PACE is inpatient hospitalizations. However, such reductions in inpatient hospitalizations will not actually translate into cost savings for Medicare unless the capitation rate is set at a level below what enrollees would have cost in FFS. It is interesting to note that Temkin-Greener, Meiners, and Gruenberg (2001) used data from 12 PACE sites to model the potential impact of the Principal Inpatient-Diagnostic Cost Group (PIP-DCG) payment model on the amount Medicare pays to PACE programs. The PIP-DCG model was predicted to result in a 38 percent reduction in Medicare payments to PACE over the rate structure prevailing at that time (1995-1998). However, risk adjustment models estimated on hospital diagnoses alone do not capture the effects of frailty on expected costs for this population or the effects of other diagnoses that may not result in hospitalizations but do affect expected future costs. Furthermore, the authors noted that under a PIP-DCG risk adjuster, PACE sites would be financially penalized for reducing hospitalizations. The PIP-DCG model was never used in reality to determine capitation payments to PACE plans.

Also, the fact that the evidence from prior studies suggests that PACE was associated with higher NH utilization and greater costs to Medicaid is clearly a cause for concern, though it also raises questions about the validity of the estimates, given the central focus of PACE on enabling enrollees to remain in the community. As mentioned above, PACE plans are responsible for covering all health care services, including long-term care utilization by their enrollees, and given capitated payments, has little incentive to promote greater use of long-term NH services. Given this apparently anomalous finding in the previous literature, and the possibility of substituting hospitalizations by short-stay NH use, investigating the differential effect of PACE on short-stay versus long-term NH utilization is a promising avenue for future research. This is especially relevant since existing studies have not distinguished between the effect of PACE on short-term versus long-term NH use. Also, in future research, it would be important to reexamine the effect of PACE on both Medicare and Medicaid costs, especially with longer followup duration of at least five years for both types of costs. Findings from such studies are likely to be especially relevant to policymakers, given recent changes in the approach to Medicare capitation payment to PACE plans.¹¹ Although there was some evidence for improvement in care quality and reduction in mortality risk under PACE, these findings are based on a single study in each case with a strong research design. Hence, there is a need for more rigorous evaluations of PACE with respect to its impact on a range of outcomes, including cost, utilization, mortality, and care quality.

However, in looking at the effect of PACE on a range of outcomes, it is important to be aware of a fundamental distinction between impact estimates for cost outcomes and all other outcomes, such as utilization, mortality, quality of care, or quality of life. All outcomes except cost are potentially affected by the care provided by the PACE program, for example, through the interventions it uses, the quality of its staff, or the degree of care coordination. Hence, PACE's impacts on these outcomes are determined by how well the PACE plans do their job. In contrast, the impact estimates for Medicare and Medicaid costs depend only on the difference between: (a) Medicare and Medicaid capitation rates for PACE; and (b) the projected costs based on the experience of the comparison group. That is, the PACE plans and the quality of their interventions have no bearing on whether PACE increases or decreases Medicare or Medicaid costs.

There are other caveats that apply to the findings from this literature review. For instance, several studies included in this review have significant limitations in terms of external validity or the generalizability of their findings and in their continued relevance for assessing the current PACE model. These limitations arise due to multiple reasons such as study setting, small sample sizes, and study timing. Several studies examined outcomes of PACE and comparison group enrollees in a single state, such as Massachusetts (Division of Health Care Finance and Policy 2005), South Carolina (Wieland et al. 2010; Wieland et al. 2012), Washington (Mancuso, Yamashiro, and Felver 2005), or Wisconsin (Kane, Homyak, and Bershadsky 2002; Kane et al. 2006a, 2006b), and had limited sample sizes. This is especially likely to be a problem for examining outcomes such as Medicaid expenditures or utilization of services covered by Medicaid. As noted by the authors of several of these studies, PACE PBPM Medicaid rates vary quite substantially across states, with some states paying three times as much as others. Similarly, Medicaid benefits and reimbursement rates also vary widely across states that could affect both costs and utilization in the comparison group. Hence, findings from studies on PACE and comparison group enrollees in a single state have limited external validity.

Even among studies focusing on PACE enrollees across multiple states, sample sizes were often quite small--especially in studies based on surveys (e.g., Chatterji et al. 1998 and Beauchamp et al. 2008 had only around a total of 1,000 treatment and comparison group members). Also, in the study by Foster, Schmitz, and Kemper (2007), the analysis had to be limited to nine states that had both PACE and HCBS programs in 1999 and that submitted Medicaid Statistical Information System data on a timely basis; hence, even for this broader-based study, the sample is unlikely to be representative of the full PACE population. As the findings from studies comparing PACE sites show, differences in site characteristics and program implementation can affect outcomes. Consequently, given the inherent differences across PACE sites, studies that focus on a limited number of sites may not offer conclusions that generalize to the broader PACE population. Several studies were also limited by either high rates of survey nonresponse, or incompleteness in claims data and their inability to link claims across multiple data sources, thereby losing significant portions of the original sample in the outcomes analysis (Chatterji et al. 1998; Beauchamp et al. 2008; Weaver et al. 2008; White 1998; White et al. 2000), potentially rendering the sample nonrepresentative.

Also, findings from prior evaluations looking at potential cost savings under PACE are likely to be outdated, given changes to the PACE financing structure over time, especially for Medicare capitation payments. Most studies examining the effect of PACE on Medicare costs were completed prior to 2007, when these changes began to be implemented. The only exception is the MedPAC (2012) study, which suffers from other methodological concerns, but is based on current cost data. Also, states differ in their approach to setting the Medicaid capitation payment, and some states have experimented with alternative approaches over time. As such, the cost findings from most of the studies included in this review have somewhat limited relevance. In general, the literature on evaluations of PACE is somewhat old, with only seven of the 22 studies published within the previous five years (2008-2012), and another seven published more than a decade back, prior to 2003. For instance, data collection for the studies by Chatterji et al. (1998), White (1998), and White et al. (2000) were conducted over 15 years ago, and thus are unlikely to be representative of current PACE programs.

REFERENCES

Beauchamp, Jody, Valerie Cheh, Robert Schmitz, Peter Kemper, and John Hall. "The Effect of the Program of All-Inclusive Care for the Elderly (PACE) on Quality." Report submitted to the Centers for Medicare and Medicaid Services. Princeton, NJ: Mathematica Policy Research, February 12, 2008.

Bloom, Shawn, Brenda Sulick, and Jennie Chin Hansen. "Picking up the PACE: The Affordable Care Act Can Grow and Expand a Proven Model of Care." Journal of the American Society on Aging, 2011, vol. 35, no. 1, pp.53-55.

Bodenheimer, T. "Long-Term Care for Frail Elderly People--The On Lok Model." New England Journal of Medicine, 1999, vol. 341, no. 17, pp. 1324-1328.

Chatterji, Pinka, Nancy R. Burstein, David Kidder, and Alan White. "Evaluation of the Program of All-Inclusive Care for the Elder (PACE) Demonstration: The Impact of PACE on Participant Outcomes." Report submitted to the Health Care Financing Administration. Cambridge, MA: Abt Associates, Inc., July 1998.

Division of Health Care Finance and Policy. "PACE Evaluation Summary." August 2005.

Eng, C. "Future Considerations for Improving End-of-Life Care for Older Persons: Program of All-Inclusive Care for the Elderly (PACE)." Journal of Palliative Medicine, 2002, vol. 5, no. 2, pp. 305-309.

Foster, Leslie, Robert Schmitz, and Peter Kemper. "The Effects of PACE on Medicare and Medicaid Expenditures." Report submitted to the Centers for Medicare and Medicaid Services. Princeton, NJ: Mathematica Policy Research, August 29, 2007.

Galantowicz, Sara. "Summary of Evidence Basis for the Program of All-Inclusive Care for the Elderly (PACE)." Memorandum to the Centers for Medicare and Medicaid Services. Thomson Reuters. August 2, 2011.

Hirth, V., J. Baskins, and M. Dever-Bumba. "Program of All-Inclusive Care (PACE): Past, Present, and Future." Journal of American Medical Directors Association, 2009, vol. 10, pp. 155-160.

Kane, Robert L., Patty Homyak, and Boris Bershadsky. "Consumer Reactions to the Wisconsin Partnership Program and Its Parent, the Program for All-Inclusive Care of the Elderly (PACE)." The Gerontologist, 2002, vol. 42, no. 3, pp. 314-320.

Kane, Robert L., Patricia Homyak, Boris Bershadsky, and Shannon Flood. "Variations on a Theme Called PACE." Journal of Gerontology, 2006a, vol. 61A, no. 7, pp. 689-693.

Kane, Robert L., Patricia Homyak, Boris Bershadsky, and Shannon Flood. "The Effects of a Variant of the Program for All-inclusive Care of the Elderly on Hospital Utilization and Outcomes." Journal of the American Geriatrics Society, 2006b, vol. 54, pp. 276-283.

Mancuso, David, Greg Yamashiro, and Barbara Felver. "PACE: An Evaluation." Washington State Department of Social and Health Services Research and Data Analysis Division. Report Number 8.26, June 29, 2005.

Medicare Payment Advisory Commission (MedPAC). "Care Coordination Program for Dual-Eligible Beneficiaries." Chapter 3. Report to the Congress: Medicare and the Health Care Delivery System. Washington, DC: MedPAC. June 2012.

Medicare Payment Advisory Commission. "Report to the Congress: Medicare and the Health Care Delivery System." June 2012. Available at: http://www.medpac.gov/documents/Jun12_EntireReport.pdf. Accessed November 2, 2012.

Meret-Hanke, Louise. "Effects of the Program of All-Inclusive Care for the Elderly on Hospital Use." The Gerontologist, 2011, vol. 51, no. 6, pp. 774-785.

Mukamel, Dana B., Helena Temkin-Greener, and Marleen L. Clark. "Stability of Disability Among PACE Enrollees: Financial and Programmatic Implications." Health Care Financing Review, Spring 1998, vol. 19, no. 3.

Mukamel, Dana B., Alina Bajorska, and Helena Temkin-Greener. "Health Care Services Utilization at the End of Life in a Managed Care Program Integrating Acute and Long-Term Care." Medical Care, 2002, vol. 40, no. 12, pp. 1136-1148.

Mukamel, Dana B., Helena Temkin-Greener, Rachel Delavan, Derick R. Peterson, Diane Gross, Stephen Kunitz, and T. Franklin Williams. "Team Performance and Risk-Adjusted Health Outcomes in the Program of All-Inclusive Care for the Elderly (PACE)." The Gerontologist, 2006, vol. 46, no. 2, pp. 227-237.

Mukamel, Dana B., Derick R. Peterson, Helena Temkin-Greener, Rachel Delavan, Diane Gross, Stephen J. Kunitz, and T. Franklin Williams. "Program Characteristics and Enrollees' Outcomes in the Program of All-Inclusive Care for the Elderly (PACE)." The Milbank Quarterly, 2007, vol. 85, no. 3, pp. 499-531.

Nadash, Pamela. "Two Models of Managed Long-Term Care: Comparing PACE With a Medicaid-Only Plan." The Gerontologist, 2004, vol. 44, no. 5, pp. 644-654.

National PACE Association. "PACE in the States." Alexandria, VA: National PACE Association. 2012. Available at: http://www.npaonline.org/website/article.asp?id=12. [List of PACE Programs by State, http://www.npaonline.org/website/navdispatch.asp?id=1741.]

Robinson, James and Sarita L. Karon. "Modeling Medicare Costs of PACE Populations." Health Care Financing Review, Spring 2000, vol. 21, no. 3.

Temkin-Greener, Helena, Mark R. Meiners, and Leonard Gruenberg. "PACE and the Medicare+Choice Risk-Adjusted Payment Model." Inquiry, Spring 2001, vol. 38, pp. 60-72.

Temkin-Greener, Helena, Alina Bajorska, and Dana B. Mukamel. "Variations in Service Use in the Program of All-Inclusive Care for the Elderly (PACE): Is More Better"? Journal of Gerontology, 2008, vol. 63A, no. 7, pp. 731-738.

Weaver, Frances M., Elaine C. Hickey, Susan L. Hughes, Vicky Parker, Dawn Fortunato, Julia Rose, Steven Cohen, Laurence Robbins, Willie Orr, Beverly Priefer, Darryl Wieland, and Judith Baskins. "Providing All-Inclusive Care for Frail Elderly Veterans: Evaluation of Three Models of Care." Journal of the American Geriatric Society, 2008, vol. 56, pp. 345-353.

White, Alan J. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE): The Effect of PACE on Costs to Medicare: A Comparison of Medicare Capitation Rates to Projected Costs in the Absence of PACE." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc. July 1998.

White, Alan J., Yvonne Abel, and David Kider. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE): A Comparison of the PACE Capitation Rates to Projected Costs in the First Year of Enrollment." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc. October 2000.

Wieland, Darryl, Vicki L. Lamb, Shae R. Sutton, Rebecca Boland, Marleen Clark, Susan Friedman, Kenneth Brummel-Smith, and G. Paul Eleazer. "Hospitalization in the Program of All-Inclusive Care for the Elderly (PACE): Rates, Concomitants, and Predictors." Journal of the American Geriatrics Society, 2000, vol. 48, pp. 1373-1380.

Wieland, Darryl, Rebecca Boland, Judith Baskins, and Bruce Kinosian. "Five-Year Survival in a Program of All-Inclusive Care For Elderly Compared With Alternative Institutional and Home- and Community-Based Care." Journal of Gerontology, 2010, vol. 65, no. 7, pp. 721-726.

Wieland, Darryl, Bruce Kinosian, Eric Stallard, and Rebecca Boland. "Does Medicaid Pay More to a Program of All-Inclusive Care for the Elderly (PACE) Than for Fee-for-Service Long-term Care?" Journal of Gerontology, 2012.¹²

APPENDIX A. SUMMARY OF PRIOR PACE EVALUATIONS AND THEIR STRENGTH OF EVIDENCE

I. Introduction

This appendix contains a summary table and detailed descriptions of 22 PACE evaluations. Studies were identified for inclusion in this literature review through a systematic search of the following databases: Ovid, PubMed, EBSCOhost, Google Scholar, and the Mathematica Integrated Library. We used keywords relevant to PACE and our outcomes of interest (costs, NH and/or hospital admission, quality of care, mortality, and participant satisfaction), and we limited the search to publications between 1997 and 2012. Through this process, we identified 58 potentially relevant papers, all of which are listed in Section IV. We then screened abstracts using criteria such as timeframe, outcomes, and comparison group design, resulting in a total of 22 relevant studies. We reviewed all aspects of each paper, include the timeframe, evaluation design, sample, data source(s), outcomes, methods of impact evaluation, and key findings. We then performed a thorough assessment of each study's research design, evaluated the potential threats to validity of the findings, and rated each paper based on the strength of its research evidence (see text for explanation of these ratings; all studies with a comparison group were classified as "moderate" or stronger). Studies included in this appendix are listed in alphabetical order by first author. The summary table below shows for each study the outcomes examined (quality of care measures include changes in functional status and mortality), the sample size, and our rating of the analytic strength of the study.

II. Summary Table

Study	Outcomes					Sample Size		Rating
Study	Costs	NursingHome Utilization	Hospital Utilization	Quality ofCare	Participant Satisfaction & Qualityof Life	Treatment	Comparison	Rating
Beauchamp et al. (2008)		X	X	X	X	740	475	Moderate to strong
Chatterji et al. (1998)		X	X	X	X	210-790	86-308	Moderate
Division of Health Care Finance & Policy (2005)			X			898	1,649-12,767	Moderateto weak
Foster, Schmitz, & Kemper (2007)	X					1,503	1,050	Moderateto strong
Kane, Homyak, & Bershadsky (2002)				X	X	326	303	Moderateto weak
Kane et al. (2006a)			X			651	634	Moderate
Kane et al. (2006b)		X	X	X		70-213	70-220	Moderateto strong
Mancuso, Yamashiro, & Felver (2005)	X			X		227	399-1,891	Moderateto strong
MedPAC (2012)	X					Not reported	Not reported	Moderateto weak
Meret-Hanke (2011)			X			3,889	3,103	Moderate
Mukamel, Temkin-Greener, & Clark (1998)				X		2,291 across 11 PACE sites		Weak
Mukamel, Bajorska, & Temkin-Greener (2002)		X	X			2,160 across 10 PACE sites		Weak
Mukamel et al. (2006)				X		1,209 across 26 PACE sites		Weak
Mukamel et al. (2007)				X		3,042 across 23 PACE sites		Weak
Nadash (2004)		X	X			1,382	1,297	Moderateto weak
Temkin-Greener, Bajorska, & Mukamel (2008)		X	X	X		9,853 across 29 PACE sites		Weak
Weaver et al. (2008)		X	X	X		85-181 in 3 VA groups with varying degrees of care under PACE		Moderateto weak
White (1998)	X					1,519	765	Moderate
White, Abel, & Kider (2000)	X					1,367	381-671	Moderate
Wieland et al. (2000)			X	X		5,478	4.7-33.4 million	Moderateto weak
Wieland et al. (2010)				X		554	468-1,018	Moderate
Wieland et al. (2012)	X					948	1,357-1,683	Moderate

III. Studies Included in Review

1. Beauchamp, Jody, Valerie Cheh, Robert Schmitz, Peter Kemper, and John Hall. "The Effect of the Program of All-Inclusive Care for the Elderly (PACE) on Quality." Report submitted to the Centers for Medicare and Medicaid Services. Princeton, NJ: Mathematica Policy Research, February 12, 2008.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	Yes
Timeframe	First survey in 2005 (1.5-5 years after program enrollment)Second survey in 2006 (3-6 years after program enrollment)
Sample	1,215 individuals in 8 study statesTreatment group: 740 PACE enrolleesComparison group: 475 HCBS enrollees
Data Source(s)	Interviews with PACE and HCBS enrollees
Methods	Propensity score matching; comparison of unadjusted and regression-adjusted group means.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: PACE enrollees had higher rates of NH admission (8 and 10 percentage points more likely to have had a NH stay in the previous year at first and second interviews respectively).
	*Hospital Utilization*: PACE enrollees had lower rates of hospitalization (11 and 9 percentage points less likely to have had a hospitalization in the previous year at first and second interviews respectively).
	*Other Service Utilization*: PACE enrollees had greater utilization of preventive health care services (21 and 14 percentage points more likely to have had a flu shot since last September at first and second interviews respectively; 29 and 25 percentage points more likely to have had a hearing screening in the previous year at the time of the first and second interviews respectively).
	*Quality of Care*: PACE enrollees had better health care management (6 percentage points more likely to have both an advanced directive and a living will at first interview; 12 and 9 percentage points more likely to have either an advanced directive or a living will at first and second interviews respectively; 11 and 8 percentage points less likely to experience pain that interferes with normal routine at first and second interviews respectively).
	*Participant Satisfaction and Quality of Life*: PACE enrollees had better self-reported health status (on a scale of 1-5, 0.21 points better self-reported health status at first interview); greater likelihood of behavioral problems (e.g., 9 percentage points more likely to be delirious, confused, or hallucinated at first interview); and lower likelihood of depression (10 percentage points less likely to be depressed in the past month at first interview, and 11 and 10 percentage points less likely to be worried a lot in the past month at first and second interviews respectively).
Threats of Validity	Unmeasured differences between the PACE and HCBS comparison samples that could remain even after propensity score matching. Nonrandom attrition due to mortality and the long gap between enrollment and first interview makes the analysis and findings representative of only those with longer survival. Since the first interview took place after enrollment, and no claims data were available, true baseline differences between the two groups in their survey responses are not captured.
Strength of Evidence	Moderate to strong

2. Chatterji, Pinka, Nancy R. Burstein, David Kidder, and Alan White. "Evaluation of the Program of All-Inclusive Care for the Elder (PACE) Demonstration: The Impact of PACE on Participant Outcomes." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates, Inc., July 1998.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	Baseline interview during 1995-1997, with followup surveys at 6, 12, 18, and 24 months after the baseline interview
Sample	1,255 individuals across 11 sites, with the sample size decreasing from 1,098 to 296 between the 6-month and 24-month followup surveysTreatment group: 881 PACE enrollees (sample size varies depending on interval, decreasing from 790 to 210 between the 6-month and 24-month followup surveys)Comparison group: 374 individuals who expressed interest in PACE and had a home visit but did not enroll (sample size varies depending on interval, decreasing from 308 to 86 between the 6-month and 24-month followup surveys)
Data Source(s)	5-round survey of PACE applicants; proxy sources surveyed as necessaryMedicare enrollment database for information on date of death
Methods	Multivariate regressions with baseline controls.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: PACE enrollees had lower rates of NH admissions (68% and 52% lower than comparison group at first and second followups); and fewer NH days (71% and 60% lower than comparison group at first and second followups).
	*Hospital Utilization*: PACE enrollees had lower rates of inpatient hospitalization (50% and 40% lower than comparison group at first and second followups); and fewer hospital days (69% and 66% lower than comparison group at first and second followups).
	*Other Service Utilization*: PACE enrollees had higher utilization of ambulatory services (25% higher at first followup than the comparison group).
	*Quality of Care*: PACE enrollees had lower mortality rates (24% lower than the comparison group over a 2.5 year followup).
	*Participant Satisfaction and Quality of Life*: PACE enrollees had better health status (16% more likely to be in good or excellent health at first followup); higher quality of life (31% and 18% more likely to report lives were at least pretty satisfying than comparison group members at first and second followups); greater satisfaction with care arrangements (24% more likely to report being very satisfied at first followup); and greater participation in social, religious, and recreational programs.
Threats of Validity	Lack of baseline equivalence between the 2 groups. Nonrandom attrition due to mortality leading to changes in sample composition over time; findings representative of those with longer survival. The very small sample size for the fourth followup limits the study's ability to estimate impacts after 2 years of enrollment.Significant variation in response rates across sites.Data over a decade old.
Strength of Evidence	Moderate

3. Division of Health Care Finance and Policy. "PACE Evaluation Summary." August 2005.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	FY 2004
Sample	15,314 individuals in MassachusettsTreatment group: 898 PACE enrolleesComparison group 1: 1,649 waiver enrolleesComparison group 2: 12,767 NH residents
Data Source(s)	FY 2004 hospital discharge data
Methods	Comparison of group means using descriptive statistics.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: PACE enrollees had similar inpatient discharge rates as NH residents, but lower discharge rates than the waiver group (14.5 preventable discharges per 100 enrollees for PACE, versus 15.6 for NH residents and 27.3 for waiver enrollees); lower inpatient days than both comparison groups; lower average lengths of stay; and lower outpatient ED visit rates.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Lack of baseline equivalence.Lack of regression adjustment for differences across groups.Sample limited to Massachusetts.
Strength of Evidence	Moderate to weak

4. Foster, Leslie, Robert Schmitz, and Peter Kemper. "The Effects of PACE on Medicare and Medicaid Expenditures." Report submitted to the Centers for Medicare and Medicaid Services. Princeton, NJ: Mathematica Policy Research, August 29, 2007.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	Yes
Timeframe	1999-2004, with 2 years of followup for Medicaid costs and 5 years of followup for Medicare costs
Sample	2,553 individuals in 9 statesTreatment group: 1,503 PACE enrolleesComparison group: 1,050 HCBS enrollees
Data Source(s)	Medicare enrollment and demographic data, Medicare claims data, Medicare county rate books for managed care plans, and state-specific Medicaid Analytic eXtract files
Methods	Propensity score matching; comparison of actual capitation payments against regression-adjusted mean expenditures over successive 6-month intervals from the month of enrollment.
Key Findings	*Costs*: Medicare capitation payments for PACE were similar to monthly Medicare expenditures for the comparison group over all 6-month intervals (except during months 7-12 when PACE payments exceeded comparison group expenditures by $180 per month, significant at the 10% level); Medicaid capitation payments for PACE were significantly greater than monthly Medicaid expenditures for the comparison group over all 4 intervals (the difference, significant at the 1% level in each time interval, decreased from $926 during months 1-6 to $536 during months 19-24); and adjusted and unadjusted estimates were comparable.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	In spite of propensity score matching, unmeasured differences between the groups could potentially bias impact estimates. Excluded 22% of potential study sample of PACE enrollees due to prior enrollment in Medicare managed care.Medicaid expenditures in the comparison group could be underestimated due to the noninclusion of certain services being in claims data. Sample may not be representative of broader PACE population. Limited followup of 2 years for Medicaid costs.
Strength of Evidence	Moderate to strong

5. Kane, Robert L., Patty Homyak, and Boris Bershadsky. "Consumer Reactions to the Wisconsin Partnership Program and Its Parent, the Program for All-Inclusive Care of the Elderly (PACE)." The Gerontologist, 2002, vol. 42, no. 3, pp. 314-320.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	Interviews conducted with PACE and WPP enrollees living in the community as of April 2000
Sample	629 individuals: 326 PACE enrollees and 303 enrollees in the WPP in 2 sites (Madison and Milwaukee)
Data Source(s)	In-person interviews with enrollees and telephone interviews with the closest caregiving family member; proxy interviews conducted as necessary
Methods	Comparison of interview responses across the 2 groups without regression adjustment.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Compared to WPP enrollees, PACE enrollees had more reliance on day care, outpatient rehabilitation, and speech therapy.
	*Quality of Care*: Compared to WPP enrollees, PACE enrollees had more disabilities including a higher rate of dementia and greater ADL and IADL dependencies.
	*Participant Satisfaction and Quality of Life*: WPP and PACE enrollees had comparable levels of unmet needs, use of advance directives, and satisfaction.
Threats of Validity	Comparison group a variation on PACE, not a true nonPACE comparison. Small sample size and limited to 2 sites in Wisconsin. Lack of baseline equivalence and regression adjustment.
Strength of Evidence	Moderate to weak

6. Kane, Robert L., Patricia Homyak, Boris Bershadsky, and Shannon Flood. "Variations on a Theme Called PACE." Journal of Gerontology, 2006a, vol. 61A, no. 7, pp. 689-693.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	1999-2001, with less than 2 years of followup
Sample	1,285 individuals enrolled in either PACE or the WPP for at least 1 month during 1999-2001 Treatment group: 651 PACE enrolleesComparison group: 634 WPP enrollees
Data Source(s)	Site billing records, enrollment and claims data from Wisconsin Medicaid and CMS for pre-enrollment utilization and diagnosis
Methods	Cross-sectional time series approach involving comparisons of average monthly rates using unadjusted means as well as regression-adjusted estimates.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: PACE enrollees had fewer hospital admissions and preventable hospital admissions; fewer hospital days; fewer ER visits and preventable ER visits; and no difference in length of hospital stays.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Comparison group a variation on PACE, not a true nonPACE comparison. Sample limited to Wisconsin.Lack of baseline equivalence. Issues with the quality and completeness of outcomes data that are based on site-specific billing records. Limited followup of 2 years or less.
Strength of Evidence	Moderate

7. Kane, Robert L., Patricia Homyak, Boris Bershadsky, and Shannon Flood. "The Effects of a Variant of the Program for All-inclusive Care of the Elderly on Hospital Utilization and Outcomes." Journal of the American Geriatrics Society, 2006b, vol. 54, pp. 276-283.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	Yes
Timeframe	1999-2001, with a maximum followup of 12 months for outcomes
Sample	862 individuals split into 2 different study cohorts as follows: (1) Direct Cohort (WPP enrollees are from waiver waiting lists) with 652 individuals Treatment group: 213 WPP enrollees Comparison group 1: 220 frail elderly in FFS Medicare, Medicaid, and community-based waiver programs living within the WPP service area Comparison group 2: 219 frail elderly in FFS Medicare, Medicaid, and community-based waiver programs living outside the WPP service area (2) Transfer Cohort (WPP enrollees transferred from waiver programs) with 210 individuals Treatment group: 70 WPP enrollees Comparison group 1: 70 frail elderly in FFS Medicare, Medicaid, and community-based waiver programs living within the WPP service area Comparison group 2: 70 frail elderly in FFS Medicare, Medicaid, and community-based waiver programs living outside the WPP service area
Data Source(s)	Administrative billing records from the sites, Medicare claims, and Wisconsin Medicaid
Methods	Longitudinal cohort design using pair-wise matched comparison group; comparison of regression-adjusted outcomes during the first 12 months after enrollment
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: WPP enrollees had no significant differences in risk of entry into NHs compared to matched comparison groups.
	*Hospital Utilization*: WPP enrollees had no significant differences in inpatient hospital admissions or ED visits compared to the matched comparison groups; for the direct cohort, WPP enrollees had fewer hospital days than comparison group 2 (124 fewer monthly hospital days per 1,000 enrollees) and fewer preventable hospital admissions than comparison group 1 (14 fewer monthly preventable hospital admissions per 1,000 enrollees).
	*Other Service Utilization*: Not studied.
	*Quality of Care*: WPP enrollees had no significant differences in mortality compared to matched comparison groups.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Treatment group a variation on PACE, not a true PACE program. Small sample size leading to potentially underpowered analyses. Study limited to Wisconsin.Data limitations in the form of different data sources for outcomes for treatment and comparison groups--no assessment of comparability of the data sources re definitions, quality or completeness.
Strength of Evidence	Moderate to strong

8. Mancuso, David, Greg Yamashiro, and Barbara Felver. "PACE: An Evaluation." Washington State Department of Social and Health Services Research and Data Analysis Division. Report Number 8.26: June 29, 2005.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	Yes
Timeframe	1998-2003, with up to 4 years of followup
Sample	2,517 individuals Treatment group: 227 PACE enrolleesComparison group 1: 1,891 HCBS enrollees Comparison group 2: 399 NH clients
Data Source(s)	Comprehensive Assessment data, Vital Registration System records, Medical Management Information System pharmacy claims, and Client Services Database
Methods	Propensity score matching; regression-adjusted outcomes comparison
Key Findings	*Costs*: PACE enrollees had greater Medicaid costs than HCBS clients ($1,442 PBPM higher in the first followup year; $1,018 PBPM higher in the fourth followup year), but similar Medicaid costs as NH clients.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: PACE enrollees had greater stability in physical functioning than HCBS clients (ADL/IADL total scores remained unchanged for PACE clients over time, while these scores increased significantly over time for the HCBS clients); and significantly lower risk of dying compared to HCBS comparison: 6 percentage points lower (13% versus 19%) after 1 year; 16 percentage points lower (29% versus 45%) after 3 years.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Small sample sizes, especially for PACE, and significant attrition over time. Assessed Medicaid costs only (Medicare not addressed). Mental health and chemical dependency treatment costs excluded.
Strength of Evidence	Moderate to strong

9. Medicare Payment Advisory Commission. "Report to the Congress: Medicare and the Health Care Delivery System." June 2012.
Design	Comparison of Medicare payments to PACE sites to Medicare spending on comparable FFS enrollees in counties where PACE providers operate
Baseline Equivalence	No
Timeframe	2010-2011
Sample	PACE enrollees and comparable Medicare FFS enrollees in counties with PACE sites
Data Source(s)	Site visits to selected PACE sites and analysis of the Medicare payment system for PACE using the CMS-HCC model
Methods	A general analysis of the Medicare capitation payments for PACE enrollees relative to predicted FFS spending on comparable enrollees using the HCC model.
Key Findings	*Costs*: Medicare spending on PACE enrollees exceeded estimated FFS spending on comparable enrollees by 17%, on average (statistical significance not reported).
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Lack of baseline equivalence and regression adjustment. Comparison of spending estimates at a single point-in-time (limited followup). Assessed Medicare costs only.
Strength of Evidence	Moderate to Weak

10. Meret-Hanke, Louise. "Effects of the Program of All-Inclusive Care for the Elderly on Hospital Use." The Gerontologist, 2011, vol. 51, no. 6, pp. 774-785.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	Yes
Timeframe	PACE enrollment during 1990-1998, and first MCBS interview during 1991-1999, with a 2-year followup period.
Sample	6,992 individualsTreatment group: 3,889 PACE enrolleesComparison group: 3,103 frail and community-dwelling older adults drawn from MCBS
Data Source(s)	DataPACE, MCBS, and Area Resource File
Methods	Propensity score matching; comparison of regression-adjusted outcomes using a 2-part model.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: PACE enrollees had higher average hospital use in the 6 months before baseline (1.58 days per month alive versus 1.04 days per month alive, significant at the 1% level); and lower average hospital use during the 2-year followup period (0.22 days per month alive versus 0.8 days per month alive, significant at the 1% level).
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Baseline differences between the 2 groups in a number of individual and regional characteristics leaves considerable scope for selection bias. Data on both individual characteristics and outcomes of the treatment and comparison groups obtained from different datasets, leading to possible noncomparability in covariates as well as differences in the coding of outcomes. Data and findings are over a decade old
Strength of Evidence	Moderate

11. Mukamel, Dana B., Helena Temkin-Greener, and Marleen L. Clark. "Stability of Disability Among PACE Enrollees: Financial and Programmatic Implications." Health Care Financing Review, Spring 1998, vol. 19, no. 3.
Design	Comparison across PACE sites
Baseline Equivalence	No
Timeframe	From program inception (varies depending on site) through 1994
Sample	2,291 individuals in 11 PACE programs
Data Source(s)	DataPACE
Methods	Comparison of outcomes at the individual and at the site level in relation to length of enrollment in the program and individual risk characteristics or the maturity of the site.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Majority of PACE enrollees remained at the same disability level 18 months after enrollment; most significant changes in ADLs occurred within the first 6 months of enrollment; the average disability level was quite stable at the plan-level among the more mature PACE plans.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	No nonPACE comparison group and limited followup of 18 months only for most sites. Sample not necessarily representative of the broader PACE population. Data over a decade old.
Strength of Evidence	Weak

12. Mukamel, Dana B., Alina Bajorska, and Helena Temkin-Greener. "Health Care Services Utilization at the End of Life in a Managed Care Program Integrating Acute and Long-Term Care." Medical Care, 2002, vol. 40, no. 12, pp. 1136-1148.
Design	Comparison across PACE sites
Baseline Equivalence	No
Timeframe	From program inception (varies depending on site) through 2000
Sample	2,160 persons in 10 PACE sites who died before January 2000
Data Source(s)	DataPACE and annual cost reports submitted by a PACE site to CMS
Methods	Multivariate regression analysis used to identify time trends in utilization as well as the relative importance of individual risk factors versus site effects in determining utilization towards the end of life
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: See Other Service utilization.
	*Hospital Utilization*: See Other Service utilization.
	*Other Service Utilization*: Utilization of health services increases as early as 7 months before death; the increases in utilization towards the end of life was attributable to increased hospital use, NH use, use of home care, primary physician care, and social services, but dominated by hospital use; and closer to the time of death, variation across program sites explains twice as much of the variation in total utilization as explained by individual characteristics.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	No nonPACE comparison group.Sample not necessarily representative of the broader PACE population.Data over a decade old.
Strength of Evidence	Weak

13. Mukamel, Dana B., Helena Temkin-Greener, Rachel Delavan, Derick R. Peterson, Diane Gross, Stephen Kunitz, and T. Franklin Williams. "Team Performance and Risk-Adjusted Health Outcomes in the Program of All-Inclusive Care for the Elderly (PACE)." The Gerontologist, 2006, vol. 46, no. 2, pp. 227-237.
Design	Comparison across PACE sites
Baseline Equivalence	No
Timeframe	Enrollment during July 1998-June 2001 with health outcomes followed until 12 months after enrollment
Sample	3,401 new PACE enrollees in 26 PACE sites 1,209 PACE team members across the 26 sites who responded to the survey on team performance (65% response rate)
Data Source(s)	DataPACE, National PACE Association records on staffing, and surveys of PACE team members
Methods	Multivariate regression analysis used to identify the relationship between team performance and risk-adjusted health outcomes.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Team performance was significantly associated with better functional status at 3 and 12 months after enrollment, and with better urinary incontinence outcomes at 12 months; and had no significant association with mortality.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	No nonPACE comparison group.Limited followup of only 12 months.
Strength of Evidence	Weak

14. Mukamel, Dana B., Derick R. Peterson, Helena Temkin-Greener, Rachel Delavan, Diane Gross, Stephen J. Kunitz, and T. Franklin Williams. "Program Characteristics and Enrollees' Outcomes in the Program of All-Inclusive Care for the Elderly (PACE)." The Milbank Quarterly, 2007, vol. 85, no. 3, pp. 499-531.
Design	Comparison across PACE sites
Baseline Equivalence	No
Timeframe	Enrollment during 1997-2001, with health outcomes followed until 12 months after enrollment
Sample	3,042 new PACE enrollees in 23 PACE programs
Data Source(s)	DataPACE, site visits and interviews with key staff, and a survey of PACE team members
Methods	Multivariate regression analysis used to identify the relationship between program characteristics and risk-adjusted health outcomes.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Several program characteristics, such as the medical director being a trained geriatrician, having more effective teams, program maturity and size, etc. were significantly associated with better functional status; fewer program characteristics, including a more diverse staff and program maturity, were associated with self-assessed health; and only 2 program characteristics--having more professionals and higher concentration of services--were associated with reduced mortality.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	No nonPACE comparison group.Limited followup of only 12 months.
Strength of Evidence	Weak

15. Nadash, Pamela. "Two Models of Managed Long-Term Care: Comparing PACE With a Medicaid-Only Plan." The Gerontologist, 2004, vol. 44, no. 5, pp. 644-654.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	PACE: individuals enrolled in PACE plans during 1996-1997MMLTC Plan: individuals enrolled in the plan during 2000Both groups are followed for the first 12 months after enrollment
Sample	2,679 individuals comprising 1,382 PACE enrollees in 12 PACE sites, and 1,297 MMLTC enrollees in the New York City area
Data Source(s)	OASIS, MMLTC plan administrative data, and DataPACE
Methods	Comparison of unadjusted means using descriptive analysis.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: PACE enrollees experienced higher NH utilization (21% versus 5.7%) with shorter median length of stay (13 days versus 22 days).
	*Hospital Utilization*: PACE enrollees experienced lower hospital utilization (33.7% versus 35.2%) during the first year after enrollment and shorter median length of stay (4 days versus 7 days).
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Lack of baseline equivalence and no regression adjustment. Potential measurement discrepancies across datasets with outcomes data obtained from DataPACE for PACE enrollees and from the plan administrative data for the MMLTC enrollees.Differences in enrollment periods and differences in geographical location between the MMLTC plan and the 12 PACE sites included in the analysis.Data and findings over a decade old.
Strength of Evidence	Moderate to weak

16. Temkin-Greener, Helena, Alina Bajorska, and Dana B. Mukamel. "Variations in Service Use in the Program of All-Inclusive Care for the Elderly (PACE): Is More Better?" Journal of Gerontology: MEDICAL SCIENCES, 2008, vol. 63A, no. 7, pp. 731-738.
Design	Comparison across PACE sites
Baseline Equivalence	No
Timeframe	The most recent 3-year period of data available for each program with the latest time period ending in December 2002
Sample	9,853 individuals in 29 PACE programs that were in operation prior to 2000
Data Source(s)	Site-level PACE administrative database
Methods	Mixed regression models for each outcome and a separate model relating change in functional status to utilization of the 5 different services.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: NH admissions were not significantly associated with change in functional status.
	*Hospital Utilization*: Only hospital admissions were significantly associated with change in functional status, with sites having higher hospital admissions also performing worse on change in risk-adjusted functional status; and sites providing more day center and therapy services had significantly fewer hospital admissions.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Observed wide variations in change in risk-adjusted functional status across sites.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	No nonPACE comparison.Inability to rule out omitted variable bias in estimating the relationship between service utilization and change in functional status.
Strength of Evidence	Weak

17. Weaver, Frances M., Elaine C. Hickey, Susan L. Hughes, Vicky Parker, Dawn Fortunato, Julia Rose, Steven Cohen, Laurence Robbins, Willie Orr, Beverly Priefer, Darryl Wieland, and Judith Baskins. "Providing All-Inclusive Care for Frail Elderly Veterans: Evaluation of Three Models of Care." Journal of the American Geriatric Society, 2008, vol. 56, pp. 345-353.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	Subjects enrolled between June 2001 and June 2003; followed up to 36 months
Sample	368 veteransGroup 1: VA as sole care provider (N = 181)Group 2: VA-community partnership with PACE (N = 102)Group 3: VA as care manager with care provided by PACE (N = 85)
Data Source(s)	National Patient Care Database, DataPACE, Medicare claims data, and patient interviews
Methods	Descriptive analysis without regression adjustment; comparison of mean outcomes in the 3 groups, before and after enrollment.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: NH admissions and days increased in all 3 groups after enrollment; however, there were higher rates of NH admissions and higher NH days under VA-community partnership with PACE and VA as care manager with care provided by PACE, compared to VA as sole care provider (statistical significance not reported).
	*Hospital Utilization*: Lower rates of inpatient admissions and fewer inpatient days under VA-community partnership with PACE and under VA as care manager with care provided by PACE, compared to VA as sole care provider (statistical significance not reported).
	*Other Service Utilization*: Percentage of patients using home care services increased significantly in groups 2 and 3 after enrollment.
	*Quality of Care*: 28-34% of enrollees in the 3 groups died by the end of the evaluation, with around half the deaths in NH (among survivors, 84-97% resided in the community, across the 3 groups).
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Small sample sizes and differences in disenrollment rates.Lack of baseline equivalence.No regression adjustment.Incomplete Medicare data.
Strength of Evidence	Moderate to weak

18. White, Alan J. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE): The Effect of PACE on Costs to Medicare: A Comparison of Medicare Capitation Rates to Projected Costs in the Absence of PACE." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc., July 1998.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	1995-1997 with a 1 year followup for Medicare costs
Sample	2,284 individualsTreatment group: 1,519 PACE enrolleesComparison group: 765 individuals who expressed interest in PACE and had a home visit but did not enroll
Data Source(s)	Survey of PACE applicants, Medicare claims data, Health Care Financing Administration enrollment database, and Medicare capitated payments to PACE sites
Methods	Comparison of projected FFS costs based on regression adjustment and actual capitation payments for PACE enrollees.
Key Findings	*Costs*: Medicare capitated payments for PACE enrollees were 38% lower in months 1-6, and 16% lower in months 7-12, than what they would have been in the absence of PACE, that is, under FFS (statistical significance not reported).
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Lack of baseline equivalence.Assessed only Medicare costs (not full capitation rate).Baseline survey response rate was 44%. Data over a decade old.
Strength of Evidence	Moderate

19. White, Alan J., Yvonne Abel, and David Kider. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE): A Comparison of the PACE Capitation Rates to Projected Costs in the First Year of Enrollment." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc., October 2000.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	1995-1997 with a 1 year followup
Sample	2,038 individuals Treatment group: 1,367 PACE enrolleesComparison group: 671 individuals who expressed interest in PACE and had a home visit but did not enroll (Medicaid data was available for only 381 comparison group members)
Data Source(s)	Survey of PACE applicants, Medicare claims data, Medicaid claims data, Health Care Financing Administration enrollment database, and Medicare and Medicaid capitated payments to PACE sites--based on audited financial statements
Methods	Comparison of projected FFS costs based on regression adjustment and actual capitation payments for PACE enrollees.
Key Findings	*Costs*: PACE enrollees had 42% lower Medicare capitation rate than projected Medicare costs, but the difference was not statistically significant; 86% higher Medicaid capitation rate than projected Medicaid costs (significant at the 5% level); and 10% higher combined Medicare and Medicaid capitation rates than projected costs (statistical significance was not reported).
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Lack of baseline equivalence and considerable scope for selection bias. Baseline survey response rate was 44% only. Restricted analysis sample to individuals who could be linked across claims data. Loss of 43% of the analysis sample for the Medicaid and total cost analyses, due to lack of Medicaid cost data. Small sample size and high variability in data.Limited followup of only 1 year. Data and findings are over a decade old.
Strength of Evidence	Moderate

20. Wieland, Darryl, Vicki L. Lamb, Shae R. Sutton, Rebecca Boland, Marleen Clark, Susan Friedman, Kenneth Brummel-Smith, and G. Paul Eleazer. "Hospitalization in the Program of All-Inclusive Care for the Elderly (PACE): Rates, Concomitants, and Predictors." Journal of the American Geriatrics Society, 2000, vol. 48, pp. 1373-1380.
Design	Comparison of inpatient hospital utilization for PACE enrollees versus the general Medicare FFS population
Baseline Equivalence	No
Timeframe	Enrollment in PACE during 1990-1997 with hospital utilization studied during 1992-1998
Sample	5,478 new PACE enrollees in any PACE site33.4 million aged and 4.7 million disabled Medicare FFS beneficiaries
Data Source(s)	DataPACE and Medicare and Medicaid Statistical Supplement
Methods	Comparison of hospital utilization between PACE and the Medicare FFS population based on comparison of means; regression-adjusted model of hospitalization for PACE enrollees to identify significant predictors of hospitalization.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: PACE enrollees had comparable numbers of bed-days per 1,000 participants (despite greater morbidity and disability than the general Medicare population); and newest PACE sites had substantially higher hospital utilization than more mature sites.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Over a third of PACE enrollees died by the end of study, with only a third of the decedents being hospitalized during the last 6 months before death and only 8 percentage of PACE enrollees dying in the hospital.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Results based on comparing hospitalizations in PACE to that in a national comparison group of FFS Medicare enrollees.Lack of baseline equivalence and no regression adjustment.Potential data comparability issues with data on hospitalizations drawn from different sources (DataPACE and the Medicare and Medicaid Statistical Supplement).Data over a decade old.
Strength of Evidence	Moderate to weak

21. Wieland, Darryl, Rebecca Boland, Judith Baskins, and Bruce Kinosian. "Five-Year Survival in a Program of All-Inclusive Care For Elderly Compared With Alternative Institutional and Home- and Community-Based Care." Journal of Gerontology, 2010, vol. 65, no. 7, pp. 721-726.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	Enrollment during 1998-2003, with a maximum of 5 years of followup
Sample	2,040 individualsTreatment group: 554 PACE enrolleesComparison group 1: 1,018 waiver program participantsComparison group 2: 468 NH entrants
Data Source(s)	State Form 1718 records, DataPACE, and state vital statistics records
Methods	Kaplan-Meier curves for survival estimation, after taking into account mortality risk at admission based on the PPI.
Key Findings	*Costs*: Not studied.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: PACE enrollees had greater mortality risk at admission than waiver participants and similar mortality risk as the NH entrants; longer median survival (4.2 years) than NH (2.3 years) or waiver (3.5 years) populations; and after accounting for mortality risk at admission, PACE enrollees still had significant survival advantage over waiver enrollees--4.7 versus 3.4 years among moderate-risk enrollees, and 3 versus 2 years among high-risk enrollees.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Lack of baseline equivalence.Mortality data obtained from different sources for enrollees in PACE versus those in the waiver program and in NHs. Sample limited to South Carolina.
Strength of Evidence	Moderate

22. Wieland, Darryl, Bruce Kinosian, Eric Stallard, and Rebecca Boland. "Does Medicaid Pay More to a Program of All-Inclusive Care for the Elderly (PACE) Than for Fee-for-Service Long-term Care?" Journal of Gerontology, 2012.
Design	Quasi-experimental design with comparison group
Baseline Equivalence	No
Timeframe	Enrollment over 1994-2005; 1-year followup for Medicaid expenditures
Sample	3,988 individualsTreatment group: 948 PACE entrants in South CarolinaComparison group 1: 1,683 waiver program entrantsComparison group 2: 1,357 NH entrants
Data Source(s)	State admission records and South Carolina's Medicaid Management Information System
Methods	Grade-of-membership models compare adjusted 1-year payments
Key Findings	*Costs*: The PBPM Medicaid capitation rates for PACE enrollees were 22-26% below the predicted FFS payments.
	*Nursing Home Utilization*: Not studied.
	*Hospital Utilization*: Not studied.
	*Other Service Utilization*: Not studied.
	*Quality of Care*: Not studied.
	*Participant Satisfaction and Quality of Life*: Not studied.
Threats of Validity	Sample limited to South Carolina.Lack of baseline equivalence.Unaccounted for changes in programs during the long (11-year) enrollment period.Limited followup of only 1 year Period of study preceded Medicare Part D, which changed drug benefit coverage and thus impacted Medicaid costs.
Strength of Evidence	Moderate

IV. All Studies Identified

Study	Included in Review
Beauchamp, Jody, Valerie Cheh, Robert Schmitz, Peter Kemper, and John Hall. "The Effect of the Program of All-Inclusive Care for the Elderly (PACE) on Quality." Report submitted to the Centers for Medicare and Medicaid Services. Princeton, NJ: Mathematica Policy Research, February 12, 2008.	X
Bloom, Shawn, Brenda Sulick, and Jennie Chin Hansen. "Picking Up the PACE: The Affordable Care Act Can Grow and Expand a Proven Model of Care." Generations: Journal of the American Society on Aging, Spring 2011, vol. 25, no. 1, pp. 53-55.
Bodenheimer, Thomas. "Long-Term Care for Frail Elderly People--The On Lok Model." New England Journal of Medicine, October 1999, vol. 341, no. 17, pp. 1324-1328.
Boult, Chad, and G. Darryl Wieland. "Comprehensive Primary Care for Older Patients With Multiple Chronic Conditions." Journal of the American Medical Association, November 2010, vol. 304, no. 17, pp. 1936-1943.
Chatterji, Pinka, Nancy R. Burstein, David Kidder, and Alan White. "Evaluation of the Program of All-Inclusive Care for the Elder (PACE) Demonstration: The Impact of PACE on Participant Outcomes." Report submitted to the Health Care Financing Administration. Cambridge, MA: Abt Associates, Inc., July 1998.	X
Damons, J. "Program of All-Inclusive Care for the Elderly (PACE) Year 2 Overview." Long Term Care, Bureau of TennCare, Tennessee, 2001.
Division of Health Care Finance and Policy. "PACE Evaluation Summary." August 2005.	X
Eleazer, P., and M. Fretwell. "The PACE Model: A Review." Emerging Systems in Long-Term Care, 1999, vol. 4, pp. 88-117.
Eng, C., J. Pedulla, G.P. Eleazer et al. "Program of All-inclusive Care for the Elderly (PACE): An Innovative Model of Integrated Geriatric Care and Financing." Journal of the American Geriatrics Society, 1997, vol. 25, pp. 223-232.
Eng, Catherine. "Future Consideration for Improving End-of-Life Care for Older Persons: Program of All-Inclusive Care for the Elderly (PACE)." Journal of Palliative Medicine, 2002, vol. 5, no. 2, pp. 305-309.
Foster, Leslie, Robert Schmitz, and Peter Kemper. "The Effects of PACE on Medicare and Medicaid Expenditures." Report submitted to the Centers for Medicare and Medicaid Services. Princeton, NJ: Mathematica Policy Research, August 29, 2007.	X
Fretwell, Marsha D. and Jane S. Old. "The PACE Program: Home-Based Care for Nursing Home-Eligible Individuals." North Carolina Medical Journal, 2011, vol. 72, no. 3, pp. 209-211.
Friedman, Susan M., Donald M. Steinwachs, Paul J. Rathouz, Lynda C. Burton, and Dana B. Mukamel. "Characteristics Predicting Nursing Home Admission in the Program of All-Inclusive Care for Elderly People." The Gerontologist, 2005, vol. 45, no. 2, pp. 157-166.
Friedman, Susan M., Donald M. Steinwachs, Helena Temkin-Greener, and Dana B. Mukamel. "Informal Caregivers and the Risk of Nursing Home Admission Among Individuals Enrolled in the Program of All-Inclusive Care for the Elderly." The Gerontologist, 2006, vol. 46, no. 4, pp. 456-463.
Gold, Marsha R., Gretchen A. Jacobson, and Rachel L. Garfield. "There Is Little Experience and Limited Data to Support Policy Making on Integrated Care For Dual Eligibles." Health Affairs, June 2012, vol. 31, no. 6, pp. 1176-1185.
Greenwood, Robert, and Jade Gong. "A PACE Financial Planning Resource: Alexian Brothers Community Services, Chattanooga Tennessee: Case Study." Fall 2002.
Gross, Diane L., Helena Temkin-Greener, Stephen Kunitz, and Dana B. Mukamel. "The Growing Pains of Integrated Health Care for the Elderly: Lessons from the Expansion of PACE." Integrated Health Care for the Elderly, 2004, vol. 82, no. 2, pp. 257-82.
Hansen, Jennie Chin. "Practical Lessons for Delivering Integrated Services in a Changing Environment: The PACE Model." Generations, Summer 1999, vol. 23, no. 2, pp. 22-28.
Hirth, Victor, Judith Baskins, and Maureen Dever-Bumba. "Program of All-Inclusive Care (PACE): Past, Present, and Future." Journal of the American Medical Directors Association, March 2009, vol. 10, pp. 155-160.
Irvin, Carol V., Susan Massey, and Turahn C. Dorsey. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE) Demonstration: Determinants of Enrollment Among Applicants to the PACE Program." Report submitted to the Health Care Financing Administration. Cambridge, MA: Abt Associates Inc., February 9, 1998.
Kane, R.L. "Setting the PACE in Chronic Care." Contemporary Gerontology, 1999, vol. 6, no. 2, pp. 47-50.
Kane, Robert L., Patty Homyak, and Boris Bershadsky. "Consumer Reactions to the Wisconsin Partnership Program and Its Parent, the Program for All-Inclusive Care of the Elderly (PACE)." The Gerontologist, 2002, vol. 42, no. 3, pp. 314-320.	X
Kane, Robert L., Patricia Homyak, Boris Bershadsky, and Shannon Flood. "Variations on a Theme Called PACE." Journal of Gerontology, 2006a, vol. 61A, no. 7, pp. 689-693.	X
Kane, Robert L., Patricia Homyak, Boris Bershadsky, and Shannon Flood. "The Effects of a Variant of the Program for All-inclusive Care of the Elderly on Hospital Utilization and Outcomes." Journal of the American Geriatrics Society, 2006b, vol. 54, pp. 276-283.	X
Leavitt, Michael O. "Interim Report to Congress: The Quality and Cost of the Program of All-Inclusive Care for the Elderly." Secretary of Health and Human Services, 2008.
Mancuso, David, Greg Yamashiro, and Barbara Felver. "PACE: An Evaluation." Washington State Department of Social and Health Services Research and Data Analysis Division. Report Number 8.26, June 29, 2005.	X
Medicare Payment Advisory Commission. "Report to the Congress: Medicare and the Health Care Delivery System." June 2012. Available at: http://www.medpac.gov/documents/Jun12_EntireReport.pdf. Accessed November 2, 2012.	X
Meret-Hanke, Louise. "Effects of the Program of All-Inclusive Care for the Elderly on Hospital Use." The Gerontologist, 2011, vol. 51, no. 6, pp. 774-785.	X
Mui, Ada C. "The Program of All-Inclusive Care for the Elderly (PACE): An Innovative Long-Term Care Model in the United States." Journal of Aging and Social Policy, 2001, vol. 13, no. 2/3, pp. 53-67.
Mukamel, Dana B., Helena Temkin-Greener, and Marleen L. Clark. "Stability of Disability Among PACE Enrollees: Financial and Programmatic Implications." Health Care Financing Review, Spring 1998, vol. 19, no. 3.	X
Mukamel, Dana B., Alina Bajorska, and Helena Temkin-Greener. "Health Care Services Utilization at the End of Life in a Managed Care Program Integrating Acute and Long-Term Care." Medical Care, 2002, vol. 40, no. 12, pp. 1136-1148.	X
Mukamel, Dana B., Helena Temkin-Greener, Rachel Delavan, Derick R. Peterson, Diane Gross, Stephen Kunitz, and T. Franklin Williams. "Team Performance and Risk-Adjusted Health Outcomes in the Program of All-Inclusive Care for the Elderly (PACE)." The Gerontologist, 2006, vol. 46, no. 2, pp. 227-237.	X
Mukamel, Dana B., Derick R. Peterson, Helena Temkin-Greener, Rachel Delavan, Diane Gross, Stephen J. Kunitz, and T. Franklin Williams. "Program Characteristics and Enrollees' Outcomes in the Program of All-Inclusive Care for the Elderly (PACE)." Milbank Quarterly, 2007, vol. 85, no. 3, pp. 499-531.	X
Nadash, Pamela. "Two Models of Managed Long-Term Care: Comparing PACE With a Medicaid-Only Plan." The Gerontologist, 2004, vol. 44, no. 5, pp. 644-654.	X
National PACE Association. "Program of All-Inclusive Care for the Elderly (PACE) Key Research Findings: Quality Care, Improved Health, Cost-Effective." 2010.
Pak, Esther, Carol Perfetto, Qilu Yu, and Matthew McNabney. "Effect of Enrollment in the Program of All-Inclusive Care for the Elderly on End-of-Life Care." Journal of the American Geriatrics Society, vol. 60, no. 1, January 2012, pp. 166-167.
Petigara, Tanaz, and Gerard Anderson. "Program of All-Inclusive Care for the Elderly." Health Policy Monitor, April 2009.
Reardon G., M. McNabney M, S. Bloom, and C. Eng. "Should Programs of All Inclusive Care for the Elderly Use Community-Based Primary Care?" Journal of the American Medical Directors Association, May 2008, vol. 9, no. 3, pp. 275-278.
Robinson, James, and Sarita L. Karon. "Modeling Medicare Costs of PACE Populations." Health Care Financing Review, Spring 2000, vol. 21, no. 3, pp. 149-170.
Rylander, C. "Expand the Use of an Effective Long-term Care Program." Recommendation of the Texas Comptroller: Chapter 8: Health and Human Services. Texas Comptroller of Public Accounts, Austin, TX, 2000.
Schimmel, Jody, Valerie Cheh, Robert Schmitz, and Peter Kemper. "The Effects of PACE on Care Quality." Princeton, NJ: Mathematica Policy Research, Inc., Draft Report.
Tan, Erwin J., Li-Yung Lui, Catherine Eng, Ashish K. Jha, and Kenneth E. Covinsky. "Differences in Mortality of Black and White Patients Enrolled in the Program of the All-Inclusive Care for the Elderly." Journal of the American Geriatrics Society, February 2003, vol. 51, no. 2, pp. 246-251.
Temkin-Greener, Helena, Mark R. Meiners, and Leonard Gruenberg. "PACE and the Medicare+Choice Risk-Adjusted Payment Model." Inquiry, Spring 2001, vol. 38, pp. 60-72.
Temkin-Greener, Helena, and Dana B. Mukamel. "Predicting Place of Death in the Program of All-Inclusive Care for the Elderly (PACE): Participant versus Program Characteristics." Journal of the American Geriatrics Society, January 2002, vol. 50, no. 1, pp. 125-135.
Temkin-Greener, Helena, Diane Gross, Stephen J. Kunitz, and Dana Mukamel. "Measuring Interdisciplinary Team Performance in a Long-Term Care Setting." Medical Care, May 2004, vol. 24, no. 5, pp. 472-481.
Temkin-Greener, Helena, Alina Bajorska, and Dana B. Mukamel. "Disenrollment From an Acute/Long-Term Managed Care Program (PACE)." Medical Care, January 2006, vol. 44, no. 1.
Temkin-Greener, Helena, Alina Bajorska, and Dana B. Mukamel. "Variations in Service Use in the Program of All-Inclusive Care for the Elderly (PACE): Is More Better?" Journal of Gerontology, 2008, vol. 63A, no. 7, pp. 731-738.	X
Thomson Reuters. "Summary of Evidence Basis for the Program of All-Inclusive Care for the Elderly (PACE)." August 2, 2011.
Trice, Laura. "PACE: A Model for Providing Comprehensive Healthcare for Frail Elders." Generations, Fall 2006, vol. 30, no. 3, pp. 90-92.
Weaver, Frances M., Elaine C. Hickey, Susan L. Hughes, Vicky Parker, Dawn Fortunato, Julia Rose, Steven Cohen, Laurence Robbins, Willie Orr, Beverly Priefer, Darryl Wieland, and Judith Baskins. "Providing All-Inclusive Care for Frail Elderly Veterans: Evaluation of Three Models of Care." Journal of the American Geriatric Society, 2008, vol. 56, pp. 345-353.	X
White, Alan J. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE): The Effect of PACE on Costs to Medicare: A Comparison of Medicare Capitation Rates to Projected Costs in the Absence of PACE." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc., July 1998.	X
White, Alan J., Yvonne Abel, and David Kider. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE): A Comparison of the PACE Capitation Rates to Projected Costs in the First Year of Enrollment." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc., October 2000.	X
Wieland, Darryl, Vicki Lamb, Huei Wang, Shae Sutton, G. Paul Eleazer, and John Egbert. "Participants in the Program of All-Inclusive Care for the Elderly (PACE) Demonstration: Developing Disease-Impairment-Disability Profiles." The Gerontologist, 2000, vol. 40, no. 2, pp. 218-227.
Wieland, Darryl, Vicki L. Lamb, Shae R. Sutton, Rebecca Boland, Marleen Clark, Susan Friedman, Kenneth Brummel-Smith, and G. Paul Eleazer. "Hospitalization in the Program of All-Inclusive Care for the Elderly (PACE): Rates, Concomitants, and Predictors." Journal of the American Geriatrics Society, 2000, vol. 48, pp. 1373-1380.	X
Wieland, Darryl, Rebecca Boland, Judith Baskins, and Bruce Kinosian. "Five-Year Survival in a Program of All-Inclusive Care For Elderly Compared With Alternative Institutional and Home- and Community-Based Care." Journal of Gerontology, July 2010, vol. 65, no. 7, pp. 721-726.	X
Wieland, Darryl, Bruce Kinosian, Eric Stallard, and Rebecca Boland. "Does Medicaid Pay More to a Program of All-Inclusive Care for the Elderly (PACE) Than for Fee-for-Service Long-term Care?" Journal of Gerontology, 2012.	X
Zimmerman, Y.A., D.A. Pemberton, T.C. Dorsey, S. Massey, and R.F. Coulam. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE) Demonstration." Annual Site Visit Report: Calendar Year 1995. Cambridge, MA: Abt Associates Inc., 1997.
Zimmerman, Yvonne, Debra Pemberton, and Laurine Thomas. "Evaluation of the Program of All-Inclusive Care for the Elderly (PACE) Demonstration: Factors Contributing to Care Management and Decision Making in the PACE Model." Report submitted to Health Care Financing Administration. Cambridge, MA: Abt Associates Inc., April 10, 1998.

NOTES

Other programs that currently provide integrated care to dually eligible beneficiaries include the 17 fully-integrated Special Needs Plans, such as Commonwealth Care Alliance in Boston, Massachusetts, but these programs operate under Medicare and Medicaid waivers, and do not have permanent provider status like PACE.
Originally, the Medicare capitation rate for each PACE organization was the Medicare Advantage (MA) (formerly Medicare+ Choice) county rate multiplied by a frailty adjuster (2.39 for all participants except those with end-stage renal disease). During 2004-2007, the Medicare capitation rate was a blend of two formulas: (1) the MA county rate; and (2) a risk-adjusted payment methodology, based on the Centers for Medicare and Medicaid Services (CMS)-hierarchical conditions category (HCC) risk adjustment model; the blended rate was then multiplied by the uniform frailty adjuster (2.39) to account for the higher Medicare costs incurred by people with functional impairments, even after accounting for chronic conditions. The weight assigned to the risk-adjusted part of the payment was gradually increased, until the blended payment transitioned to a fully HCC score-adjusted payment in 2008. Since 2008, the Medicare PACE capitation payment has been computed as the HCC score-adjusted MA payment, multiplied by a plan-specific frailty factor, rather than the uniform frailty factor used previously. This plan-specific frailty factor is calculated by CMS, varies across PACE organizations and over time, and has been gradually transitioned during 2008-2012 from being a weighted average of the uniform adjuster and a plan-specific adjustment factor to entirely being based on the plan-specific factor.
Medicaid capitation rates for PACE plans are subject to an "upper payment limit" (UPL), intended to be the cost of a comparable FFS equivalent population. The UPL is determined separately by each state, and most states calculate the UPL by using a blend of FFS costs for NH residents and for home and community-based services (HCBS) waiver enrollees. States vary in how heavily they weight the costs of the HCBS and NH groups for developing an UPL; for example, Pennsylvania sets the UPL at about 80 percent of the NH daily rate, whereas Florida sets its UPL primarily on FFS claims data for HCBS waiver program recipients. The monthly capitation rate to PACE plans is then set as a percentage of the UPL, for example, between 85 percent and 95 percent. However, states are experimenting with alternative approaches to setting capitation rates, for example, using risk-based models.
States have discretion on whether or not to include PACE in the Medicaid program. For example, Minnesota has not adopted PACE as part of its Medicaid program, and there are no PACE plans currently in Minnesota.
For instance, PACE requires enrollees to have sufficient support at home so they can remain safely in the community at night.
Several studies have examined health care utilization outcomes such as hospitalizations and NH admissions using DataPACE--an administrative database created by PACE plans with information on health care utilization and health status of enrollees in each plan. However, the quality of the utilization data in DataPACE has not been independently assessed, nor has its comparability to the data sources used for the comparison group selected been evaluated. Several other studies administered surveys to both PACE participants and nonparticipants (comparison group) to collect data on health care utilization and quality of care. However, given the frail health of beneficiaries served by PACE, nonresponse rates were high, especially for studies with multiple rounds of followup surveys. Moreover, systematic differences between the two groups in the beneficiaries' ability to recall past service use could bias impact estimates. The more recent availability of the Timeline File based on the Long-Term Care Minimum Dataset should enable researchers to examine NH utilization for PACE enrollees.
We are not aware of any significant evaluations of PACE based on either a randomized or a nonrandomized study design published prior to 1997.
The sizeable difference (42 percent) between the projected Medicare cost ($1,844) and the capitation payment ($1,072) was not statistically significant at the 5 percent level. The 95 percent confidence interval for the projected Medicare cost was $1,050-$2,638, and therefore, contained the capitation payment amount. The wide confidence interval around projected Medicare costs was due to the small comparison group sample size of 662 members used in projecting Medicare costs.
Predicted FFS payments for PACE enrollees were $36,620 (95 percent confidence interval: $35,662-$37,580). Actual Medicaid capitation payment to PACE was $27,648, which is 24.5 percent below the mean predicted FFS payment; the estimated savings from PACE, therefore, ranges between 22 percent and 26 percent, given the 95 percent confidence interval of predicted FFS payments. The authors, perhaps erroneously, report that the Medicaid capitation payment was 28 percent below the lower limit of predicted FFS payments.
For the studies comparing PACE to WPP, it is useful to remember that WPP is not a true nonPACE alternative but rather an enhanced or less rigid version of PACE that allows enrollees to retain their primary care physicians and does not require the use of the day care center. Thus, findings from these studies need to be interpreted as evaluating the effect of the additional features of WPP.
Changes in the Medicare capitation rates also include changes to the frailty factor, which as mentioned earlier has transitioned from a uniform frailty adjuster (2.39) to a plan-specific frailty factor. Robinson and Karon (2000) used a nine-state sample of PACE-eligible (NH certifiable) individuals to assess the appropriateness of the uniform PACE frailty adjuster (2.39). They found that, depending on the level of risk adjustment applied to the base cost estimate, the appropriate frailty adjuster would range from 1.00 to 1.83. However, their estimates did not account for the possibly greater impairment of actual PACE enrollees or changes in enrollment profiles over time. Hence, the uncertainty in future enrollment processes led the authors to conclude that the capitation rate calculations should reflect actual risk characteristics of enrollees, and should thus vary by site rather than be based on a national number. The current Medicare capitation payment calculations for PACE, which incorporate site-specific frailty factors that vary across programs and over time, do reflect the authors' recommendations.
The online version of this article was published in 2012, and the full journal article was published in 2013: Wieland, Darryl, Bruce Kinosian, Eric Stallard, and Rebecca Boland. "Does Medicaid Pay More to a Program of All-Inclusive Care for the Elderly (PACE) Than for Fee-for-Service Long-term Care?" Journal of Gerontology, 2013, vol. 68, no. 1, pp. 47-56.

CENTER OF EXCELLENCE IN RESEARCH ON DISABILITY SERVICES AND CARE COORDINATION AND INTEGRATION (CERDS)

REPORTS AVAILABLE

Abstracted List of Tasks and Reports

HTML http://aspe.hhs.gov/daltcp/reports/2014/CERDS.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/CERDS.pdf

--- --- --- --- ---

Association between NCQA Patient-Centered Medical Home Recognition for Primary Care Practices and Quality of Care for Children with Disabilities and Special Health Care Needs

HTML http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV3.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV3.pdf

Children with Disabilities and Special Health Care Needs in NCQA-Recognized Patient-Centered Medical Homes: Health Care Utilization, Provider Perspectives and Parental Expectations Executive Summary

HTML http://aspe.hhs.gov/daltcp/reports/2014/ChildDisES.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/ChildDisES.pdf

Descriptive Study of Three Disability Competent Managed Care Plans for Medicaid Enrollees

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/3MCPlanses.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/3MCPlans.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/3MCPlans.pdf

Effect of PACE on Costs, Nursing Home Admissions, and Mortality: 2006-2011

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/PACEeffectes.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/PACEeffect.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/PACEeffect.pdf

Effectiveness of Alternative Ways of Implementing Care Management Components in Medicare D-SNPs: The Brand New Day Study

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/OrthoV2s.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/OrthoV2.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/OrthoV2.pdf

Effectiveness of Alternative Ways of Implementing Care Management Components in Medicare D-SNPs: The Care Wisconsin and Gateway Study

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/OrthoV1es.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/OrthoV1.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/OrthoV1.pdf

Evaluating PACE: A Review of the Literature

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/PACELitReves.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/PACELitRev.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/PACELitRev.pdf

Factors Predicting Transitions from Medicare-Only to Medicare-Medicaid Enrollee Status

HTML http://aspe.hhs.gov/daltcp/reports/2014/MMTransV2.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/MMTransV2.shtml

Identifying Medicare Beneficiaries with Disabilities: Improving on Claims-Based Algorithms

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/algorithmes.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/algorithm.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/algorithm.pdf

Impacts of Waiting Periods for Home and Community-Based Services on Consumers and Medicaid Long-Term Care Costs in Iowa

HTML http://aspe.hhs.gov/daltcp/reports/2014/IAWaitPd.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/IAWaitPd.pdf

Integrating Physical Health Care in Behavioral Health Agencies in Rural Pennsylvania

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/ruralPAes.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/ruralPA.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/ruralPA.pdf

Non-Elderly Disabled Category 2 Housing Choice Voucher Program: An Implementation and Impact Analysis

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/Cat2Housinges.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/Cat2Housing.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/Cat2Housing.pdf

Parent Perspectives on Care Received at Patient-Centered Medical Homes for Their Children with Special Health Care Needs

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV2es.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV2.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV2.pdf

Physician Perspectives on the Influence of Medical Home Recognition on Practice Transformation and Care Quality for Children with Special Health Care Needs

HTML http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV1.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/ChildDisV1.pdf

Strategies for Integrating and Coordinating Care for Behavioral Health Populations: Case Studies of Four States

Executive Summary http://aspe.hhs.gov/daltcp/reports/2014/4CaseStudes.shtml

HTML http://aspe.hhs.gov/daltcp/reports/2014/4CaseStud.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/4CaseStud.pdf

Transitions from Medicare-Only to Medicare-Medicaid Enrollment

HTML http://aspe.hhs.gov/daltcp/reports/2014/MMTransV1.shtml

PDF http://aspe.hhs.gov/daltcp/reports/2014/MMTransV1.pdf

To obtain a printed copy of this report, send the full report title and your mailing information to:

U.S. Department of Health and Human ServicesOffice of Disability, Aging and Long-Term Care PolicyRoom 424E, H.H. Humphrey Building200 Independence Avenue, S.W.Washington, D.C. 20201FAX: 202-401-7733Email: webmaster.DALTCP@hhs.gov

RETURN TO:

Office of Disability, Aging and Long-Term Care Policy (DALTCP) Home [http://aspe.hhs.gov/office_specific/daltcp.cfm]Assistant Secretary for Planning and Evaluation (ASPE) Home [http://aspe.hhs.gov]U.S. Department of Health and Human Services (HHS) Home [http://www.hhs.gov]

Last Updated: 07/15/2014

Files

PACELitRev.pdf (pdf, 751.68 KB)

Topics

Program Evaluation | Nursing Homes & Facilities | Long-Term Services & Supports (LTSS)

Product Type

Literature Review

Populations

Older Adults

Program

Program of All-Inclusive Care for the Elderly (PACE)

Evaluating PACE: A Review of the Literature

Arkadipta Ghosh, Cara Orfield and Robert Schmitz

Mathematica Policy Research

January 2014

TABLE OF CONTENTS

ACRONYMS

EXECUTIVE SUMMARY

I. INTRODUCTION

II. CHALLENGES IN EVALUATING PACE

III. SCREENING CRITERIA AND STUDY RATING SCHEME

IV. FINDINGS FROM THE REVIEW OF PRIOR EVALUATIONS OF PACE

A. Costs

B. Nursing Home and/or Hospital Utilization

C. Quality of Care

D. Participant Satisfaction and Quality of Life

V. CONCLUSIONS

REFERENCES

APPENDIX A. SUMMARY OF PRIOR PACE EVALUATIONS AND THEIR STRENGTH OF EVIDENCE

I. Introduction

II. Summary Table

III. Studies Included in Review

IV. All Studies Identified

NOTES

CENTER OF EXCELLENCE IN RESEARCH ON DISABILITY SERVICES AND CARE COORDINATION AND INTEGRATION (CERDS)

REPORTS AVAILABLE

ASPE.hhs.gov

Evaluating PACE: A Review of the Literature

Arkadipta Ghosh, Cara Orfield and Robert Schmitz

Mathematica Policy Research

January 2014

TABLE OF CONTENTS

A. Costs

B. Nursing Home and/or Hospital Utilization

C. Quality of Care

D. Participant Satisfaction and Quality of Life

I. Introduction

II. Summary Table

III. Studies Included in Review

CENTER OF EXCELLENCE IN RESEARCH ON DISABILITY SERVICES AND CARE COORDINATION AND INTEGRATION (CERDS)

REPORTS AVAILABLE

Connect with Us