Evaluation of the SAMHSA Primary and Behavioral Health Care Integration (PBHCI) Grant Program: Final Report

December 2013

Printer Friendly Version in PDF Format: http://aspe.hhs.gov/daltcp/reports/2013/PBHCIfr.pdf (129 PDF pages)Free PDF Reader: http://get.adobe.com/reader/  

---

This report was prepared under contracts #HHSP23320095649WC and #HHSP23337015T between the U.S. Department of Health and Human Services (HHS), Office of Disability, Aging and Long-Term Care Policy (DALTCP) and the RAND Corporation. For additional information about the study, you may visit the DALTCP home page at http://aspe.hhs.gov/office_specific/daltcp.cfm or contact the ASPE Project Officer, David de Voursney, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, SW, Washington, DC 20201. His e-mail address is: David.DeVoursney@hhs.gov.

The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.

---

TABLE OF CONTENTS

PREFACE

ACKNOWLEDGMENTS

ACRONYMS

EXECUTIVE SUMMARY

1. INTRODUCTION

Overview

The Problem

Toward a Potential Solution

Primary and Behavioral Health Care Integration Grants

Primary and Behavioral Health Care Integration Program Evaluation Design

Report Structure

2. METHODS

Overview

Grantee Population

Evaluation Methods and Measures

3. PROCESS EVALUATION I: PROGRAM STRUCTURES AND PROCEDURES

Overview

Methods

Results

Summary

4. PROCESS EVALUATION II: CONSUMERS, CARE NEEDS, AND SERVICE UTILIZATION

Overview

Methods

Results

Summary

5. COMPARATIVE EFFECTIVENESS STUDY

Overview

Methods

Results

Discussion

6. MODEL FEATURES EVALUATION

Overview

Methods and Analyses

Results

Summary

7. SUMMARY, CONCLUSIONS AND IMPLICATIONS

Summary of Results

Conclusions and Implications for Integrating Behavioral Health and Physical Health Services

Program-Level Implications

Implications for the Field

Conclusions

REFERENCES

APPENDIX

NOTES

LIST OF FIGURES

FIGURE 1.1: Four Quadrants of Clinical Integration Based on Client Need

FIGURE 1.2: Structure-Process-Outcome Framework Guiding the PBHCI Evaluation

FIGURE 3.1: Grantee Scores on Four Dimensions of Integration and Overall

FIGURE 5.1: Change Trends in Case-Mix Adjusted Physical Health Indicators, Separately and Combined for Intervention/Control Site Pairs

FIGURE 6.1: Integrated Care Program Features Predict Consumer Use of Integrated Services; then Consumer Service Use Predicts Consumer Outcomes

LIST OF TABLES

TABLE 1.1: Standard Framework for Levels of Integrated Health Care

TABLE 2.1: Between-Cohort Differences for Grantees Included in and Excluded from the Multisite Evaluation

TABLE 2.2: Grantee Data Collection Requirements and Additional Data Sources for the PBHCI National Evaluation

TABLE 3.1: Web Survey Respondents by Job Type and Number of Grantees with Job Type Represented

TABLE 3.2: PBHCI Program Staff: Funded and Not Funded by PBHCI Grant

TABLE 3.3: Wellness Services Available over the Past Year in Individual or Group Settings

TABLE 3.4: Barriers to Implementing PC in BH Integration

TABLE 3.5: Grantee Levels of Integration Across Four Dimensions of Integration

TABLE 4.1: Planned and Actual Numbers of Consumers Receiving PC Services, Year 1 and Years 1-4, by Cohort

TABLE 4.2: Consumers Enrolled and Discontinued from PBHCI Across Grantees and for all Granees Combined, by Cohort

TABLE 4.3: Consumer Demographics, Percentage, by Cohort

TABLE 4.4: Physical Heath Indicator Values Showing Consumer Risk

TABLE 4.5: Consumers at Risk for Physical Health Conditions at Baseline, by Cohort

TABLE 4.6: Consumer Rates of Probable Chronic Physical Conditions, Percentage, by Cohort

TABLE 4.7: Percentage of Consumers Having Provider Contact Within 12 months of Enrolling in PBHCI, by Provider Type and Cohort

TABLE 4.8: PC Provider Encounters for Consumers With and Without Identified Physical Health Risk, by Cohort

TABLE 4.9: Consumer Physical Health Service Utilization, by Cohort

TABLE 4.10: Mental Health Service Utilization, All PBHCI Consumers, by Cohort

TABLE 4.11: Substance Use Service Utilization, all PBHCI Consumers, by Cohort

TABLE 4.12: Wellness Service Utilization, All PBHCI Consumers, by Cohort

TABLE 4.13: Quality of Care for Selected Physical Health Conditions, Percentage, by Cohort

TABLE 4.14: Operational Definitions of Basic and Comprehensive Integrated Care

TABLE 4.15: Consumer Access of Integrated Care During the First 12 Months in PBHCI

TABLE 5.1: Baseline and Follow-Up Data Sources and Timing

TABLE 5.2: Difference-in-Difference Analysis

TABLE 5.3: Number of SMI Consumers Services and Location of Intervention and Control Sites

TABLE 5.4: BH Services Provided at Intervention and Control Sites

TABLE 5.5: Demographics of Participating Consumers Within Intervention and Control-Site Pairs

TABLE 5.6: Proportions of Comparative Effectiveness Study Participants "At Risk" for Chronic Physical Illness at Study Baseline or Enrollment in PBHCI

TABLE 5.7: Characteristics of PC at Participating PBHCI Sites

TABLE 5.8: Integrated Care Program Structural Features of PBHCI Intervention Sties

TABLE 5.9: Proportion of Consumers with a PBHCI Provider Contact Within One Year of Enrollment

TABLE 5.10: Proportion of Consumers with Physical Health Service Contacts Within One Year of Enrolling in PBHCI

TABLE 5.11: Proportion of Consumers with Wellness Service Utilization Contacts Within One Year of Enrolling in PBHCI

TABLE 5.12: Physical Health Outcomes Combined Across All Consumers, Clinics

TABLE 5.13: Physical Health Outcomes for Persons with Baseline Risk for Physical Health Conditions, Combined Across All Participating PBHCI and Control Clinics

TABLE 5.14: Percentage of Consumers Who Showed Improvement, No Change, or Worsening Physical Health Risk from Baseline to One-Year Follow-Up

TABLE 5.15: Comparative Change in BH Indicators Among Persons Served at PBHCI and Control Clinics during the One-Year Study Period

TABLE 6.1: Operational Definitions of Basic and Comprehensive Integrated Care Service Use

TABLE 6.2: Step 1: Program Structures as Multivariate Predictors of Consumer Access to Basic and Comprehensive Integrated Care

TABLE 6.3: Program Integration Features as Multivariate Predictors of Consumer Access to Basic and Comprehensive Integrated Care

TABLE 6.4: Step 2: Change in Physical Health Indicators as predicted by Access to Integrated Care, Comparative Effectiveness Sample

TABLE A.1: Baseline and Follow-Up Data Sources and Timing

TABLE A.2: Change in Physical Health Indicators Between Persons Served at All Participating PBHCI and Control Clinics, by Intervention-Control Site Pair

TABLE A.3: Change in Physical Health Indicators from Intake to 12-Month Follow-Up, All PBHCI Consumers, Cohorts I-III

TABLE A.4: Change Physical Health Indicators as Predicted by Access to Integrated Care and PC Provider Contacts, PBHCI Cohorts I-III

PREFACE

The Substance Abuse and Mental Health Services Administration's (SAMHSA's) Primary and Behavioral Health Care Integration (PBHCI) grants program is intended to improve the overall wellness and physical health status of people with serious mental illness (SMI), including individuals with co-occurring substance use disorders (SUDs), by making available an array of coordinated primary care (PC) services in community mental health and other community-based behavioral health (BH) settings. In September 2009, SAMHSA partnered with the Office of the Assistant Secretary for Planning and Evaluation (ASPE) to oversee a one-year contract with the RAND Corporation to design an evaluation of the PBHCI grants program. In September 2010, SAMHSA and ASPE entered into a new, three-year contract with the RAND Corporation to execute the evaluation that RAND designed. This is the final report from RAND's evaluation of the PBHCI grants program.

The RAND evaluation of the PBHCI grants program had three main components, each designed to answer one of three research questions:

• Research Question 1 (Process Evaluation): Is it possible to integrate the services provided by PC providers and community-based BH agencies (i.e., what are the different structural and clinical approaches to integration being implemented)?

• Research Question 2 (Outcomes Evaluation): Does the integration of primary and BH care lead to improvements in the mental and physical health of the population with SMI and/or SUDs served by these models?

• Research Question 3 (Model Features Evaluation): Which models and/or model features of integrated primary and BH care lead to better mental and physical health outcomes?

To address these questions, RAND conducted a program-wide process evaluation that assesses PBHCI program structures, procedures, consumers, and their care needs and service utilization (Research Question 1); a small, comparative effectiveness study testing the physical health and BH outcomes of consumers served in PBHCI clinics (Research Question 2); and analysis designed to link PBHCI program features to consumer outcomes (Research Question 3). RAND discusses implications of the study results for programs and the broader field. RAND also provides options for future PBHCI-related research.

This report will be of interest to national and state policymakers, health care organizations and clinical practitioners, patient advocacy organizations, health researchers, and others responsible for ensuring that individuals with SMI receive appropriate preventive and primary health care services.

RAND's evaluation of the PBHCI grants program was sponsored by SAMHSA and ASPE contract No. OS-42345. Trina Dutta (SAMHSA), and David DeVoursney (ASPE) were the project officers. The research was conducted in RAND Health, a division of the RAND Corporation. A profile of RAND Health, abstracts of its publications, and ordering information can be found at http://www.rand.org/health.

ACKNOWLEDGMENTS

We would like to thank Sarah Hauer for her administrative support; Judy Perlman and Kim Todd for their work on the comparative effectiveness study; Benjamin Druss, Rebecca Collins, Jeanne Ringel, and Paul Koegel for reviewing and providing valuable feedback on draft versions of this report; and Donna Keyser for her work on the PBHCI evaluation design. We would also like to thank the project sponsors at the Office of the Assistant Secretary for Planning and Evaluation (David DeVoursney, Hakan Aykan, Kirsten Beronio, and Vidhya Alakeson) and at SAMHSA (Trina Dutta, Lisa Patton, Melanie Brown, and Crystal Blyler), as well as their technical assistance colleagues at the Center for Integrated Health Solutions. Most importantly, we would like to thank all of the grantee programs and staff that collected, cleaned, and submitted data; worked through the web survey; hosted site visits; and otherwise supported evaluation activities.

ACRONYMS

The following acronyms are mentioned in this report and/or appendix.

A1c	Glycated Hemoglobin
AHRQ	HHS Agency for Healthcare Research and Quality
ASPE	HHS Office of the Assistant Secretary for Planning and Evaluation
ATP	Adult Treatment Panel
BH	Behavioral Health
BL	Baseline
BMI	Body Mass Index
BP	Blood Pressure
CCM	Chronic Care Model
CDC	HHS Centers for Disease Control and Prevention
CEO	Chief Executive Officer
CFO	Chief Financial Officer
CHD	Coronary Heart Disease
CI	Confidence Interval
CIHS	HRSA Center for Integrated Health Solutions
CLIA	Clinical Laboratory Improvement Amendments
CM	Centimetre
CMHS	SAMHSA Center for Mental Health Services
CMS	HHS Centers for Medicare and Medicaid Services
CO	Carbon Monoxide
CQI	Continuous Quality Improvement
DBP	Diastolic Blood Pressure
dL	Deciliter
EBP	Evidence-Based Practice
EHR	Electronic Health Record
FPG	Fasting Plasma Glucose
FQHC	Federally Qualified Health Center
FTE	Full-Time Equivalent
FU	Follow-up
GAF	Global Assessment of Functioning
GED	General Educational Development
GPRA	Government and Performance Results Act
HDL	High-Density Lipoprotein
HDL-C	High-Density Lipoprotein Cholesterol
Hg	Hectogram
HHS	U.S. Department of Health and Human Services
HIT	Health Information Technology
HRSA	HHS Health Resources and Services Administration
ICSI	Institute for Clinical Systems Improvement
IOM	Institute of Medicine
kg	kilogram
LDL	Low-Density Lipoprotein
LDL-C	Low-Density Lipoprotein Cholesterol
LPN	Licensed Nurse Practitioner
mg	Milligram
mm	Millimetre
MOA	Memorandum of Agreement
NCQA	National Committee for Quality Assurance
NHLBI	NIH National Heart, Lung, and Blood Institute
NIH	HHS National Institutes of Health
NOM	National Outcome Measure
NP	Nurse Practitioner
OHD	Onsite Health Diagnostics
OMB	U.S. Office of Management and Budget
PA	Physician Assistant
PART	Program Analysis Review Tool
PBHCI	Primary and Behavioral Health Care Integration
PC	Primary Care
PCMH	Patient-Centered Medical Home
PCPI	Physician Consortium for Performance Improvement
PPM	Parts Per Million
RFA	Request for Applications
RN	Registered Nurse
SAMHSA	HHS Substance Abuse and Mental Health Services
SBP	Systolic Blood Pressure
SMI	Serious Mental Illness
Smok	Self-Reported Smoking Status
STD	Standard
SUD	Substance Use Disorder
TC	Total Cholesterol
TRAC	TRansformation ACcountability
Trig	Triglycerides
WC	Waist Circumference

EXECUTIVE SUMMARY

This report describes the RAND Corporation's evaluation of the Substance Abuse and Mental Health Services Administration's (SAMHSA's) Primary and Behavioral Health Care Integration (PBHCI) grants program. The PBHCI grants were designed to improve the overall wellness and physical health status of people with serious mental illness (SMI) and/or co-occurring substance use disorders (SUDs) by supporting the integration of primary care (PC) and preventive physical health services into community behavioral health (BH) centers where individuals already receive care. This evaluation provides information about the grantees' implementation of PBHCI, consumer outcomes, and PBHCI program features associated with consumer-level processes and outcomes of care. It also includes implications for programs and the boarder field, plus suggestions for future evaluation that may strengthen ongoing and future implementation of PBHCI.

Background

Excess morbidity and mortality in persons with SMI is a public health crisis. Compared with people without mental illness, individuals with SMI (e.g., schizophrenia, other psychoses, bipolar disorder, and severe depression) have higher rates of chronic medical conditions, including hypertension, diabetes, obesity, cardiovascular disease, and HIV/AIDS; higher frequency of multiple general medical conditions; and more than twice the rate of premature death resulting from these conditions (Kelly, Boggs, and Conley, 2007; Mauer, 2006; Parks et al., 2006; Sokal et al., 2004; Saha, Chant, and McGrath, 2007; Laursen et al., 2013).

Numerous factors contribute to the excess burden of general medical conditions among persons with SMI, including low levels of self-care, medication side effects, substance abuse comorbidity, unhealthy lifestyles, and socioeconomic disadvantage (Burnam and Watkins, 2006; CDC, 2012; Druss, 2007). The organizational and financial separation of the behavioral and general health care sectors contributes to disparities in access to and the quality of general medical care for people with SMI (Alakeson, Frank, and Katz, 2010; Bao, Casalino, and Pincus, 2013; Druss, 2007; Horvitz-Lennon, Kilbourne, and Pincus, 2006).

SAMHSA's PBHCI service grant program is intended to improve the health status among adults with SMI and/or co-occurring SUDs by making available an array of coordinated PC services in community mental health centers and other community-based BH settings. The PBHCI grantees evaluated in this report received $500,000 per year to coordinate access to PC and/or services for which there was no funding source, including four core (required) program features:

1. screening/referral for needed physical health prevention and treatment;
2. developing a registry/tracking system for physical health needs/outcomes;
3. care management;
4. prevention and wellness support services.

Grantees could also implement six optional program features (same day physical and BH visits; co-located, routine PC services; a supervising PC physician; an embedded nurse care manager; evidence-based practices for preventive care; and wellness programs), infrastructure development, and performance measurement activities.

In 2009-2010, RAND designed the PBHCI evaluation around a structure-process-outcomes framework (Donabedian, 1966, 1980). The evaluation had three evaluation components, each designed to answer one of three research questions:

• Research Question 1 (Process Evaluation): Is it possible to integrate the services provided by PC providers and community-based BH agencies (i.e., what are the different structural and clinical approaches to integration being implemented)?

• Research Question 2 (Outcomes Evaluation): Does the integration of primary and BH care lead to improvements in the mental and physical health of the population with SMI and/or SUDs served by these models?

• Research Question 3 (Model Features Evaluation): Which models and/or model features of integrated primary and BH care lead to better mental and physical health outcomes?

RAND then won a separate three-year contract to conduct this evaluation work (2010-2013). The results of this PBHCI evaluation are described below.

Results

Research Question 1 (Process Evaluation)

To answer this descriptive, process-oriented question, we measured the extent to which key integration features and strategies were present at each grantee site (program and staff-level analyses) and the degree to which individuals with SMI received appropriate integrated services (consumer-level analysis). Data showed that PBHCI programs had multidisciplinary teams with different staff mixes, and that they had different infrastructures and offered different packages of services. Programs also varied in the extent to which their structures and procedures reflected integrated care, with programs offering variable levels of co-located services, structures, and systems shared by primary and BH care providers, integrated practices, and clinic cultures.

PBHCI programs also served a diverse population of consumers with high rates of need for integrated primary and BH care services. Once enrolled in PBHCI, most consumers had some primary and BH care contact during their first year in the program, and more than half accessed a basic package of integrated services, including screening or treatment planning, PC, and case management; consumers were less likely to have accessed substance abuse-related services and wellness services targeting smoking and weight. Improving consumer access to the full array of PBHCI services, particularly among consumers with identified physical health needs, could be a target for future improvements to PBHCI.

Research Question 2 (Outcomes Evaluation)

We conducted a small, comparative effectiveness study consisting of three matched PBHCI and control clinic pairs. Results of a difference-in-difference analysis showed that, relative to consumers receiving services at control clinics, PBHCI consumers showed improvements in some (diastolic blood pressure, total cholesterol, LDL cholesterol and fasting plasma glucose) but not all (systolic blood pressure, body mass index, HDL cholesterol, glycated hemoglobin, triglycerides, self-reported smoking) of the physical health indicators examined. Compared with consumers served at control-sites, consumers served through PBHCI showed no benefit in terms of indicators of BH.

Research Question 3 (Model Features Evaluation)

Instead of implementing different integrated care models in their entirety (e.g., Cherokee model, Chronic Care Model), our initial work showed that many programs implemented "bits and pieces" or combinations of several integration models (Scharf et al., 2013). As such, our approach to Research Question 3 focused on model features whose presence or absence could be reliably assessed. To answer Research Question 3, we used the full sample of 56 grantee data to first identify program-level predictors of consumer access to PC providers and packages of integrated care. Then we used data from the three intervention sites included in the comparative effectiveness evaluation (Research Question 1) to test the relationship between consumer access to primary, integrated care and consumer physical health outcomes. Overall, results showed that several program features had an effect on consumer access to integrated care (e.g., the number of days a PC clinic was open per week, regularly scheduled integrated staff meetings, and other aspects of program-level integration increased access; rural location decreased access), but consumer access to PC and integrated care was not clearly associated with physical health outcomes.

Conclusions

PBHCI programs were successful in several ways, such as building integrated, multidisciplinary teams that offer an array of integrated primary, BH, and wellness services, and across PBHCI grantee programs, these services were provided to a diverse clientele with high rates of need for integrated care. PBHCI programs also experienced several challenges, including lower-than-expected rates of consumer enrollment, financial sustainability, intra-team communication, and creating an integrated clinic culture. These programs also experienced challenges related to implementing wellness programs and improving consumer smoking and weight outcomes. Ongoing and future cohorts of grantees could consider several options to improve program implementation, such as maximizing data-driven, continuous quality improvement; monitoring implementation fidelity to evidence-based wellness programs; and investing in strategies that improve consumer access to integrated services, among others. Stakeholders in the field of integrated care could benefit from consensus around program performance expectations, and the establishment of national quality indicators for integrated care accountability and core performance monitoring requirements. Finally, technical assistance providers could consider continuing dissemination of emerging best care practices for adults with SMI and supporting grantees navigating concurrent health care reforms. Future evaluations, such as an evaluation of PBHCI utilization and costs, strategies to improve sustainability, and a prospective trial of alternative models of integrated care could help SAMHSA and grantees demonstrate the value of their PBHCI work.

1. INTRODUCTION

Overview

This report describes the RAND Corporation's evaluation of Primary and Behavioral Health Care Integration (PBHCI), one of the Substance Abuse and Mental Health Services Administration's (SAMHSA's) service grant programs. PBHCI is intended to improve the overall wellness and physical health status of people with serious mental illness (SMI), including individuals with co-occurring substance use disorders (SUDs), by making available an array of coordinated primary care (PC) services in community mental health centers and other community-based behavioral health (BH) settings. In particular, better coordination and integration of PC and BH services, improved prevention, early identification and intervention to reduce chronic diseases, and the enhanced capacity to holistically serve those with mental and/or SUDs are expected to better the overall health status of the population served. The first 13 PBHCI grants were awarded in September 2009. At the time of this report, 100 community BH agencies had received PBHCI grants (through a total of six funding waves across two funding announcements) to provide integrated PC and BH services to their adult clients with SMI.

In 2009-2010, RAND designed the PBHCI evaluation to provide information on the program's implementation strategies and processes, whether the program leads to improvements in outcomes, and which program models and/or model features lead to better outcomes. The evaluation design was rooted in a structure-process-outcomes framework, following the assumption that health care outcomes (e.g., symptoms, quality of life, functional status) are influenced both by the structure of care (e.g., what services are available) and the processes of care (e.g., to what degree the services are implemented and appropriately delivered to clients) (Donabedian, 1966, 1980). In 2010, RAND won a separate contract to conduct an evaluation of the PBHCI grants program, which at the time included the first 56 grantees awarded through September 2010.

In this chapter, we introduce the reader to the problem of chronic physical illness among adults with SMI, how integrated care--including PBHCI-funded programs--might help to reduce these disparities, and how RAND's evaluation of the PBHCI program illustrates the implementation and impact of the program overall.

The Problem

Public Health Crisis

Excess morbidity and mortality in persons with SMI is a public health crisis. Compared with people without mental illness, individuals with SMI (e.g., schizophrenia, other psychoses, bipolar disorder, and severe depression) have higher rates of chronic medical conditions, including hypertension, diabetes, obesity, cardiovascular disease, and HIV/AIDS; a higher frequency of multiple general medical conditions; and more than twice the rate of premature death resulting from these conditions (Kelly, Boggs, and Conley, 2007; Mauer, 2006; Parks et al., 2006; Sokal et al., 2004; Saha, Chant, and McGrath, 2007; Laursen et al., 2013). Physical comorbidities adversely impact quality of life and can have a detrimental effect on BH, compounding already high levels of functional impairment among persons with SMI (Dixon et al., 1999; Druss et al., 2000; Pirraglia et al., 2009; Proudfoot et al., 2012).

Factors Underlying the Crisis

Numerous factors contribute to the excess burden of general medical conditions among persons with SMI. These include mental illness-related factors such as low motivation and neglect of self-care; medication side effects; substance abuse comorbidity; unhealthy lifestyles, including inadequate physical activity, poor nutrition, and smoking; fears related to using general medical services; and socioeconomic disadvantage (Burnam and Watkins, 2006; CDC, 2012; Druss, 2007). Limited access to and poor quality of general medical care also contribute to this disparity (Lawrence and Kisely, 2009). For example, people with SMI are less likely to have health insurance than people without mental illness, they perceive more barriers to obtaining preventive and general health care, and they report that providers dismiss their somatic complaints (Salsberry, Chipps, and Kennedy, 2005; Mechanic and Bilder, 2004; Katon and Unutzer, 2013).

For many people with SMI, specialty BH settings are a trusted point of contact with the broader health system (Alakeson et al., 2010); however, specialty mental health providers often lack expertise or comfort in diagnosing or treating medical conditions (Golomb et al., 2000; Shore, 1996; Lawrence et al., 2009; Bao, Casalino, and Pincus, 2013). Community mental health centers also tend not to have the capacity to conduct routine health promotion activities, PC screening, monitoring, or on-site treatment nor the infrastructure and incentives to coordinate care with patients' medical providers (Samet, Friedmann, and Saitz, 2001; Bao, Casalino, and Pincus, 2013). As a result, many people with SMI resort to seeking general medical care in emergency rooms, resulting in overcrowding and high costs, as well as inappropriate care and poor health outcomes. Even when people with SMI do make contact with the general medical system, they receive less comprehensive services and lower quality care than other groups (Druss et al., 2000; Druss et al., 2001; Druss, 2007). People with SMI who are older and/or who abuse drugs (populations with particularly high rates of general medical conditions) are at even higher risk for substandard general health care (Dickey et al., 2002; Druss et al., 2001).

The organizational and financial separation of the behavioral and general health care sectors is commonly recognized as a key contributor to disparities in access to and quality of general medical care, especially for people with SMI (Alakeson et al., 2010; Bao, Casalino, and Pincus, 2013; Druss, 2007; Horvitz-Lennon, Kilbourne, and Pincus, 2006). As a result, specialty mental health clinics in the public sector (e.g., community mental health centers and substance abuse programs) are typically accountable only for the treatment of BH conditions rather than the full-scope of health issues affecting people with those conditions. As noted by the Institute of Medicine (IOM) nearly a decade ago, the general separation of medical and BH care makes coordinated health care especially unlikely for persons with SMI (IOM, 2006).

Toward a Potential Solution

Achieving greater clinical and health system integration between service providers and systems is a centerpiece of the IOM strategy for improving the quality of care for people with mental health and SUDs (IOM, 2006). It is also a strategy endorsed by the President's New Freedom Commission on Mental Health (2003) and the Bazelon Center for Mental Health Law (2004). Clinical integration is also a fundamental component of national health care reform (Rittenhouse and Shortell, 2009; Katon and Unutzer, 2013).

Integrated primary and BH care is expected to address a number of clinical and health system issues creating disparities in health and health care for adults with SMI. For example, issues such as lower medication adherence, higher incidence of co-occurring chronic medical conditions, higher incidence of co-occurring alcohol and drug abuse problems, the lack of a stable medical home, and the need for more complex medical plans, common among adults with SMI (Mauer, 2006), are expected to be addressed in integrated care settings. In particular, these settings provide more time for providers to address complex medical issues; stronger collaborative relationships between PC and BH providers; improved capacity for PC providers to distinguish between physical health and BH problems; and opportunities for same day, same facility appointments, which could improve adherence to treatment recommendations such as routine screenings and medication checks (Koyanagi, 2004). While simply combining streams of primary and BH care services is unlikely to create an environment in which all of these optimal conditions are necessarily met, several approaches to integration have been attempted and specific integration models with specific strategies for effective integration have been proposed.

Integration Models, Standards, and Implementation

Integrated primary and BH care can work in two directions, either: (1) specialty BH care is introduced into PC settings; or (2) PC is introduced into specialty BH settings. There is growing consensus that adults with SMI, particularly those with comorbid chronic physical health conditions, should receive care based in specialty BH settings based on perspectives that people are best served in the settings where they have principal connections with the health care system (Alakeson et al., 2010), or by their degree of medical and BH care needs; this population and its service needs are represented in Quadrant Four of Mauer's (2006) Four Quadrant Model of Clinical Integration (Figure 1.1).

FIGURE 1.1. Four Quadrants of Clinical Integration Based on Client Need
Behavioral Health Needs	Physical Health Needs
	Low	High
Low	Quadrant IServe in primary care settinge.g., persons with moderate alcohol abuse and fibromyalgia	Quadrant IIIServe in primary care settinge.g., persons with moderate depression and uncontrolled diabetes
High	Quadrant IIServe in primary care and specialty mental health settingse.g., persons with bipolar disorder and chronic pain	Quadrant IVServe in primary care and specialty mental health settingse.g., persons with severe depression and uncontrolled diabetes
SOURCE: Mauer, 2006; Collins et al., 2010.

Yet, the academic literature on BH-based integrated care is in its infancy and strategies for integrating BH-based programs of integrated care come primarily from PC-based integration experiences. Some of the lessons learned that may be relevant to BH-based programs suggest that implementation will require practice and policy changes at multiple levels (e.g., service delivery, information exchange, health care workforce, financing of care, quality oversight) (Horvitz-Lennon, Kilbourne, and Pincus, 2006) and that multidimensional efforts to improve integrated care are more likely to achieve positive results (IOM, 2006). According to the IOM, successful integration efforts typically include screening for co-occurring conditions, making a formal determination to either treat or refer individuals for treatment of co-occurring conditions, implementing more effective mechanisms for linking providers of different services to enable joint planning and coordinated treatment, and providing organizational supports for collaboration between clinicians on-site and off-site. The extent to which responsibility and accountability are explicitly assigned among providers and delivery systems involved in the integrated care process is critically important to its overall success.

Some of the most well-known, explicit models for integrating care have grown successively out of the Chronic Care Model (CCM). The CCM incorporates six elements for improving the quality of chronic illness care: (1) providing chronic illness self-management support to patients and their families; (2) redesigning care delivery structures and operations; (3) linking patients and their care with community resources to support the management of their illness; (4) providing decision support to clinicians; (5) using computerized clinical information systems to support compliance with treatment protocols and monitor patients' health indicators; and (6) aligning the health care organization's (or provider's) structures, goals, and values to support chronic care (Bodenheimer et al., 2002). The CCM's emphasis on the use of interdisciplinary structures and practices in which a clear division of the roles and responsibilities of the various team members fosters their collaboration is a noteworthy innovation of the model (IOM, 2006); however, it may also constitute an implementation challenge, as it requires new roles and divisions of labor among clinicians with differing training and expertise (IOM, 2006).

The Patient-Centered Medical Home (PCMH), rooted in the CCM, includes seven principles for providing comprehensive care that facilitates partnerships between individual patients and their personal physicians (Joint Principles of the PCMH, 2007; Croghan and Brown, 2010). These principles are: (1) personal physician; (2) physician-directed medical practice; (3) whole-person orientation; (4) coordinated and/or integrated care across the health system; (5) quality and safety; (6) enhanced access to care; and (7) appropriate payment structure. The National Committee for Quality Assurance (NCQA) has now published standards and guidelines for a PCMH certification process (NCQA, 2011a, 2011b), and through this process there is the opportunity to increase consistency of care across PCMHs. As of April 2013, 43 states have adopted a policy to advance medical homes (National Academy for State Health Policy, 2013) and a large Medicare demonstration project is currently underway (Bao, Casalino, and Pincus, 2013).

The newest of the models reviewed here--the Health Home--was established as an incentivized option for state Medicaid programs under section 2703 of the Affordable Care Act and builds on the PCMH to provide accessible and accountable services for individuals with multiple chronic conditions. The Health Home includes key PCMH characteristics such as access to and coordination of services, including preventive care, and the adoption of recovery orientation, among others (Smith and Sederer, 2009; Alakeson et al., 2010). Health Homes can be specialized to meet the needs of a particular population, such as adults with SMI. Standards for Behavioral Health Home certification targeting adults with SMI are scheduled to be released by the Joint Commission in early 2014 and will provide a framework to help organizations provide quality care, meet state Medicaid requirements, and improve their reimbursement structure for integrated primary and BH care (Joint Commission, 2013).

While accreditation standardizes the way in which integrated care is implemented, variability in model implementation can still occur. Even among organizations endorsing a single model and sharing several key components of care, the degree to which primary and BH care is truly integrated in practice can vary. Recently, the SAMHSA Health Resources and Services Administration (HRSA) Center for Integrated Health Solutions (CIHS)--the first "national home" for information and resources dedicated to bidirectional integration of BH and PC--endorsed a standard framework to describe levels of integration along a continuum (CIHS, 2013). Based on the pioneering work of others (Doherty, 1995; Blount, 2003), CIHS provides a six-level framework for classifying within-program integration (Table 1.1).

TABLE 1.1. Standard Framework for Levels of Integrated Health Care
Integration  Categories	Integration  Levels	Description
Coordinated Care	Level 1--Minimal Collaboration	BH and PC providers work at separate facilities and have separate systems. Providers communicate rarely about cases. When communication occurs, it is usually based on a particular provider's need for specific information about a mutual consumer.
	Level 2--Basic Collaboration at a Distance	BH and PC providers maintain separate facilities and separate systems. Providers view each other as resources and communicate periodically about shared consumers. These communications are typically driven by specific issues. For example, a PC physician may request a copy of a psychiatric evaluation to know if there is a confirmed psychiatric diagnosis. BH is most often viewed as specialty care.
Co-located Care	Level 3--Basic Collaboration On-site	BH and PC providers are co-located in the same facility but may or may not share the same practice space. Providers still use separate systems, but communication becomes more regular due to close proximity, especially by phone or email, with an occasional meeting to discuss shared consumers. Movement of consumers between practices is most often through a referral process that has a higher likelihood of success because the practices are in the same location. Providers may feel like they are part of a larger team, but the team and how it operates are not clearly defined, leaving most decisions about consumer care to be made independently by individual providers.
	Level 4--Close Collaboration with Some System Integration	There is closer collaboration between PC and BH providers due to co-location in the same practice space, and there is the beginning of integration through some shared systems. A typical model may involve a PC setting embedding a BH provider. In an embedded practice, the PC front desk schedules all appointments and the BH provider has access and enters notes in the medical record. Often, complex consumers with multiple health care issues drive the need for consultation, which is done through personal communication. As processionals have more opportunity to share consumers, they have a better basic understanding of each other's roles.
Integrated Care	Level 5--Close Collaboration Approaching an Integrated Practice	There are high levels of collaboration and integration between BH and PC providers. The providers begin to function as a true team, with frequent personal communication. The team actively seeks system solutions, as it recognizes barriers to care integration for a broader range of consumers. However, some issues, like the availability of an integrated medical record, may not be readily resolved. Providers understand the different roles team members need to play and they have started to change their practice and the structure of care to achieve consumer goals.
	Level 6--Full Collaboration in a Transformed/Merged Practice	The highest level of integration involves the greatest amount of practice change. Fuller collaboration between providers has allowed antecedent system cultures (whether from two separate systems or from one evolving system) to blur into a single transformed or merged practice. Providers and consumers view the operation as a single health system treating the whole person. The principle of treating the whole person is applied to all consumers, not just targeted groups.
SOURCE: CIHS, 2013.NOTES: BH = behavioral health; PC = primary care.

These aspects of how resources are brought together and how services are framed and delivered can have significant consequences for the nature and quality of care that consumers receive.

Evidence to Date and Remaining Knowledge Gaps

Multiple randomized, controlled trials have shown that variants of PC-based integrated and/or collaborative care improve quality of care for and outcomes of substance use and mental health disorders (Druss and Mauer, 2010; Katon and Unutzer, 2013). While there are many fewer studies of BH-based integrated care, reviews suggest that this approach is also promising (Druss and von Esenwein, 2006; Butler et al., 2008). Broadly, studies of BH-based integrated care found improvements in consumers' general medical health and BH and in the quality of care provided (Butler et al., 2008; Druss et al., 2006). For instance, consumers served at BH-based integrated care programs have shown an increase in PC visits, improved attainment of performance measures related to metabolic and cardiovascular risk, and reduced emergency department use (Pirraglia et al., 2012; Druss et al., 2010; McGuire et al., 2009; Saxon et al., 2006; Zappe and Danton, 2004).

The extant literature, however, does not reflect a significant range of BH-based integration approaches that are currently occurring in the field. Specifically, most published studies have been conducted in large, integrated health systems, such as the Veterans Health Administration or other large health maintenance organizations in which PC providers and BH staff were already working together to provide care. Furthermore, such integrated health systems operate as single fiscal entities in which all providers are paid by the same organization, which limits the administrative and financial barriers to integrated care that these programs face. Less systematic accounts of integration of PC in smaller BH systems describe greater and more unique challenges, such as those related to inadequate space for PC activities and difficulty integrating PC activities into the existing organizational workflow (Boardman, 2006). Finally, most research on integrated care has focused on care for people with depression, not SMI more broadly.

More research is needed on the benefits and challenges of integrating PC into diverse BH settings, the approaches and processes by which care is delivered, and their outcomes for adults with SMI. This report, describing SAMHSA and the Office of the Assistant Secretary for Planning and Evaluation's (ASPE's) assessment of the PBHCI grants program, seeks to provide much needed information about this appealing yet understudied and underspecified approach to integrated care.

Primary and Behavioral Health Care Integration Grants

The PBHCI grants program is intended to improve the overall wellness and physical health status of people with SMI, including individuals with co-occurring SUDs, by making available an array of coordinated PC services in community mental health centers and other community-based BH settings. In particular, better coordination and integration of PC and BH services, improved prevention, early identification and intervention to reduce chronic diseases, and enhanced capacity to holistically serve those with mental and/or SUDs are expected to lead to better overall health status of the population served.

More specifically, PBHCI grants were intended to support services for adults with SMI receiving care in the public mental health system. While later waves of PBCHI grantees responded to different requests for applications (RFAs) with slightly different foci, requirements, and funding conditions, the first three cohorts of PBHCI (n=56 of now 100) grantees are the focus of this report, since they were funded at the time that the evaluation (described below) began. These first three cohorts received $500,000 per year over four years to implement four core and six optional program features comprising integrated primary and BH services for adults with SMI as described below. Additional detail about the similarities and differences between the early and later grantee cohorts is given in Chapter Two (see Table 2.1).

Core and Optional Program Features

The first 56 PBHCI grantees implemented core (required) and optional program features identified in the RFA, as well as a range of other activities that were not specified in the RFA. Therefore, while programs had some features in common, they also varied widely in terms of how integration was conceptualized and operationalized in practice.

The four core program features represented services targeted to coordinate access to PC and/or services for which there was no funding source. These features were:

1. Screening/referral for necessary PC prevention and treatment, including screening/assessment/treatment and referral for hypertension, obesity, smoking, and substance abuse.

2. Developing a registry/tracking system for all PC needs and outcomes for consumers with SMI.

3. Care management, understood as individualized, person-centered planning and coordination to increase consumer participation and follow-up with all PC screening, assessment, and treatment services, including the involvement of consumers and family members in service development and implementation and peer/support management services.

4. Incorporation of prevention and wellness support services, including nutrition consultation, health education and literacy, peer specialists, and self-help/management programs, into individualized wellness plans for each person receiving services through the grant.

Core program features could be implemented through any strategy proposed by the grantee. Grantees could provide the PC services themselves, purchase them through contracts with other providers, or make them available through a memorandum of agreement (MOA) with other providers.

Six optional strategies for integrating PC services into community mental health settings were also proposed in the RFA:

1. Undertake regular screening and registry tracking/outcome measurement at the time of psychiatric visits for all individuals receiving psychotropic medications.

2. Co-locate medical nurse practitioners (NPs)/PC physicians in BH facilities whose charge is to provide routine PC services.

3. Identify a PC supervising physician within the full-scope health care home to provide consultation on complex health issues for the psychiatrist, medical NPs, and/or nurse care manager.

4. Embed nurse care managers within the PC team working in the BH setting to support specific individuals (i.e., those with significantly elevated levels of glucose, lipids, blood pressure (BP), and weight/body mass index [BMI]).

5. Use evidence-based practices (EBPs) in clinical preventive services developed to improve the health status of the general population, adapting these practices for use in the BH system.

6. Create wellness programs that utilize proven methods/materials developed for engaging individuals in managing their health conditions, adopting these programs for use in the mental health setting with peers serving as group facilitators.

Infrastructure Development

SAMHSA anticipated that some infrastructure development could be necessary for grantees to successfully implement their PBHCI programs. As such, up to 25% of the total grant award could be used for infrastructure such as interagency coordination mechanisms and partnerships with other service providers for service delivery (e.g., building provider networks and linkages among service partners); policies to support needed collaborative service system improvement (e.g., changes in standards of practice and data sharing); workforce development (e.g., training, support for certification/licensure, or credentialing); enhanced computer systems, management information systems, and electronic health records (EHRs); training/workforce development to assist in the provision of effective services consistent with the purposes of the grant, as well as coordinating access to and enrollment in public/private insurance; and process redesign to enhance effectiveness, efficiency, and optimal collaboration between PC and BH provider staff.

Data Collection, Performance Measurement, and Assessment

Grantees could designate up to 20% of the total grant award for expenses related to data collection, performance measurement, and assessment. PBHCI grantees, especially in the first three cohorts, had considerable data collection responsibilities (see Chapter Two). Briefly, grantees collected data on client-level BH, physical health, and service utilization, as well as program-level infrastructure development, prevention and mental health promotion, and program innovation and implementation.

They were also asked to provide an assessment of the PC needs of the consumers with SMI served by their agency, to describe how their project data would be analyzed by racial/ethnic group to ensure that appropriate populations were being served and disparities were minimized, and to develop a plan to sustain integrated services beyond the life of the grant.

Primary and Behavioral Health Care Integration Evaluation Design

Administrative Context

The PBHCI grants program is of interest to several federal agencies, including SAMHSA, HRSA, ASPE, and the Centers for Medicare and Medicaid Services (CMS). As such, several aspects of the broader PBHCI initiative have resulted from collaborations between each of these agencies. For instance, the Technical Assistance Center is jointly funded by SAMHSA and HRSA, and the PBHCI evaluation design and evaluation were jointly funded and administered by SAMHSA and ASPE. PBHCI was also mentioned in CMS' 2703 State Medicaid Director letter to states as a source of information on how to integrate primary and BH care services. In short, the PBHCI grants program is the result of targeted interagency collaboration specifically focused on integrated care for adults with SMI (see Alakeson et al., 2010). The evaluation design and procedures approved and supported by SAMHSA and ASPE are described below.

Overview and Conceptual Framework

The purpose of this project was to evaluate the PBHCI grants program in order to understand: (1) implementation strategies and processes; (2) whether the program leads to improvements in outcomes; and (3) which program models and/or model features lead to better outcomes. To answer these questions, RAND designed the PBHCI evaluation around a structure-process-outcomes framework, following the assumption that health care outcomes (e.g., symptoms, quality of life, and functional status) are influenced both by the structure of care (e.g., what services are available) and the processes of care (e.g., to what degree the services are implemented and appropriately delivered to clients) (Figure 1.2) (Donabedian, 1966, 1980). In 2009, RAND won the contract to design the evaluation for the first 13 PBHCI grantees concurrent with the first year of program implementation. As additional grant funds became available, the design was adapted and expanded to accommodate the larger pool of grantees as much as possible.

FIGURE 1.2. Structure-Process-Outcome Framework Guiding the PBHCI Evaluation
Independent Variables				Dependent Variables
Structure of Care		Process of Care		Outcomes of Care
What services are available?Core (required) features: Screening/referral/treatment Registry/tracking system Care management Prevention/wellness approach Examples of optional features: Co-location of nurse practitioners/PCPs Identification of primary care supervising physician Embedded nurse care managers Use of EBPs Use of SBIRT		To what degree are services implementation?Examples: Numbers/proportions of SMI patients screened/referred/treated & for what Number of records/types of information tracked in registry Number of SMI patients connected with care manager & for what Number of individualized wellness plans developed and utilized		Does it make a difference?Improved overall health status of clients Physical health outcomes (e.g., BMI, blood pressure, blood glucose or HbAlc, lipid provide) Mental health outcomes
Environment/Admistrative Context(e.g., size, suspice, location, health IT)

The structure of PBHCI care previously included four core features of integration and an unlimited number of optional features that grantees could choose to implement. Each of these features represented some standard set of components, functionalities, and/or practices that may or may not have been present at grantee sites. Moreover, the strategies with which the grantees implemented these features, the comprehensiveness of implementation and what happened as a result of their implementation efforts could also be different across sites, due in part to the actual components, functionalities, and/or practices of each feature present at each site, as well as their different environments and administrative contexts. This report makes a unique contribution to the empirical literature, as it provides new information on what works in BH-based PBHCI in myriad settings; it also describes how it works.

SAMHSA and its collaborating agencies identified three main research questions to be answered through this evaluation.

Research Question 1: Process Evaluation

Is it possible to integrate the services provided by PC providers and community-based BH agencies (i.e., what are the different structural and clinical approaches to integration being implemented)? To answer this descriptive, process-oriented question, we measured the extent to which key integration features and strategies were present at each grantee site (program and staff-level analyses) and the degree to which individuals with SMI received appropriate integrated services (consumer-level analysis). The data used to answer this research question came from both quantitative and qualitative sources.

Quantitative information describing program structural features and clinical processes came from a program and staff-level web-based survey designed to capture information on program implementation and operation, as well as from client-level service utilization data collected from grantees. Client-level service utilization data were used to characterize the types and amount of services received by PBHCI consumers, as well as to construct quality metrics of care wherever possible. These data were supplemented by qualitative information collected through quarterly reports submitted by all grantees for all quarters and RAND-conducted site visits at six sites selected to be broadly representative of the program as a whole. PBHCI processes were therefore described quantitatively and supplemented with specific examples of program implementation and conduct that were noted by RAND researchers while on-site at PBHCI programs around the country.

Research Question 2: Outcomes Evaluation

Does the integration of primary and BH care lead to improvements in the mental and physical health of the population with SMI and/or substance use disorders served by these models? To answer this question, we compared individuals at three selected intervention (PBHCI) sites and three matched control-sites on changes in physical health and BH indicators over time (a quasi-experimental design). The comparative effectiveness study was limited to three intervention sites due to resource constraints.

Within the context of this quasi-experimental design, we used a difference-in-difference analysis to estimate the size of PBHCI effects on various outcomes among the population served. This method entails comparing the differences of the two repeated outcome measurements in each individual served at intervention and control-sites. For example, we compared the differences in baseline and follow-up measurements of BP between consumers served at intervention and control-sites. To ensure that these comparisons were fair and meaningful, we balanced the intervention and control-site populations using propensity score matching. A propensity score (i.e., the probability that an individual with known characteristics was in the intervention group) was used to select or weight control cases so that they provided an estimate of what the outcomes would have been for intervention cases had they not received PBHCI services. Intervention outcome effects were then estimated by comparing individuals treated at PBHCI sites with these weighted control cases. These data provide a non-representative but rigorous assessment of PBHCI effects on consumer physical health and BH as compared to conditions in a non-integrated setting.

Research Question 3: Model Evaluation

Which models and/or model features of integrated primary and BH care lead to better mental and physical health outcomes? Our initial work describing the PBHCI grantees suggested that grantees selected implementation models (e.g., CCM, Cherokee, etc.) that were not reliably associated program structures or processes (i.e., many programs implemented "bits and pieces" or combinations of several integration models). Therefore, our approach to Research Question 3 focuses on model features whose presence or absence could be assessed more reliably and their relationship to outcomes.

We used data from the web-based survey (collected from 55 of the 56 participating grantee sites) to examine correlational relationships between model features, processes of care, and outcomes. Specifically, since there were only three PBHCI sites included in the comparative effectiveness study (i.e., outcomes evaluation--Research Question 2), we could not conduct an analysis that included model features/strategies, processes of care, and outcomes in a single analytic modeling effort. To circumvent this sample size issue, we separately analyzed data from the 55 participating sites to examine the relationship between structure (model features measured at the program level) and process of care indicators (measured at the client level). The results of this analysis identified specific model features associated with higher rates of appropriate care. We then conducted a separate, second set of analyses that was restricted to the three programs in the comparative effectiveness evaluation and extended the individual-level difference-in-difference analysis to include processes of care indicators that predict outcomes and may help to explain differences in outcomes among the three program sites. This two-step approach enabled us to draw tentative conclusions about which model features are most likely to result in processes of care that predict positive outcomes.

Report Structure

In Chapter Two, we provide basic information about the project methodology, supplemented by additional methodological detail in the Appendix. We then answer Research Question 1 over two sequential chapters, each describing fundamental components of the process evaluation. Chapter Three describes the PBHCI programs' locations, physical structures, staff teams, and processes. Chapter Four describes PBHCI consumers' care needs and service utilization. Specifically, we describe the characteristics of consumers served, their degree of assessed risk for chronic physical health conditions, and the services that they received while enrolled in PBHCI during their first year in care. The process evaluation results provide important contextual information. In Chapter Five, we describe the outcomes evaluation (Research Question 2) that tested the effects of PBHCI on consumer physical health indicators in the context of a small comparative effectiveness study. Since our approach to answering Research Question 3 integrates data and findings from Research Questions 1 and 2, we answer this research question last by relating model features to consumer outcomes in Chapter Six. In Chapter Seven, we summarize our results and provide implications for programs and the field at large.

2. METHODS

Overview

This chapter provides an overview of the sample of PBHCI grantees included in the evaluation, the data collection methods, and the measures used to answer Research Questions 1 (Process), 2 (Outcomes), and 3 (Model Features). Additional methodological information about data sources and analysis is provided throughout the report, as well as in the Appendix.

All PBHCI evaluation procedures were reviewed and approved by RAND's Human Subjects Protection Committee and the federal U.S. Office of Management and Budget (OMB).

Grantee Population

At the time that this evaluation was designed, SAMHSA had funded 13 PBHCI grantees. Midway through the evaluation design, RAND received notice that the evaluation would be expanded by nine grantees for a total of 22 grantees. At the time of this report, SAMHSA had awarded 100 PBHCI grants. Of these 100 grants, the first 56 programs (awarded across three subsequent waves, or cohorts, of funding) were included in this evaluation because they were funded at the time that the evaluation began. Cohort I includes 13 grantees (awarded in September 2009), Cohort II includes nine grantees (awarded in September 2010), and Cohort III includes 34 grantees (also awarded in September 2010). Cohort IV (awarded in September 2011), Cohort V (awarded in September 2012), and Cohort VI (awarded in July 2013) were not included in the evaluation. Cohorts I-III were funded from the same RFA based on priority score. Additional detail about PBHCI Cohorts I-VI appears in Table 2.1. Specifically, Table 2.1 shows the number of grantees in each cohort, the cohorts of grantees included in this evaluation, and those cohorts funded from the original RFA released in 2009. It also describes some between-cohort differences, including the requirement of a tobacco cessation program and related data collection activities, the receipt of a health information technology (HIT) supplement to achieve federal standards for HIT meaningful use--Stage 1, and additional detail about the foci of cohorts funded after the evaluation began.

TABLE 2.1. Between-Cohort Differences for Grantees Included in and Excluded from the Multisite Evaluation
Cohort	I	II	III	IV	V	VI
Number of grantees	13	9	34	8	30	6
Included in this multisite evaluation	X	X	X
Funded from 2009 RFA	X	X	X	X
Experienced change in data reporting requirements mid-project	X
Tobacco cessation program required		X	X	X	X	X
Eligible to have received HIT supplement ($200,000) to achieve federal standards for HIT Meaningful Use, Stage 1a	X	X	X	X
Funded from 2012 RFA [including emphasis on health homes and Meaningful Use HIT standards, requirements around consumers served per year 9,600 consumers by end of Year 4), lower total funding per year ($400,000)]					X	X
Optional (O) or Required (R) recording of breath CO (tobacco smoke exposure) and waist circumference (metabolic syndrome)	O	O	O	O	R	R
NOTES: CO = carbon monoxide; HIT = health information technology; RFA = request for applications. 47 of 56 eligible grantees received the HIT supplement.

Evaluation Methods and Measures

A goal of the RAND evaluation design was to minimize grantee burden and, as such, the majority of data used in the evaluation were collected by PBHCI grantees in service of their grant agreements with SAMHSA. Beyond these SAMHSA data reporting requirements, RAND requested (but did not require) additional data from grantees. RAND also conducted limited primary data collection. Table 2.2 describes the methods and measures used to collect data about PBHCI programs and how the data were used to answer the evaluation's main research questions. Additional information about each method and its measures is also detailed below.

TRAC Consumer-Level Data

TRAC is the web-based system through which all grantees funded by the SAMHSA Center for Mental Health Services (CMHS) are required to report performance measurement data. The TRAC system is a web-based data entry and reporting system that provides a data repository for CMHS program performance measures. Performance measures are collected as part of a CMHS effort to promote accountability within its programs. This effort is mandated by the Government and Performance Results Act (GPRA) and the OMB's Program Analysis Review Tool (PART).¹ This evaluation drew on two sets of data submitted by grantees to TRAC: SAMHSA's National Outcome Measures (NOMs) and a new NOMs section designed for this grant program (Section H) that includes physical health indicator data.

TABLE 2.2. Grantee Data Collection Requirements and Additional Data Sources for the PBHCI National Evaluation
Method	Measures	Level of  Observation	Participating  Sites	SAMHSA  Required?	Data  Collection  Period	Research  Question
TRAC	NOMs (including Section H)	Consumer	All sites (n=56)	Yes	Life of grant	Process, Outcomes, Model Features
SharePoint	Individual service use (registry)	Consumer	All sites (n=56)	No	Winter 2011-April 2013	Process, Model Features
	Quarterly reports	Program	All sites (n=56)	Yes	Life of grant	Process, Model Features
Site Visit Interviews	Semi-structured staff interviews	Program	PBHCI sites (n=6)Matched control-sites (n=3)	No	Spring 2013	Outcomes
Web Survey	Structured self-report	Provider	All but one site (n=55)a	No	Spring 2013	Process, Model Features
Biometric Screening	Physical exam and survey	Consumer	PBHCI sites (n=3)Matched control-sites (n=3)	No	Spring 2012 (Cohort I)Spring 2013 (Cohort III)	Outcomes
NOTES: NOM = National Outcome Measure; PBHCI = primary and behavioral health care integration; SAMHSA = Substance Abuse and Mental Health Services Administration; TRAC = TRansformation ACcountability. The web survey was administrated to all 56 sites, but no responses were received from one site.

NOMs

SAMHSA's NOMs tool is a standardized questionnaire that captures consumer-level BH information. NOMs domains used in the evaluation include demographics, functioning, stability in housing, education and employment, crime and criminal justice status, perception of care, social connectedness, services received, and status at reassessment and clinical discharge. Additional information about trauma and military service are now also included in the NOMs, but these fields were added after the evaluation was designed and underway; therefore they were not included in any analyses in this report. The NOMs is completed when consumers enroll in the PBHCI program, every six months thereafter, and at discharge.

NOMs Section H--Physical Health Indicators

PBHCI grantees also completed a program-specific section of the NOMS (Section H) for tracking physical health indicators, which facilitated standardized reporting and consolidation of physical health data across grantees. Data reported through Section H included height, weight, glycated hemoglobin (A1c) and/or blood glucose, BP, triglycerides, and cholesterol. These data were used as biomarkers for obesity, diabetes,² hypertension, hyperlipidemia, and hypercholesterolemia, respectively. Grantees could report optional indicators such as waist circumference and breath CO, which were used as indicators of metabolic syndrome and smoking status, respectively. Grantees could also indicate if a fast occurred prior to a consumer's blood tests.

SharePoint

RAND created a PBHCI Microsoft SharePoint site, a secure, online repository for PBHCI project data not submitted through TRAC. Grantees had individual log-ins and passwords and, once logged into the system, they only had access to directories pertaining to their PBHCI site. RAND staff, SAMHSA grant program officers, and technical assistance staff also had limited access to the SharePoint site and could communicate and share documents securely with grantees through this mechanism. All grantees submitted two main types of data to RAND through SharePoint: quarterly reports and individual service use data.

Quarterly Reports

Quarterly reports, designed and required by SAMHSA, were intended to capture emergent information about the innovative approaches grantees take to accomplish PBHCI program goals. Quarterly reports contained qualitative program-level information about accomplishments, changes in staffing, the involvement of consumers and families, barriers to program accomplishment, infrastructure activities, wellness activities, progress with data collection, program components implemented through grant activity, funding sources, eligible program consumers, staff involvement in SAMHSA group activities, and other grant programs. Quarterly reports could also include supplementary materials such as press releases or résumés of new hires, as appropriate. Quarterly report data were reviewed to develop the content of the web survey and site visit interviews, with particular attention to the challenges identified by grantees.

Individual Service Use

Individual service utilization data (sometimes referred to as "registry data," as these data may have come from programs' clinical registries) were requested by SAMHSA on behalf of RAND. The individual service use data include quantitative information about service use for physical health, mental health, substance use, and wellness programs, as well as contacts with various providers of care. Anecdotally, physical health service use data appear more systematically collected across sites, whereas mental health and substance use data are less complete and reliable.

The data received by RAND were consumer day-level summaries of all services received. Specifically, programs logged all services received by a consumer on any day that the consumer made contact with the program. Note that multiple contacts with a provider or multiple services received on the same day were counted as a single contact, as data were aggregated at the level of consumer days. Individual service utilization data were collected in any standard spreadsheet or database format and then uploaded to SharePoint. Individual service use data were submitted quarterly by grantees and at the same time as quarterly reports, as the two data sources were intended to be complementary.

Site Visit Interviews

RAND conducted in-depth, in-person interviews with select staff from the three PBHCI and three non-PBHCI sites taking part in the outcomes evaluation (Research Question 2; see Chapter One and Chapter Five) plus three additional PBHCI sites, for a total of nine site visits (n=6 PBHCI, n=3 control). Overall, sites were selected for diversity in their approaches to implementing PBHCI (e.g., geographical location, urbanicity, client demographics, services provided, PC partner agency) and at least moderate early implementation success (e.g., providing higher-than-average rates of PC provider and case manager contacts, reliable data reporting, endorsement by SAMHSA project officers after the first two years of the grant). Site visits at intervention sites occurred after approximately two years of program implementation. Matched control-sites were selected because they were within the same state as their PBHCI pair and because they offered similar programs and served similar clientele as their PBHCI pair without offering PC (i.e., PC was not part of their program, nor did they have a formal referral relationship with a PC provider).

Site visit interviews were conducted with select staff from the following domains: program leadership (administrators, which may include program managers, medical directors, chief financial officers, key administrators, and evaluators/data managers), care coordinators, PC providers (physicians, NPs, physician assistants [PAs], and wellness educators), and BH providers (psychiatrists, psychologists, social workers, case managers, and/or peer specialists). These interviews addressed topics including BH/PC collaboration, program structural features, screening and referral, registry and consumer tracking, performance monitoring, care management, EBPs, wellness/prevention/early intervention, self-management support, consumer involvement, electronic capabilities, women and minority health cultural competency, and program implementation. Staff interviews lasted one to two hours each.

The purpose of the site visits was to collect more detailed and qualitative information about program structures, staff, and processes to supplement the quantitative data collected through the web-based survey.

Additional detail about these site visits is given in the Appendix.

Web-Based Survey

RAND conducted a web-based survey of select administrative, PC, BH, and care management PBHCI staff in the third year of the evaluation. The web-based survey included information about BH/PC collaboration, program structural features, screening and referral, registry and consumer tracking, performance monitoring, care management, wellness/prevention/early intervention, self-management support, consumer involvement, electronic capabilities, women and minority health cultural competency, and program implementation. Given that there are no existing/validated standards for evaluating the integration of PC into BH settings, we adapted relevant content from the 2011 NCQA Standards and Guidelines for Patient-Centered Medical Homes, which involve the integration of BH and care management into PC settings.

Further details about the web survey appear in Chapter Three.

Physical Health Screening

RAND issued a subcontract to Onsite Health Diagnostics (OHD), a biometric screening contractor, to administer a slightly modified version of the NOMs and the NOMs Section H physical health indicators data for the comparative effectiveness evaluation at three control (baseline and one-year follow-up) and three intervention (follow-up only) sites. This additional primary data collection was necessary to obtain information from control-sites (who were not collecting or submitting data to TRAC) and to provide comparable follow-up data collected within the needed time frame for participants at the participating PBHCI sites. Additional detail about the site selection, sample recruitment, and biometric screening procedures is given in the "Comparative Effectiveness Study" section in Chapter Five.

3. PROCESS EVALUATION I: PROGRAM STRUCTURES AND PROCEDURES

Overview

In this chapter, we describe the first half of the process evaluation, designed to answer Research Question 1, Is it possible to integrate the services provided by PC providers and community-based BH agencies (i.e., what are the different structural and clinical approaches to integration being implemented)? In particular, the focus of this chapter is on describing the structures and procedures of PBHCI programs of care and assessing how program structures and procedures constitute integrated care. We complete the discussion of the process evaluation in the next chapter, in which we describe the nature and extent of primary and BH care services used by PBHCI consumers and how those service utilization patterns can be used to operationalize and interpret programs' success at integrating primary and BH care.

Methods

The data in this chapter come primarily from the staff web survey and are supplemented with data from quarterly reports and site visit interviews (see the Chapter Two and the Appendix).³ Since this is the primary section of the report in which web survey data are used, we provide additional detail about survey development, administration, and data analysis here. Note that, although the web survey questions were aimed at gathering information about grantees' PBHCI programs and consumers (as defined in the following section), we do not know which program features existed prior to the PBHCI grant and which were newly added or expanded using PBHCI resources.

Web Survey Development

The web survey was developed to assess variation in PBHCI program implementation of core and optional PBHCI program features such as screening/referral, registry/tracking, care management, wellness and preventive services, and staffing. It was also designed to capture the degree of integration with respect to a variety of structural and clinical program components (e.g., the management of health records and treatment planning). Web survey respondents were instructed to consider PBHCI consumers as any "adult with serious mental illness [with or without co-occurring substance use disorder] who received PC, physical health, or wellness services through your agency's PBHCI grant-funded program" and their PBHCI program as "your agency's integrated BH, PC, physical health, and wellness services for adult consumers with serious mental illness (and, in some cases, co-occurring substance use disorder)."

Given that there are no existing/validated standards for evaluating the integration of PC into BH settings, we adapted relevant content from the 2011 NCQA Standards and Guidelines for Patient-Centered Medical Homes, which involve the integration of BH and care management into PC settings. The NCQA standards provide criteria for categorizing practices into one of three levels of PCMH recognition, and we adapted items from these criteria in the following domains: consumer access to care; documentation of consumer data; care management services; tracking and following up on tests, referrals, and care at other facilities; and the use of performance data for quality improvement. Additional questions were developed based on themes that emerged from the quarterly reports and site visits, such as integration culture and barriers to providing integrated care. To refine the survey content, wording, and administration, we pilot tested survey questions during in-person interviews at two Cohort IV program sites (i.e., those not taking part in this evaluation) and included respondents from each staff type group of interest--administrators, BH providers, PC providers, and care managers.

Survey Administration

PBHCI program directors were contacted and asked to provide email addresses for persons holding various positions in their PBHCI program, including administrators, PC providers, mental health providers, and care coordinators. A total of 669 email addresses were provided from all 56 programs queried. A total of 633 invitations to complete the survey were successfully sent; in some cases, an email address had a typographical error or the staff member no longer worked at the agency. Staff received specific sets of questions based on the combination of job roles that they endorsed at the start of the survey.

Data Analysis

We collected grantee-level data (e.g., services provided within the program) from multiple respondents within each program. Quantitative continuous data were averaged within programs. Categorical responses were aggregated within programs by using the most commonly endorsed (i.e., modal) non-missing response. In cases in which programs had data but no unique mode, we assigned a value of "disagree" to differentiate these cases from those in which no program-level data were available. However, for some items related to program administration (e.g., type of agreement with PC partner) we expected PBHCI program leaders to provide the most reliable information, and we used their responses as "tiebreakers" to resolve within-program disagreements. Results using this method are clearly labeled as such.

We note finding a high level of disagreement within programs about "fact-based" program-level features (e.g., the menu of available services). While this may be a function of the major systemic changes associated with implementing integrated care or changes in program components over time, disagreement or inconsistency in the staff's understanding of their programs' features is itself a variable that may be of interest in future study.

Web Survey Respondents

A total of 388 staff members representing 55 of the 56 PBHCI grantees responded to the web survey.⁴ Between 1 and 15 staff members per grantee responded. The distribution of respondents across job types and the number of grantees represented is summarized in Table 3.1.

TABLE 3.1. Web Survey Respondents by Job Type and Number of Grantees with Job Type Represented
Job Type	Respondents N  (Percentage of 388)a	Grantees  N (Percentage of 55)b
PBHCI program director or manager	70 (18)	54 (98)
Also care providerc	27 (7)	23 (42)
Other administrator (e.g., medical director, CEO, CFO)	42 (11)	27 (49)
Also care providerc	18 (5)	16 (29)
Administrative support (e.g., evaluator, data manager, receptionist)	74 (19)	43 (78)
BH provider	71 (18)	34 (62)
PC provider	98 (25)	46 (84)
Care manager or coordinator	69 (18)	40 (73)
Wellness specialist	8 (2)	7 (13)
Peer or mentor	22 (6)	17 (31)
NOTES: BH = behavioral health; CEO = Chief Executive Officer; CFO = Chief Financial Officer; PBHCI = primary and behavioral health care integration; PC = primary care. The number of respondents across job types does not total 388 because respondents can be in more than one category. The total number of grantees is 55 for all web survey results, since one grantee provided no web survey responses. Care provider = either BH provider, PC provider, or care manager/coordinator.

Results

Program Structures

PBHCI grantees demonstrated variation in the structure of their integrated care programs across several dimensions, including organizational partnerships, the physical location and structure of clinics, the multidisciplinary staff mix, and staff training and expertise.

Organization Partnerships

To incorporate PC into BH settings, BH agencies could choose to partner with other health care agencies or clinics that could provide them with PC staff and infrastructure, such as facilities, equipment, and record systems, or they could hire individual PC providers directly into their agency. While partnerships with other organizations provide useful resources, they may also provide challenges for grantees due to the additional rules and regulations of the partner organization that must be navigated (Bao, Casalino, and Pincus, 2013).

Results of the web survey showed that most grantees (n=45, 82%) partnered with other health care organizations to provide PC.⁵ Among grantees with PC partnerships, 13 (29%) had developed a formal contract with their partner organizations, 28 (63%) used letters of commitment, or MOA/Memorandum of Understanding, and one relied on an informal unwritten agreement. Partnership agreements typically included policies about communication between organizations, such as how clinical consumer information would be shared (n=40, 73%); coordination of services, such as scheduling BH and PC visits on the same day (n=34, 62%); and financial details (n=36, 65%).

Physical Locations

All participating PBHCI grantees brought some PC services and staff into BH campuses. Co-location requires that BH agencies provide and prepare space in their facilities for PC and wellness services, including exam rooms and private space for obtaining test results and other sensitive health information. It may also involve having on-site laboratories, pharmacies, or other services that make it easier and more convenient for consumers to adhere to provider recommendations. Most grantees (n=43, 78%) described their main BH and PC facilities as located in the same building.⁶ In addition, 21 grantees (38%) reported having a pharmacy on-site, 43 (78%) had on-site phlebotomy, and 4 (7%) had Clinical Laboratory Improvement Amendments (CLIA)-accredited lab testing capabilities (i.e., meeting federally regulated standards), which enable programs to perform a range of common lab tests on-site (e.g., cholesterol and glucose). Three grantees (5%) reported having none of these additional capabilities on-site.

Multidisciplinary Staff

PBHCI programs require multidisciplinary staff. Table 3.2 details the staff mix across PBHCI programs, as well as the quantity of staff time funded by the PBHCI grant versus other sources. Most PBHCI programs included PC physicians (n=44, 80%) who provided services directly to consumers or who supervised NPs or PAs providing direct consumer care. Most programs (n=46; 84%) also included either NPs or PAs. Many programs included registered nurses (RNs) or licensed practical nurses (LPNs) to provide PC support (n=44, 80%); fewer included medical assistants (n=32, 58%).

TABLE 3.2. PBHCI Program Staff: Funded and Not Funded by PBHCI Grant
Type of Staff	Grantees with  This Staff Type  N (%)	Funded by PBHCI		Not Funded by PBHCI
		Grantees  N (%)	FTEs Median  (Range)	Grantees  N (%)	FTEs Median  (Range)
PC provider	44 (80)	30 (55)	0.4 (0.1-1.5)	20 (36)	0.8 (0.1-2.0)
NP or PA	46 (84)	39 (71)	1.0 (0.2-2.6)	18 (33)	1.0 (0.2-2.0)
RN or LPN	44 (80)	32 (58)	1.0 (0.2-4.0)	16 (29)	1.0 (0.5-4.5)
Medical assistant	32 (58)	25 (45)	1.0 (0.1-2.0)	20 (36)	1.0 (0.1-6.0)
Nurse care coordinatora	39 (71)	36 (65)	1.2 (0.3-3.0)	7 (13)	1.1 (1.0-3.0)
Non-nurse care coordinatora	36 (65)	23 (42)	1.0 (0.5-3.8)	7 (13)	2.0 (1.0-6.0)
Case managera	48 (87)	31 (56)	1.0 (0.1-13.0)	21 (38)	5.0 (0.2-80.0)
BH provider	48 (87)	20 (36)	1.0 (0.1-4.0)	26 (47)	3.2 (0.4-20.0)
Peer	41 (75)	31 (56)	0.4 (0.1-1.5)	18 (33)	1.0 (0.1-5.0)
NOTES: BH = behavioral health; FTE = full-time equivalent; LPN = licensed practical nurse; NP = nurse practitioner; PA = physician assistant; PBHCI = primary and behavioral health care integration; PC = primary care; RN = registered nurse. FTEs are calculated using within-program means of web survey respondent estimates. Program inclusion of staff type is based on the modal response, with PBHCI program leaders as tiebreakers to resolve disagreements. For this question, care coordinators were defined as linking PC and BH services, and case managers were defined as linking consumers to community services.

PBHCI grantees were required to provide care management, defined as individualized person-centered planning and coordination to increase consumer participation and follow-up with all PC screening, assessment, and treatment services. Some BH agencies used existing BH staff, such as case managers, to fill this role, whereas others recruited new personnel. Most grantees (n=39, 71%) staffed nurses as care coordinators, and 36 (65%) included non-nurse care coordinators. To link consumers to non-health community services (e.g., housing and employment), 48 grantees (87%) included case managers.

Many programs (n=41, 75%) employed peer specialists who supported PBHCI through their insights and ability to build rapport and trust with other consumers. Survey respondents indicated that peers helped grantees design their PBHCI programs, helped with consumer recruitment and engagement, and helped with provider-consumer communication across an array of services. In the "other" free-text category of care team staff members, seven (15%) of PBHCI programs reported hiring wellness specialists, and less than four (7%) grantees specified a smoking cessation coach, a diabetes educator, a nutritionist, an occupational therapist, or an employment specialist to work with participating consumers.

Staff Experience, Training, and Supervision

To ensure that PC and BH providers had the skills and knowledge necessary for providing quality integrated care, programs could provide education or training opportunities for members of their care teams. Training could be provided by grantees or by external resources, and the examples provided to web survey respondents included education to BH providers about physical health conditions or treatments and education to PC providers about BH conditions or treatment services. Among the grantees that employed each of the following staff types, 84% (37 of 44) provided training for PC physicians, 89% (41 of 46) for NPs or PAs, 93% (41 of 44) for RNs or LPNs, 66% (21 of 32) for medical assistants, 87% (34 of 39) for nurse care coordinators, 89% (32 of 36) for non-nurse care coordinators, 94% (45 of 48) for case managers, and 88% (42 of 48) for BH providers. Programs could also assign supervisors to facilitate the integration of care. Supervision of integrated care activities (e.g., regular meetings to discuss consumer cases that are focused on building clinician expertise and ensuring the quality of integrated services) was reported by 51 grantees (93%).

Clinical Delivery Systems

Programs also varied in how they delivered care to participating PBHCI consumers in terms of PC-BH provider collaboration, screening, primary and preventive care, case management and coordination, processes for medication reconciliation and referrals to outside providers, wellness and self-management support services, information systems, approaches to performance monitoring, and plans and approaches to funding and sustainability.

PC-BH Provider Collaboration

Programs could facilitate PC-BH collaboration by implementing regularly scheduled care team meetings to discuss cases and develop integrated treatment plans. Five grantees (9%) reported no regularly scheduled meetings among PC and BH providers, whereas 19 (35%) reported regularly scheduled meetings at least weekly. Among the 284 respondents who reported their opinion about the adequacy of communication between PC and BH providers in their program, 73 (26%) from 35 different programs described the level of communication as inadequate. On average, reports of inadequate communication were higher for programs that did not have regularly scheduled meetings (mean=50% of respondents per program) than for those that met weekly (mean=16% of respondents per program).

PC and BH providers could also collaborate on the development of integrated treatment plans for PBHCI consumers. Overall, 14 (25%) grantees reported using an integrated treatment plan. Within this group, an average 26% of survey respondents (SD= 9%) described the level of PC and BH provider collaboration on consumer treatment plans as close or usual. Among grantees with separate treatment plans, average reports of close or usual collaboration among providers was only slightly lower, at 24% (SD=24%). In other words, the existence of shared treatment plans was not necessarily associated with collaboration across specialties. Most grantees (n=45, 82%) reported that PC and BH providers collaborated to achieve specific consumer goals for the majority (i.e., 51% or more) of their PBHCI consumers. Three grantees (5%) reported PC-BH collaboration for less than 25% of consumers. The degree to which shared PC and BH information systems facilitate provider collaboration is described below.

Screening

Grantees were required to provide screening to all enrolled consumers for physical health conditions. A full set of screenings included assessments of BP, weight, height, waist circumference, blood glucose, cholesterol, and triglycerides. Most grantees (n=42, 76%) reported that all adults with SMI on their caseloads were eligible for the full set of screenings. Three grantees (5%) targeted consumers with no recent PC contacts and two grantees (4%) targeted consumers with known physical health conditions. Among the 41 grantees that provided web survey information about the indicators included in their PBHCI consumer physical health screening, 95% (n=39) included BP and cholesterol; 93% (n=38) included height, weight, and triglycerides; and 90% (n=37) included glucose or A1c. Waist circumference and breath CO were optional indicators and were collected by 27 (66%) and 14 (34%) grantees, respectively.

Primary and Preventive Care

Programs differed widely in terms of the scope and availability of PC services on-site. Some programs offered a wide range of PC services at the co-located site, including preventive services such as immunizations and gynecological exams and even minor surgeries. In contrast, some programs provided only basic screening and evaluation on-site and referred consumers to other facilities for PC treatment and services.

Most grantees (n=38, 69%) provided PC services at a co-located site five days per week. PC services were available 3-4 days per week in 12 programs (22%) and 1-2 days per week in three (5%) programs. Two grantees (4%) did not provide PC treatment or services beyond basic screening and evaluation on-site (i.e., PC services were not co-located). Seven grantees (13%) provided PC services during the evenings (i.e., after 6 p.m.) either one or two days per week. Two grantees (4%) also provided PC services on the weekend.

Some programs also provided PC clinical advice to consumers by telephone or email. During regular office hours, 38 grantees (69%) provided clinical advice by phone and six (11%) provided it electronically. After office hours, 19 grantees (35%) offered either of these services.

Care Management and Coordination

Care management and coordination services provide individualized person-centered support to help consumers navigate health and community resources. Staff providing these services may have BH, medical, or non-professional backgrounds and may be referred to as case managers, care managers, or care coordinators. The term case management is often used in BH settings to describe the coordination of non-health community services for consumers (e.g., housing and employment), whereas care management or care coordination has been used to emphasize the coordination of health or clinical services. In practice, however, we found the distinction to be less clear. All survey respondents who identified themselves as case managers described the coordination of PC and BH services for consumers as part of their job roles, and 33% of those who identified exclusively as case managers (n=10) described themselves as primarily PC providers. Similarly, 26% of those who identified exclusively as care coordinators (n=8) described themselves as primarily BH providers. In this section, we use the term care coordinator to refer to any staff that identified coordination of PC and BH services for consumers as part of their job roles, regardless of their job title.

Care coordinator caseloads varied widely from small to large (e.g., over 700) or unspecified (e.g., some care coordinators are individually responsible for all program participants). Survey respondents included 69 care coordinators from 40 grantee programs (73%). Among them, 42 (61%) reported having a caseload of PBHCI consumers. Caseloads ranged from three to 750, with a mean of 141 (SD=194) and a median of 51. Among care coordinators with caseloads, eight (20%) reported that their caseloads were too high; interestingly, this group did not include those with the five highest caseloads (ranging from 386 to 750). Caseload size was also unrelated to whether a care coordinator identified as a case or care manager or coordinator.

Among care coordinator survey respondents, 11 (16%) had some college education, 28 (41%) had bachelor's degrees, 22 (32%) had master's degrees, and four (6%) had doctoral degrees (e.g., M.D., Ph.D., or Psy.D). Twenty-three care coordinators (33%) reported being licensed health providers, 37 (54%) reported not being licensed, and five (7%) indicated that their profession did not involve licensing. Fifteen care coordinators from 13 programs were nurses (either NPs, RNs, LPNs, or licensed vocational nurses).

At the program level, 30 grantees (55%) reported regularly identifying consumers in need of additional care management support. Initial visits with care coordinators for PBHCI consumers were available within a day for 16 grantees (29%), within a week for 30 (55%), and within two weeks for three (5%). One grantee (2%) reported a wait time of 15-30 days, and the remaining programs disagreed about care coordinator wait times.⁷ Consultation by phone with care managers after hours was available in 16 programs (29%).

Medication Reconciliation and Referrals to Outside Providers

PBHCI consumers are likely to be prescribed medications for both PC and BH conditions, making medication reconciliation an integral component of integrated care. Almost all grantees reported tracking medications prescribed to consumers by providers within and outside the organization and assessing client adherence to prescriptions (n=54, 98%), including contacting pharmacies to obtain necessary information about consumer medications. Most grantees also provided consumers with educational materials about their medications (n=46, 84%).

Similarly, for referrals to external health care providers, 45 grantees (82%) reported typically providing external providers with a clinical reason for referral and additional relevant information, and 33 (60%) reported regularly tracking whether consumers attended referred appointments. Electronic or paper-based systems to track consumer lab tests were used by 49 grantees (89%). Lab test-tracking systems flagged missing results and followed up as necessary for 42 grantees (76%), flagged abnormal results and notified clinicians for 41 grantees (75%), and notified consumers of results for 33 grantees (60%).

Wellness and Self-Management Support

PBHCI grantees were required to compliment traditional PC with wellness services, although specific types of wellness services were not required. Table 3.3 details a range of wellness services that programs offered over the last year of the grant and the availability of these services both in terms of the number of months per year and the number of hours per week. The most common wellness services, reported by 53 grantees (96%), involved nutrition and diet education and exercise, such as walking groups, yoga, Pilates, Zumba, water aerobics, and Tai Chi. Other common wellness services provided by 50 grantees (91%) included instructions for cooking healthy foods, stress management or relaxation training, and diabetes education. Wellness services reported by less than three grantees (5%) included acupuncture, employment and education support, financial wellness, and gender-specific groups.

Among grantees providing various wellness services, at least 50% provided these services throughout the year; other grantees provided wellness services more sporadically. For example, although smoking cessation services were provided by 52 grantees (95%), one grantee (2%) provided the services for only three months, and four others (7%) provided them for less than six months.⁸ A number of other wellness services were only available for one month in some programs, including social support and sexual health education. Wellness services that are not offered regularly may have limited impact on improving consumer health--for example, smoking cessation may require more than three months of support.

TABLE 3.3. Wellness Services Available over the Past Year in Individual or Group Settings
Wellness Service	Grantees  N (%)	Months per  Year Median  (Range)	Hours per  Week Median  (Range)
Peer facilitators/supports	48 (87)	12 (3-12)	15 (1-60)
Nutrition/diet education	53 (96)	12 (5-12)	6 (1-40)
Cooking healthy foods	50 (91)	11 (1-12)	2 (1-40)
Exercise	53 (96)	12 (5-12)	5 (1-35)
Social support	51 (93)	12 (1-12)	12 (1-40)
Stress management/relaxation training	50 (91)	12 (5-12)	4 (1-30)
Diabetes management/education	50 (91)	12 (3-12)	6 (1-40)
Other physical health condition management/education	47 (85)	12 (3-12)	5 (1-40)
Chronic mental health condition management/education	48 (87)	12 (6-12)	13 (1-45)
SUD support	43 (78)	12 (8-12)	11 (1-40)
Smoking cessation	52 (95)	12 (3-12)	5 (1-40)
Sexual health education	31 (56)	12 (1-12)	2 (1-40)
NOTES: SUD = substance use disorder. The availability of wellness services was based on the modal response among grantees.

Grantees described several approaches to recruiting consumers for wellness services, including enlisting peers to provide outreach and program information to consumers and having PC providers write "wellness prescriptions"--formalized recommendations that consumers take part in specific services.

Information Systems

Integrated health records may promote information sharing across provider types. Forty-seven (n=47) of the 56 grantees (84%) in Cohorts I-III (i.e., included in this evaluation) received a $200,000 grant to implement/enhance EHRs. At the time the web survey data were collected (late spring 2013), most grantees (n=43, 78%) reported their PC and BH records were separate; only ten (18%) reported integrated records. Most grantees (n=45, 82%) used EHRs for both PC and BH records; however, four (7%) had EHRs for BH records only, and two (4%) had EHRs for PC records only. Three programs (5%) used paper-based records for both PC and BH. All of the ten grantees (18%) that reported integrated PC and BH records used EHRs.

Regardless of whether programs had paper-based records or EHRs, registries were a core component of the PBHCI grant program. Registries contain sets of health information about all consumers in a program and are structured to be searchable so that lists can be generated for subsets of consumers based on diagnoses or service needs. Seven grantees (13%) reported that they did not have a registry.⁹ Two grantees (4%) used paper-based registries, 19 (35%) had electronic registries integrated with their EHRs, and 26 (47%) used electronic registries that were not integrated with EHRs. Among the 45 grantees (82%) who used an electronic registry, 44 (98%) included current and active diagnoses; 40 (89%) included allergies; 41 (91%) included BP, height, and weight; and 37 (82%) included status of tobacco use.

Similarly, 43 grantees (78%) used electronic prescribing. Other electronic system capabilities included health record safeguards against medication interactions or incompatibility (n=41, 75%), preventive care need reminders to clinicians at the time of consumer visits (n=23, 42%), and consumer clinical information exchanges with health providers outside of PBHCI programs (n=11, 20%). Electronic systems to track lab tests were used by 40 grantees (73%); among them, 30 (75%) used the system to order tests, 36 (90%) retrieved test results using the electronic system, and 32 (80%) stored test results in structured fields in consumer health records.

Performance Monitoring

Ongoing performance monitoring is critical to ensure that PBHCI grantees work strategically towards their initial goals and continue to perform effectively in the long term. Most grantees (n=51, 93%) reported using data for ongoing quality improvement processes. Almost all grantees (n=54, 98%) used consumer feedback to gauge program performance, which could include suggestion boxes or consumer surveys. When asked about the types of performance measures used, 49 grantees (89%) reported having used consumer-level data to monitor program performance and 38 (69%) reported having computed program-level measures (e.g., the percentage of eligible consumers who received appropriate immunizations). Thirty-eight grantees (69%) reported using consumer data to track program performance on a "regular basis." However, the frequency of regularity was not defined.

Funding and Sustainability

PBHCI programs must find ways to sustain integrated care services beyond the life of the grant. Accompanying the passage of the Affordable Care Act, there are a number of state and federal policies and initiatives being developed to provide funding opportunities to support integrated care (e.g., Health Homes and PCMH). More than half of the grantees (n=35, 64%) reported their programs had been influenced by initiatives other than PBHCI to support integrated care:¹⁰ 20 grantees (36%) were affected by Health Homes, 12 (22%) by Medical Homes, 15 (27%) by state Medicaid initiatives (e.g., Medicaid expansion, demonstration projects for dual eligible, and the creation of certified peer specialist wellness billable service), 15 (27%) by state health or mental health authority initiatives (e.g., smoking cessation initiatives, California's Mental Health Services Act county funding, and Massachusetts's Community-Based Flexible Supports program), and 14 (25%) by grants from foundations or other sources. More than half of all programs were also influenced by major state or local budget cuts (n=37, 67%), the initiation of new Medicaid managed care arrangements (n=30, 55%), or changes in payer reimbursement policies (n=32, 58%). One program (2%) reported being influenced by tax breaks.

Barriers

Barriers to implementing and providing integrated care endorsed by web survey respondents are detailed in Table 3.4. Difficulties with hiring or staffing (n=53, 96%) and high consumer no-show rates (n=53; 96%) were among the most common and persistent barriers mentioned. Recruiting and engaging consumers in wellness, prevention, or PC (n=50, 91%) and recruiting consumers for PBHCI program participation (n=47, 85%) were also among the most common barriers cited, although some grantees were concerned about their capacity to serve consumer needs due to limited space or staff. Some grantees reported difficulty recruiting consumers because their consumers had existing relationships with PC providers or alternative care options with more flexible availability. Limited transportation for consumers was reported by 51 grantees (93%), which may contribute to the high no-show rates and other consumer engagement barriers described above.

TABLE 3.4. Barriers to Implementing PC in BH Integration
Barrier	Barrier Experienceda  N (%)	Barrier Resolvedb  N (% of Experienced)	Barrier Ongoingb  N (% of Experienced)
Hiring/staffingc	53 (96)	---	---
Adequate space for PC	49 (89)	14 (29)	24 (49)
High consumer no-show rates	53 (96)	4 (8)	48 (91)
Engaging consumers in wellness, prevention, or PC follow-up	50 (91)	5 (10)	42 (84)
Recruiting consumers for PBHCI	47 (85)	8 (17)	32 (68)
Transportation for consumers	51 (93)	5 (10)	42 (82)
Tracking consumer health information	50 (91)	11 (22)	36 (72)
Sharing consumer health information	50 (91)	9 (18)	35 (70)
Implementing EHRs	47 (85)	12 (26)	32 (68)
Meeting data collection requirements	52 (95)	10 (19)	29 (56)
Shared PC-BH leadership decisionmaking	43 (78)	6 (14)	32 (74)
Shared PC-BH provider decisionmaking	45 (82)	5 (11)	33 (73)
Billing or funding	50 (91)	5 (10)	40 (80)
Consumer health insurance limitations	50 (91)	7 (14)	39 (78)
NOTES: BH = behavioral health; EHR = electronic health record; PBHCI = primary and behavioral health care integration; PC = primary care. Barriers were considered experienced by a grantee if reported by any program staff. Barriers were designated as ongoing or resolved based on the modal response within the program. Percentages do not sum to 100 due to missing data or disagreements within the program. No data are available for whether hiring/staff barriers were resolved or ongoing because this issue was presented in a different question format.

Barriers related to tracking and sharing consumer health information were common (n=50, 91%), as were challenges to implementing EHRs (n=47, 85%). Challenges related to billing or funding or consumer health insurance limitations were also reported by 50 grantees (91%), 80% of whom reported ongoing billing or funding challenges (n=40). Finally, related to the collaborative culture of integrated programs, 43 grantees (78%) reported barriers related to shared decisionmaking among PC and BH leadership, and 45 (82%) reported this barrier among PC and BH providers.

Level of Integration

To summarize the degree to which grantees had successfully integrated PC into their BH settings, we created a framework based on four dimensions of integration that were developed from existing descriptions of the degree of integrated care (e.g., Doherty, 1995) and informed by variations in integrated care program elements that we observed during site visits. These four dimensions were co-location, shared structures and systems, integrated practice, and culture. Each dimension included 4-6 web survey items whose results we used to classify grantees into one of three levels of integration: low, medium, or high. The distribution of grantees across items within each of the four dimensions, as well as an explanation of how grantees were classified as low, medium, or high, are provided in Table 3.5.

TABLE 3.5. Grantee Levels of Integration Across Four Dimensions of Integration
Dimension	Services	Level Details	Level of Integration  N (%)			Missing
			High	Med	Low
Co-locationa	Basic PC and BH services	High: PC and BH in same buildingLow: Not same building	43(78)	---	10(18)	2(4)
	Pharmacy	High: Pharmacy on-siteLow: Not on-site	22(40)	---	33(60)	0(0)
	Phlebotomy	High: Phlebotomy on-siteLow: Not on-site	43(78)	---	12(22)	0(0)
	Lab testing	High: CLIA-accredited lab testing on-siteLow: Not on-site	7(13)	---	46(84)	2(4)
Shared Structures and Systemsb	Shared health records	High: Shared/integrated PC-BH health recordsLow: Separate PC and BH records	10(18)	---	43(78)	2(4)
	EHRs	High: PC and BH records are electronicMed: PC or BH records are electronicLow: No electronic records	45(82)	6(11)	3(5)	1(2)
	Shared treatment plans	High: Consumer has single integrated PC-BH treatment planLow: Treatment plans are separate	14(25)	---	37(67)	4(7)
	Regularly scheduled PC-BH provider meetings	High: At least weekly to discuss consumer clinical infoMed: Less than weekly, once a month or moreLow: Never or less than once a month	19(35)	12(22)	9(16)	15(27)
	Care coordination (medical)	High: Any nurse care coordinator/manager on staffLow: None	39(71)	---	14(25)	2(4)
	Care/case management	High: Any respondent staff focused on non-health community resources (e.g., legal, housing)Low: None	31(56)	---	4(9)	19(35)
Integrated Practicec	BH to PC communication	High: More than two meetings per weekMed: 1-2 meetings per weekLow: Less than one meeting per week	7(13)	8(15)	12(22)	28(51)
	PC to BH communication	High: More than two meetings per weekMed: 1-2 meetings per weekLow: Less than one meeting per week	15(27)	15(27)	7(13)	18(33)
	Care/case manager to PC and BH communication	High: Two meetings or more per week with PC and BH providerMed: At least one meeting per week with PC and BHLow: Less than one meeting per week with PC and/or BH providers	12(22)	15(27)	8(15)	20(36)
	PC access to BH records	High: More than 80% of PC respondents regularly access BH recordsMed: 50%-80%Low: Less than 50%	21(38)	4(7)	9(16)	21(38)
	BH access to PC records	High: More than 80% of BH respondents regularly access PC recordsMed: 50%-80%Low: Less than 50%	10(18)	1(2)	12(22)	32(58)
Cultured	PC-BH collaboration on treatment plans	High: More than 80% of respondents describe collaboration on treatment plans as "close" or "usual"Med: 50%-80% describe collaboration as aboveLow: Less than 50%	1(2)	10(18)	42(76)	2(4)
	PC-BH collaboration on consumer goals	High: PC and BH work together on specific goals for more than75% of PBHCI consumersMed: For 50%-75% of consumersLow: For less than 50% of consumers	11(20)	34(62)	10(18)	0(0)
	Overall PC-BH leadership collaboration	High: More than 80% of respondents report no challenges related to PC-BH leadership shared decisionmaking  Med: 50%-80%Low: Less than 50% report no barrier	14(25)	23(42)	18(33)	0(0)
	Overall PC-BH provider collaboration	High: more than 80% of respondents report no challenges related to PC-BH provider shared decisionmakingMed: 50%-80%Low: Less than 50% report no barrier	14(25)	18(33)	23(42)	0(0)
NOTES: BH = behavioral health; CLIA = Clinical Laboratory Improvement Amendments; EHR = electronic health record; PBHCI = primary and behavioral health care integration; PC = primary care. Co-location subscale raw scores could range from 0 to 8. Shared Structures and Systems subscale raw scores could range from 0 to 12. Integrated Practice subscale raw scores could range from 0 to 10. Culture subscale raw scores could range from 0 to 8.

To summarize programs' level of integration within each of the four dimensions, we scored each item result as low, medium, or high, or as 0, 1, or 2, respectively. We then summed grantee scores across all relevant items. If programs were missing data for less than 40% of the items within a dimension, we imputed the missing item scores using the mean of observed scores across programs. Programs missing data for more than 40% of the items within a dimension were not included in the summary of scores presented below. To standardize the four dimension scores, we converted raw scores to percentages of highest possible score so that each ranged from 0 to 100. A total integration score was computed based on the average of the four standardized dimension scores.

Co-location

Physical co-location is important because: (1) BH consumers who are already on-site for BH services are more likely to access PC services if they do not have to travel elsewhere to get it; (2) it enables providers to do "warm hand-offs" so that consumers feel comfortable with other service providers; and (3) it provides an opportunity for PC and BH providers and staff to communicate frequently and in person (Blount, 2003). The degree of integration based on physical location was determined with reference to the co-location items described earlier in this chapter: the co-location of PC and BH within a shared building and additional on-site resources such as a pharmacy, phlebotomy, and lab testing. Data on all four items were available for 51 programs (93%). The remaining four programs (7%) were missing one item each, and missing data were imputed using observed item means. One program (2%) received the maximum score of 100% and one program received the minimum score of 0%. The remaining programs were evenly distributed around a mean score of 53% (SD=21%) and a median of 50%.

Shared Structures and Systems

Shared structures and systems refer to staff, infrastructure, and policies in place to promote shared information across PC and BH, such as shared health records, EHRs, shared treatment plans, regularly scheduled meetings including both PC and BH providers, and the inclusion of a nurse care coordinator, as well as a care/case manager focused on community services. Data on all six items were available for 24 programs (44%), 20 programs (36%) were missing data on one item, and ten programs (18%) were missing data on two items. Missing data were imputed using observed item means for programs missing data on 1-2 items (n=30, 55%). Of the 54 programs (98%) for which dimension subscale scores were computed, one program (2%) received the maximum score of 100% and one program received the lowest observed score of 17%. The remaining programs were fairly evenly distributed around a mean score of 60% (SD=18%) and a median of 59%.

Integrated Practice

In contrast to shared structures, indicators of integrated practice were derived from staff's self-reported practice behaviors. While we considered the existence of regularly scheduled meetings a structural component of programs, providers' self-reported frequency of communication with other provider types was considered an indicator of integrated practice. Similarly, providers' self-reported frequency of accessing health records from the other domain (i.e., PC or BH) was also included as an integrated practice indicator. Data were available for all five items for ten programs (18%), ten programs (18%) were missing data for one item, and 13 programs (24%) were missing data for two items. Missing data were imputed using observed item means for programs missing data on 1-2 items (n=23, 42%).¹¹ Among the 33 programs (60%) that had data for at least two of the five subscale items, one program (3%) received the maximum score of 100% and one (3%) received the lowest observed score of 26%. The mean score was 59% (SD=21%) and the median was 61%.

Culture

In contrast to specific behaviors involved in care delivery, items assessing culture were focused on staff perceptions of PC-BH collaboration in their programs. We included measures of culture because, as we learned from open-field responses to the web survey, although objective structures and systems may be in place to support integration, staff often report needing to "feel like part of the team" in order to engage one another in coordinating and providing whole-person care. To this end, the culture assessment included staff perceptions about the degree of collaboration with regard to treatment plans, the percentage of consumers for whom PC and BH providers collaborated to achieve consumer goals, and any challenges related to shared decisionmaking among PC and BH leaders or providers. Data for all four culture items were available for 53 programs (96%), and the remaining two programs (4%) were missing data for one item; these missing data were imputed using observed item means. One program (2%) earned the maximum score of 100%. Five programs (9%) received the minimum score of 0%. The mean and median score was 38% (SD=23%).

Overall Degree of Integration

The distributions of the four dimension scores and overall score are shown in Figure 3.1. Overall scores were computed for the 33 programs (66%) that had at least 60% non-missing data for all four dimension subscales and ranged from 28% to 81%, with a mean and median of 51% (SD=12%). The limited range of the overall scale indicates that no grantee scored above 81% or below 28% on all four subscales. There were no significant correlations among subscale scores, which suggests that programs may be well integrated in terms of some dimensions, but not others.

FIGURE 3.1. Grantee Scores on Four Dimensions of Integration and Overall

Our results show that grantees varied widely in the degree of implementation of their programs with regards to each of the four subscales we examined and that the majority of programs are only moderately integrated, since the overall integration score was less than 62% for the 75% of grantees for which overall scores could be computed. Some of the low scoring may be attributable to missing data, but even among the seven programs (13%) with no missing data, the highest score was only 73%, and five programs scored under 50%.

Summary

Before we discuss our findings related to program structures and procedures, we note some limitations of the web survey data. First, PBHCI programs were not equally represented in the web survey because some programs had more staff complete the survey than others: 1-15 staff members per program responded. In particular, programs with different types of respondents contributed more perspectives to the overall analysis. Since the web survey data were self-reported, other possible limitations of the data come from the risk of systematic respondent bias. For instance, respondents may have been biased toward presenting programs in a positive light (e.g., wanting their program to "look good"), or struggling programs whose staff may be less engaged may have not responded to the survey. Alternatively, disgruntled staff may have been eager to complete the survey and air grievances, thus contributing negative bias to the survey results. In any case, the web survey data must be considered within the general limitations of data that were incomplete within and across programs and were also self-reported.

Noting those limitations, we found variation with regard to the structure of PBHCI grantees' integrated care programs across several dimensions, including organizational partnerships, the physical location and structure of clinics, the multidisciplinary staff mix, and staff training and expertise. Our observations during site visits were consistent with these findings and provided further details about how programs integrate care. For instance, during site visits we learned that co-location is more nuanced than as measured in the web survey (i.e., PC and BH may or may not share reception and hallways); a finer-grained analysis of program features and procedures may provide further information about components that facilitate or impede integrated care. The free-text option in the web survey also showed the importance of fit and personality of care team members to the success of the program overall. For example, some PC staff reported feeling "unwelcome" at BH sites, which likely affects how well team members work together to provide integrated care.

PBHCI programs also varied in how they delivered care to consumers in terms of PC-BH provider collaboration, screening, primary and preventive care, case management and coordination, processes for medication reconciliation and referrals to outside providers, wellness and self-management support services, information systems, approaches to performance monitoring, and plans and approaches to funding and sustainability. In terms of record keeping, for example, we found that the extent of actual collaboration between PC and BH varies widely, especially when considering all members of the care team. During site visits, we found that some BH providers were frustrated because they could not access PC notes, and that separate PC and BH records created extra work for transcribing and sharing information relevant to the entire care team. While those with electronic systems reported fewer of these particular barriers, they also noted that implementing new electronic systems was time consuming and challenging, and that many staff experience steep learning curves once new electronic systems are in place. Finally, systems for communication and coordination across programs were also quite varied. While many site visit interviewees and web survey respondents reported wanting regularly scheduled provider meetings, some, especially those who share hallways, did not make such a request. Staff disagreement about the availability of PBHCI services (e.g., the type and timing of PC, wellness classes) suggests that many programs need to continue co-training PC and BH staff and developing systems for ongoing cross-specialty communication.

Despite variability in their approaches to PBHCI, respondents across programs reported many of the same barriers to implementing and delivering integrated care services, including staff and consumer recruitment and retention. Staff also frequently reported financial barriers, such as worries about financing nurse care managers, peer staff, and wellness services after the grant funds run out. From the site visits however, it was evident that financial barriers differed widely based on consumers' insurance status and state-specific Medicaid regulations about the type and intensity of reimbursable services. Finally, across programs, consumer transportation to PBHCI service appointments was a common problem, particularly in rural areas.

Last, to describe the degree to which grantee programs are offering integrated PC and BH services, we coded structural and procedural components of integration along four dimensions (co-location, shared structures and systems, integrated practice, and culture) based on broadly accepted standards, as well as in a single summary score. When we assessed programs on these dimensions, we found that integration varied widely both across and within programs, such that programs were often integrated along some dimensions, but not others. Further investigation into the nuances of PBHCI integrated care may help future programs anticipate problems and implement more completely integrated services from the start.

4. PROCESS EVALUATION II: CONSUMERS, CARE NEEDS, AND SERVICE UTILIZATION

Overview

In this chapter, we describe the second half of the process evaluation, designed to answer Research Question 1, Is it possible to integrate the services provided by PC providers and community-based BH agencies (i.e., what are the different structural and clinical approaches to integration being implemented)? In the prior chapter, we described the structures and procedures of PBHCI programs of care, along with how these structures and procedures may or may not be supporting integration. In this chapter, we describe how consumers and their needs interface with available PBHCI program services. As described in Chapter One, numerous factors contribute to the excess burden of general medical conditions among persons with SMI, including mental illness-related factors, fears related to using general medical services, and socioeconomic disadvantage (Burnam and Watkins, 2006; CDC, 2011; Druss, 2007). Importantly, limited access to and poor quality of general medical care also play a role (Lawrence et al., 2009; Salsberry, Chipps, and Kennedy, 2005; Mechanic and Bilder, 2004; Katon and Unutzer, 2013). To show how PBHCI addresses these issues, we present data illustrating programs' success at enrolling and engaging consumers in care, characteristics of the PBHCI consumer population served, consumers' care needs, and the range and intensity of primary and BH services they receive. We also use service utilization patterns to characterize the extent to which consumers receive integrated care.

Methods

Consumer demographics, consumer health, and service utilization data come from two sources: SAMHSA's NOMs and consumer-level service utilization data requested by RAND (see Chapter Two). NOMs and service utilization data were collected by grantees and come from the entire pool of PBHCI grantees (n=56) included in the evaluation.

Whenever possible, we present results separately for Cohort I (n=13) because it was in its fourth year of program implementation at the time of this research, while Cohorts II (n=9) and III (n=34) were in their third year, having received their grant funding one year later (Table 2.1). Our focus is on results from all cohorts combined, although we note differences between Cohort I and Cohorts II and III where such differences have implications for overall PBHCI processes.

Results

Enrollment and Retention of Target Population

In their initial proposal to SAMHSA, grantees indicated the number of persons they planned to enroll both in the first year of the grant and over the life of the grant (Years 1-4). To measure programs' success at enrolling consumers, we examined the ratio of actual versus planned consumers receiving services from a PC provider (Table 4.1). Overall, programs provided PC services to more consumers in Year 1 than they had initially planned (median=137%, range 11%-535%), but over the life of the grant, programs served fewer consumers than they had anticipated. While noting that Cohorts II and III still had more than one year to meet their service targets at the time the data were collected, Cohort II and III grantees had provided PC services to only 27% of their target population.¹²

TABLE 4.1. Planned and Actual Numbers of Consumers Receiving PC Services, Year 1 and Years 1-4, by Cohorta
Year		Cohort I	Cohorts II/III	All Cohorts
Year 1b	Actual totalMedian (range) across sites	3,298505 (440-1,143)	11,971253 (17-948)	18,896294 (17-1,143)
	Planned TotalMedian (range) across sites	2,475400 (250-800)	13,025220 (40-2,000)	15,500250 (40-2,000)
	Ratio overallMedian (range) across sites	1.331.52 (0.63-3.03)	0.92  1.31 (0.11-5.35)	1.221.37 (0.11-5.35)
Years 1-4c	Actual totalMedian (range) across sites	N/AN/A	12,247265 (17-948)	Cannot estimate
	Planned totalMedian (range) across sites	MissingN/A	44,568662 (203-4,140)	Cannot estimateN/A
	Ratio overallMedian (range) across sites	Cannot estimate  N/A	0.270.39 (0.02-2.14)	Cannot estimateN/A
NOTES: PBHCI = primary and behavioral health care integration; PC = primary care. Because all PBHCI consumers need to see a PC provider at least once, "actual" numbers are equivalent to the total number of consumers enrolled. Actual Year 1 data and the ratio are for the five out of 13 Cohort I sites with available planned Year 1 data (eight sites have missing data) and for the 38 out of 42 Cohort II and III sites with available planned data for Years 1-4 (four sites have missing data). None of the 13 Cohort I sites have planned data for Years 1-4; hence, actual data are not presented and the ratio cannot be estimated. Data for Planned Years 1-4 for Cohort II and III sites are missing for three sites; actual data and the ratio are for the sites with available planned data only.

Among those enrolled in PBHCI, we also calculated the number of consumers who stayed engaged in and were discontinued from PBHCI services (Table 4.2). Combined, grantees enrolled 25,648 persons, 22% (n=5,755) of whom discontinued treatment after approximately seven months. Individual grantees typically enrolled an average of 445 (range 73-1,143) consumers and discontinued an average of 66 (range 0-453) of them after an average of 6.3 (range 0.03-26.4) months of care.¹³

TABLE 4.2. Consumers Enrolled and Discontinued from PBHCI Across Grantees and for all Grantees Combined, by Cohort
PBHCI Program		Number of Consumers  Enrolled (baseline)	Number of Consumers  Who Discontinue (d/c)	Median (range)  Follow-Up Before d/c
Cohort I	Median (range) across grantees	612 (174-1,143)	187 (18-453)	8.1 (0.03-38.1)
	All grantees combined	8,816	2,625
Cohorts II, III	Median (range) across grantees	357.5 (73-1,141)	52.5 (0-393)	6.3 (0.03-26.4)
	All grantees combined	16,832	3,130
All Cohorts	Median (range) across grantees	445 (73-1,143)	66 (0-453)	7.1 (0.03-38.1)
	All grantees combined	25,648	5,755
NOTES: PBHCI = primary and behavioral health care integration. Discontinuation includes all documented causes, such as lack of contact with the program; withdrawal or refusal to receive treatment; insurance issues or ineligibility for services; moved out of area, incarceration, admission to long-term hospital, etc. Consumers who only had baseline data were not included in the computation of discontinuation rates because we were interested in describing consumers' typical length of time in treatment, and consumers with no follow-up would significantly skew this distribution, thus biasing potential estimates of treatment effect.

Consumer Characteristics

Demographics

PBHCI consumers were men and women, predominantly non-elderly adults, who were racially and ethnically diverse (Table 4.3).

Psychosocial and Socioeconomic Characteristics

At baseline, approximately half of all PBHCI consumers reported positive well-being, including feeling healthy overall (41.8%), functioning well in everyday life (42.0%), and feeling socially connected (55.9%) while generally free of serious psychological distress (58.7%). Although a minority of consumers had multiple emergency department contacts in the previous month, the majority had none (median=0, range 0-31). Consumers' median clinician-assessed Global Assessment of Functioning (GAF) score was 51, indicating moderate to serious psychiatric symptoms or disability, including scores broadly dispersed over most of the GAF scale (0-90).

TABLE 4.3. Consumer Demographics, Percentage (95%, CI), by Cohort
Demographic		Cohort I(N=8,816)	Cohorts II/III(N=16,832)	All Cohorts(N=25,648)
Gender	Male	44.8 (43.8, 45.9)	47.3 (46.5, 48.0)	46.4 (45.8, 47.0)
	Female	55.0 (53.9, 56.0)	52.4 (51.7, 53.2)	53.3 (52.7, 53.9)
	Transgender/Other	0.2 (0.1, 0.3)	0.3 (0.2, 0.4)	0.3 (0.2, 0.3)
Ethnicity	Latino	13.3 (12.6, 14.0)	15.6 (15.0, 16.1)	14.8 (14.3, 15.2)
	Non-Latino, Black	15.8 (15.0, 16.6)	25.1 (24.4, 25.7)	21.9 (21.4, 22.4)
	Non-Latino, White	62.5 (61.4, 63.5)	45.1 (44.4, 45.9)	51.1 (50.5, 51.7)
	Other	8.5 (7.9, 9.1)	14.2 (13.7, 14.7)	12.2 (11.8, 12.6)
Race	Black	18.5 (17.7, 19.3)	29.4 (28.7, 30.1)	25.6 (25.0, 26.1)
	White	70.4 (69.4, 71.4)	56.5 (55.7, 57.3)	61.3 (60.7, 61.9)
	Asian	1.5 (1.2, 1.7)	5.7 (5.4, 6.1)	4.2 (4.0, 4.5)
	Native Hawaiian/Pacific Islander	2.1 (1.8, 2.4)	1.7 (1.5, 1.9)	1.9 (1.7, 2.0)
	Alaska native	0.3 (0.2, 0.4)	0.8 (0.7, 0.9)	0.6 (0.5, 0.7)
	American Indian	5.1 (4.7, 5.6)	8.3 (7.9, 8.7)	7.2 (6.9, 7.5)
Age group	18-34	28.3 (27.3, 29.2)	24.2 (23.6, 24.9)	25.6 (25.1, 26.1)
	35-49	40.5 (39.5, 41.6)	37.9 (37.2, 38.7)	38.8 (38.2, 39.4)
	50-64	29.7 (28.7, 30.6)	34.3 (33.5, 35.0)	32.7 (32.1, 33.3)
	65+	1.5 (1.3, 1.8)	3.6 (3.3, 3.9)	2.9 (2.7, 3.1)
Age, mean (SD)		42.4 (11.8)	44.5 (12.3)	43.8 (12.2)
NOTE: CI = confidence interval.

PBHCI consumers also reported high rates of social disadvantage and substance use. Few consumers were employed or pursuing education or training (11.4%), few had a high school education or GED (29.2%), and a notable minority had experienced homelessness in the past 30 days (7%). Consumers were also likely to report current tobacco use (59.8%), but reports of recent binge drinking (10%) and illegal substance use (21.4%) were not uncommon.

Physical Health

Consumers were considered to be "at risk" for physical health conditions if their physical health indicator values exceeded the cutoffs detailed in Table 4.4. We describe consumers here as "at risk" for physical health conditions because we did not have access to physician-rendered diagnoses. Indeed, while some physical health indicator values in the "at risk" range could indicate the presence of physical illness, in some cases, out-of-range values could be attributable to other factors (e.g., body builders often have BMIs in the "at risk" range, but physicians would be unlikely to give those persons an obesity diagnosis) (Kruschitz et al., 2013). Therefore, our "at risk" language reflects the fact that we have only limited information about consumers' physical health.

Importantly, while some thresholds denote risk for one condition (e.g., the threshold for BMI denotes risk for obesity), others, such as FPG, denote risk for more than one condition. Specifically, the FPG threshold indicates pre-diabetes and thus, the threshold denotes risk for diabetes.¹⁴ We note that a subset of plasma glucose values were non-fasting; when this was the case (Cohort I=17%; Cohorts II and III=16%; Overall=16%) we used the non-FPG threshold of >140 mg/dL. The FPG threshold, along with the "at risk" thresholds for BP, waist circumference, triglycerides, and high-density lipoprotein (HDL), also denotes risk for metabolic syndrome--a cluster of clinical features that confer high risk for cardiovascular disease (i.e., coronary heart disease (CHD), cerebrovascular disease, and other vascular disorders). Although metabolic syndrome has been variously named and defined, we use the widely used operational definition put forth by the National Cholesterol Education Program in its Adult Treatment Panel III report (ATP III) and modified by the National Heart, Lung, and Blood Institute (NHLBI) and the American Heart Association (Grundy et al., 2004). The "at risk" threshold for low-density lipoprotein (LDL) denotes risk for CHD and Type 2 diabetes, and the threshold for breath CO denotes current tobacco use.

TABLE 4.4. Physical Health Indicator Values Showing Consumer Risk
Indicator	Condition	"At Risk" Range
BPa	Hypertension
SBP		>130
DBP		>85
WC (cm)b	Metabolic syndrome
Men		>102
Women		>88
BMI	Obesity	>25
Breath CO (ppm)	Smoking	>10
FPG	Diabetes	>100
A1c	Diabetes	>5.7
HDL-C	Hypercholesterolemia	<40
LDL-C	Hypercholesterolemia	>130
Trig	Hyperlipidemia	>50
NOTES: A1c = glycated hemoglobin;BP = blood pressure; BMI = body mass index; CO = carbon monoxide; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; SBP = systolic blood pressure; Trig = Triglycerides; WC = waist circumference. The population "at risk" has elevated SBP and/or DBP levels, and "at risk" levels were defined as risk for metabolic syndrome. "At risk" levels are different for men and women.

A large number of consumers were at risk for physical health conditions (Table 4.5). Consumers were at risk for obesity (77%), diabetes (FPG 37%, A1c 53%), hypertension (45%), dyslipidemia (triglycerides 40%; HDL-C, women 49.1%, men 39.7%), and tobacco-associated health conditions (breath CO data suggest that 52% were current smokers¹⁵). High proportions of male and particularly female PBHCI consumers met abdominal obesity criteria (waist circumference, women 72.6%, men 43.3%), which substantially elevates their risk for metabolic syndrome. These rates were all higher than national adult averages for obesity (36%; Ogden et al., 2012), pre-diabetes (35%; CDC, 2011), hypertension (32%; CDC, 2012), dyslipidemia (LDL-C, 34%; CDC, 2011 and triglycerides 30%; Toth, Potter, and Ming, 2012), tobacco use (19%; CDC, 2012), and metabolic syndrome (34%, no significant difference by sex; Ervin, 2009).

TABLE 4.5. Consumers At Risk for Physical Health Conditions at Baseline, by Cohort
	At Riska	Cohort I(N=8,816)		Cohorts II/III(N=16,832)		All Cohorts(N=25,648)
		Mean  (SD)	Percentage  At Risk	Mean  (SD)	Percentage  At Risk	Mean  (SD)	Percentage  At Risk
BMI	>25	31.9(26.6)	75.4(74.3, 76.5)	33.5(29.7)	77.0(76.3, 77.7)	33(28.8)	76.5(75.9, 77.1)
FPG (mg/dL)b	> 100	103(51.7)	30.9(29.0, 32.8)	108(50.1)	39.4(38.3, 40.6)	107(50.5)	37.4(36.5, 38.4)
A1c (%)	> 5.7	6.3(5.2)	49.9(47.5, 52.4)	6.2(2.7)	53.5(52.1, 54.9)	6.2(3.5)	52.6(51.4, 53.8)
BP (mm/Hg)
SBP	>130	124(22.6)	34.7(33.5, 36.0)	126(20.3)	39.2(38.4, 40.0)	125(21.0)	37.9(37.2, 38.6)
DBP	>85	79.6(12.3)	30.0(28.8, 31.2)	80(12.0)	31.4(30.7, 32.2)	79.9(12.1)	31.0(30.4, 31.7)
SBP/DBP	>130/85	N/A	42.3(41.0, 43.6)	N/A	46.2(45.3, 47.0)	N/A	45.0(44.3, 45.7)
WC (cm)
Men	>102	101(19.7)	44.3(41.7, 46.9)	101(20.2)	42.8(41.1, 44.6)	101(20.1)	43.3(41.8, 44.8)
Women	>88	101(21.3)	72.8(70.6, 75.0)	100(22.6)	73.0(71.5, 74.4)	101(22.2)	72.9(71.7, 74.1)
Triglycerides (mg/dL)	>150	167(144)	42.0(40.3, 43.6)	158(127)	39.1(38.2, 40.0)	160(131)	39.8(39.0, 40.6)
HDL (mg/dL)
Men	<50	44.7(14.7)	41.2(38.8, 43.7)	45.9(18.2)	39.2(37.8, 40.6)	45.6(17.4)	39.7(38.5, 40.9)
Women	<40	52.4(16.7)	49.2(46.9, 51.5)	52.7(17.2)	49.1(47.8, 50.5)	52.6(17.1)	49.1(48.0, 50.3)
All	<40	48.9(16.3)	30.9(29.3, 32.4)	49.4(18.0)	29.8(28.9, 30.6)	49.3(17.6)	30.0(29.3, 30.8)
LDL (mg/dL)	>130	110(37.1)	26.8(25.3, 28.4)	110(85.2)	25.9(25.1, 26.8)	110(76.4)	26.1(25.4, 26.9)
Breath CO (ppm)	>10	13.6(13.3)	55.1(52.6, 57.5)	13.9(20.6)	49.2(47.3, 51.2)	13.8(18.1)	51.5(50.0, 53.0)
NOTES: A1c = glycated hemoglobin; BP = blood pressure; BMI = body mass index; CO = carbon monoxide; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL = high-density lipoprotein; LDL = low-density lipoprotein; SBP = systolic blood pressure; WC = waist circumference. At risk thresholds are presented by gender only when these vary by gender. For consumers who did not have FPG and only had non-fasting glucose, we used the at risk threshold for non-fasting glucose (>140) (Cohort I=17%; Cohorts II and III=16%; Overall=16%). Non-fasting glucose values are included in the data presented above.

Since we did not have access to any physician-rendered diagnoses or prescribed medications, we used physical health indicators to estimate rates of metabolic syndrome, obesity, Type 2 diabetes, dyslipidemia, and hypertension (Table 4.6). Although this is consistent with methods used by other authors (Chen et al., 2008; McEvoy et al., 2005), we are cognizant that the validity of these estimates may be compromised both by false positives and false negatives, particularly for Type 2 diabetes and hypertension. In ascending order of prevalence, consumers had PC needs related to Type 2 diabetes (15.8%), hypertension (27.7%), metabolic syndrome (29.5%), obesity (50.8%), and dyslipidemia (58.0%).

TABLE 4.6. Consumer Rates of Probable Chronic Physical Conditions, Percentage (95%, CI), by Cohort
Conditiona		Cohort I	Cohorts II/III	All Cohorts
Metabolic syndrome	Modified ATP III	28.5  (26.8, 30.2)	29.9  (28.9, 30.8)	29.5  (28.7, 30.4)
Obesity	BMI >30	48.2(46.9, 49.5)	51.8(51.0, 52.6)	50.8(50.1, 51.5)
Diabetes (Type 2)	FPG >126 mg/dLNon-fasting glucose >200mg/dL  A1c>6.5	13.1(12.0, 14.2)	16.6(15.9, 17.3)	15.8(15.2, 16.4)
Dyslipidemia	TC >240 mg/dLLDL >160 mg/dLHDL <40 (men) or <50 (women)Triglycerides >200	60.7(59.0, 62.4)	57.2(56.2, 58.1)	58.0(57.2, 58.8)
Hypertension	BP>140/90	26.3(25.2, 27.5)	28.2(27.5, 29.0)	27.7(27.1, 28.3)
NOTES: ATP = adult treatment panel;BP = blood pressure; BMI = body mass index; CI = confidence interval; FPG = fasting plasma glucose; HDL = high-density lipoprotein; LDL = low-density lipoprotein; TC = total cholesterol. Missing data: Metabolic Syndrome, N missing = 14,517; Obesity, N Missing = 6,227; Diabetes, N missing = 11,367; Dyslipidemia, N missing = 12,214; Hypertension, N missing = 5,857.

Service Use

In this section, we present results in terms of standardized rates (e.g., 500 consumers/grantee) or units of time (e.g., 12-month follow-up) to account for variability in programs' operational history and size, as appropriate. Overall, 81.3% of consumers had contact with PBHCI providers and services at least once per month. Within the first 12 months of enrolling, consumers were likely to have at least one contact with PBHCI PC providers (75.3%), psychiatrists or psychiatric nurses (72.4%), and case managers (69.3%). While nearly half of all consumers saw counselors (49.2%), rates of contact with peer specialists (24.6%) or "other specialists" (22.3%) were low (Table 4.7). We note that roughly one in four consumers did not see a PC provider or psychiatrist/psychiatric nurse and that, while the median monthly visit rate for these providers was approximately 0.5, at the low end of the distribution, consumers received 0.03 monthly visits. Likewise, while most PBHCI consumers saw a case manager during their first 12 months in the program (69.3%), the relatively low median number of monthly visits (0.88) and broad distribution (0.03-33.3) suggests generally low but variable grantee capacity to meet consumers' case management needs. However, in some cases, grantees enrolled clients with external PC providers to give them access to wellness programming.

TABLE 4.7. Percentage of Consumers Having Provider Contact Within 12 months of Enrolling in PBHCI, by Provider Type and Cohort
		Cohort I  (N=8,816)	Cohorts II/III  (N=16,832)	All Cohorts  (N=25,648)
Percent (95% CI) of Consumers Seeing Provider at Least Once During First 12 Months	Case Managers	78.1 (77.2, 78.9)	64.6 (63.9, 65.4)	69.3 (68.7, 69.8)
	PC Provider	79.1 (78.2, 79.9)	73.3 (72.6, 74.0)	75.3 (74.8, 75.9)
	Psychiatrist/Psychiatric Nurse	76.4 (75.5, 77.3)	70.3 (69.6, 71.0)	72.4 (71.9, 73.0)
	Counselors	41.4 (40.4, 42.4)	53.4 (52.6, 54.1)	49.2 (48.6, 49.8)
	Peer Specialists	18.0 (17.2, 18.8)	27.9 (27.2, 28.6)	24.6 (24.0, 25.1)
	Other Specialists	27.7 (26.8, 28.7)	19.5 (18.9, 20.1)	22.3 (21.7, 22.8)
Median (Range) Visits per Month Among Those Seeing Provider	Case Managers	0.87 (0.03-33.33)	0.89 (0.03-33.33)	0.88 (0.03-33.33)
	PC Provider	0.59 (0.03-33.33)	0.42 (0.03-33.33)	0.48 (0.03-33.33)
	Psychiatrist/Psychiatric Nurse	0.62 (0.03-33.33)	0.52 (0.04-33.33)	0.56 (0.03-33.33)
	Counselors	0.58 (0.03-33.33)	1.39 (0.04-33.33)	1.08 (0.03-33.33)
	Peer Specialists	0.35 (0.03-33.33)	0.51 (0.04-33.33)	0.44 (0.03-33.33)
	Other Specialists	0.25 (0.03-33.33)	0.41 (0.04-33.33)	0.33 (0.03-33.33)
Median (Range) Provider Visits Per Month Per Grantee/500 Consumers	Case Managers	657 (239-2,196)	1,202 (7-8,196)	986 (7-8,196)
	PC Provider	686 (93-4,401)	557 (6-9,244)	573 (6-9,244)
	Psychiatrist/Psychiatric Nurse	468 (234-1,358)	361 (1-8,528)	398 (1-8,528)
	Counselors	276 (86-4,248)	825 (2-4,357)	646 (2-4,357)
	Peer Specialists	34 (0-421)	171 (0-4,010)	147 (0-4,010)
	Other Specialists	186 (11-926)	74 (0-4,514)	82 (0-4,514)
NOTES: CI = confidence interval; PBHCI = primary and behavioral health care integration; PC = primary care. The proportion of consumers seeing a PC provider is slightly different between tables. This is because the sample of consumers included in various analyses changes somewhat as a result of missing physical health indicator data.

Within the first 12 months of enrolling in PBHCI, approximately 80% of consumers with and without risk for chronic physical health conditions had at least one contact with a PBHCI PC provider (Table 4.8). PC provider contact rates were somewhat higher for consumers with probable chronic physical illness (median contacts per month=0.48) than those without (median=0.43) (Table 4.9).

TABLE 4.8. PC Provider Encounters for Consumers With and Without Identified Physical Health Risk,a by Cohort
	Cohort I	Cohorts II/III	All Cohorts
Percentage (95% CI) of Consumers Seeing PC Provider, by Condition, During First 12 Months
Metabolic Syndrome	83.0 (80.4, 85.7)	82.8 (81.3, 84.3)	82.0 (80.7, 83.4)
Obesity	79.8 (78.3, 81.3)	80.4 (79.5, 81.4)	80.3 (79.5, 81.1)
Diabetes Mellitus	83.4 (80.0, 86.8)	78.2 (76.2, 80.1)	79.2 (77.6, 80.9)
Dyslipidemia	82.7 (81.0, 84.3)	81.8 (80.8, 82.8)	82.0 (81.1, 82.8)
Hypertension	85.6 (83.0, 88.3)	80.0 (78.0, 81.8)	81.5 (80.0, 83.1)
Any Condition	81.3 (80.1, 82.5)	79.3 (78.5, 80.1)	79.8 (79.2, 80.5)
None of the Conditions	81.7 (79.0, 84.5)	79.8 (78.1, 81.4)	80.3 (78.8, 81.7)
Median (Range) Visits per Month Among Those Seeing PC Provider
Metabolic Syndrome	0.56 (0.04-33.33)	0.48 (0.03-33.33)	0.51 (0.03-33.33)
Obesity	0.59 (0.04-33.33)	0.43 (0.03-33.33)	0.47 (0.03-33.33)
Diabetes Mellitus	0.62 (0.04-33.33)	0.48 (0.04-33.33)	0.51 (0.04-33.33)
Dyslipidemia	0.54 (0.04-33.33)	0.44 (0.03-33.33)	0.48 (0.03-33.33)
Hypertension	0.72 (0.04-33.33)	0.49 (0.03-33.33)	0.56 (0.03-33.33)
Any Condition	0.58 (0.04-33.33)	0.44 (0.03-33.33)	0.48 (0.03-33.33)
None of the Conditions	0.56 (0.04-33.33)	0.40 (0.04-33.33)	0.43 (0.04-33.33)
NOTES: CI = confidence interval; PC = primary care. Ranges are similar and large across indicators based on extreme outliers, possibly related to consumers in residential or partial-hospitalization settings. The proportion of consumers seeing a PC provider is slightly different between tables. This is because the sample of consumers included in various analyses changes somewhat as a result of missing physical health indicator data. High need for PC was defined based on the presence of one or more of the selected physical health conditions listed in the table. Low risk was defined as absence of all the conditions--if the subject had missing data, only one condition with missing data was allowed for the subject to qualify for this category.

Rates of consumer utilization of screening/assessment, referral, treatment planning, and medication management services, as well as hospitalization for physical health, mental health, and substance use conditions are shown in Tables 4.9-4.11. Rates of utilization of wellness services appear in Table 4.12.

TABLE 4.9. Consumer Physical Health Service Utilization, by Cohort
	Cohort I(N=8,816)	Cohorts II/III(N=16,832)	All Cohorts(N=25,648)
Percent (95% CI) of Consumers Receiving PC Services During First 12 Months
Screening/assessment	89.8 (89.2, 90.4)	84.2 (83.7, 84.8)	86.2 (85.7, 86.6)
Referral	44.0 (42.9, 45.1)	36.1 (35.3, 36.8)	38.6 (38.0, 39.2)
Planning	80.1 (79.2, 81.0)	69.3 (68.6, 70.0)	72.7 (72.1, 73.3)
Medication Management	74.4 (73.4, 75.3)	60.1 (59.3, 60.8)	64.8 (64.2, 65.4)
Hospitalization	15.8 (14.9, 16.7)	8.6 (8.1, 9.0)	10.7 (10.3, 11.2)
Median (Range) Visits per Month Among Those Using Service
Screening/assessment	0.57 (0.03-33.33)	0.49 (0.04-33.33)	0.52 (0.03-33.33)
Referral	0.18 (0.03-33.33)	0.18 (0.03-33.33)	0.18 (0.03-33.33)
Planning	0.43 (0.03-33.33)	0.38 (0.04-33.33)	0.40 (0.03-33.33)
Medication Management	0.34 (0.03-33.33)	0.34 (0.03-33.33)	0.34 (0.03-33.33)
Hospitalization	0.28 (0.03-33.33)	0.16 (0.03-33.33)	0.20 (0.03-33.33)
Median (Range) Provider Visits Per Month Per Grantee/500 Consumers
Screening/assessment	921 (113-4391)	832 (89-6280)	860 (89-6280)
Referral	136 (11-3565)	145 (10-5678)	145 (10-5678)
Planning	575 (58-4400)	543 (4-8496)	553 (4-8496)
Medication Management	366 (110-1793)	374 (2-8619)	374 (2-8619)
Hospitalization	22 (1-249)	5 (0-1162)	5 (0-1162)
NOTE: CI = confidence interval; PC = primary care.

During their first 12 months of enrollment in PBHCI, more than 85% of consumers received physical health screenings (Table 4.9). Appropriately, high proportions of consumers also received physical health treatment planning and medication management services (72.7% and 64.8%, respectively), and fewer consumers received referrals or hospitalizations related to physical health (38.6% and 10.7%, respectively). Consistent with project requirements, consumers received physical health screening/assessment and treatment planning approximately once every two months.

TABLE 4.10. Mental Health Service Utilization, All PBHCI Consumers, by Cohort
	Cohort I(N=8,816)	Cohorts II/III(N=16,832)	All Cohorts(N=25,648)
Percent (95% CI) of Consumers Using Service During First 12 Months
Screening/assessment	71.5 (70.5, 72.4)	86.3 (85.8, 86.8)	81.2 (80.7, 81.7)
Referral	21.2 (20.2, 22.2)	25.0 (24.3, 25.7)	23.9 (23.3, 24.5)
Planning	63.5 (62.5, 64.6)	62.5 (61.7, 63.2)	62.8 (62.3, 63.4)
Medication Management	76.8 (75.9, 77.6)	70.3 (69.6, 71.0)	72.6 (72.0, 73.1)
Hospitalization	16.6 (15.8, 17.5)	14.3 (13.7, 14.8)	15.0 (14.6, 15.5)
Median (Range) Visits per Month Among Those Using Service
Screening/assessment	1.10 (0.03-33.33)	0.75 (0.04-33.33)	0.85 (0.03-33.33)
Referral	0.28 (0.03-33.33)	0.25 (0.03-33.33)	0.26 (0.03-33.33)
Planning	0.48 (0.03-33.33)	0.45 (0.04-33.33)	0.46 (0.03-33.33)
Medication Management	0.48 (0.03-33.33)	0.57 (0.03-33.33)	0.53 (0.03-33.33)
Hospitalization	0.35 (0.03-33.33)	0.24 (0.04-33.33)	0.27 (0.03-33.33)
Median (Range) Provider Visits Per Month Per Grantee/500 Consumers
Screening/assessment	626 (79-2370)	1534 (14-9253)	1431 (14-9253)
Referral	15 (0-357)	25 (0-7734)	22 (0-7734)
Planning	230 (18-2098)	240 (32-7495)	240 (18-7495)
Medication Management	353 (153-1428)	403 (3-7362)	397 (3-7362)
Hospitalization	30 (0-875)	6 (0-700)	6 (0-875)
NOTE: CI = confidence interval; PBHCI = primary and behavioral health care integration.

Similarly, during their first 12 months of enrollment, PBHCI consumers were likely to receive mental health screening (81.2%), medication management (72.6%), and treatment planning (62.8%); but they were less likely to receive a referral for a mental health problem (23.9%) or hospitalizations (15.0%) (Table 4.10).

TABLE 4.11. Substance Use Service Utilization, all PBHCI Consumers, by Cohort
	Cohort I(N=8,816)	Cohorts II/III(N=16,832)	All Cohorts(N=25,648)
Percent (95% CI) of Consumers Using Service During First 12 Months
Screening/assessment	58.7 (57.6, 59.8)	55.0 (54.3, 55.8)	56.3 (55.6, 56.9)
Referral	5.8 (5.2, 6.4)	14.1 (13.6, 14.7)	11.7 (11.2, 12.1)
Planning	11.8 (11.1, 12.5)	13.2 (12.6, 13.7)	12.7 (12.3, 13.2)
Medication Management	13.9 (13.1, 14.7)	5.3 (5.0, 5.7)	8.2 (7.8, 8.5)
Hospitalization	0.6 (0.4, 0.8)	5.9 (5.5, 6.3)	4.1 (3.9, 4.4)
Counseling	12.7 (12.0, 13.5)	18.7 (18.1, 19.3)	16.8 (16.3, 17.3)
Median (Range) Visits per Month Among Those Using Service
Screening/assessment	0.72 (0.03-33.33)	0.36 (0.03-33.33)	0.43 (0.03-33.33)
Referral	0.14 (0.03-33.33)	0.25 (0.04-33.33)	0.23 (0.03-33.33)
Planning	0.33 (0.03-33.33)	0.31 (0.04-33.33)	0.32 (0.03-33.33)
Medication Management	0.41 (0.03-33.33)	0.25 (0.04-33.33)	0.36 (0.03-33.33)
Hospitalization	0.09 (0.03-33.33)	0.20 (0.04-33.33)	0.19 (0.03-33.33)
Counseling	0.15 (0.03-33.33)	0.66 (0.04-33.33)	0.43 (0.03-33.33)
Median (Range) Provider Visits Per Month Per Grantee/500 Consumers
Screening/assessment	244 (4-2,284)	440 (0-8,602)	422 (0-8,602)
Referral	6 (1-98)	13 (0-2,753)	9 (0-2,753)
Planning	59 (0-367)	38 (0-3,950)	43 (0-3,950)
Medication Management	8 (0-296)	8 (0-3,538)	8 (0-3,538)
Hospitalization	1 (0-47)	1 (0-861)	1 (0-861)
Counseling	46 (0-216)	133 (1-3,542)	79 (0-3,542)
NOTES: CI = confidence interval; PBHCI = primary and behavioral health care integration. Substance use service utilization does not include tobacco services; Counseling includes active engagement with a consumer by a health care professional or paraprofessional to provide specific information about concepts or skills associated with recovery from substance abuse. This could include screening, brief intervention, and referral to treatment brief counseling.

Compared to PC and mental health services, consumers used fewer substance abuse-related services: Just over half (56.3%) of consumers were screened/assessed for substance use, 16.8% received substance use counseling, and less than 15% of consumers received referrals, treatment planning, medication management, or hospitalizations (Table 4.11). These rates are relatively consistent with TRAC levels of self-reported need, in which 10% reported recent binge drinking and 21.4% reported using illegal substances.

Most (78.4%) consumers accessed (i.e., had at least one contact with) wellness services during their first 12 months enrolled in PBHCI (Table 4.12). Consumers were most likely to access coping/skills support (55.9%) and wellness education (50.7%) and receive referrals to any wellness service during a clinic visit (42.1%). The wellness service with the highest intensity of use at both the consumer and grantee levels was coping/skills support, although the intensity of use varied widely, both among users and across grantees.

TABLE 4.12. Wellness Service Utilization, All PBHCI Consumers, by Cohort
	Cohort I(N=8,816)	Cohorts II/III(N=16,832)	All Cohorts(N=25,648)
Percent (95% CI) of Consumers Using Service During the First 12 Months
Any Wellness Service	72.0 (71.1, 73.0)	81.7 (81.1, 82.3)	78.4 (77.9, 78.9)
Referral	51.6 (50.4, 52.8)	38.2 (37.5, 39.0)	42.1 (41.5, 42.8)
Medication management	22.2 (21.3, 23.2)	36.2 (35.5, 37.0)	31.6 (31.0, 32.2)
Smoking cessation	25.8 (24.8, 26.8)	32.5 (31.8, 33.2)	30.3 (29.7, 30.9)
Wellness education	46.1 (45.0, 47.1)	53.1 (52.3, 53.8)	50.7 (50.0, 51.3)
Exercise	22.8 (21.9, 23.7)	20.6 (19.9, 21.2)	21.3 (20.8, 21.8)
Coping/Skills support	44.7 (43.7, 45.8)	61.8 (61.1, 62.5)	55.9 (55.3, 56.5)
Median (Range) Visits per Month Among Those Using Service
Any Wellness Service	0.58 (0.03-33.33)	1.19 (0.04-33.33)	0.92 (0.03-33.33)
Referral	0.30 (0.03-33.33)	0.23 (0.03-33.33)	0.26 (0.03-33.33)
Medication management	0.35 (0.03-33.33)	0.44 (0.04-33.33)	0.42 (0.03-33.33)
Smoking cessation	0.14 (0.03-33.33)	0.23 (0.04-33.33)	0.19 (0.03-33.33)
Wellness education	0.27 (0.03-33.33)	0.41 (0.03-33.33)	0.36 (0.03-33.33)
Exercise	0.23 (0.03-33.33)	0.35 (0.04-33.33)	0.30 (0.03-33.33)
Coping/Skills support	0.48 (0.03-33.33)	0.99 (0.04-33.33)	0.81 (0.03-33.33)
Median (Range) Provider Visits Per Month Per Grantee/500 Consumers
Any Wellness Service	595 (48-4,318)	1635 (106-8,980)	1418 (48-8,980)
Referral	245 (0-2,626)	243 (1-7,200)	244 (0-7,200)
Medication management	73 (0-1,039)	255 (1-8,337)	191 (0-8,337)
Smoking cessation	89 (1-340)	109 (0-6,025)	105 (0-6,025)
Wellness education	236 (12-1,649)	387 (29-8,459)	346 (12-8,459)
Exercise	72 (2-1,010)	157 (0-4,702)	113 (0-4,702)
Coping/Skills support	229 (17-3,380)	863 (0-8,218)	687 (0-8,218)
NOTES: CI = confidence interval; PBHCI = primary and behavioral health care integration. Wellness Medication Management includes medication education, support for medication adherence, and other medication-related topics.

Preliminary Indicators of Care Quality

We assessed the quality of PC for consumers with a high need for such care (i.e., Type 2 diabetes, hypertension, obesity, and current tobacco use) through a selection of process and intermediate outcome measures indicating the appropriateness of care (Table 4.13). Here too, we used available physical health indicator data to identify consumers with the four primary conditions under study. There was a high degree of variability across the diabetes and hypertension care quality indicators examined. For example, the proportions of PBHCI consumers who met different quality indicators for diabetes care within the first 12 months of enrolling in PBHCI ranged from 9.2% (weight loss of >10 lbs among overweight consumers) to 68.9% (good BP control among consumers with comorbid hypertension). The proportion of consumers who met criteria for quality indicators for hypertension care ranged from 37.8% (receipt of education on the usage of non-pharmacological treatments) to 74.5% (LDL under the "at risk" threshold). Single obesity and tobacco use quality indicators suggested that half (obesity) or less than half (tobacco) of consumers received appropriate care for these conditions.

TABLE 4.13. Quality of Care for Selected Physical Health Conditions, Percentage (95% CI), by Cohort
Indicator	Type ofIndicator	IndicatorSource	CohortI	CohortsII/III	AllCohorts
Diabetes
Consumers with diabetes and LDL <100 mg/dLa	Intermediate outcome	NCQA--modified	48.5  (38.5, 58.5)	55.1  (50.0, 60.3)	53.7  (49.1, 58.3)
Consumers with diabetes and BP <130/80 mm/Hga	Intermediate outcome	NCQA--modified	30.8(22.9, 38.8)	37.4(33.3, 41.5)	36.1(32.5, 39.7)
Consumers with diabetes and BP <140/90 mm/Hga	Intermediate outcome	NCQA	62.4(54.1, 70.7)	70.4(66.6, 74.3)	68.9(65.4, 72.4)
Consumers with diabetes, BMI >25 at baseline and who lost 10 lbsa	Intermediate outcome	NHLBI	9.9(7.0, 12.9)	9.0(7.5, 10.4)	9.2(7.9, 10.5)
Consumers with diabetes who received education about diabetes, nutrition, cooking, physical activity, or exercisea	Process	ICSI--modified	66.8(62.5, 71.1)	68.5(66.3, 70.6)	68.1(66.2, 70.1)
Hypertension
Percentage of hypertensive consumers who received education on the usage of non-pharmacological treatmentsa,b	Process	ICSI--modified	36.1(32.2, 39.9)	38.5(36.2, 40.8)	37.8(35.9, 39.8)
Consumers with hypertension and LDL <130 mg/dLa	Intermediate outcome	Literature--modified	72.3(64.1, 80.4)	75.4(70.4, 80.5)	74.5(70.2, 78.8)
Consumers with hypertension who received education services related to hypertension, nutrition, cooking, physical activity, or exercisea	Process	ICSI--modified	52.2(48.4, 55.9)	54.7(52.4, 57.0)	54.0(52.0, 56.0)
Obesity
Counseling on physical activity and/or nutrition for those with documented elevated BMIa	Process	ICSI--modified	49.7(47.8, 51.6)	55.7(54.6, 56.9)	54.1(53.1, 55.1)
Tobacco use
Consumers identified as tobacco users who received cessation intervention during a two-year measurement period	Process	PCPI	35.3(33.9, 36.8)	44.2(43.2, 45.2)	41.4(40.6, 42.3)
NOTES: BP = blood pressure; BMI = body mass index; CI = confidence interval; ICSI = Institute for Clinical Systems Improvement; LDL = low-density lipoprotein; NCQA = National Committee for Quality Assurance; NHLBI = National Heart, Lung, and Blood Institute; PBHCI = primary and behavioral health care integration; PCPI = Physician Consortium for Performance Improvement. Most modifications reflect the fact that the indicator was restricted to consumers' first 12 months within enrolling in PBHCI. Includes nutrition, cooking, and physical activity wellness services.

Use of Integrated Services

We evaluated the proportion of PBHCI consumers who accessed basic and comprehensive integrated care services. We operationalized integrated care separately and differently for the general sample of PBHCI consumers and for those with probable chronic physical health conditions, since those with physical illness have more complex needs (Table 4.14). Specifically, for the general sample of consumers, we defined basic integrated care as including: (1) a physical health screening or assessment or treatment planning session; (2) contact with a PC provider; and (3) contact with case management, all within the first 12 months of enrolling in PBHCI.

TABLE 4.14. Operational Definitions of Basic and Comprehensive Integrated Care
	General Sample		Physical Health Condition
	Basic	Comprehensive	Basic	Comprehensive
Physical health service
Screening/assessment	Xa	X	Xb	X
Treatment planning	Xa	X	Xb	X
PC Provider	X	X	X	X
Medication management			Xb	X
Referral				X
Wellness		X		X
Case management	X	X	X	X
NOTES: PC = primary care; PBHCI = primary and behavioral health care integration. All indicated services must have been received within the first 12 months of enrollment in PBHCI. All persons receiving comprehensive integrated care necessarily receive basic integrated care. Either one of screening/assessment or treatment planning. Any one of screening/assessment, treatment planning or medication management.

We defined comprehensive integrated care as consisting of: (1) a physical health screening or assessment; (2) a treatment planning session; (3) contact with a PC provider; (4) use of a wellness service; and (5) contact with case management, all within the first 12 months of enrolling in PBHCI. All persons receiving comprehensive integrated care necessarily received basic integrated care.

We also defined basic and comprehensive services separately and more conservatively for consumers with an identified physical health risk. Basic integrated care for this group included: (1) a physical health screening or assessment, treatment planning session, or medication management session; (2) contact with a PC provider; and (3) contact with case management, all within the first 12 months of enrolling in PBHCI. This differs from how we defined basic integrated care for the general population because the first criterion (screening/assessment, treatment planning) can also be satisfied with a medication management session.

The comprehensive package of services for persons with identified physical health risk included: (1) a physical health screening or assessment; (2) treatment planning; (3) contact with a PC provider; (4) medication management; (5) referral; (6) use of a wellness service; and (7) contact with case management. This differs from how we defined comprehensive integrated care for the general population because it also includes medication management and referral.

Just over half of the general sample of PBHCI consumers received basic integrated care and just over one-quarter received comprehensive integrated care (Table 4.15). Among those with probable physical health conditions, approximately one-third received comprehensive integrated services and just over one-half received basic integrated services.

TABLE 4.15. Consumer Access of Integrated Care During the First 12 Months in PBHCI
		Cohort I% (95% CI)	Cohorts II/III% (95% CI)	All Cohorts% (95% CI)
General Sample	Comprehensive	33.7 (32.1, 35.3)	24.8 (23.7, 25.8)	28.0 (27.1, 28.9)
	Basic	68.7 (67.1, 70.2)	46.9 (45.6, 48.1)	54.7 (53.7, 55.7)
Metabolic Syndrome (N= 1,345)	Comprehensive	48.3 (42.6, 54.0)	26.4 (23.7, 29.1)	31.2 (28.7, 33.7)
	Basic	80.4 (75.9, 84.9)	50.4 (47.4, 53.5)	57.0 (54.4, 59.7)
Obesity (N= 3,684)	Comprehensive	40.5 (37.4, 43.6)	26.2 (24.5, 27.8)	29.9 (28.4, 31.3)
	Basic	73.6 (70.8, 76.4)	50.1 (48.2, 51.9)	56.1 (54.5, 57.7)
Diabetes (N=921)	Comprehensive	44.5 (37.3, 51.7)	30.9 (27.5, 34.2)	33.6 (30.5, 36.6)
	Basic	79.1 (73.2, 85.0)	52.5 (48.9, 56.1)	57.8 (54.6, 61.0)
Dyslipidemia (N= 3,094)	Comprehensive	43.2 (39.6, 46.7)	27.3 (25.5, 29.1)	31.1 (29.5, 32.8)
	Basic	79.1 (76.2, 82.0)	51.7 (49.7, 53.7)	58.3 (56.6, 60.0)
Hypertension (N= 905)	Comprehensive	41.8 (35.5, 48.1)	25.9 (22.6, 29.2)	30.1 (27.1, 33.0)
	Basic	77.2 (71.9, 82.6)	49.7 (45.9, 53.5)	56.9 (53.7, 60.1)
NOTES: CI = confidence interval; PBHCI = primary and behavioral health care integration. Comprehensive and basic integrated care are operationalized differently for the general sample and for consumers with chronic physical health conditions. See definitions in Table 4.14. All consumers receiving comprehensive integrated care necessarily receive basic integrated care, too.

Summary

This chapter presented results from the second half of the process evaluation, describing how consumers interface with grantees' PBHCI programs of care. In particular, we described grantees' success at enrolling and engaging consumers in their programs and enrolled consumers' demographic and psychosocial characteristics, their care needs, and the match between those needs and the services they received. We also described consumers' receipt of integrated care.

We found that grantee sites approached the integration of primary and BH care in different ways, leading to significant variability in the reach and appropriateness of the services provided. This variability notwithstanding, several results are worth noting because of their probable capacity to impact outcomes. We discuss these results in connection with the process domain they describe. PBHCI programs reported high rates of consumer enrollment in their first year of operation, often exceeding estimated enrollment for that year. On the other hand, programs largely fell short of long-term enrollment targets, enrolling just over one-quarter of their total anticipated clientele after three of four grant years. Taken together, the results suggest that programs may have success identifying and enrolling the most willing consumers in integrated care, but that they may require technical assistance to identify and engage other members of their consumer population with suspected PC needs.

PBHCI consumers were demographically diverse, suggesting that PBHCI programs were able to engage a wide variety of clientele. PBHCI consumers also had high rates of health care needs. Approximately half of the consumers reported low levels of well-being and nearly one-quarter reported use of illegal substances. Rates of consumers at risk for chronic physical health conditions were consistently higher than general population rates and they affirm the need for intensive, integrated primary and BH care systems in communities and nationwide.

Service utilization data showed that PBHCI programs are making progress toward addressing consumers' multispecialty health care needs. Appropriately, the majority of consumers had contact with physical, BH, and care management providers, and they received physical and mental health assessments or screenings, medication management, and treatment planning. However, rates of substance abuse screening, medication management, and treatment planning were lower; in part, this may reflect lower rates of substance use service needs, but it may also reflect low rates of identified substance use problems. Given the high rates of comorbid substance abuse and SMI in national samples (SAMHSA, 2012), PBHCI programs could work to ensure they are reliably assessing/screening and effectively managing consumers' substance use–related needs.

Service utilization data also showed that, although the majority of consumers received at least one contact with a variety of providers and services during their first year of care, a sizable proportion (22%) of enrollees discontinued PBHCI treatment, with the average length of enrollment prior to discontinuation being just seven months. Although problematic, difficulties engaging people with SMI in integrated care may reflect the relative newness of PBHCI programs. However, since programs did most of their consumer enrollment during the first year of operation, these data suggest that programs can continue to improve their strategies for engaging consumers in long-term care.

Similarly, access to and the intensity of PC provider contacts were not substantially different for consumers with probable chronic physical health conditions (who necessarily have a greater degree of PC needs) compared to those at lower risk (i.e., consumers not meeting any of the risk indicators assessed). These findings raise questions about the capacity of grantees to allocate resources to those consumers with the greatest physical health needs.

With regard to the quality of PC, overall, programs showed low rates of meeting evidence-based quality indicators for obesity and tobacco use--two of the major preventable causes of morbidity and mortality in the United States (Danaei et al., 2009).

Finally, the data showed that some consumers can receive a rich array of behavioral and PC services. Nearly half of the enrolled consumers received at least some physical and care management services within one year of enrolling in PBHCI. Given that the majority of these programs of integrated care are fairly new (less than four years old), the PBHCI programs' coordination of systems, providers, and services to manage complex comorbid conditions is noteworthy. At the same time, some PBHCI programs are not yet providing basic (about 50%) or comprehensive (about 75%) integrated services to many consumers in need, suggesting the need for programs to continue to develop and refine their strategies for improving consumer access to and use of primary and secondary preventive care services.

In sum, grantees' implementation of PBHCI was highly variable, with some grantees providing high quality comprehensive services to consumers and others struggling to provide services likely to achieve the desired outcome of improving the physical health and overall well-being of their target consumer population.

5. COMPARATIVE EFFECTIVENESS STUDY

Overview

In this chapter, we describe the results of the comparative effectiveness study, which was designed to answer Research Question 2, Does the integration of primary and BH care lead to improvements in the mental and physical health of the population with SMI and/or substance use disorders served by these integrated care models? This comparative effectiveness study consisted of a quasi-experimental design through which we compared differences in individuals' physical health and BH outcomes at three intervention (PBHCI) sites and three matched control-sites after one year's time. We hypothesized that consumers served at PBHCI clinics would show greater improvements in physical health and BH during the study period than those served in usual (control) clinic settings.

Methods

Additional methodological detail about the comparative effectiveness study is provided in the Appendix.

Participants and Sites

We selected three PBHCI (intervention) sites that were large (ideally serving at least 750 SMI consumers each, although actual numbers suggest that programs were serving 450 or more consumers each) and that undertook diverse approaches to implementing PBHCI (e.g., geographical location, urbanicity, client demographics, services provided, and PC partner agency). The programs needed to be "high implementers" of PBHCI so that we could illustrate PBHCI outcomes for programs implementing integrated care with better-than-average success. In short, the sample of PBHCI sites included in this comparative effectiveness evaluation is diverse but intentionally not representative of the larger pool of grantees.

Matched control-sites were identified via a web search; suggestions from SAMHSA, state mental health authorities, and other agencies; and suggestions from prospective intervention sites themselves. Control-sites were required to be located within the same state as their matched PBHCI intervention site and be similar to the intervention site in their selection criteria, except that they offered no or low PC (i.e., they did not offer PC as part of their program or had no formal referral relationship with a PC provider). The final sample includes the original Cohort I site and two new Cohort III sites that satisfied the study inclusion criteria outlined above.

Intervention sites were compensated up to $10,000 and control-sites were compensated up to $25,000 for participating in the comparative effectiveness evaluation; compensation was prorated based on the number of consumers identified, recruited, and enrolled in the study.

Consumers

Eligible consumers were adults at least 18 years of age whose primary psychiatric diagnosis was an SMI (see Chapter One) and who were enrolled in integrated care services at a participating PBHCI program or received BH care at a matched control-site. We also attempted to enroll consumers at intervention sites who had their first contact with PBHCI within one year (+/- six months) from the follow-up data collection event so that we could quantify the impact of PBHCI on individuals' outcomes after approximately one year of treatment.

Data Collection

Data for the comparative effectiveness study included physical health indicators and a slightly abbreviated version of the NOMs (see Chapter Two). Data sources and the timing of data collection at the control and intervention site pairs are provided in Table 5.1 and the Appendix.

TABLE 5.1. Baseline and Follow-Up Data Sources and Timing
Sites		Baseline	1-Year Follow-Up
Control	Site 1	November 2011	November 2012
	Data Source	RAND Subcontractor OHD	RAND Subcontractor OHD
	Sites 2 and 3	March, April, and July 2012	February-May 2013
	Data Source	RAND subcontractor OHD	RAND subcontractor OHD
Intervention	Site 1	February 2011	February 2012
	Data Source	TRAC	RAND subcontractor OHD
	Sites 2 and 3	February 2012	February-May 2013
	Data Source	TRAC	RAND subcontractor OHD
NOTE: OHD = Onsite Health Diagnostics; TRAC = TRansformation ACcountability.

Participant Recruitment

Prospective participants learned about the study from trained case management staff and advertising (e.g., posters in waiting rooms, write-ups in clinic newsletters, etc.) at participating control clinics and received further information by mail and by phone. Participants received $10 for completing the survey and $10 for completing the biometric screening procedures at the baseline and follow-up assessment, respectively.

Procedures

Consumers provided written informed consent upon arriving at their respective study site and then took part in a physical health exam (conducted by RAND subcontractor OHD). The exam included measures of height, weight, BMI (calculated from height and weight), BP, waist circumference, and breath CO. OHD' licensed and trained phlebotomists also collected a blood sample for the following tests: FPG; A1c; and several lipids, including cholesterol (total, HDL, and LDL) and triglycerides. On-site staff asked participants if they had successfully fasted for eight hours prior to the health exam, and although an eight-hour fast was necessary prior to the blood tests for FPG and lipids, blood samples were drawn from individuals whether or not they were fasting. Fasting information was recorded and accounted for in the data analysis.

Individual participants and the medical directors of participating intervention and control-sites received results of the physical health exams. Test results falling outside of the normal range were clearly indicated. Along with test results, consumers received a booklet with information about each test and the meaning of out-of-range values. Consumers with out-of-range values were instructed to contact their health care provider, and consumers without health care providers were instructed to contact their BH care provider for assistance with connecting to PC services. Participating clinics agreed to facilitate consumer referrals to local PC providers, as needed, following receipt of screening results. This evaluation has no information, however, about the frequency with which control-site referrals were made or completed.

Analytic Approach

We used a difference-in-difference analysis to estimate the causal association of PBHCI with participant outcomes (Ashenfelier and Card, 1985; Meyer, 1995; Imbens and Wooldridge, 2009). Difference-in-difference is one of the most popular approaches to estimating treatment effects from quasi-experimental and observational studies because it mitigates biases in intervention-control group comparisons that could be the result of permanent differences between those groups and/or biases in the pre-post comparison resulting from secular trends unrelated to the intervention. Table 5.2 illustrates how a difference-in-difference analysis is computed. In the simplest case, program effects are assessed by comparing before-and-after differences in outcomes between persons exposed to an intervention and persons with no exposure (the control group).

TABLE 5.2. Difference-in-Difference Analysis
yst	c = 2(control consumers)	c = 1(PBHCI consumers)
t = 2 (follow-up)	y22	y12
t = 1 (baseline)	y21	y11
Difference	y22 - y21	y12 - y11
Difference-in-difference	(y11 - y11) - (y22 - y21)
NOTE: PBHCI = primary and behavioral health care integration.

We calculated PBHCI effects on consumer health outcomes using a semiparametric, causal difference-in-difference analysis (Abadie, 2005). Specifically, we compared differences in outcomes between persons served at PBHCI and control clinics after one year, and we created a case-mix adjustment that balanced the observed characteristics¹⁶ of PBHCI and control participants via propensity score matching (i.e., we weighted control data based on the probability that an individual, based on his/her observed characteristics, received the PBHCI intervention). A strength of this semiparametric, flexible model is that it can accommodate patterns in the data that could create model misspecification when more rigid, parametric approaches are applied. Given standard technical assumptions (i.e., Rubin's [1974] causal model¹⁷), this method provides the best unbiased estimate for the average treatment effect on persons in the treatment group relative to the control group.

Results

Match Within Intervention and Control-Site Pairs

The following set of analyses shows the degree of balance within intervention and control-site pairs on program features that could affect service utilization and consumer health and well-being. Since the pool of possible control clinic participants was limited, we expected selected sites to be similar in some regards but different in others. While clinic differences (as opposed to consumer differences) cannot be accounted for in the analyses per se, we consider them in the discussion of program results.

Table 5.3 and Table 5.4 describe characteristics of the PBHCI intervention and control-sites prior to and independent of the PBHCI program. An important feature of Table 5.3 is that participants from control-sites could include all adult clients with SMI, while the PBHCI consumers may not reflect the entire pool of adults with SMI served by the BH grantee.

TABLE 5.3. Number of SMI Consumers Served and Location of Intervention and Control Sites
Site Pair	Type	Total SMI  Clients	Urbanicity	PBHCI Target Clientele
1	Intervention	3,120	Rural	Clients without PC provider or dissatisfied with PC provider
	Control	1,050	Rural	None
2	Intervention	1,600	Urban	All clients offered
	Control	2,000	Rural	None
3	Intervention	3,000	Urban	All existing adult clients with SMI at BH clinic and partner FQHC
	Control	1,130	Urban	None
NOTE: BH = behavioral health; FQHC = federally qualified health center; PBHCI =primary and behavioral health care integration; PC = primary care; SMI = serious mental illness.

Overall, data show that the intervention and control-sites were well matched in terms of program structure; that is, paired sites were in similar locations (but see Site Pair 2), provided similar BH services, and served similarly sized client populations.

TABLE 5.4. BH Services Provided at Intervention and Control Sites
Site Pair	Type	Outpatient	Residential	Crisis/  Emergency	SUD  Treatment
1	Intervention	X		X
	Control	X	X	X	X
2	Intervention	X	X	X	Detox only
	Control	X	X	X	X
3	Intervention	X	X	X	X
	Control	X	X	X	X
NOTE: BH = behavioral health; SUD = substance use disorder.

Participant baseline data also showed that, within site pairs, clinics tended to serve similar clientele but with some differences (Table 5.5). Specifically, within Site Pair 1, the intervention site served a population that was nearly ten years younger than that of the control-site. In Pair 2, the control sample had more female and Caucasian participants than the intervention site (which had nearly equal proportions of men and women and African American and Caucasian participants). We note these differences, in particular, as older age, gender, and race are associated with increased risk for chronic physical health conditions (CDC, 2012, 2013). We accounted for within-pair consumer demographic differences in the outcomes analysis using propensity score matching.

TABLE 5.5. Demographics of Participating Consumers Within Intervention and Control-Site Pairs
Demographics	Site Pair 1		Site Pair 2		Site Pair 3		All  Grantees  N=56
	Intervention  %	Control  %	Intervention  %	Control  %	Intervention  %	Control  %
Gender
Male	41.6	40.6	47.3a	30.6	40.0	37.4	46.4
Female	58.4	59.4	52.5a	68.6	60.0	62.3	53.3
Ethnicity
Hispanic/Latino	0.8	1.1	3.1	4.4	11.9	8.5	14.8
Black	2.7	3.0	45.1a	11.0	3.9a	1.2	25.6
Asian	2.0a	0.0	0.3	1.5	0.8	0.4	4.2
Hawaiian Native	3.5a	0.0	1.2	1.1	0.3	1.5	1.9
Alaska Native	0.2	0.0	0.2	0.0	0.6	0.0	0.6
White	93.1	95.1	51.4a	84.2	85.8a	92.7	61.3
American Indian	4.4a	8.2	0.7a	10.3	3.9	3.1	7.2
Age: mean	39.7a	47.4	41.5a	44.0	41.1a	38.3	43.8
Within-pair difference p<0.05.

Baseline Physical Health

Table 5.6 shows the proportion of consumers identified as "at risk" for chronic physical illness at control-sites and at intervention and other PBHCI sites at baseline or at enrollment in PBHCI, respectively. The purpose of this table is to illustrate any differences in physical health risk between consumers served at intervention and control clinics and between intervention sites and the PBHCI population at large.

TABLE 5.6. Proportions of Comparative Effectiveness Study Participants "At Risk" for Chronic Physical Illness at Study Baseline (controls) or Enrollment in PBHCI (intervention)
	Site 1		Site 2		Site 3		All  Sites  N=56
	Intervention	Control	Intervention	Control	Intervention	Control
SBP	37.4a	27.4	40.9	38.0	34.4a	22.4	37.9
DBP	30.8	26.3	34.8a	24.0	32.7a	18.0	31.0
BMI	75.6	81.7	74.3	74.1	74.4	78.0	76.5
TC	10.9	10.9	11.1	16.4	10.2	7.1	12.3
HDL-C	44.6a	30.5	25.6	26.9	31.9	29.9	30.0
LDL-C	26.2	18.9	26.7	25.1	21.5	18.2	26.1
FPG	11.3a	25.5	22.3	18.3	18.9	9.6	30.5
A1c	55.4	59.9	37.9	38.6	30.3	36.0	52.6
Trig	46.1	53.8	33.4a	47.0	40.2	47.2	39.8
Smok	72.5a	58.1	62.7	64.2	55.5	54.1	59.8
NOTES: A1c = glycated hemoglobin; BMI = body mass index; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; SBP = systolic blood pressure; Smok = self-reported smoking status; TC = total cholesterol; Trig = Triglycerides. Within site pair difference of >10%.

In general, the data showed similar rates of chronic physical illness risk at intervention and other PBHCI sites (except for higher rates of elevated plasma glucose). The match between intervention and control-site pairs was also generally good, although intervention sites had consistently higher incidence of hypertension risk (elevated SBP and DBP) and marginally lower incidence of elevated triglycerides (this difference was most pronounced within Site Pair 2). Also notable was that within Site Pair 1, the intervention group showed higher rates of elevated HDL-C and smoking compared to the control. Finally, we also note that some differences may be due to PBHCI programs' selection of adults with SMI into integrated care services (see Table 5.3). We used propensity score matching to account for these differences within site pairs in the outcomes analysis below.

PC at Participating PBHCI Sites

In this section, we describe the PC at participating intervention sites to show how PBHCI programs were intentionally and systematically different from their matched control. This comparison suggests program features that could contribute to observed PBHCI outcome effects.

We also describe PC offered at the three participating intervention sites relative to the PBHCI program in general in order to show how the intervention sites may (or may not) resemble the "typical" PBHCI grantee. This information is meant to help the reader consider the extent to which the results of this study may generalize to outcomes across the larger pool of PBHCI grantees.

PC Program Structures

The PC services available at participating intervention sites are shown in Table 5.7 and Table 5.8.

TABLE 5.7. Characteristics of PC at Participating PBHCI Sites
PBHCI  Program	PC Partner  Agency	FQHC	FQHC Annual  Patient Volume	EHR
1		N/A	N/A	N/A
2	X	X	10,750	X
3	X	X	27,000	X
NOTE: EHR = electronic health record; FQHC = federally qualified health center; PBHCI = primary and behavioral health care integration; PC = primary care.

As intended, participating programs implemented PBHCI in very different ways. Two intervention sites (2 and 3) had existing partnerships with PC agencies prior to receiving the PBHCI grant, while Intervention Site 1 hired its own PC providers. All three programs provided individual and group wellness programs, including smoking cessation services, although one site did not offer nicotine replacement medications to help smokers quit.

TABLE 5.8. Integrated Care Program Structural Features of PBHCI Intervention Sites
PBHCI  Program	Co-location  NPs/PC Provider	PC Supervising  Physician	Embedded Nurse  Care Managers	Use of  EBPs	Unique Structural  Features
1	X	X	X	X	On-site phlebotomy; tobacco recovery across the continuum
2	X	X	a	X
3	X	X	X	X	Pharmacy and lab on-site
NOTES: EBP = evidence-based practice; NP = nurse practitioner; PBHCI = primary and behavioral health care integration; PC = primary care. Non-nurse care manager.

Importantly, although control-sites were selected for having no or low PC (among other reasons), we note that Control-Site 3 had an unofficial referral relationship with a local FQHC (the same FQHC providing PC to Intervention Site 3) and that it also offered some consumer wellness services (smoking cessation, a course entitled "Living Well with Chronic Conditions," Zumba, and Wii Fitness). We take these unintended interventions-control-site similarities into account when considering the impact of Intervention Site 3 on outcomes.

PC Service Access and Utilization

Next we describe the proportion of consumers accessing (i.e., having at least one contact with) different PBHCI providers and services (including referrals to specialists or other related but ancillary services) within 12 months of enrolling in PBHCI. We present these data alongside proportions of consumers with provider and service contacts at all other grantee sites to illustrate how the selected intervention sites may or may not resemble the "typical" PBHCI grantee.

Data on provider contacts appear in Table 5.9. As intended by the site selection process, the proportion of consumers having contact with PC providers and care managers within 12 months of enrolling in PBHCI was generally higher at intervention sites than at other PBHCI sites (with the exception of low rates of care manager contacts at Site 3). Unexpectedly, the proportion of consumers with peer specialist and other specialist contacts within 12 months of enrolling in PBHCI was lower at intervention versus other PBHCI sites.

TABLE 5.9. Proportion of Consumers with a PBHCI Provider Contact Within One Year of Enrollment
	Site 1	Site 2	Site 3	All Sites  (N=56)
Percentage of Clients Seeing Provider During First 12 Months
Care manager	87.6	98.1	16.5	69.3
PC provider	90.0	87.1	89.3	75.3
Psychiatrist/psychiatric nurse	87.1	62.3	71.3	72.4
Counselor	74.6	24.6	74.3	49.2
Peer specialist	15.1	8.9	4.7	24.6
Other specialist	46.0	0.0	13.2	22.3
NOTE: PBHCI = primary and behavioral health care integration; PC = primary care.

Rates of consumer access to PBHCI physical health services within 12 months of enrollment are presented in Table 5.10. Rates of access to ambulatory physical health services tended to be higher at intervention sites than at other PBHCI sites. In addition, fewer consumers were hospitalized for a physical illness at Intervention Sites 1 and 3 (but not Site 2) than at other PBHCI sites.

TABLE 5.10. Proportion of Consumers with Physical Health Service Contacts Within One Year of Enrolling in PBHCI
	Site 1	Site 2	Site 3	All Sites
Percentage of Clients Using Service During First 12 Months
Screening/assessment	99.9	89.0	89.3	86.2
Referral	50.1	40.5	0.0	38.6
Planning	94.6	79.9	89.3	72.7
Medication management	94.4	79.9	83.5	64.8
Hospitalization	0.0	10.2	0.0	10.7
NOTE: PBHCI = primary and behavioral health care integration.

Finally, the proportion of consumers using wellness services within 12 months of enrollment in PBHCI (Table 5.11) varied between intervention sites. First, not all sites offered every wellness service. For instance, Intervention Sites 1 and 3 did not offer medication management, and Intervention Site 3 did not offer consumers referrals to wellness programs. Intervention Site 2 did not offer exercise. Among the wellness services available, consumers at Intervention Sites 1 and 2 were as likely as or more likely than consumers at other sites to access smoking cessation, wellness education, and coping/skills support programs. Expectedly, wellness service access at Intervention Site 3 was consistently low. For instance, while 91% of consumers received smoking cessation services at Intervention Site 1, only 3% of consumers received this service at Intervention Site 3.

TABLE 5.11. Proportion of Consumers with Wellness Service Utilization Contacts Within One Year of Enrolling in PBHCI
	Site 1	Site 2	Site 3	All Sites
Any wellness service	54.5	91.5	14.2	78.4
Referral	17.1	17.6	n/a	42.1
Medication management	n/a	79.7	n/a	31.6
Smoking cessation	91.1	61.4	3.2	30.3
Wellness education	48.7	84.7	10.1	50.7
Exercise	48.1	0.2	7.9	21.3
Coping/skills support	49.5	85.4	4.3	55.9
NOTE: PBHCI = primary and behavioral health care integration.

Attrition

Analysis of attrition at control-sites showed that we were able to recruit 65% of consumers who participated in the baseline assessment to come back for the one-year follow-up assessment. Attrition analysis at the intervention sites is more difficult because the baseline sample includes all consumers enrolled between six and 18 months prior to the data collection session, some of whom would not have volunteered for the follow-up research screen. Overall, 25% of potentially eligible intervention site consumers participated in the study. To determine how selection effects may have impacted the results, we ran several rounds of sensitivity analyses. Fortunately, these sensitivity analyses showed that the outcome results reported below were likely not biased by consumer selection at intervention sites or attrition at control-sites.

Physical Health Outcomes

In this section, we test our hypothesis that consumers served at PBHCI clinics would show greater improvements in physical health and BH during the study period than those served at usual care (control) clinic settings (Table 5.12). Overall, the results showed that PBHCI consumers showed greater improvements in DBP, TC (primarily due to reductions in LDL-C), and plasma glucose compared to controls. However, there were no statistically significant differences between PBHCI and control consumers in changes in SBP, BMI, and A1c; and in the case of self-reported smoking, outcomes at control clinics were more favorable than those for PBHCI.¹⁸ We also did not detect any effect of PBHCI on triglycerides; however, this was expected because the analysis was underpowered.

TABLE 5.12. Physical Health Outcomes Combined Across All Consumers, Clinics
Indicator	N	Unadjusted Mean Change			Case-Mix Adjusted  (estimated) Difference	STD  Error	p-value
		PBHCI	Control	Difference
SBP	881	-2.05	0.02	-2.17	-2.44	1.34	0.07
DBP	881	-4.38	-1.05	-3.33	-2.60	0.94	0.01
BMI	869	-0.11	-0.33	0.22	-0.02	0.50	0.97
TC	736	-3.78	1.83	-5.62	-7.07	3.16	0.03
HDL-C	795	1.68	0.96	0.72	-0.14	1.09	0.90
LDL-C	739	-5.44	2.70	-8.14	-9.30	2.55	0.00
FPG	752	0.01	0.06	-0.06	-0.07	0.04	0.04
A1c	532	0.16	0.00	0.16	0.22	0.21	0.28
Trig	794	-5.05	-4.09	-0.96	0.87	8.80	0.92
Smok	906	0.02	-0.03	0.05	0.05	0.02	0.03
NOTE: A1c = glycated hemoglobin; BMI = body mass index; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; SBP = systolic blood pressure; Smok = self-reported smoking status; STD = standard; TC = total cholesterol; Trig = Triglycerides.

We replicated this analysis among the subset of consumers who were "at risk" for chronic physical illness at baseline. A limitation of this analysis is that the number of consumers in the "at risk" range in both the baseline and follow-up data was often too small to produce detectable effects. However, given the importance of assessing outcomes for consumers identified as "at risk" upon enrollment in PBHCI (i.e., those who show a need for treatment), we briefly report the outcomes for the "at risk" sample here. The data showed that PBHCI consumers had greater improvements than the controls in HDL, LDL, and TC (Table 5.13; Figure 5.1). No other effects were significantly different between PBHCI and control consumers.

TABLE 5.13. Physical Health Outcomes for Persons with Baseline Risk for Physical Health Conditions, Combined Across All Participating PBHCI and Control Clinics
Indicator	N	Unadjusted Mean Change			Case-Mix  Estimated Difference	STD  Error	p-value
		PBHCI	Control	Difference
SBP	299	-13.90	-12.96	-0.94	-2.22	2.15	0.30
DBP	237	-12.44	-10.14	-2.30	-1.41	1.79	0.43
BMI	668	-0.78	-0.72	-0.06	-0.34	0.62	0.59
TC	85	-54.25	-18.38	-35.87	-33.02	9.13	0.00
HDL-C	248	7.24	3.15	4.09	3.90	1.32	0.00
LDL-C	177	-35.28	-2.52	-32.76	-33.78	5.46	<0.0001
FPG	136	-0.49	-0.37	-0.12	-0.15	0.09	0.13
A1c	254	-0.85	-0.03	-0.83	-0.65	0.40	0.10
Trig	376	-54.22	-34.82	-19.40	-8.98	14.07	0.52
Smok	517	-0.09	-0.13	0.04	0.05	0.03	0.11
NOTE: A1c = glycated hemoglobin; BMI = body mass index; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; SBP = systolic blood pressure; Smok = self-reported smoking status; STD = standard; TC = total cholesterol; Trig = Triglycerides.

To provide greater context for the continuous data presented above, we also calculated the proportion of PBHCI and control consumers who showed improvement, no change, or worsening in physical health indicators during the study period (Table 5.14). Individuals were classified as "improved" if an indicator was in the "at risk" range at baseline but not at follow-up; individuals were classified as having "no change" if an indicator did not change in range from baseline to follow-up; and individuals were classified as "worsened" if an indicator was in the "not at risk" range at baseline but in the "at risk" range at follow-up. For the indicators showing statistically significant PBHCI benefit above, the rates of improvement for PBHCI versus controls, respectively, are as follows: DBP (63.2% versus 53.6%), TC (53.2% versus 44.2%), and LDL-C (55.4% versus 43.5%). Note also that although rates of improvement for PBHCI versus controls are similar for FPG, rates of FPG worsening were higher for control consumers than PBHCI consumers.

TABLE 5.14. Percentage of Consumers Who Showed Improvement, No Change, or Worsening Physical Health Risk from Baseline to One-Year Follow-Up
Indicator	Improveda(%)		No Changeb(%)		Worsenedc(%)
	PBHCI	Control	PBHCI	Control	PBHCI	Control
SBP	54.2	45.5	1.7	3.2	44.1	51.3
DBP	63.2	53.6	3.4	4.1	33.4	42.3
BMI	45.7	53.0	0.0	1.1	54.3	45.9
TC	53.2	44.2	0.7	0.4	46.1	55.3
HDL-C	56.4	52.1	3.2	3.8	40.4	44.1
LDL-C	55.4	43.5	1.9	1.9	42.7	54.6
FPG	8.0	7.1	83.3	79.4	8.7	13.5
A1c	25.3	47.2	3.6	14.3	71.1	38.5
Trig	45.9	52.4	0.3	0.4	53.8	47.1
Smok	5.5	7.3	87.2	88.4	7.3	4.3
NOTES: A1c = glycated hemoglobin; BMI = body mass index; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; SBP = systolic blood pressure; Smok = self-reported smoking status; STD = standard; TC = total cholesterol; Trig = Triglycerides. Indicator was in the "at risk" range at baseline and was no longer in the "at risk" range at follow-up. Indicator did not change from "at risk" or "not at risk" from baseline to follow-up. Indicator was in the "not at risk" range at baseline and was in the "at risk" range at follow-up.

To better understand the PBHCI-control differences reported above, we also examined physical health outcomes separately for each site pair (Figure 5.1; also see the tabular results in the Appendix). Since individual site samples were smaller and had reduced statistical power compared to the pooled analysis above, we note within-site pair differences as significant at the p<0.10 level to facilitate detection of PBHCI program effects.

Across the site pairs, PBHCI programs were generally associated with improvements in SBP and DBP, TC, HDL-C (except Site Pair 2), LDL-C, and FPG (except Site Pair 1). Programs showed mixed effects for BMI, with the results for Site Pairs 1 and 2 favoring PBHCI (although not significantly so), and trends unexpectedly suggested greater improvement in A1c, triglycerides (Site Pair 3), and smoking for controls.

Finally, as a point of comparison, we also analyzed the change in consumer physical health indicators from baseline to one-year follow-up, as reported in TRAC. These analyses are presented in the Appendix.

FIGURE 5.1. Change Trends in Case-Mix Adjusted Physical Health Indicators, Separately and Combined for Intervention/Control-Site Pairs

NOTES: Green favors PBHCI, Orange shows no advantage for PBHCI or controls, and Red favors controls. A plus sign indicates a statistically significant difference (p<0.05) for analyses of the "at risk" sample.

BH Outcomes

Since changes in physical health and BH service use and outcomes may be interrelated (e.g., consumers who feel better physically may be better able to engage in social relationships), we also investigated changes in BH outcomes for persons served at intervention and control clinics (Table 5.15). Overall, the data did not suggest a clear relationship between PBHCI and BH outcomes. Specifically, the changes in self-reported social connectedness and self-reported overall health were not different between PBHCI and control consumers; and while the changes in self-reported rates of binge drinking and substance abuse were different for PBHCI and control consumers, the difference in substance abuse favored PBHCI, while the difference in binge drinking favored the controls.

TABLE 5.15. Comparative Change in BH Indicators Among Persons Served at PBHCI and Control Clinics during the One-Year Study Period
Indicator	N	Unadjusted Mean Change			Estimated  Difference	STD  Error	p-value
		PBHCI	Control	Difference
Binge drinking	948	0.12	-0.01	0.13	0.11	0.03	<0.0001
Substance use	951	-0.12	0.00	-0.11	-0.13	0.03	<0.0001
Social connectedness	1,013	0.05	0.03	0.02	0.03	0.04	0.43
Healthy overall	935	0.05	0.02	0.03	0.05	0.04	0.24
NOTE: BH = behavioral health; PBHCI = primary and behavioral health care integration; STD = standard.

Discussion

The purpose of this chapter was to present the results of a comparative effectiveness evaluation of PBHCI. The evaluation compared changes in physical health and BH indicators at three PBHCI and control clinic pairs over one year. The results indicate that, compared with the controls, consumers treated at PBHCI clinics had greater reductions in select indicators of risk for metabolic syndrome and several physical health conditions, including hypertension, dyslipidemia, diabetes, and cardiovascular disease. No similar benefit of PBHCI was observed for other indicators, including triglycerides, obesity, and smoking. Consistent with Druss et al. (2001) and with a research design that did not test an intervention designed to improve delivery of BH services, results also showed no reliable benefit of PBHCI on indicators of BH.

Overall, this mixed set of results was largely expected; rarely do dramatic changes to health care delivery, particularly in the early stages of implementation, result in consistent improvements in all health outcomes examined. As with other recent, related health care reforms (e.g., Nutting et al., 2009; Felland, Lechner, and Sommers, 2013), issues related to program implementation, quality of care delivery, patient engagement/adherence to treatment, and a lack of focus on other factors that could affect outcomes (e.g., physical activity and diet) (see Chapter Three and Chapter Four regarding process evaluation) likely mitigated the effects of early PBHCI implementation on outcomes. Nonetheless, we consider the meaning of these outcomes individually by comparing them with other intervention studies for adults with SMI, and within the limitations of the current design. Before discussing these results, however, we describe the methodological limitations of this portion of the study.

Limitations

Several limitations of this analysis are worth mentioning. First, the PBHCI clinics included in this analysis are not representative of PBHCI as a whole. The comparative effectiveness evaluation sample was small (due to budget constraints) and sites were highly selected both for program and population diversity and to support design features of the evaluation (e.g., a need for a larger population of PBHCI consumers for statistical power). There were also some challenges with control-site selection. In addition to disparity in terms of structural features (e.g., location in Site Pair 2), site visits revealed significantly more provision of PC than anticipated (Site Pair 3).

Relatedly, consumers were not randomized to conditions; consequently, control programs were eligible to enroll all adult consumers with SMI, while intervention sites could only enroll those already enrolled in PBHCI, who may have been selected (intentionally or not) for extant physical illness, use of psychotropic medications, or other reasons. Other limitations include limited statistical power for some outcomes of interest (e.g., HDL-C, A1c, triglycerides), causing potential treatment effects (i.e., HDL-C) to have gone undetected. We were also unable to include other physical health indicators of interest (breath CO, waist circumference) because the data were incomplete (both were optional indicators not collected in full by the sites included in the evaluation).

Several key pieces of information about control-sites were also unavailable. For instance, we could not account for control participants' baseline PC service utilization before-and-after the study period. This is important because there may have been some slight clinical improvement in physical health indicators following the baseline assessment and receipt of screening results. In other words, the baseline screen that was part of this study could have acted as an intervention that made the comparison of PBHCI to controls particularly stringent (e.g., Baker et al., 2013). Further, several variables of interest were not measured as part of PBHCI, such as receipt of medications, treatment adherence, and others.

To properly consider observed differences in baseline data, we note that baseline data were collected differently at intervention sites (as part of clinical care; reported by grantee) that at control-sites (collected by OHD; single data collection event). Indeed, it is unclear whether differences in the baseline assessed risk are true differences or somehow related to data collection procedures.

Finally, the study follow-up period was somewhat limited. Particular physical health indicators are known to be "quick responders" to appropriate medication (e.g., BP), while others that might also require intensive lifestyle modification (e.g., smoking cessation or weight loss) could take longer to improve, so longer term treatment impacts were not observed. Finally, since it is unclear whether consumers received medication and/or behavioral interventions, we were unable to identifying the "active" and/or missing components of treatment.

Dyslipidemia

Treatment of dyslipidemia is well studied in general populations of adults and effective treatment consists of diet modification and statins (e.g., Jellinger et al., 2012). At the same time, lipids have not been a primary target for intervention studies of adults with SMI, whose lipids are often affected by psychotropic medications (Gierisch et al., 2013), and best practices for the population have not yet been established. Therefore, while there is still much to learn about how providers should treat dyslipidemia in adults with SMI, at least 15 studies have reported lipid levels as secondary outcomes of other interventions (Gierisch et al., 2013), and we consider the results of this study alongside the available literature.

Cholesterol

Cholesterol-related diseases (e.g., cardiovascular disease) are major contributors to the disparity in life expectancy between adults with and without SMI (Gierisch et al., 2013). The results of this study showed that PBHCI was associated with greater reductions in cholesterol than controls. Importantly, among consumers with cholesterol in the "at risk" range at baseline, cholesterol reductions were large enough to potentially result in clinical improvements in consumer physical health. Specifically, multiple studies have shown that each 10-mg/dL reduction in LDL-C, for example, is associated with an approximately 10% reduction in cardiovascular risk in adults (reviewed in Rahilly-Tierney et al., 2009). In this study, LDL-C was reduced by 35 mg/dL in the PBHCI "at risk" group (versus 2.52 mg/dL in "at risk" controls), suggesting a potential reduction in cholesterol-related cardiovascular risk of up to 35%.

This result is consistent with other published trials showing that cholesterol levels can be effectively managed with evidence-based lifestyle modification and/or pharmacotherapy (Carrizo et al., 2009; McKibbin et al., 2006; Wu et al., 2008; Fernandez-San-Marin et al., 2013). However, since we do not have any information about which consumers received cholesterol-lowering medications or any other medications, including psychotropics, associated with lipid changes (e.g., antipsychotic drugs) or which consumers made changes to their diet, further evaluation is needed to identify factors associated with reductions in cholesterol occurring in PBHCI consumers.

Triglycerides

The results of this study showed no clear statistical or clinical relationship between PBHCI participation and change in consumer triglyceride levels. Since no studies have evaluated standard pharmacotherapy for hyperlipidemia in adults with SMI (Gierisich et al., 2013), and the larger literature suggests reduced pharmacological options for statin-treated patients with persistent high triglycerides (Wierzbicki et al., 2012), it is difficult to contextualize this null finding. Nonetheless, to provide some guidance, we note that the effects of treating triglycerides as secondary outcomes tend to be small (Carrizo et al., 2009; McKibbin et al., 2006; Wu et al., 2007), suggesting that although there was a non-significant trend in the "at risk" sample that showed possible clinical favor for PBHCI, this relatively small, quasi-experimental trial was unlikely to detect treatment effects on triglycerides (i.e., statistical tests for this outcome were likely underpowered).

Hypertension

Individuals with SMI are at risk for hypertension because of sedentary lifestyle, smoking, and complications from antipsychotic medication (Gierisich et al., 2013). Results of this study show that PBHCI was associated with reductions in DBP, with 63% of PBHCI consumers (relative to 52% of controls) who were "at risk" at baseline transitioning into the "normal" range at one-year follow-up.

How do the PBHCI effects observed here compare to other studies? Unfortunately, very few studies have directly tested the effects of hypertension treatment in adults with SMI. One potential comparison comes from a randomized controlled trial testing the effects of integrated hypertension and depression pharmacotherapy in a sample of older patients (Bogner and DeVries, 2008). Results of that study showed that consumers receiving integrated care had lower SBP (14 mm/Hg) and DBP (10 mm/Hg) than controls, suggesting a larger treatment effect on SBP and DBP than that observed for PBHCI (2 and 3 mm/Hg, respectively).

However, clinical and epidemiologic studies from the general population provide some information about the potential health impact of PBHCI-related reductions in BP observed. Specifically, large prospective intervention studies investigating hypertension-related morbidity and mortality show that reductions in resting SBP and DBP as small as 3 mm/Hg can reduce CHD risk by 5%, stroke by 8%, and all-cause mortality by 4% (reviewed in Cornelissen et al., 2011). For the general sample of PBHCI consumers who saw a mean DBP reduction of 4.38 mm/Hg, PBHCI is therefore likely to be associated with a 3%-4% reduction in the risk for CHD. And although there were no statistical differences in BP rates between PBHCI and control consumers with baseline "risk," BP reductions were even larger in the baseline "at risk" group (SBP=14 mm/Hg and DBP=12 mm/Hg).

In any case, future studies might continue to find ways to improve hypertension outcomes for adults with SMI, particularly focusing on systolic hypertension, which for most patients is more important to control than diastolic hypertension and is also more difficult to control (Grundy et al., 2004).

Diabetes

Adults with SMI are at increased risk for diabetes due to lifestyle factors (e.g., sedentary lifestyle and poor diet) and complications related to psychotropic medication use (reviewed in Gierisich et al., 2013). Results of this study showed only modest improvements in diabetes risk. In particular, the data showed no PBHCI-related improvement in A1c relative to controls. Further, while PBHCI consumers showed statistically significant reductions in FPG, this indicator remained unchanged for most participants during the study year, although PBHCI consumers in the general sample were less likely than controls to have FPG levels get worse. In any case, observed changes in FPG and A1c for the general and "at risk" samples were unlikely to be associated with any reduced risk for diabetes-related complications such as ischemic heart disease and stroke (Singh et al., 2013). These modest observed treatment effects on diabetes control are consistent with other studies of attempts to improve diabetes outcomes among adults with SMI. Specifically, a recent Agency for Healthcare Research and Quality (AHRQ) meta-analysis found seven total studies of behavioral, peer and family, or pharmacological interventions to improve glycemic control among adults with SMI (Gierisich et al., 2013). Among these, just two studies showed modest treatment advantages, both of which included the biguanide agent, metformin (Carrizo et al., 2009; Hoffman, Case, and Jacobson, 2012). In a similar review of metformin for prevention of weight gain in psychiatric populations (Newall et al., 2012), data showed that metformin primarily has effects on A1c (and not FPG), making the results of this study difficult to compare. We note that a significant challenge of this evaluation is that programs were able to report either FPG or A1c and, due to convenience, cost, or other factors, many programs changed indicators midway through the project; this led to low rates of complete (baseline and follow-up) data for both glucose and A1c, which limited the statistical power and stability of results reported for these indicators. A further challenge is that PBHCI programs may have attempted to control consumers' diabetes in several ways, including diabetes-specific medication, antipsychotic medication switching, and exercise and nutrition education, and since it is unclear whether programs (or providers within programs) employed evidence-based interventions for primary and secondary prevention of diabetes risk or which interventions were employed, whether or not they were included among the routine PBHCI screening activities, it is difficult to know the potential mechanisms producing change.

Obesity

As with diabetes, adults with SMI are at increased risk for obesity due to lifestyle factors (e.g., sedentary lifestyle and poor diet) and complications related to psychotropic medication use (Gierisich et al., 2013). The results of this study showed that PBHCI did not have a statistical or clinically meaningful impact on obesity as measured by BMI. This null outcome is somewhat disappointing when compared to the published literature on weight control for adults with SMI (reviewed in Fernandez-San-Martin et al., 2013; Gierisich et al., 2013). A recent AHRQ meta-analysis of more than 30 studies of weight control interventions for adults with SMI showed that the net effect of these studies is typically positive, albeit small (about 3 kg), even though treatment effects may be short-lived beyond the intervention period (e.g., not exceeding three months [Fernandez-San-Martin et al., 2013]).

Aspects of how PBHCI weight control programs were implemented may explain why there were no consistent PBHCI effects on weight. For instance, published studies tended to be specifically weight focused (as opposed to wellness-focused, more broadly), with interventions delivered by highly trained personnel implementing manualized interventions, often with high fidelity. Published studies often include exercise and diet interventions and, in some cases, additions or modifications to psychotropic medication that also affect weight (e.g., adding the anticonvulsant topiramate and zonisamide, or adding metformin) (Gierisich et al., 2013). Among the PBHCI sites selected for this evaluation, almost no consumers participated in exercise interventions (except at Site 1), and participation in related wellness services (e.g., nutrition classes) was also likely low, since wellness participation overall was limited (e.g., at one site, only 14% of consumers took part in any wellness service at all). In the future, PBHCI programs may better promote weight control by implementing weight control programs with greater rigor.

Cigarette Smoking

Several reasons likely contribute to the disproportionately high rates of cigarette smoking among adults with SMI, including enhanced dopamine reinforcement, metabolic effects of antipsychotic medication, and reduced opportunities for other rewards (e.g., CDC, 2012). The results of this study showed that consumers treated at control clinics were more likely to be smoking abstinent than those treated at PBHCI sites. While smoking cessation is notoriously difficult for adults with or without SMI, studies show that adults with SMI can quit when provided with intensive and appropriate treatment (Tsoi, Porwal, and Webster, 2013), therefore several study design and treatment factors that could explain this finding are worth considering. First, utilization data show that while smoking cessation services were widely used by consumers at some sites, they were virtually unused by consumers at another site. Specifically, Intervention Site 3 did not offer smoking cessation services while its paired control clinic did. However, negative PBHCI effects are unlikely to be entirely related to low rates of utilization, since outcomes were disappointing even at sites where most consumers used the service. More specifically, PBHCI programs might need to consider changes to improve the impact of their cessation interventions. For instance, recent meta-analyses show that, although adults with SMI can quit smoking, behavioral interventions and nicotine replacement are generally not effective, and that treatment with bupropion is more likely to help adults with SMI to quit (Tsoi, Porwal, and Webster, 2013). While the programs included in this evaluation offered nicotine replacement, none explicitly mentioned bupropion.

Fortunately, SAMHSA has already begun to improve the quality of its PBHCI smoking cessation programs. Through CIHS (the PBHCI Technical Assistance Center), PBHCI grantees were offered multi-session trainings on best practices for treatment, educating staff and increasing staff competence for asking about tobacco use and intervening appropriately, and implementing smoke-free policies. SAMHSA has also since improved its reporting requirements around tobacco use. All new cohorts are now required to collect breath CO to measure tobacco smoke exposure. Not only is breath CO an objective measure of tobacco smoke exposure, it can capture greater variability in smoking outcomes, such as reductions in smoking that may be occurring among adults who are preparing, but are not yet ready, to completely quit.

6. MODEL FEATURES EVALUATION

Overview

In this chapter, we describe the results of Research Question 3, Which models and/or model features of integrated primary and BH care lead to better mental and physical health outcomes? Early on, we learned that grantees implemented "bits and pieces" or combinations of integration models (e.g., Cherokee and Chronic Care) and few were implemented in whole or with direct evidence of fidelity (Scharf et al., 2013). As such, our approach to Research Question 3 focuses on model features whose presence or absence could be assessed with greater objectivity. Further, since there were too few sites in the comparative effectiveness study (n=3) to associate model features with consumer outcomes, we used a two-step process to create sequential links between structures, processes, and outcomes instead. In Step 1, we used programs' features (structures and procedures) to predict consumer access to integrated care (process). In Step 2, we tested the association between consumer access to integrated care (process) and change in physical health indicators (outcomes) relative to controls. This two-step approach was designed to suggest, albeit indirectly, links between PBHCI model features and objective improvements in consumer health.

Methods and Analyses

Our analytic approach to answering Research Question 3 is illustrated in Figure 6.1.

FIGURE 6.1. Integrated Care Program Features Predict Consumer Use of Integrated Services; then Consumer Service Use Predicts Consumer Outcomes
	Step 1	Step 2
Program Features	Consumer Outcomes	Process Indicators
Structures/Procedures Rural PC Partner Agency PC-BH Meetings/Mos PC Service Days/Week PC Clin Advice / Phone Integration Co-location Shared structures/systems;   Integrated practice; Culture Summary Score	Physical Health Indicators Diastolic Blood Pressure   Total Cholesterol LDL-Cholesterol Fasting Plasma Glucose	Integrated Care Access   Basic Comprehensive PC Provider Contacts

First, we used regression analysis to predict consumer access (in the general sample) to basic and comprehensive integrated care within one year of enrolling in PBHCI (Table 6.1; see also Chapter Four for a comprehensive explanation of these service packages) from select program structures for which we had reliable data from the web survey (see Chapter Three): rural (versus non-rural); the presence of a PC partner agency (versus hiring PC providers into the BH agency); the number of regularly scheduled PC-BH provider meetings per month; the number of days per week that PBHCI PC services are available; and whether PC clinical advice is provided by phone) plus a single summary score reflecting integration structural features and provider processes. As in Chapter Four, we defined basic integrated care as including: (1) a physical health screening or assessment or treatment planning session; (2) contact with a PC provider; and (3) contact with case management within the first 12 months of enrolling in PBHCI. We then defined comprehensive integrated care as consisting of: (1) a physical health screening or assessment; (2) a treatment planning session; (3) contact with a PC provider; (4) use of a wellness service; and (5) contact with case management within the first 12 months of enrolling in PBHCI. Consumers who received comprehensive integrated care necessarily received the basic package as well.

TABLE 6.1. Operational Definitions of Basic and Comprehensive Integrated Care Service Use
	General Sampleb
	Basic	Comprehensive
Physical Health Service
Screening/assessment	Xa	X
Treatment planning	Xa	X
PC Provider	X	X
Wellness		X
Case management	X	X
NOTES: PC = primary care. All indicated services must have been received within the first 12 months of enrollment in PBHCI. Either one of screening/assessment or treatment planning. General sample refers not just those individuals with physical health indicators in the "at risk" range (see Chapter Four).

To better understand the relationship between integrated care program features and service access, we also tested the relationship between integrated service access and the individual components comprising the integration summary score (co-location, shared structures and systems, integrated provider practice, clinic culture, total integration score) (Table 3.5). More specifically, the aim of this analysis was to determine whether clusters of program features reflecting integrated care would be better predictors of consumer access to integrated care than individual program structures or processes.

Then, in a second set of analyses limited to the comparative effectiveness study sample (see Chapter Five), we tested the relationship between PBHCI consumer PC visits and access to integrated care (basic and comprehensive) and the degree of change in PBHCI consumer physical health indicators relative to controls. Specifically, we used regression analysis to estimate the relationship between the number of PC contacts and consumer access to integrated services and estimates of change in physical health indicators derived from the case-mix--adjusted difference-in-difference analyses described in Chapter Five. Analyses were limited to only those physical health indicators that showed improvement among PBHCI consumers relative to controls. These were DBP, TC, LDL-C, and FPG.

Results

Step 1: Predicting Consumer Service Access from Program Features

The first step toward identifying PBHCI program features associated with consumer physical health outcomes was to test the association between program features (including integrated structures and provider processes) and consumer access to integrated care. Table 6.2 shows that consumers served at sites with PC available on more days per week and whose PC-BH staff attended more regularly scheduled, shared meetings were more likely to receive both basic and comprehensive integrated care within one year of enrolling in PBHCI. Results also showed that consumers served at rural sites were less likely to receive basic and comprehensive integrated care within their first year enrolled in PBHCI. However, the relationship between predictors and integrated care access was not always consistent. Specifically, physician-provided phone/email advice, the presence of a PC partner agency, and total integration score were all associated with lower consumer access to basic integrated care but greater consumer access to comprehensive integrated care, respectively.

TABLE 6.2. Step 1: Program Structures as Multivariate Predictors of Consumer Access of Basic and Comprehensive Integrated Carea
	Basic Integrationd		Comprehensive Integratione
	Point Estimate	95% CI	Point Estimate	95% CI
Total integration score	0.97c	0.96-0.98	1.04c	1.03-1.05
PC Advice by phone or email	0.63c	0.50-0.80	2.18c	1.65-2.88
PC/BH provider meetings/ month	1.16c	1.07-1.24	1.10c	1.03-1.18
PC partner agency	0.61c	0.47-0.79	3.38c	2.60-4.40
Rural	0.23c	0.18-0.28	0.11c	0.09-0.15
PC service days/week	1.72c	1.60-1.86	1.21c	1.13-1.30
NOTES: BH = behavioral health; CI = confidence interval; PBHCI = primary and behavioral health care integration; PC = primary care. We also ran this model without the total integration variable and the direction and relative magnitude of all predictors remained the same. Point estimate=odds ratio. Values <1 should be interpreted as a negative association. Within 12 months of enrolling in PBHCI. p<0.05. p<0.001. N=5,897 observations used; includes only consumers with 12 months of care; 3,337 received basic integrated care, 2,560 did not receive basic integrated care. N=5,897 observations used; includes only consumers with 12 months of care; 1,670 consumer received comprehensive integrated care; 4,227 consumers did not.

To better understand the specific integrated structures and provider processes associated with consumer access to integrated care, we tested the relationship between components of integrated programs and consumer access to integrated care in a single, multivariate model (Table 6.3). The direction of predictor-integration access relationships were the same for basic and comprehensive integrated care: Co-location, integrated practice, and culture were positively associated with access to integrated care during consumers' first year in PBHCI, while shared structures and systems were negatively associated with consumer access to integrated care.

TABLE 6.3. Program Integration Features as Multivariate Predictors of Consumer Access to Basic and Comprehensive Integrated Carea
	Basic Integrationd		Comprehensive Integratione
	Point Estimate	95% CI	Point Estimate	95% CI
Co-location	1.01c	1.01-1.01	1.01c	1.01-1.02
Shared structures/systems	0.98c	0.98-0.98	0.98c	0.98-0.98
Integrated practice	1.02c	1.02-1.03	1.02c	1.01-1.02
Culture	1.01c	1.01-1.01	1.02c	1.02-1.02
NOTES: CI = confidence interval; PBHCI = primary and behavioral health care integration. Univariate analyses produced effects in the same direction as in the multivariate model. Within 12 months of enrolling in PBHCI. p<0.05. p<0.001. N=5,897 observations used; includes only consumers with 12 months of care; 3,337 received basic integrated care, 2,560 did not receive basic integrated care. N=5,897 observations used; includes only consumers with 12 months of care; 1,670 consumers received comprehensive integrated care; 4,227 consumers did not.

Predicting Physical Health Outcomes from Consumer Processes

As a second step toward predicting consumer physical health outcomes from program features, we tested the association between consumer process measures (i.e., consumer access of integrated care and the number of contacts with a PC provider) and outcomes (change in physical health indicators relative to controls). Although we restricted our analysis to comparative effectiveness study indicators that showed significant improvement among PBHCI consumers compared to controls (see Chapter Five), the data showed no association between consumer access to integrated care or PC provider contacts and relative change in consumer physical health (Table 6.4). When we repeated these analyses in the subsample of consumers with physical health indicators in the "at risk" range at baseline, we similarly found no relationship between access to integrated care or PC provider contacts and change in consumer indicators of physical health.

TABLE 6.4. Step 2: Change in Physical Health Indicators as Predicted by Access to Integrated Care, Comparative Effectiveness Sample
	DBP	TC	LDL-C	FPG
N	281	171	211	192
Integration--Basic
Estimate (SE)	-0.61 (1.69)	-7.54 (6.38)	-5.47 (5.03)	0.11 (0.06)
Integration--Comprehensive
Estimate (SE)	-2.34 (1.60)	7.50 (6.72)	-0.61 (4.95)	0.04 (0.06)
PC Provider Contacts
Estimate (SE)	0.07 (0.11)	-0.16 (0.38)	-0.25 (0.31)	-0.00 (0.00)
NOTES: DBP = diastolic blood pressure; FPG = fasting plasma glucose; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; PC = primary care; TC = total cholesterol. Negative values favor PBHCI.

For comparison purposes, we repeated these analyses in the general sample of grantees from all participating sites (n=56) using registry data to define PC provider contacts and access to integrated care and TRAC data to derive consumer physical health outcomes (change from baseline to 12-month follow-up). Overall, results were similar, showing no reliable relationship between PC provider contacts or access to integrated care and consumer physical health outcomes in TRAC. Detailed results of these analyses are provided in the Appendix.

Summary

We conducted a two-step analysis designed to sequentially (albeit indirectly) link PBHCI program features to consumer outcomes. The first step of the analysis tested the relationship between program features and consumer access to basic and comprehensive integrated care. The second step tested the relationship between consumer access to basic and integrated care and comparative effectiveness study analyses of PBHCI consumer change in physical health outcomes relative to controls.

Overall, we are unable to draw conclusions about the relationship between program features and consumer outcomes because the results of the second step of the analyses were entirely null; that is, PBHCI consumer access to integrated care was unrelated to improvements in physical health as observed in the comparative effectiveness study.¹⁹ This finding is not entirely unexpected. For several analyses, both the sample and degree of PBHCI-related indicator change was small and may have resulted in underpowered statistical tests. Additionally, while we were able to describe consumer access to integrated care, the level of process data detail available for evaluation precluded rigorous tests of what was occurring during the provision of integrated care. For instance, we did not have access to consumer medications and/or the degree to which services comprising integrated care (e.g., wellness, care management, and referral) conformed to any particular standards (i.e., did consumers attend referral appointments?). So, while we know the broad categories of services that consumers received, we are unaware of the appropriateness and quality of those services, which could limit the impact of integration on consumer health. Further still, the possibility remains that some consumer-level characteristics could be obscuring the expected relationship between service access and outcomes. For example, consumers who accessed more services may have gotten better; however, consumers who were not getting better may have also accessed more services when their conditions were showing signs of being more difficult to treat. In short, there may be no simple, linear relationship between service access and change in outcomes for all PBHCI consumers, and more specific subgroup analyses may be needed to capture all of the ways that service use impacts outcomes.

In any case, tests of the relationship between program features and consumers' processes were well powered, and they provide new information about program features that appear to be associated with consumer access to integrated care. Data showing several negative associations between program features and access to basic integrated care were puzzling, challenging the validity of the findings to a certain extent. However, since the relationships between program features and comprehensive integrated care were as expected, as were the majority of those between program-level markers of integration and both basic and comprehensive integrated care, we do our best to summarize and synthesize what we found below.

PC Clinic Days and BH-PC Regularly Scheduled Meetings

Results showed that consumers receiving services at clinics offering more days of PC per week were more likely to access basic and comprehensive integrated care. This important finding is intuitive: Consumers have different and often changing schedules, and a variety of available appointments likely increases the chances that a consumer can schedule an appointment that is convenient and more likely to be attended. This finding can also help programs plan for and implement their PC services more effectively. For instance, instead of offering PC provider hours concentrated on a few days of the week (e.g., by having many PC staff work simultaneously), programs can create greater consumer access to integrated care by offering PC over several days of the week.

Data also showed that consumers served at programs with regularly scheduled joint BH-PC meetings were more likely to access basic and comprehensive integrated care. Indeed, our experience visiting PBHCI programs was that when integrated care teams met to discuss whole consumer care, the team became more aware of consumers' full range of needs and the potential array of services (across domains) available to them.

PC Partner Agency and Physician Advice by Phone/Email

Data also showed that consumers served by programs including a PC partner agency and whose physicians provided advice by phone/email had lower rates of access to basic integrated care but greater access to comprehensive care. While these findings may initially appear somewhat difficult to explain, one possibility is that accessing PC care at a program with a partner PC agency may be more difficult initially (e.g., it may entail different procedures and paperwork than consumers' BH agency), but once consumers have established entry into the system, they have improved access to a wider array of services. Future evaluations of PBHCI could include qualitative analysis of provider and consumer experiences related to making contact and engaging in services when PC is delivered by independent providers versus a partnering FQHC.

Rural Versus Urban Setting

The results showed that consumers served at rural sites (compared to urban or suburban sites) were unlikely to receive either basic or comprehensive integrated care. Several factors could account for this finding. Consumers served at rural locations may experience more transportation-related barriers to care (e.g., low or no bus service and the need to travel long distances to care), particularly if appointments are not scheduled within a single day, transportation barriers could uniquely impede consumer access to integrated care at rural sites. Similarly, in many rural areas, accessing providers may be difficult because there are too few physicians (particularly specialists) to serve all persons in need. A closer analysis of physician-provider ratios across PBHCI programs could shed light on factors contributing to this rural/non-rural program difference.

Integration

Unexpectedly, our integration summary score was differentially associated with access to basic and comprehensive care. Specifically, consumers receiving services from programs with greater integration total scores were less likely to have accessed basic integrated care but were more likely to have accessed comprehensive integrated services. To understand more about the relationship between program and provider features and consumer access to integrate care, we looked at the specific relationships between program components of integrated care and consumer access to services.

Co-location

Consumers who received services at programs with co-located primary and BH care were more likely to have accessed basic and comprehensive integrated care. This finding is intuitive and consistent with our site visit experiences, suggesting that when staff can facilitate warm hand-offs between new providers or consumers can have all care needs met at one location (e.g., having labs drawn and prescriptions filled) they are less likely to encounter barriers to service access such as transportation limitations or discomfort navigating new clinics and settings, and they are more likely to follow through with multiple aspects of needed care.

Shared Structures and Systems

Unexpectedly, data showed that consumer access to basic and comprehensive integrated care was lower at sites with integrated structures and systems (i.e., shared records, including electronic records; shared treatment plans, and regularly scheduled meetings). Indeed, SAMHSA and individual programs made considerable investments in integrated structures and systems specifically to improve consumer access of integrated services. What could explain this finding? One possibility is that integrated structures and systems may reduce face time between providers of different specialties (because less direct communication is needed when providers can read and review one another's notes in a chart) but that reduced face time also reduces communication about availability, as well as consumer need and use of varied services; thus reducing the likelihood that consumers will be referred to or follow-up with all their care needs. Another possibility is that providers have become more efficient and better able to improve consumer outcomes with fewer consumer-provider contacts. More research is needed on the specific types and functionality of shared structures and systems implemented by PBHCI programs, as well as how these specific integrated systems and structures affect provider practices and consumer outcomes in PBHCI clinic settings.

Integrated Practice

Consumers who received services at programs whose practices were well integrated (i.e., had high rates of PC-BH contacts, PC and BH contacts with care managers, and use of cross-specialty integrated records) were likely to receive both basic and integrated care. This finding was expected: Providers who work closely together, who check one another's work, and who ensure that consumers have help attending and implementing treatment recommendations (e.g., attending wellness classes) are likely to ensure that consumers receive an array of services. Future research could consider the degree to which each of these components of integrated practice (alone or in combination) may be particularly important for promoting ongoing consumer access to integrated services and what rates and types of contact create the greatest program impacts and efficiencies.

Culture

Finally, we found that consumers receiving services at programs in which staff perceived a greater culture of integrated care (e.g., collaboration by PC-BH leadership and PC-BH providers to reach goals) were more likely to access basic and comprehensive integrated care within one year of enrolling in PBHCI. Anecdotally, we heard from several web survey respondents that although some programs had many program structures and processes in place to facilitate integrated care, some providers often felt unwelcome or disrespected by other members of the team. It follows that only providers working at programs with a collaborative culture or "integrated care culture" would become motivated to learn about and help consumers follow through with treatment recommendations made by professionals identifying and treating problems outside of his or her own area of expertise.

7. SUMMARY, CONCLUSIONS AND IMPLICATIONS

Summary of Results

Research Question 1: Is it possible to integrate the services provided by PC providers and community-based BH agencies (i.e., what are the different structural and clinical approaches to integration being implemented)?

We addressed Research Question 1 by first examining the array of program structures and features implemented by PBHCI grantees (Chapter Three) and then by describing consumers, their care needs, and the fit between those needs and the PBHCI services used (Chapter Four).

PBHCI grantees demonstrated variation in the structure of their integrated care programs across several dimensions, including organizational partnerships, the physical location and structure of clinics, multidisciplinary staff mix, and staff training and expertise. We also found variation in implementation of these structural features (e.g., during site visits, we saw that PC and BH may or may not share reception and hallways) and that less tangible aspects of PBHCI programs (e.g., PC providers feeling unwelcome at BH sites) might also affect the provision of integrated care.

PBHCI programs also varied in how care was delivered to consumers in terms of PC-BH provider collaboration, screening, primary and preventive care, case management and coordination, processes for medication reconciliation and referrals to outside providers, wellness and self-management support services, information systems, approaches to performance monitoring, and plans and approaches to funding and sustainability. Variations and challenges associated with record keeping were widespread. Program staff using separate records struggled to gain access to necessary information, while those with new electronic systems often found integrated systems difficult to use. Importantly, systems for communication and coordination across programs were also quite varied, with many requiring improvements: Some programs did not include regularly scheduled PC-BH meetings, nor were staff aware of the scope of services available to their PBHCI clientele.

Despite variability in approaches to PBHCI, respondents reported many of the same barriers, such as staff and consumer recruitment and retention, financial barriers related to program sustainability for non-billable services (e.g., peer staff and wellness programs), and reliable transportation to appointments.

In any case, when we assessed programs on dimensions of structural and procedural integration, we found that integration varied widely across programs and also within sites, such that programs were often integrated along some dimensions but not others. Further investigation into the nuances of PBHCI integrated care may help future programs anticipate problems and implement more completely integrated services from the start.

In our examination of consumers, their care needs, and service use (Chapter Four), we also found considerable between grantee variation leading to significant variability in reach and appropriateness of services provided. Nonetheless, some overall trends emerged. For instance, PBHCI programs reported high rates of consumer enrollment in their first year of operation but largely fell short of long-term enrollment targets. However, consumers enrolled in PBHCI were demographically diverse, suggesting that PBHCI programs were able to engage a wide variety of clientele. These clients had high rates of health care needs. Approximately half of consumers reported low levels of well-being, nearly one-quarter reported using illegal substances, and (depending on the indicator) between one in eight (hypertension) and three in four (obesity) were at risk for chronic physical health conditions. Taken together, these high rates of mental health, substance use, and chronic PC needs affirm the need for intensive, integrated PC and BH services.

Service utilization data showed that PBHCI programs are making progress toward addressing consumers' multispecialty health care needs. Appropriately, the majority of consumers had contact with physical, BH, and care management providers, and they received physical and mental health assessments or screenings, medication management, and treatment planning. However, rates of substance abuse screening, medication management, and treatment planning were lower; in part, this may reflect lower rates of substance use service needs. Nonetheless, given the high rates of comorbid substance abuse and SMI (Kessler et al., 2005), PBHCI programs should ensure that they are reliably assessing/screening and effectively managing consumers' substance use-related needs.

Service utilization data also showed that although the majority of consumers received at least one contact with a variety of providers and services during their first year of care, a sizable proportion (22%) of enrollees discontinued PBHCI treatment, with the average length of enrollment prior to discontinuation being just seven months. Although problematic, difficulties engaging people with SMI in integrated care may reflect the relative newness of PBHCI programs. However, since programs did most of their consumer enrollment during the first year of operation, these data suggest that programs could continue to improve strategies for engaging consumers in care for the long term.

Similarly, access to and the intensity of PC provider contacts were not substantially different for consumers with probable chronic physical health conditions (who necessarily have a greater degree of PC needs) compared with those at lower risk (i.e., consumers not meeting any of the risk indicators assessed). These findings raise questions about the capacity of grantees to allocate resources to those consumers with the greatest physical health needs.

With regard to the quality of PC, overall, programs showed low rates of meeting evidence-based quality indicators for obesity and tobacco use--the two major preventable causes of morbidity and mortality in the United States (CDC, 2012; Ogden et al., 2012).

Finally, programs showed that some consumers can receive a rich array of behavioral and PC services: Nearly half of enrolled consumers receive at least some physical health and care management services within one year of enrolling in PBHCI. Given that the majority of these programs of integrated care are fairly new (less than four years old), PBHCI programs' coordination of system, providers, and services to manage complex, comorbid conditions is noteworthy. At the same time, PBHCI programs are not yet providing basic (about 50%) or comprehensive (about 75%) integrated services to many consumers in need, suggesting the need for programs to continue to develop and refine strategies for improving consumer access to and use of primary and secondary preventive care services.

In sum, grantees' implementation of PBHCI was highly variable, but there are sufficient challenges to warrant concerns over grantees' ability to achieve the desired outcomes of improving the physical health and overall well-being of their target consumer population.

Research Question 2: Does the integration of primary and BH care lead to improvements in the mental and physical health of the population with SMI and/or substance use disorders served by these integrated care models?

The results of this relatively small, comparative effectiveness study showed that compared with controls, consumers treated at PBHCI clinics had greater reductions in select indicators of risk for metabolic syndrome and several physical health conditions, including hypertension, dyslipidemia, diabetes, and cardiovascular disease. No similar benefit of PBHCI was observed for other indicators, including triglycerides, obesity, and smoking. Consistent with other studies of integrated care not directly targeting changes to BH service delivery (Druss et al., 2001), results also showed no reliable benefit of PBHCI on indicators of BH.

Overall, this mixed set of results was largely expected; rarely do dramatic changes to health care delivery, particularly in the early stages of implementation, result in consistent improvements in all health outcomes examined (e.g., Nutting et al., 2009). Similarly, since best practices for treating many physical health conditions (e.g., dyslipidemia and hypertension) in adults with SMI have not been established, mixed and modest outcomes were not surprising. However, best practices for general populations of adults are well established, and a major limitation of this evaluation is that we were unable to determine which consumers received key components of those interventions (e.g., medications and evidence-based wellness interventions) or whether or not null effects are a result of incomplete/inappropriate care or comparatively low treatment response among PBHCI consumers.

In any case, while the overall results of this early and small scale comparative effectiveness study were modest, PBHCI programs resulted in comparatively more consumer improvements in the physical health indicators than controls, and with further implementation support from SAMHSA and the Technical Assistance Center, improved quality improvement efforts, and other strategies to ensure rigorous implementation of the program, PBHCI programs may continue to improve their delivery of quality integrated care to adults with SMI.

Research Question 3: What model features are associated with outcomes?

We used a two-step approach to answer Research Question 3 that involved: (1) linking program features (e.g., structures and provider practices) with process measures (consumer access of integrated care); and (2) linking these same process measures with outcomes (comparative change in PBHCI consumer indicators of physical health). Ultimately, we were unable to answer Research Question 3 because Step 2 of the analysis did not show any associations between consumer contacts with PC providers or access of integrated care and physical health outcomes (possibly due in part to small sample size and smaller-than-expected PBHCI effects on physical health).

However, analysis for Step 1 (linking program features with consumer processes) suggested that several program structures, plus structural and procedural components of integrated care, were associated with consumer access of integrated services. For example, clinics that offer more days per week of PC, that have more regularly scheduled PC-BH meetings to discuss clinical cases, whose services were co-located, whose provider practices were integrated, and whose clinic culture was better integrated were more likely to provide consumers with access to basic and comprehensive integrated care. Consumers served at clinics in rural locations, however, were less likely to have access to integrated services. Taken together, these results suggest that programs may be able to monitor specific processes and then make concrete changes to several of these features to promote consumer access to integrated care. And while programs are unlikely to change their physical location, SAMHSA and the Technical Assistance Center might consider working closely with rural programs to clarify consumers' specific barriers to accessing integrated care and then taking steps to reduce those barriers.

Conclusions and Implications for Integrating Behavioral Health and Physical Health Services

The results of this multisite, multi-method evaluation suggest a number of ways that systems of integrated care for adults with SMI might be improved, as well as needs for future evaluation activities that could inform system improvements. Indeed, PBHCI is only one among many new of programs of integrated care, and many of the learnings from this evaluation have broader implications for the field.

Program-Level Implications

Programs may consider the following strategies for integrating BH and physical health services.

Conduct a Systematic Needs Assessment

In this study, we identified several barriers that programs implementing a program of integrated care might mitigate by conducting a thorough needs assessment prior to implementation. As such, new programs might consider conducting a needs assessment before undertaking an integrated care initiative. Such a needs assessment could include systematic efforts to understand the types and extent of consumer physical health care needs, preferences, attitudes, and beliefs about integrated PC and BH services (e.g., do they see value in preventive PC or do they prefer the emergency department because care is available 24 hours a day). In addition, the assessment could include an analysis of potential barriers to integrated care at multiple levels (Pincus et al., 2005) and an environmental scan identifying, for example, services already within an agency or elsewhere in the community intended to and/or already meeting consumer needs.²⁰ Detailed information about the number of consumers in need of care could also help programs appropriately budget for staff and resources with sufficient capacity to effectively meet consumers' needs. Needs information can also help programs to advocate for financial support.

Importantly, needs assessments may be more effective if they identify specific issues, such as the processes by which programs will identify individuals with health risks (e.g., registries and connected referral and care coordination processes) and connect them with the appropriate health services (e.g., connecting consumers with elevated breath CO to smoking cessation services). Needs assessments may also identify environmental program characteristics, such as integration culture, which have been shown in this study to be an important determinant of consumer access to integrated care.

Improve Program Performance Through Data-Driven Continuous Quality Improvement (CQI) Activities

PBHCI grantees were required to collect data that can support CQI activities. Indeed, programs can improve their performance if their goals and drivers of progress toward those goals are well specified, if they collect data on key performance indicators illustrating progress toward goals, and if they use these performance indicators to highlight areas of program performance that need to be improved. For instance, data from this evaluation showed that approximately 45% of PBHCI consumers were not receiving basic integrated services and that rates of integrated care access were similar for persons with and without identified physical health risk. The use of registry data in CQI processes can better ensure that PC and wellness services are targeted to consumers with identified physical health conditions and that consumers are routinely receiving the full array of available services.

Several existing models and frameworks describe how grantees can implement CQI, such as Plan Do Study Act cycles and RAND's Getting to Outcomes model, among others. Appropriately, the SAMHSA/HRSA Technical Assistance Center has several CQI resources available to grantees, and it routinely provides support to programs implementing CQI and using data to implement program reforms. CQI efforts may be strengthened by programs taking advantage of technical assistance that helps them specify which processes and tools can be used to support CQI, as well as provides lessons learned about how to implement and use them successfully.

Consider Expanding Use of or Adapting Evidence-Based Practices When Appropriate, and Assess Fidelity to Those Practices When They Are Used

EBPs exist for smoking cessation interventions and diet and weight management activities, and many PBHCI grantees proposed to use smoking and weight-related EBPs in their programs of care. Yet, PBHCI consumers failed to show improvements on these indicators compared with controls in the three sites included in our matched sample. Moving forward, integrated care programs could consider the following strategies to potentially improve impacts on smoking and weight, and other physical health outcomes.

1. Ensure that evidence-based care for physical health conditions is offered to all consumers on an ongoing basis, whenever such practices are available and appropriate for the population being served. While not all EBPs developed for the general population may be appropriate for adults with SMI, many may be adopted in-whole, or adapted to retain the central components of the intervention.

2. Consider ongoing assessment of the degree to which providers implement the EBP with fidelity to optimize treatment effects. An important component of this option is that services be delivered by staff with sufficient training and expertise to understand, implement, and explain to consumers, key components of an intervention. Although we were unable to monitor fidelity to many of these practices (many required medication information that was not available to the evaluation team), programs themselves likely have sufficient access to information to be able to monitor provider-level fidelity to EBPs.

3. In addition to implementing appropriate programs with fidelity, work to change the culture of the of the provider organization to support broader program goals like healthy lifestyles and wellness. This can set expectations for program staff and reinforce the services provided.

Invest in Strategies that Facilitate Consumer Access to Care

Our analyses showed that several program features were associated with increased consumer access to care. These included co-located PC and BH services, the number of days per week that PC services were available, and the availability of transportation. Since several programs identified low rates of consumer enrollment and engagement as barriers to successful integration, programs may consider co-locating PC and BH services, making PC available during more days, and providing transportation. Indeed, programs could conduct an analysis of the costs associated with providing transportation to select consumers compared to the additional PC billing that would be possible with anticipated reductions in appointment no-show rates. Alternatively, programs could consider offering expanded days/hours of care or offering telephone consultation or triage (e.g., with a nurse) when direct PC provider access is unavailable.

Provide Ongoing Education to Staff About Available PC and BH Services

Truly integrated care means that there is "no wrong door" into the care system and that consumers can learn about (or be referred to appropriate) services from anyone on their care team. Data from site visits and the web survey in particular suggested that there was much disagreement/misunderstanding among providers about what services were available within a particular grantee program. Particularly when services are not co-located or agencies partner to provide integrated care, programs could consider providing ongoing trainings to all persons involved in care so that they become intimately familiar with the array of PC, BH, and wellness services available; how information can and should be shared across provider types; what referral mechanisms are available; and eligibility and/or access information (e.g., whether copay is required) relevant to all types of potentially beneficial care. While we note that many programs rely on case managers to be the "keepers" of this information, data from this study showed that programs with higher rates of PC-BH provider meetings had increased consumer access to integrated care. In other words, direct provider-to-provider communication is likely important for consumer access to integrated care.

Build Partnerships with Other Community Organizations

Data from this study showed that having an FQHC partner increased consumer access to comprehensive integrated care. Indeed, FQHC staff has established procedures and networks of medical providers and resources that individual BH grantee providers may not have. However, barriers data from this evaluation suggest that such partnerships are more likely to be successful when they are implemented with clear expectations, mechanisms for data sharing and communication, data collection responsibilities, roles, and shared accountability for consumer care. Included in an integrated network of community partners could be hospitals (that can work with programs to coordinate the use of appropriate services and coordinate hospital and discharge care) and other social services organizations (e.g., housing and substance use treatment facilities) to ensure that clients do not drop out of services because some other basic need is unmet. Indeed, having flexibility or broadly defined case manager/care coordinator roles may enable staff with regular consumer contact to follow-up with consumers on these issues and make warm hand-offs to other agencies offering the appropriate services.

Hire Staff Who Are a Good Fit for an Integrated Care Environment

Data from the web survey showed that consumers served at clinics with integrated PC-BH cultures were more likely to access integrated services. We also heard anecdotally from administrators and providers that having "the right personality" for the job was key to providers' professional well-being and consumer engagement in care. Many programs reported that PC positions were very difficult to fill because few providers enjoyed the challenges (or recognized the benefits) of working with adults with SMI, and because other local agencies (e.g., hospitals and universities) could pay PC providers more for similar work; at the same time, we also heard that agencies that waited to hire staff with experience serving adults with SMI, homeless populations, or other related groups (even if key positions went temporarily unfilled) had greater success, including lower staff turnover. Programs may reduce the risk of having unfilled positions if they partner with local educational institutions by offering internships and other placements to ensure that there are local recent graduates who have ties to the program and the necessary skills to provide consumers with high quality integrated care.

Implications for the Field

The PBHCI program and evaluation suggests some additional implications for the broader field of integrated care.

Build Consensus Around Performance Expectations

PBHCI programs varied widely in the types and numbers of services they provided, the numbers of consumers served, and their success in engaging consumers in comprehensive care that addressed their physical health and wellness needs. Although programs proposed to use evidence-based wellness interventions, our evaluation suggested that these services were often implemented loosely, such that the core components of the intervention were not reliably implemented nor were clinic cultural changes made to support consumer behavior change consistently in place.

One strategy that could help lead to program improvements in wellness and other services would be the availability of clearer integrated care program performance expectations, tools for monitoring performance, and use of incentives that encourage programs to meet performance expectations. Providers and stakeholder organizations could work together to develop more explicit guidelines and performance expectations for integrated care that might include the following: guidelines for adoption of recognized best practices for PC and wellness services; minimum standards for provider caseloads, service availability, and provider qualifications.

Develop National Quality Indicators for the Accountability of Integrated Care

The field of integrated care is challenged to gauge the overall success of the PBHCI program because there are few similar studies that include readily comparable results. Given that there are several other initiatives currently underway that include the provision of integrated care to adults with SMI, stakeholders in the field may benefit when these programs use a standardized, national set of quality indicators to monitor and maintain the quality of integrated care. Importantly, such data may be useful for comparing and contrasting existing models and approaches to integration. Wider use of existing quality indicators, when they are available from organizations like NCQA, and continued work to support the development of common quality indicators would advance broader efforts to integrate care for people with SMI.

Establish Core Performance-Monitoring Requirements

A system of performance monitoring that is standardized across integrated care programs is a necessary step toward assessing program success in meeting minimal performance expectations and implementing national quality indicators for accountability in integrated care. In addition, performance monitoring is an essential component of program-level capacity to engage in quality improvement.

We offer several suggestions for stakeholders in the field tasked with integrated care performance-monitoring:

1. Monitor program implementation and model features, including specific implementation of screening and assessment protocols, wellness programs, and care management. Importantly, programs should demonstrate the existence and functionality of their client registries that include tracking of health assessments and treatment plan follow-ups.

2. Monitor provision of services at the consumer level, including providers seen, services utilized, medications prescribed, and prescriptions filled. This information could be used by programs to assess the appropriateness of services for clients with varying health needs. Consumer-level utilization data would also support reporting of national quality indicators.

3. Work with others to establish standardized web-based clinical registries for integrated care. Advantages to having a standardized clinical registry tool for consumer-level data include standardized data for quality reporting, and an invaluable data resource that could be used to support the development of best practices for the treatment of physical health conditions among adults with SMI. A challenge of this recommendation is that programs may be required to use other EHRs to interact with other systems (e.g., a county-wide registry), and thus field-wide registry could require double-entering data. Most new systems, however, can be designed to extract electronic data to auto-populate another on-line system and so this challenge in many cases could be overcome.

Ensuring that programs have access to and utilize a standardized clinical registry pre-populated with fields for outcome analysis is central to improving outcomes. The experience of our evaluation team is that data that are not prioritized for monitoring are unlikely to drive quality improvements.

Technical Assistance Needs for Programs Implementing Integrated Care for Adults with SMI ²¹

Our study suggests that new and continuing programs of integrated care may benefit from technical assistance in several areas, such as those listed below.

1. Emerging best practices for adults with SMI. A limited number of best practices exist for the treatment of physical health conditions among adults with SMI. As such, programs may benefit from technical assistance efforts to monitor the development of and widely disseminate best practices to PBHCI and other integrated care programs as soon as they are available. This could be done through informational webinars and other materials, as well as direct technical assistance to programs.

2. Navigation of multiple health care reforms. Programs may be operating in states that are also implementing other integration-related health care reforms. For instance, many PBHCI programs in Ohio and New York State (among at least four others) reported that they were struggling to implement PBHCI while also meeting implementation and performance criteria to become Medicaid Health Homes. Technical assistance may identify any overlap between integrated care program implementation and other ongoing health care reforms to maximize program implementation efficiencies while maintaining programs' high quality care. These reforms may also support program sustainability so bear consideration during sustainability planning.

Future Evaluation Activities

The results of this evaluation suggest several additional evaluation activities that could benefit the larger field of integrated care.

1. Evaluate integrated care service utilization patterns and their costs. The results of this evaluation showed that PBHCI programs were often challenged to find ways to become sustainable beyond the grant period; other grant-funded integrated care programs may experience this challenge, as well. Funders may be more likely to continue to invest in integrated care programs if they can show that: (a) shifts service utilization in a positive way; (b) integrated care is cost-effective or cost-neutral; and (c) integrated care provides funders with a return on their investment. A comprehensive analysis of integrated care service utilization patterns and their associated costs could give funders and individual programs important information to attract further investment in their programs despite persistent and widespread issues with parity and payment.

2. Evaluate past, current, and future strategies for sustaining integrated care. Prior health service innovations show that clinical innovation typically precedes financial innovation (e.g., Nutting et al., 2009; Felland et al., 2013). Integrated care programs may benefit from engaging others within their localities and states to identify and develop resources to sustain integrated care, but information about the most promising approaches to sustainability is not widely available and may not be accessible. Research that describes historical, ongoing, and proposed payment models that show promise (or significant limitations) for improving integrated care payment reform may help programs advocate for resources to sustain their integrated services.

3. Conduct a prospective trial of alternative models of integrated care. There are currently several, large ongoing trials of health care innovations including integrated care models. As such, stakeholders in the field could conduct a prospective, comparative effectiveness trial to assess the comparative clinical impact and costs associated with these models when serving similar and/or overlapping populations of adults with SMI.

Conclusions

Over the course of this three-year evaluation, RAND assessed early PBHCI program processes, outcomes, and PBHCI model features associated with implementation and outcome success. Overall, PBHCI programs had several successes, such as building integrated, multidisciplinary teams that offered an array of integrated PC, BH, and wellness services; these services were then provided to a diverse clientele with high rates of need for integrated care. PBHCI programs also experienced several challenges, including lower-than-expected rates of consumer enrollment, sustainability challenges, intra-team communication, and creating an integrated clinic culture. Programs also experienced challenges related to implementing wellness programs and improving outcomes, particularly with regard to smoking and weight. Current and future grantee cohorts can consider improving their programs via comprehensive, data-driven, continuous quality improvement; monitoring implementation fidelity to evidence-based wellness programs; and investing in strategies that improve access to integrated services, among others. In the future, integrated care programs could be improved by having clearer performance expectations, national quality indicators for accountability in integrated care programs, and core performance monitoring requirements. Finally, technical assistance needs include continuing the dissemination of emerging best care practices for adults with SMI and supporting grantees navigating concurrent health care reforms. Future evaluations, including an assessment of integrated care program service utilization patterns and costs, strategies to improve sustainability, and a prospective trial of alternative models of integrated care could help stakeholders demonstrate the value of integrated care.

REFERENCES

Aarons, G.A., A.E. Green, L.A. Palinkas, S. Self-Brown, D.J. Whitaker, J.R. Lutzker, J.F. Silovsky, D.B. Hecht, and M.J. Chaffin, "Dynamic Adaptation Process to Implement an Evidence-Based Child Maltreatment Intervention," Implementation Science, Vol. 7, 2012, pp. 1-9.

Abadie, A., "Semiparametric Difference-in-Differences Estimators," Review of Economic Studies, Vol. 72, No. 250, 2005, pp. 1-19.

Agency for Healthcare Research and Quality, Care Coordination Measures Atlas, Washington, DC: U.S. Department of Health and Human Services, 2010. As of September 3, 2013: http://www.ahrq.gov/professionals/systems/long-term-care/resources/coordination/atlas/care-coordination-measures-atlas.pdf.

Alakeson, V., R.G. Frank, and R.E. Katz, "Specialty Care Medical Homes for People with Severe, Persistent Mental Disorders," Health Affairs (Millwood), Vol. 29, No. 5, May 2010, pp. 867-873.

American Academy of Family Physicians, AAoPA, American College of Physicians, American Osteopathic Association, Joint Principles of the Patient-Centered Medical Home, 2007.

Ashenfelier, O., and D. Card, "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," Review of Economics and Statistics, Vol. 67, No. 4, 1985, pp. 648-660.

Baker, J.R., A.C. Vandal, J. Yeoh, I. Zeng, S. Wong, and S.N. Ryan, "Clinical Trial Participation Improves Outcome: A Matched Historical Cohort Study," Clinical Trials, 2013.

Bao, Y., L.P. Casalino, and H.A. Pincus, "Behavioral Health and Health Care Reform Models: Patient-Centered Medical Home, Health Home, and Accountable Care Organization," Journal of Behavioral Health Services and Research, Vol. 40, No. 1, 2013, pp. 121-132.

Baranowski, T., and G. Stables, "Process Evaluations of the 5-a-Day Projects," Health Education and Behavior, Vol. 27, No. 2, 2000, pp. 157-166.

Bartels, S.J., S.I. Pratt, K.A. Aschbrenner, L.K. Barre, K. Jue, R.S. Wolfe, H. Xie, G. McHugo, M. Santos, G.E. Williams, J.A. Naslund, and K.T. Mueser, "Clinically Significant Improved Fitness and Weight Loss Among Overweight Persons with Serious Mental Illness," Psychiatric Services, Vol. 64, No. 8., 2013, pp. 729-736.

Blount, A., "Integrated Primary Care: Organizing the Evidence," Families, Systems, and Health, Vol. 21, No. 12, 2003, pp. 121-133.

Boardman, J.B., "Health Access and Integration for Adults with Serious and Persistent Mental Illness," Families, Systems, and Health, Vol. 24, No. 1, 2006, pp. 3-18.

Bodenheimer, T., K. Lorig, H. Holman, and K. Grumbach, "Patient Self-Management Of Chronic Disease In Primary Care," JAMA, Vol. 288, No. 19, November 20, 2002, pp. 2469-2475.

Bogner, H.R., and H.F. de Vries, "An Intervention to Improve Adherence Through Integration of Depression and Hypertension Treatment: A Pilot Randomized Controlled Trial,"Annals of Family Medicine, Vol. 6, No. 4, 2008, pp. 295-301.

Bouffard, J.A., F.S. Taxman, and R. Silverman, "Improving Process Evaluations of Correctional Programs by Using a Comprehensive Evaluation Methodology," Evaluation and Program Planning, Vol. 26, No. 2, 2003, pp. 149-161.

Burnam, M.A., and K.E. Watkins, "Substance Abuse with Mental Disorders: Specialized Public Systems and Integrated Care," Health Affairs, Vol. 25, No. 3, 2006, pp. 648-658.

Butler, M., R.L. Kane, D. McAlpin, R.G. Kathol, S.S. Fu, H. Hagedorn, and T.J. Wilt, Integration of Mental Health/Substance Abuse and Primary Care, Rockville, MD: Agency for Healthcare Research and Quality, Report Number 09-E003, 2008.

Carrizo E., V. Fernández, L. Connell, I. Sandia, D. Prieto, J. Mogollón, D. Valbuena, I. Fernández, E.A. de Baptista, and T. Baptista, "Extended Release Metformin for Metabolic Control Assistance During Prolonged Clozapine Administration: A 14 Week, Double-Blind, Parallel Group, Placebo-Controlled Study," Schizophrenia Research, Vol. 113, No. 1, 2009, pp. 19-26.

Carroll, C., M. Patterson, S. Wood, A. Booth, J. Rick, and S. Balain, "A Conceptual Framework for Implementation Fidelity," Implementation Science, Vol. 2, 2007, pp. 40-49.

CDC--See Centers for Disease Control and Prevention.

Centers for Disease Control and Prevention, National Diabetes Fact Sheet, 2011, Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2011a. As of September 2013: http://www.cdc.gov/diabetes/pubs/pdf/ndfs_2011.pdf.

_____, "Vital Signs: Prevalence, Treatment, and Control of High Levels of Low-Density Lipoprotein Cholesterol--United States, 1999-2002 and 2005-2008," Morbidity and Mortality Weekly Report, Vol. 60, No. 4, 2011b, pp. 109-14.

_____, "Current Cigarette Smoking Among Adults--United States, 2011," Morbidity and Mortality Weekly Report, Vol. 61, No. 44, 2012, pp. 889-894.

_____, "Hypertension Morbidity, 2009-2010," FastStats, May 2013. As of September 2013: http://www.cdc.gov/nchs/fastats/hyprtens.htm.

Center for Integrated Health Solutions, A Standard Framework for Levels of Integrated Healthcare, April 2013.

Chen, K., J.B. Lindsey, A. Khera, J.A. De Lemos, C.R. Ayers, A. Goyal, G.L. Vega, S.A. Murphy, S.M. Grundy, and D.K. McGuire, "Independent Associations Between Metabolic Syndrome, Diabetes Mellitus and Atherosclerosis: Observations from the Dallas Heart Study," Peripheral Vascular Disease, Vol. 5, No. 2, 2008, pp. 96-101.

CIHS--See Substance Abuse and Mental Health Services Administration, HRSA Center for Integrated Health Solutions.

Collins, C., D.L. Hewson, R. Munger, and T. Wade, Evolving Models of Behavioral Health Integration in Primary Care, New York, NY: Milbank Memorial Fund, 2010.

Cornelissen, V.A., R.H Fagard, E. Coeckelberghs, and L. Vanhees, "Impact of Resistance Training on Blood Pressure and Other Cardiovascular Risk Factors," Hypertension, Vol. 58, 2011, pp. 950-958.

Croghan, T., and J. Brown, Integrating Mental Health Treatment into the Patient-Centered Medical Home, Patient Centered Primary Care Collaborative, 2010.

Danaei, G., E.L. Ding, D. Mozaffarian, B. Taylor, J. Rehm, C.J. Murray, and M. Ezzati, "The Preventable Causes of Death in the United States: Comparative Risk Assessment of Dietary, Lifestyle, and Metabolic Risk Factors," PLoS Medicine, Vol. 6, No. 4, 2009.

Daumit, G.L., A.T. Dalcin, G.J. Jerome, D.R. Young, J. Charleston, R.M. Crum, C. Anthony, J.H. Hayes, P.B. McCarron, E. Khaykin, and L.J. Appel, "A Behavioral Weight-Loss Intervention for Persons with Serious Mental Illness in Psychiatric Rehabilitation Centers," International Journal of Obesity, Vol. 35, No. 8, 2011, pp. 1114-1123.

Daumit, G.L., F.B. Dickerson, N.Y. Wang, A. Dalcin, G.J. Jerome, C.A. Anderson, D.R. Young, K.D. Frick, A. Yu, J.V. Gennusa III, M. Oefinger, R.M. Crum, J. Charleston, S.S. Casagrande, E. Guallar, R.W. Goldberg, L.M. Campbell, and L.J. Appel, "A Behavioral Weight-Loss Intervention in Persons with Serious Mental Illness," New England Journal of Medicine, Vol. 368, No. 17, 2013, pp. 1594-1602.

Dickey, B., S.-L.T. Normand, R.D. Weiss, R.E. Drake, and H. Azeni, "Medical Morbidity, Mental Illness, and Substance Use Disorders,"Psychiatric Services, Vol. 53, No. 7, 2002, pp. 861-867.

Dixon, L.B., J.A. Kreyenbuhl, F.B. Dickerson, T.W. Donner, C.H. Brown, K. Wolheiter, L. Postrada, R.W. Goldberg, L. Fang, C. Marano, and E. Messias, "A Comparison of Type 2 Diabetes Outcomes Among Persons with and Without Severe Mental Illnesses," Psychiatric Services, Vol. 55, No. 8, 2004, pp. 892-900.

Dixon, L.B., L. Postrado, J. Delahanty, P.J. Fischer, and A. Lehman, "The Association of Medical Comorbidity in Schizophrenia with Poor Physical and Mental Health," Journal of Nervous and Mental Disease, Vol. 187, No. 8, 1999, pp. 496-502.

Doherty, W.J., "The Why's and Levels of Collaborative Family Health Care," Family Systems Medicine, Vol. 13, No. 3-4, 1995, pp. 275-281.

Donabedian, A., "Evaluating the Quality of Medical Care," Milbank Memorial Fund Quarterly, Vol. 44 (Suppl), 1966, pp. 166-206.

Donabedian, A., "Methods for Deriving Criteria for Assessing the Quality of Medical Care," Medical Care Review, Vol. 37, No. 7, 1980, pp. 653-698.

Druss, B.G., "Improving Medical Care for Persons with Serious Mental Illness: Challenges and Solutions," Journal of Clinical Psychiatry, Vol. 68, No. 4 (Suppl), 2007, pp. 40-44.

Druss, B.G., W.D. Bradford, R.A. Rosenheck, M.J. Radford, and H.M. Krumholz, "Mental Disorders and Use of Cardiovascular Procedures After Myocardial Infarction," Journal of the American Medical Association, Vol. 283, No. 4, 2000, pp. 506-611.

Druss, B.G., W.D. Bradford, R.A. Rosenheck, M.J. Radford, and H.M. Krumholz, "Quality of Medical Care and Excess Mortality in Older Patients with Mental Disorders," Archives of General Psychiatry, Vol. 58, No. 6, 2001, pp. 565-572.

Druss B.G., S.A. von Esenwein, "Improving General Medical Care for Persons with Mental and Addictive Disorders: Systematic Review," General Hospital Psychiatry, Vol. 28, 2006, pp. 145-153.

Druss, B.G., S.A. von Esenwein, M.T. Compton, K.J. Rask, L. Zhao, and R.M. Parker, "A Randomized Trial of Medical Care Management for Community Mental Health Settings: The Primary Care Access, Referral, and Evaluation (PCARE) Study," American Journal of Psychiatry, Vol. 167, No. 2, 2010, pp. 151-159.

Druss, B.G., and B.J. Mauer, "Health Care Reform and Care at the Behavioral Health-Primary Care Interface," Psychiatric Services, Vol. 61, No. 11, November 2010, pp. 1087-1092.

Druss, B.G., R.M. Rohrbaugh, C.M. Levinson, and R.A. Rosenheck, "Integrated Medical Care for Patients with Serious Psychiatric Illness: A Randomized Trial," Archives of General Psychiatry, Vol. 58, No. 9, 2001, pp. 861-868.

Ervin, R.B., "Prevalence of Metabolic Syndrome Among Adults 20 Years of Age and Over, by Sex, Age, Race and Ethnicity, and Body Mass Index: United States, 2003-2006," National Health Statistics Reports, No. 13, May 5, 2009. As of September 2013: http://www.cdc.gov/nchs/data/nhsr/nhsr013.pdf.

Felland, L.E., A.E. Lechner, and A. Sommers, "Improving Access to Specialty Care for Medicaid Patients: Policy Issues and Options," Commonwealth Fund, June 2013.

Fernández-San-Martín, M.I., L.M. Martín-López, R. Masa-Font, N. Olona-Tabueña, Y. Roman, J. Martin-Royo, S. Oller-Canet, S. González-Tejón, L. San-Emeterio, A. Barroso-Garcia, L. Viñas-Cabrera, and G. Flores-Mateo, "The Effectiveness of Lifestyle Interventions to Reduce Cardiovascular Risk in Patients with Severe Mental Disorders: Meta-Analysis of Intervention Studies," Community Mental Health Journal, June 6, 2013.

Fiore, M.C., C.R. Jaen, T.B. Baker et al., "A Clinical Practice Guideline Treating Tobacco Use and Dependence: 2008 Update: A U.S. Public Health Service Report," American Journal of Preventive Medicine, Vol. 35, No. 2, 2008, pp. 158-176.

Gierisch, J.M., J.A. Nieuwsma, D.W. Bradford, C.M. Wilder, M.C. Mann-Wrobel, A.J. McBroom, L. Wing, M.D. Musty, M.M. Chobot, V. Hasselblad, and J.W. Williams Jr., Interventions to Improve Cardiovascular Risk Factors in People with Serious Mental Illness, Rockville, MD: Agency for Healthcare Research and Quality, Report No.: 13-EHC063-EF, April 2013.

Golomb, B.A., J.M. Pyne, B. Wright, B. Jaworski, J.B. Lohr, and S.A. Bozzette, "The Role of Psychiatrists in Primary Care of Patients with Severe Mental Illness," Psychiatric Services, Vol. 51, No. 6, 2000, pp. 766-773.

Grundy, S.M., H.B. Brewer Jr., J.L. Cleeman, S.C. Smith Jr., and C. Lefant, "Definition of Metabolic Syndrome: Report of the National Heart, Lung, and Blood Institute/American Heart Association Conference on Scientific Issues Related to Definition," Circulation, Vol. 109, No. 3, 2004, pp. 433-438.

Helitzer, D., S.-J. Yoon, N. Wallerstein, and L. Dow y Garcia-Verlarde, "The Role of Process Evaluation in the Training of Facilitators for an Adolescent Health Education Program," Journal of School Health, Vol. 70, No. 4, 2000, pp. 141-147.

Hoffmann, V.P., M. Case, and J.G. Jacobson, "Assessment of Treatment Algorithms Including Amantadine, Metformin, and Zonisamide for the Prevention of Weight Gain with Olanzapine: A Randomized Controlled Open-Label Study," Journal of Clinical Psychiatry, Vol. 73, No. 2, 2012, pp. 216-223.

Horvitz-Lennon, M., A.M. Kilbourne, and H.A. Pincus, "From Silos to Bridges: Meeting the General Health Care Needs of Adults with Serious Mental Illness," Health Affairs, Vol. 25, No. 3, 2006, pp. 659-669.

Hulscher, M.E.J.L., M.G.H. Laurant, and R.P.T. Grol, "Process Evaluation on Quality Improvement Interventions," Quality and Safety in Health Care, Vol. 12, 2003, pp. 40-46.

Imbens, G.W., and J.M. Wooldridge, "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, Vol. 47, No. 1, 2009, pp. 5-86.

Improving Chronic Illness Care, ACIC Survey, 2013. As of September 3, 2013: http://www.improvingchroniccare.org/index.php?p=ACIC_Survey&s=35.

Institute of Medicine, Improving the Quality of Health Care for Mental and Substance-Use Conditions, Washington, DC: National Academies Press, 2006.

IOM--See Institute of Medicine.

Jellinger, P., D. Smith, A. Mehta, O. Ganda, Y. Handelsman, H. Rodbard, M. Shepherd, and J. Seibel, The AACE Task Force for Management of Dyslipidemia and Prevention of Atherosclerosis, "American Association of Clinical Endocrinologists' Guidelines for Management of Dyslipidemia and Prevention of Atherosclerosis," Endocrine Practice, Vol. 18, Suppl. 1, March/April 2012.

Joint Commission, Facts About the Joint Commission's Behavioral Health Home Certification, 2013. As of August 30, 2013: http://www.jointcommission.org/assets/1/18/Facts_BHH_certification.pdf.

Katon, W.J, and J. Unützer, "Health Reform and the Affordable Care Act: The Importance of Mental Health Treatment to Achieving the Triple Aim," Journal of Psychosomatic Research, Vol. 74, No. 6, June 2013.

Kelly, D.L., D.L. Boggs, and R.R. Conley, "Reaching for Wellness in Schizophrenia," Psychiatric Clinics of North America, Vol. 30, No. 3, 2007, pp. 453-479.

Kessler, R.C., W.T. Chiu, O. Demler, and E.E. Walters, "Prevalence, Severity and Comorbidity of 12-month DSM-IV Disorders in the National Comorbidity Survey Replication," Archives of General Psychiatry, Vol. 62, No. 6, 2005, pp. 617-627.

Koran, L.M., H.C. Sox Jr., K.I. Marton, S. Moltzen, C.H. Sox, H.C. Kraemer, K. Imai, T.G. Kelsey, T.G. Rose, L.C. Levin, and S. Chandra, "Medical Evaluation of Psychiatric Patients: I. Results in a State Mental Health System," Archives of General Psychiatry, Vol. 46, No. 8, 1989, pp. 733-740.

Koyanagi, C., Get It Together: How to Integrate Physical and Mental Health Care for People with Serious Mental Disorders, Washington, DC: Bazelon Center for Mental Health Law, 2004. As of December 18, 2013: http://www.bazelon.org/issues/mentalhealth/publications/getittogether/.

Kruschitz, R., S.J. Wallner-Liebmann, M.J. Mahlin, M. Moser, B. Ludvik, W.J. Schnedl, and E. Tafeit, "Detecting Body Fat--A Weighty Problem BMI versus Subcutaneous Fat Patterns in Athletes and Non-Athletes," PLoS One, Vol. 8, No. 8, 2013.

Laursen, T.M., K. Wahlbeck, J. Hällgren, J. Westman, U. Ösby, H. Alinaghizadeh, M. Gissler, and M. Nordentoft, "Life Expectancy and Death by Diseases of the Circulatory System in Patients with Bipolar Disorder or Schizophrenia in the Nordic Countries," PLoS One, Vol. 8, No. 6, 2013, pp. 1-7.

Lawrence, D., and S. Kisely, "Inequalities in Healthcare Provision for People with Severe Mental Illness," Journal of Psychopharmacology, Vol. 24, No. 11, Supplement 4, 2010, pp. 61-68.

Lin, E.H.B., W. Katon, M. Von Korff, C. Rutter, G.E. Simon, M. Oliver, P. Ciechanowski, E.J. Ludman, T. Bush, and B. Yound, "Relationship of Depression and Diabetes Self-Care, Medication Adherence, and Preventive Care," Diabetes Care, Vol. 27, No. 9, 2004, pp. 2154-2160.

McGuire, J., L. Gelberg, J. Blue-Howells, and R.A. Rosenheck, "Access to Primary Care for Homeless Veterans with Serious Mental Illness or Substance Abuse: A Follow-Up Evaluation of Co-Located Primary Care and Homeless Social Services," Administration and Policy in Mental Health, Vol. 36, No. 4, 2009, pp. 255-264.

Mauer, B., Behavioral Health/Primary Care Integration: The Four Quadrant Model and Evidence-Based Practices, National Council for Community Behavioral Healthcare, 2006.

Marder, S.R., S.M. Essock, A.L. Miller, R.W. Buchanan, D.E. Casey, J.M. Davis, J.M. Kane, J.A. Lieberman, N.R. Schooler, N. Covell, S. Stroup, E.M. Weissman, D.A. Wirshing, C.S. Hall, L. Pogach, X. Pi-Sunyer, T. Bigger, A. Friedman, D. Kleinberg, S.J. Yevich, B. David, and S. Shon, "Physical Health Monitoring of Patients with Schizophrenia," American Journal of Psychiatry, Vol. 161, No. 8, 2004, pp. 1334-1349.

McEvoy, J.P., J.M. Meyer, D.C. Goff, H.A. Nasrallah, S.M. Davis, L. Sullivan, H.Y. Meltzer, J. Hsiao, T.S. Stroup, and J.A. Lieberman, "Prevalence of the Metabolic Syndrome in Patients with Schizophrenia: Baseline Results from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Schizophrenia Trial and Comparison with National Estimates from NHANES III," Schizophrenia Research, Vol. 80, No. 1, 2005, pp. 19-32.

McKibbin, C.L., T.L. Patterson, G. Norman, et al., "A Lifestyle Intervention for Older Schizophrenia Patients with Diabetes Mellitus: A Randomized Controlled Trial," Schizophrenia Research, Vol. 86, No. 1-3, 2006, pp. 36-44.

Mechanic, D., and S. Bilder, "Treatment of People with Mental Illness: A Decade-Long Perspective," Health Affairs, Vol. 23, No. 4, 2004, pp. 84-95.

Meyer, B., Natural and Quasi-Natural Experiments in Economics," Journal of Business and Economic Statistics, Vol. 13, 1995, pp. 151-162.

Monroe-DeVita, M., G.B. Teague, and L.L. Moser, "The TMACT: A New Tool for Measuring Fidelity to Assertive Community Treatment," Journal of the American Psychiatric Nurses Association, Vol. 17, No. 1, 2011, pp. 17-29.

National Academy for State Health Policy, Medical Home and Patient-Centered Care, 2013. As of August 30, 2013: http://www.nashp.org/med-home-map.

National Alliance to End Homelessness, State of Homelessness in America 2012, Washington, DC: Homeless Research Institute, 2012. As of September 3, 2013: http://www.endhomelessness.org/library/entry/the-state-of-homelessness-in-america-2012.

National Committee for Quality Assurance, Comparison: PPC-PCMH 2008 with PCMH 2011, 2011a. As of December 18, 2013: http://www.ncqa.org/portals/0/Programs/Recognition/PPC-PCMH%202008%20vs%20PCMH%202011Crosswalk%20FINAL.pdf.

_____, Standards and Guidelines for NCQA's Patient-Centered Medical Home (PCMH), March 2011b. As of April 4, 2012: http://www.chcact.org/images/customer-files//Appendix3_PCMH2011glossary.pdf.

National Heart, Lung, and Blood Institute, "Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) Final Report," Circulation, Vol. 106, No. 25, 2002, pp. 3143-3421.

NCQA--See National Committee for Quality Assurance.

Newall, H., N. Myles, P.B. Ward, K. Samaras, D. Shiers, and J. Curtis, "Efficacy of Metformin for Prevention of Weight Gain in Psychiatric Populations: A Review," International Clinical Psychopharmacology, Vol. 27, No. 2, 2012, pp. 69-75.

Nutting, P.A., W.L. Miller, B.F. Crabtree, C.R. Jaen, E.E. Stewart, and K.C. Stange, "Initial Lessons from the First National Demonstration Project on Pratice Transformation to a Patient-Centered Medical Home," Annals of Family Medicine, Vol. 7, No. 3, 2009, pp. 254-260.

Oakley, A., V. Strange, C. Bonell, E. Allen, J. Stephenson, and RIPPLE Study Team, "Process Evaluation in Randomised Controlled Trials of Complex Interventions," BMJ, Vol. 332, No. 7538, 2006, pp. 413-416.

Ogden, C.L., M.D. Carroll, B.K. Kit, and K.M. Flegal, "Prevalence of Obesity in the United States, 2009-2010," NCHS Data Brief, No. 82, 2012.

Parks, J., D. Svendsen, P. Singer, and M.E. Foi, eds.,Morbidity and Mortality in People with Serious Mental Illness, Alexandria, VA: NASMHPD, 2006.

Pincus, H.A., J. Houtsinger, J. Bachman, and D. Keyser, "Depression in Primary Care: Bringing Behavioral Health Care into the Mainstream," Health Affairs, Vol. 24, No. 1, pp 271-276.

Pirraglia, P.A., K. Biswas, A.M. Kilbourne, H. Fenn, and M.S. Bauer, "A Prospective Study of the Impact of Comorbid Medical Disease on Bipolar Disorder Outcomes," Journal of Affective Disorders, Vol. 115, No. 3, 2009, pp. 355-359.

Pirraglia, P.A., E. Rowalnd, W.-C. Wu, P.D. Friedmann, T.P. O'Toole, L.B. Cohen, and T.H. Taveira, "Benefits of a Primary Care Clinic Co-Located and Integrated in a Mental Health Setting for Veterans with Serious Mental Illness," Preventing Chronic Disease, Vol. 9, 2012, pp. 1-8.

President's New Freedom Commission on Mental Health, Achieving the Promise: Transforming Mental Health Care in America, 2003.

Proudfoot, J., A. Whitton, G. Parker, J. Doran, V. Manicavasagar, and K. Delmas, "Triggers of Mania and Depression in Young Adults with Bipolar Disorder," Journal of Affective Disorders, Vol. 143, No. 1-3, 2012, pp. 196-202.

Public Law 111-148, Section 2703, Guaranteed Renewability of Coverage, March 23. 2010.

Rahilly-Tierney, C.R., E.V. Lawler, R.E. Scranton, and J.M. Gaziano, "Cardiovascular Benefit of Magnitude of Low-Density Lipoprotein Cholesterol Reduction: A Comparison of Subgroups by Age," Circulation, Vol. 120, No. 15, 2009, pp. 1491-1497.

Rittenhouse, D.R., and S.M. Shortell, "The Patient-Centered Medical Home: Will It Stand the Test of Health Reform?" JAMA, Vol. 301, No. 19, May 20, 2009, pp. 2038-2040.

Rubin, D.B., "Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies," Journal of Educational Psychology, Vol. 86, No. 5, 1974, pp. 686-701.

Saha, S., D. Chant, and J. McGrath, "A Systematic Review of Mortality in Schizophrenia: Is the Differential Mortality Gap Worsening Over Time?" Archives of General Psychiatry, Vol. 64, No. 10, 2007.

Salsberry, P.J., E. Chipps, and C. Kennedy, "Use of General Medical Services Among Medicaid Patients with Severe and Persistent Mental Illness," Psychiatric Services, Vol. 56, No. 4, 2005, pp. 458-462.

Samet, J.H., P. Friedmann, and R. Saitz, "Benefits of Linking Primary Medical Care and Substance Abuse Services: Patient, Provider, and Societal Perspectives," Archives of Internal Medicine, Vol. 161, No. 1, 2001, pp. 85-91.

Saunders, R.P., M.H. Evans, and P. Joshi, "Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide," Health Promotion Practice, Vol. 6, No. 2, 2005, pp. 134-147.

Saxon, A.J., C.A. Malte, K.L. Sloan, J.S. Baer, D.A. Calsyn, P. Nichol, M.K. Chapko, and D.R. Kivlahan, "Randomized Trial of Onsite Versus Referral Primary Medical Care for Veterans in Addictions Treatment," Medical Care, Vol. 44, No. 4, 2006, pp. 334-342.

Scharf, D.M., N.K. Eberhart, N. Schmidt, C.A.Vaughan, T. Dutta, H.A. Pincus, and M.A. Burnam, "Integrating Primary Care into Community Behavioral Health Settings: Programs and Early Implementation Experiences," Psychiatric Services, Vol. 64, No. 7, 2013, pp. 660-665.

Shore, J.H., "Psychiatry at a Crossroad: Our Role in Primary Care," American Journal of Psychiatry, Vol. 153, No. 11, 1996, pp. 1398-1403.

Singh, G.M., G. Danaei, F. Farzadfar, G.A. Stevens, M. Woodward et al., "The Age-Specific Quantitative Effects of Metabolic Risk Factors on Cardiovascular Diseases and Diabetes: A Pooled Analysis," PLoS ONE, Vol. 8, No. 7, 2013, pp. e65174.

Smith, T.E., and L. Sederer, "A New Kind of Homelessness for Individuals with Serious Mental Illness? The Need for a Mental Health Home," Psychiatric Services, Vol. 60, No. 4, pp. 528-533.

Sokal, J., E. Messias, F.B. Dickerson, J. Kreyenbuhl, C.H. Brown, R.W. Goldberg, and L.B. Dixon, "Comorbidity of Medical Illnesses Among Adults with Serious Mental Illness Who Are Receiving Community Psychiatric Services," Journal of Nervous and Mental Disease, Vol. 192, No. 6, 2004, pp. 421-427.

Stetler, C.B., M.W. Legro, C.M. Wallace, C. Bowman, M. Guihan, H. Hagedorn, B. Kimmel, N.D. Sharp, and J.L. Smith, "The Role of Formative Evaluation in Implementation Research and the QUERI Experience," Journal of General Internal Medicine, Vol. 21, No. S2, 2006, pp. S1-S8.

Substance Abuse and Mental Health Services Administration, Rates of Co-Occurring Mental and Substance Use Disorders, undated. As of December 18, 2013: http://www.samhsa.gov/co-occurring/topics/data/disorders.aspx.

Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, Transformation Accountability, undated. As of December 5, 2013: https://www.cmhs-gpra.samhsa.gov/TracPRD/.

Substance Abuse and Mental Health Services Administration, HRSA Center for Integrated Health Solutions, SAMHSA PBHCI Program, 2013a. As of December 18, 2013: http://www.integration.samhsa.gov/about-us/pbhci.

_____, SAMHSA PBHCI Grantees, July 25, 2013b.

Teague, G.B., G.R. Bond, and R.E. Drake, "Program Fidelity in Assertive Community Treatment," American Journal of Orthopsychiatry, Vol. 68, No. 2, 1998, pp. 216-233.

Toth, P.P., D. Potter, and E.E. Ming, "Prevalence of Lipid Abnormalities in the United States: the National Health and Nutrition Examination Survey 2003-2006," Journal of Clinical Lipidology, Vol. 4, No. 6, pp. 325-330.

Tsoi, D.T., M. Porwal, and A.C. Webster, "Interventions for Smoking Cessation and Reduction in Individuals with Schizophrenia," Cochrane Database Systems Review, 2013.

Wierzbicki, A.S., R.E. Clarke, A. Viljoen, and D.P. Mikhailidis, "Triglycerides: A Case for Treatment?" Curr Opin Cardio, Vol. 27, No. 4, 2012, pp. 398-404. As of August 30, 2013: http://journals.lww.com/co-cardiology/Abstract/2012/07000/Triglycerides___a_case_for_treatment_.14.aspx.

Wu, R.R., J.P. Zhao, H. Jin, et al., "Lifestyle Intervention and Metformin for Treatment of Antipsychotic-Induced Weight Gain: A Randomized Controlled Trial," JAMA, Vol. 299, No. 2, 2008, pp. 185-193.

Zappe, C., and W. Danton, "Integrated Mental Health and Primary Care: A Model of Coordinated Services," Federal Practitioner, June 2004, pp. 74-81.

APPENDIX

Site Visits

RAND conducted in-depth, in-person interviews with select staff from the three PBHCI and three non-PBHCI sites taking part in the comparative effectiveness evaluation (see below), plus three additional PBHCI sites, for a total of nine site visits (n=6 PBHCI, n=3 control). These site visits consisted of brief facility tours and in-person interviews with administrative, PC, BH, and care management staff. The main selection criteria for site visits to grantees were the same as the criteria for selecting sites to participate in the comparative effectiveness study. Staff interviews lasted 1-2 hours each and site visits at intervention sites occurred after approximately two years of program implementation.

The main selection criteria for site visits to grantees were the same as the criteria for selecting sites to participate in the comparative effectiveness component: diversity in geographical location, urbanicity, consumer demographics, services provided, and approaches to implementing PBHCI (i.e., the presence and/or type of PC partnership, chosen integration model, data infrastructure, and EBPs used). Sites also had to be willing and able to host the RAND team for a site visit during the study period.

Site visit interviews were conducted with select staff from the following domains: program leadership (administrators, which may include program managers, medical directors, chief financial officers, key administrators, and evaluators/data managers), care coordinators, PC providers (physicians, NPs, PAs, and wellness educators), and BH providers (psychiatrists, psychologists, social workers, case managers and/or peer specialists). Interviews addressed topics such as BH-PC collaboration, program structural features, screening and referral, registry and consumer tracking, performance monitoring, care management, EBPs, wellness/prevention/early intervention, self-management support, consumer involvement, electronic capabilities, women and minority health cultural competency, and program implementation. The topics discussed during site visit interviews were designed to parallel the content of the web-based survey discussed below, but provide greater depth than was possible in survey format.

Comparative Effectiveness Study

Additional Methodologic Detail

Sites

We selected PBHCI (intervention) sites to create a sample with diversity in geographical location, urbanicity, client demographics, services provided, and approaches to implementing PBHCI (i.e., the presence and/or type of PC partnership, chosen integration model, data infrastructure, and EBPs used). We also selected sites that served at least 750 adults meeting SAMHSA's criteria for SMI so that we could obtain adequate statistical power for the analysis of physical health outcomes; sites whose project data were of sufficient quality and completeness to support meaningful evaluation; sites that were willing and able to serve as an intervention site; and sites for which we could identify and engage an appropriately matched control. We also selected sites that were "high implementers" of PBHCI (i.e., those identified as "performing well" by SAMHSA project officers and whose early service utilization data also showed higher-than-average consumer rates of PC provider and case management contacts); we specifically sought out high implementers for the comparative effectiveness study so that we could illustrate PBHCI outcomes for programs implementing integrated care with better-than-average success. We note, therefore, that the sample of PBHCI sites included in this comparative effectiveness evaluation is diverse but intentionally not representative of the larger pool of grantees.

Matched control-sites were identified via web search, suggestions from SAMHSA, state mental health authorities or other agencies, and through suggestions from prospective intervention sites themselves. Control-sites were required to be located within the same state as the PBHCI intervention site and be similar in terms of agency size, urbanicity, client demographics, and services provided, with the exception of no or low PC (i.e., they did not offer PC as part of their program or had no formal referral relationship with a PC provider).

One PBHCI site each was initially selected from Cohorts I, II, and III. However, we were unable to proceed with the original Cohort II and III sites because, on further investigation, no sites from Cohort II ultimately met the inclusion criteria for this part of the project and because we were unable to find a suitable control for the site we had selected from Cohort III. As such, the final sample includes the original Cohort I site and two new Cohort III sites that satisfied the study inclusion criteria outlined above.

Intervention sites were compensated up to $10,000 and control-sites were compensated up to $25,000 for participating in the comparative effectiveness evaluation; compensation was prorated based on the number of consumers identified, recruited, and enrolled in the study.

Consumers

Eligible consumers were adults at least 18 years of age whose primary psychiatric diagnosis was an SMI (see Chapter One) and who were enrolled in integrated care services at a participating PBHCI program or received BH care at a matched control-site. Participants were also required to be able to provide written, informed consent on their own behalf and to be sufficiently fluent in English or Spanish to complete the study questionnaires. Prospective participants were excluded if they were unwilling or unable to provide informed consent.

While not a requirement of study participation per se, we also attempted to enroll consumers at intervention sites who had their first contact with PBHCI within one year (+/- six months) from the follow-up data collection event so that we could quantify the impact of PBHCI on individuals' outcomes after approximately one year of treatment. While we did not exclude individuals who had slightly longer or shorter exposure to the program overall, we did control for these differences in our analysis of treatment effects. Intervention site participants' median and range of time in PBHCI is reported in the "Results" section of this appendix.

Data Collection

Data for the comparative effectiveness study included physical health indicators (detailed in Chapter Four) and a slightly abbreviated version of the NOMs (see Chapter Two for further detail). Data sources and the timing of data collection at the control and intervention site pairs are detailed in Table A.1. Both baseline and follow-up data for the control-sites were collected by RAND subcontractor, OHD (a biometric screening company), as these data were not available through any other source. In contrast, intervention (PBHCI) site data came from two sources: baseline data were taken from the SAMHSA TRAC system (see Chapter Two) and follow-up data were collected by OHD following the same data collection procedures used at the control-sites. Our evaluation design used TRAC data at baseline for the intervention sites to reduce grantee and participant burden and to keep evaluation costs low. OHD collected intervention site follow-up data so that all data could be collected at a single point in time (necessary for the efficient conclusion of the evaluation) and so that the data collection methods were the same for both the control and intervention sites.

OHD used data collection and transmittal procedures that were fully Health Insurance Portability and Accountability Act of 1996 compliant, and it abided by all RAND Human Subjects Protections Committee requirements for the study.

TABLE A.1. Baseline and Follow-Up Data Sources and Timing
Sites		Baseline	1-Year Follow-Up
Control	Site 1	November 2011	November 2012
	Data Source	OHD	OHD
	Sites 2 and 3	March, April, and July 2012	February-May 2013
	Data Source	OHD	OHD
Intervention	Site 1	February 2011	February 2012
	Data Source	TRAC	OHD
	Sites 2 and 3	February 2012	February-May 2013
	Data Source	TRAC	OHD
NOTE: OHD = Onsite Health Diagnostics; TRAC = TRansformation ACcountability.

Training for Research Protocol

Onsite Health Diagnostics

RAND staff provided a half-day, in-person training to OHD team leads (a nurse or similar staff with extensive experience working with psychiatric populations) and periodic training updates throughout the data collection period. Training included instructions for following the research protocol and suggestions for working effectively with persons with SMI. RAND staff attended the first data collection event to ensure that procedures were conducted according to protocol.

Case Managers

Since case management staff provided initial outreach to prospective study participants (see "Participant Recruitment," below), RAND staff also provided a 45-minute web-based training to case management staff at each participating intervention and control clinic. Case manager trainings included information about the purpose of the study, consumer eligibility requirements, procedures for recruiting prospective participants, and suggestions for increasing enrollment (e.g., reducing anxiety associated with the blood draw).

Participant Recruitment

Prospective participants learned about the study from case management staff and advertising (e.g., posters in waiting rooms, write-ups in clinic newsletters, etc.) at participating clinics. Case managers obtained signed releases and contact information from prospective participants so that OHD could then send them a letter describing the study and inviting them to attend the study screening event. Letters were followed by telephone calls from OHD designed to review key study information, answer any remaining questions, and schedule study appointments for interested individuals. With participants' permission, case managers were notified of participants' study appointments so that they could help participants attend the study session (e.g., provide transportation and schedule other appointments adjacent to study sessions). Participants received $10 for completing the survey and $10 for completing the biometric screening procedures. Participants could therefore receive up to $20 for completing one study session. Consumers at control-sites who were asked to complete the survey and physical health screening at baseline and one-year follow-up could therefore earn up to $40 for participating.

Participating sites (n=6) aimed to screen 300 consumers at baseline and to retain at least 250 consumers at follow-up, for a total of 1,800 baseline and 1,500 follow-up screens.

Procedures

Individuals provided written informed consent upon arriving at their respective study site. Individuals who needed assistance reading or understanding the consent forms were assisted by trained OHD staff. OHD staff then conducted the physical health exam, which included measures of height, weight, BMI (calculated from height and weight), BP, waist circumference, and breath CO. OHD licensed and trained phlebotomists also collected a blood sample for the following tests: FPG; A1c; and several lipids, including cholesterol (total, HDL, and LDL) and triglycerides. OHD staff asked participants if they successfully fasted for eight hours prior to the health exam, and although an eight-hour fast was necessary prior to the blood tests for FPG and lipids, blood samples were drawn from individuals whether or not they were fasting. Fasting information was recorded and accounted for in the data analysis.

Individual participants and the medical directors of participating intervention and control-sites received results of the physical health exams. Test results falling outside of the normal range were clearly indicated. Along with test results, consumers received a booklet with information about each test and the meaning of out-of-range values. Consumers with out-of-range values were instructed to contact their health care provider, and consumers without health care providers were instructed to contact their BH care provider for assistance connecting to PC services. Participating clinics agreed to facilitate consumer referrals to local PC providers, as needed, following receipt of screening results. This evaluation does not include information, however, about the frequency with which control-site referrals were made or completed.

Supplemental Analyses--Comparative Effectiveness Evaluation

Like the comparative effectiveness sample, our analysis of TRAC data showed PBHCI consumer improvements in DBP, TC, and LDL-C, and it showed no change in HDL-C or triglycerides. Unlike the comparative effectiveness sample, TRAC data showed consumer improvement in SBP, A1c, and smoking; TRAC data also showed worsening BMI and FPG. While these analyses are based on a large and rich data set, we caution the reader not to over-interpret these results, as it is impossible to determine how much of the observed effects are attributable to PBHCI (versus other factors, such as population trends, impacts of national policy, etc.); the results also do not show how PBHCI consumers fared during the study period relative to the general population. For example, although FPG appears to be getting worse in this analysis, the comparative effectiveness analysis data show that the PBHCI sample fared better than (i.e., did get as worse as) controls during the study period.

TABLE A.2. Change in Physical Health Indicators Between Persons Served at All Participating PBHCI and Control Clinics, by Intervention-Control Site Pair
Indicator	Site Pair 1					Site Pair 2					Site Pair 3
	Unadjusted Mean Change				Case-Mix  AdjustedEstimate	Unadjusted Mean Change				Case-Mix  AdjustedEstimate	Unadjusted Mean Change				Case-Mix  AdjustedEstimate
	N	PBHCI	Control	Diff		N	PBHCI	Control	Diff		N	PBHCI	Control	Diff
SBP	310	-3.63	2.85	-6.48	-4.95a	282	-2.25	-2.48	0.23	-2.43	292	-0.23	-0.37	0.13	-0.63
DBP	310	-3.91	-0.52	-3.38	-0.90	282	-4.47	-0.72	-3.75	-3.74a	292	-5.26	-2.08	-3.18	-4.04a
BMI	301	0.37	-0.09	0.47	-0.30	281	-0.34	-0.27	-0.07	-0.44	290	-0.65	-0.68	0.03	0.15
TC	201	0.68	3.05	-2.37	-4.88	267	-5.77	-4.61	-1.16	-0.38	271	-3.33	8.46	-11.79	-15.77a
HDL-C	260	3.20	2.18	1.02	-0.68	268	1.62	-1.12	2.75	2.38	270	0.03	2.01	-1.98	-3.07a
LDL-C	244	-5.63	1.06	-6.69	-7.51a	249	-5.81	-1.84	-3.97	-3.00	249	-2.22	10.03	-12.26	-16.87a
FPG	273	0.09	0.05	0.04	0.00	218	-0.13	0.10	-0.22	-0.18a	267	0.00	0.04	-0.04	-0.05
A1c	160	-0.08	-0.08	0.00	0.27	221	0.00	0.17	-0.18	0.06	148	1.48	-0.10	1.58	1.57a
Trig	259	-1.93	0.13	-2.06	4.20	267	-3.53	0.86	-4.39	1.69	271	-8.63	-10.87	2.25	5.55a
Smok	311	-0.01	0.00	-0.01	0.00	304	0.01	-0.04	0.06	0.04	294	0.05	-0.05	0.10	0.09
NOTES: A1c = glycated hemoglobin; BMI = body mass index; DBP = diastolic blood pressure; FPG = fasting plasma glucose; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; SBP = systolic blood pressure; Smok = self-reported smoking status; TC = total cholesterol; Trig = Triglycerides. p<0.10.

Supplemental Analyses--Model Features Evaluation

As a comparison to the analyses presented in Table 6.4, we examined the relationship between integrated care access and physical health change (baseline to one-year follow-up) in TRAC data for consumers at all 56 sites in Cohorts I-III. As in our analysis of the comparative effectiveness study data, we examined the relationship between integrated care access and physical health indicators showing improvement over time in TRAC. As above (see Table A.3), this includes SBP, DBP, LDL-C, TC, and A1c. We also examined the relationship between the number of consumer-PC provider contacts and physical health indicators showing improvement over time. Results of this analysis (Table A.4) show that although greater access to integrated care was associated with improvements in SBP, there was no relationship between access to integrated care and improvement in DBP, TC, LDL-C, or A1c.

TABLE A.3. Change in Physical Health Indicators from Intake to 12-Month Follow-Up, All PBHCI Consumers, Cohorts I-III
Indicator	Both Records  (N)	BaselineMean (SD)	Follow-UpMean (SD)	Difference  (FU - BL)	T
SBP	5069	125.41 (18.02)	124.52 (16.87)	-0.87	-3.32c
DBP	5070	79.78 (11.68)	79.1 (10.99)	-0.68	-3.83c
BMI	4662	31.62 (8.37)	31.8 (8.54)	0.17	1.99a
TC	2819	188.35 (44.82)	185.19 (43.15)	-3.16	-4.32c
HDL-C	2874	48.32 (16.64)	48.32 (17.20)	-0.00	-0.01
LDL-C	2680	107.99 (36.35)	105.25 (36.72)	-2.74	-4.25c
FPG	1439	100.78 (27.13)	102.24 (27.68)	1.67	2.37a
A1c	1155	6.28 (1.75)	6.17 (1.69)	-0.10	-2.56a
Trig	2820	154.84 (88.38)	152.98 (84.48)	-1.86	-1.29
Smok	4860	0.55 (0.50)	0.53 (0.50)	-0.02	-5.12c
NOTES: A1c = glycated hemoglobin; BL = baseline;BMI = body mass index; CO = carbon monoxide; DBP = diastolic blood pressure; FPG = fasting plasma glucose; FU = follow-up; HDL-C = high-density lipoprotein cholesterol; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; NOM = National Outcome Measure; SBP = systolic blood pressure; Smok = self-reported smoking status; TC = total cholesterol; Trig = Triglycerides; WC = waist circumference. Breath CO and WC were not included due to high rates of missing/unreliable data. We reported self-reported smoking from the NOMs instead, where 1=any smoking in last 30 days and 0=no smoking in past 30 days. p<0.05. p<0.01. p<0.001.

To further understand how service use might be related to outcomes, we also looked at the association between PC contacts and change in TRAC physical health indicators from baseline to one-year follow-up. Data showed no relationship between consumer-PC contacts and access to integrated care and change in DBP, TC, LDL-C, and A1c; although access to integrated care appeared to be associated with improvements in SBP.

TABLE A.4. Change Physical Health Indicators as Predicted by Access to Integrated Care and PC Provider Contacts, PBHCI Cohorts I-III
	SBP	DBP	TC	LDL-C	A1c
N	4,235	4,236	2,360	2,634	898
Integration--Basic
Estimate (SE)	-1.72a (0.58)	-0.54 (0.40)	-1.84 (1.65)	-1.04 (1.44)	-0.09 (0.10)
Integration--Comprehensive
Estimate (SE)	-1.55a (0.60)	-0.59 (0.41)	1.15 (1.69)	-0.42 (1.45)	-0.03 (0.09)
PC Provider Contacts
Estimate (SE)	0.02 (0.02)	0.03 (0.01)*	-0.02 (0.05)	-0.00 (0.04)	0.00 (0.00)
NOTES: A1c = glycated hemoglobin; DBP = diastolic blood pressure; LDL-C = low-density lipoprotein cholesterol; PBHCI = primary and behavioral health care integration; PC = primary care; SBP = systolic blood pressure; TC = total cholesterol. Negative values favor PBHCI. p<0.05.

Overall, our analysis of TRAC data shows that there are few or no simple or consistent relationships between the quantity of service use and change in physical health indicators during consumers' first year in PBHCI. This could be because treatment effects are obscured by greater use of services both by consumers who get better and those whose illnesses are more difficult to treat. Further, analysis of longer treatment periods could also show different results. In any case, additional analyses investigating other ways that service use might be related to outcomes could make an important contribution to the field.

NOTES

1. For more information about the TRAC system, see SAMHSA (undated).

2. Two different kinds of tests were used to diagnose pre-diabetes. The fasting plasma glucose (FPG) test measures glucose that is floating free in the blood after fasting and only shows the blood glucose level at the time of the test. The A1c test measures the amount of glucose attached to hemoglobin (the oxygen-carrying protein in red blood cells), and because the lifespan of red blood cells is approximately 120 days, A1c reflects long-term glycemic exposure, representing the average glucose concentration over the preceding 8-12 weeks.

3. All web survey data are presented out of a total of 55 because one of the 56 grantees had no survey respondents.

4. For the one grantee with no web survey respondents, none of the six staff members with valid email addresses responded to the survey, despite receiving five reminders each. As a result, all web survey data are presented out of a total of 55 grantees.

5. Based on our review of grantee proposals, 30 grantees (55%) had partnered with federally qualified health centers (FQHCs); however, some of these relationships changed over the course of the grant. More recent data about grantee partnerships with FQHCs is provided in a July 25, 2013 SAMHSA-HRSA CIHS document titled SAMHSA PBHCI Grantees.

6. Results are based on mode of respondents within programs, with PBHCI program leaders as tiebreakers if there were ties for mode within a program.

7. Results are based on mode of respondents within programs, with PBHCI program leaders as tiebreakers if there were ties for mode within a program. Remaining disagreement is due to disagreement among program leaders or missing program leader data.

8. The web survey did not distinguish between wellness services that were available for less than 12 months over the past year because they were initially implemented part-way through the year, and those that were provided sporadically (i.e., that began, ended, and possibly began again within the year). However, quarterly report data and site visit interviews indicated that wellness services were often provided sporadically, sometimes due to limited availability of wellness instructors or perceived lack of consumer interest.

9. Mode of respondents within programs, with PBHCI program leaders as tiebreakers if there were ties for mode within program.

10. The type of influence was not specified in the survey.

11. The high rates of missing data for this dimension were due to the fact that PC or BH providers from many programs did not respond to the survey.

12. Program enrollment targets were taken from grantee proposals. More recent and conservative targets may have been reported by grantees to TRAC (suggesting that higher proportions of target clientele were enrolled in the program). However, RAND did not receive program-level target information from TRAC, and as such, those data are not included in this report.

13. Cohort I grantees have enrolled more individuals than those in Cohorts II and III but they have also been operating longer, and the newer cohorts will likely add more people to their patient caseloads. Moreover, although a smaller proportion of the patients enrolled in Cohort II and III sites have discontinued, the median treatment tenure prior to discontinuation is shorter for those sites relative to Cohort I sites, and sites in Cohorts II and III may have more individuals discontinue treatment over time.

14. A1c is also used to diagnose pre-diabetes and identify risk for diabetes. While FPG reflects current glycemic control (i.e., glucose level at the time of the test), A1c reflects glycemic control over the preceding 8-12 weeks. Pre-diabetes and diabetes may be diagnosed using either FPG or A1c but the methods do not identify the exact same populations, and at the time of this writing, no consensus exists as to which method should be preferred.

15. Self-reported smoking rates were slightly higher (60%) than those indicated by breath CO. This suggests that even if breath CO was not administered systematically to all consumers (as some sites may have used it exclusively with consumers who reported that they smoked), breath CO rates were not artificially inflating the overall rate of smoking prevalence within the PBHCI population.

16. Observed characteristics included in the model reported here were primary mental health diagnosis (bipolar disorder, schizophrenia, major depressive disorder, anxiety, or other) and several variables from the NOMs (gender, race, age, education, school/training, employment, criminal justice contact, binge drinking, substance use, healthy overall, social connectedness, and housing).

17. Rubin's causal model illustrates ways that carefully controlled, nonrandomized data can be used when randomized data are not available.

18. We ran these analyses several ways, including and excluding individuals with missing data at baseline or follow-up and including and excluding individuals who received less than the desired 6-18 months of PBHCI treatment. Results were robust across analyses. The results presented here include individuals for whom baseline and follow-up data were available. Also see the above paragraph on attrition.

19. We do note in our discussion of the comparative effectiveness evaluation (Chapter Five) that the absence of some effects may have been related to an absence of service availability (e.g., smoking outcomes were poor because some programs did not offer smoking cessation services).

20. We learned during a site visit that one PBHCI program struggled to meet its enrollment targets because there was a clinic nearby that offered PC on more days per week.

21. Dr. Scharf provides technical assistance to the SAMHSA HRSA CIHS around data collection and grantee-level evaluation activities.

---

To obtain a printed copy of this report, send the full report title and your mailing information to:

U.S. Department of Health and Human ServicesOffice of Disability, Aging and Long-Term Care PolicyRoom 424E, H.H. Humphrey Building200 Independence Avenue, S.W.Washington, D.C. 20201FAX:  202-401-7733Email:  webmaster.DALTCP@hhs.gov

---

RETURN TO:

Office of Disability, Aging and Long-Term Care Policy (DALTCP) Home [http://aspe.hhs.gov/office_specific/daltcp.cfm]Assistant Secretary for Planning and Evaluation (ASPE) Home [http://aspe.hhs.gov]U.S. Department of Health and Human Services (HHS) Home [http://www.hhs.gov]

Last Updated: 04/09/2014