Implementation Findings from the National Evaluation of the Certified Community Behavioral Health Clinic Demonstration. IV. CONCLUSIONS AND FUTURE EVALUATION ACTIVITIES


In the demonstration's second year, CCBHCs and states built on and further refined efforts to hire and maintain staff, increase access to care, sustain the full scope of CCBHC services, and ensure coordinated care for CCBHC clients. Although some CCBHCs experienced challenges related to staffing or the implementation of new services, state officials reported that CCBHCs generally addressed these challenges and, since then, have consistently adhered to the demonstration criteria.

With few exceptions, CCBHCs were able to hire and maintain the required types of staff throughout the demonstration. The first and second years of the demonstration saw little difference in the proportion of CCBHCs that employed most required staff types. In the categories in which fewer clinics employed staff in DY2 than in DY1, reductions in staff employment were minimal. Such changes in staffing may suggest clinics' efforts to experiment and identifying ways to use staff and resources more efficiently. CCBHCs and states reported that clinics faced several ongoing challenges associated with hiring and retaining staff, including, for example, uncertainty around the future of the demonstration, retaining enough of each staff type to meet increased demand for services, and increases in caseloads and responsibilities leading to staff burnout. However, officials generally perceived that clinics effectively used strategies such as increased salaries and benefits to overcome challenges.

In the second demonstration year, CCBHCs and states continued to focus on making services more accessible and increasing consumer engagement. States reported that the most common strategy that CCBHCs used to increase service access was the introduction of open-access scheduling. CCBHCs also have continued to provide services in locations outside of the clinic and make broad use of telehealth to extend the reach of CCBHC services. Stakeholder organizations representing consumers and families reported that the strategies adopted by CCBHCs, such as open-access scheduling and expanded hours of service provision, have significantly improved access to care for CCBHC clients in their states.

Officials in all states perceived that clinics were able to sustain delivery of the nine core CCBHC services throughout the demonstration, a finding confirmed by clinics in the progress report. Nearly all CCBHCs in both DY1 and DY2 reported that they provided the required services, with the exception of intensive community-based mental health services for members of the armed forces and veterans; about 70 percent of clinics provided those services in both years. States speculated that the armed forces/veteran populations did not comprise a large percentage of CCBHC clients and that CCBHCs may have struggled to engage members of these groups and to develop strong referral relationships and care coordination agreements with VA providers. Though not required by the demonstration, a smaller number of clinics provided on-site primary care; only about half of clinics provided this service in either demonstration year.

CCBHCs were able to add and sustain a range of EBPs across demonstration years. In addition, provision of many EBPs by DCOs increased substantially in the second demonstration year. Early in the demonstration, CCBHCs generally addressed the challenges to maintaining EBPs and providing the full scope of CCBHC services, although officials continued to explore ways to support clinics' efforts to offer the full range of services. For example, officials granted CCBHCs increased flexibility to tailor EBPs and other services more precisely to the needs and preferences of their client populations.

CCBHCs have used a variety of strategies to improve care coordination, including the addition of various provider types to treatment teams and the expansion of targeted care coordination strategies to different populations and service lines. Improvements to EHR and HIT systems in the early stages of the demonstration aided clinics' care coordination efforts, in some cases permitting CCBHCs to better integrate care plans, create linkages with external providers, and receive alerts about clients' care transitions.

CCBHCs did not, for the most part, engage DCOs to provide services; instead, they elected to offer the full scope of CCBHCs services directly, although reliance on DCOs did increase slightly in the second demonstration year. Officials suggested that CCBHCs preferred to provide services directly out of a clear desire to embrace the model fully and a reluctance to assume responsibility for the oversight of another provider's services. CCBHCs did, however, continue to provide and expand services in collateral agencies such as schools and shelters, and they built and sustained close formal and informal relationships with a range of external providers.

States and clinics alike described a need for flexibility within the CCBHC model to adjust requirements and practices to best suit the needs of the consumers over the course of the demonstration. For example, some states and clinics found that consumers were not availing themselves of certain required EBPs or access requirements as frequently as expected, and modified these practices to better reflect actual patterns of use. Other findings in the report may point to additional experimentation and fine-tuning of demonstration practices from DY1 to DY2. For instance, some changes in staffing or the composition of care teams may be the result of clinics identifying more efficient and effective ways of providing required CCBHC services.

A. Future Evaluation Activities

This report updated the initial snapshot of early implementation of the demonstration based on interviews with state officials and progress reports submitted by CCBHCs. The update includes data from additional interviews with state officials and consumer and family organizations, site visits to CCBHCs, and progress reports submitted by CCBHCs.

In August 2019, we will submit a report summarizing information from the first year of CCBHC cost reports. Drawing on information from our interviews and site visits, the report will provide an overview of clinics' experience with the PPS and the progress made by CCBHCs and states as they work toward submission of the required quality measures. We will update the report in August 2020 to include information from the second year of CCBHC cost reports and will summarize the quality of care provided to CCBHC consumers by using data from the CCBHC-reported and state-reported quality measures. Our plans to submit these reports as scheduled are dependent on our receipt of the cost reports and quality measures without substantial delays.

We are in the process of obtaining Medicaid claims and encounter data from states to examine changes in service utilization and costs. We plan to examine the impacts of CCBHC services on: (1) hospitalization rates; (2) emergency department service utilization; and (3) ambulatory care relative to within-state comparison groups (Medicaid beneficiaries with similar diagnostic and demographic characteristics who did not receive care from CCBHCs). Depending on the availability of data within each state, we expect that the impact analyses will use approximately four years of Medicaid claims/encounter data (up to a two-year pre-demonstration period and a two-year post-implementation period). We will report these findings in our final report in May 2021, along with updated findings that draw on both years of CCBHC cost reports and quality measures. Table IV.1 provides an overview of the timeline for submission of future deliverables and findings.

TABLE IV.1. Timeline for Reporting Future Evaluation Findings
Reports to Congress Mathematica/RAND Deliverable(s) to Inform Reports to Congress
(submission month and year)
Data Available for Deliverables
(date of data collection)
3 (December 2019) Second implementation memorandum (June 2019)
  • Third-round state interviews and consumer/family organization representative interviews (March 2019)
  • CCBHC site visits (December 2018-February 2019)
  • Second CCBHC progress reports (March 2019)
  Initial cost and quality report (August 2019)
  • First-round cost reports (March 2019) and pre-demonstration claims (December 2018) in addition to interviews and site visits listed above
4 (December 2020) Final cost and quality report (August 2020)
  • First and second-round cost reports (March 2019 and March 2020) and Year 1 quality measures (June 2019)
5 (December 2021) Final report (May 2021)
  • All above, Year 2 quality measures (June 2020) and Medicaid claims/encounter data