Strategies for Measuring the Quality of Psychotherapy: A White Paper to Inform Measure Development and Implementation. I. Introduction


A. Policy Context

Several types of psychotherapy can benefit individuals with the most prevalent mental disorders, including depression, anxiety, and post-traumatic stress disorder (PTSD). These psychotherapies include cognitive behavioral therapy (CBT), interpersonal therapy (IPT), psychodynamic therapy, and several others. Although the strength of the evidence supporting each varies, these therapies have generally demonstrated positive outcomes for adults and children (Jakobsen et al. 2012; Cape et al. 2010). Several clinical guidelines for the treatment of depression, anxiety, and PTSD recommend psychotherapy either alone or in combination with medications (National Institute for Health and Clinical Excellence 2005, 2009, 2011; American Psychiatric Association 2006).

As psychotropic medications have become widely used to treat common mental disorders in recent decades, data from national surveys suggest that the use of psychotherapy either alone or in combination with medications has decreased substantially (Olfson and Marcus 2010). Studies have found that only half of treatment episodes for depression include psychotherapy (Horvitz-Lennon et al. 2003), and fewer than 25 percent of older adults with depression receive psychotherapy (Wei et al. 2005). As few as 6 percent of veterans receiving outpatient care for PTSD receive evidence-based psychotherapy in the first six months of treatment (Shiner et al. 2013). Studies also suggest that few people receive a course of psychotherapy that is consistent with clinical trials; 40 percent of people do not return for their second psychotherapy visit, and fewer than 25 percent complete five therapy sessions (Simon et al. 2012a). Although there are no national data that point specifically to the content of psychotherapy and how it relates to protocols tested in research settings, these and other studies suggest that the majority of individuals with mental disorders are unlikely to receive psychotherapy consistent with the treatments tested in clinical trials.

Several factors may impede the receipt of evidence-based psychotherapy, including: (1) a lack of financial incentives to encourage quality and outcomes in psychotherapy; (2) inadequate training in evidence-based psychotherapies; (3) providers' attitudes toward the adoption of evidence-based treatments; and (4) barriers related to consumers' ability to access and sustain participation in psychotherapy. Although Medicare, Medicaid, and commercial insurers cover various forms of psychotherapy, in many states and communities the reimbursement is based largely on the number of visits or amount of time devoted to care rather than on the actual content (e.g., whether a specific evidence-based protocol was followed) or outcomes of the treatment (Weisz et al. 2013). It is important to note that many different types of providers--psychiatrists, psychologists, social workers, and other types of counselors--deliver psychotherapy in a wide range of treatment settings, including primary care offices, public sector mental health clinics, and small or solo private practices. Many of these providers are not affiliated with larger health care delivery systems that can offer clinical support and training on how to deliver care that is consistent with the latest research. These providers come from a variety of training programs and clinical orientations, and many have not received training in specific evidence-based psychotherapies (Arean et al. 2012). A frequently cited survey of accredited training programs in psychiatry, psychology, and social work in the United States found that very few programs required both didactic training and clinical supervision in common evidence-based therapies (Weissman et al. 2006). In addition, the requirements for provider certification to receive reimbursement differ across states, Medicaid programs, and health plans. In some states and communities, there are minimal standards that often rely on state licensing rules and providers' self-reported education and credentials without verification or rigorous continuing education requirements. Finally, research has found that mental health providers vary in terms of their willingness to adopt new treatment approaches that are inconsistent with their existing clinical orientation or that could threaten their autonomy (Aarons et al. 2009, 2011, 2012). Compounding these issues, many consumers face financial barriers to accessing and sustaining their participation in psychotherapy, including high-deductible health plans and limits on the number of covered psychotherapy visits. They may also discontinue therapy before achieving its full benefits due to their time constraints related to employment and family obligations.

Efforts to systematically monitor the delivery of psychotherapy are hindered by the fact that there are few widely accepted clinical quality measures that assess the receipt of psychotherapy, the degree to which it resembles the psychotherapy tested in trials, or the outcomes of care. We conducted an extensive review of behavioral health quality measures in early 2012 and found that most measures of psychotherapy focused on access to care or quantifying the number of visits rather than on the content or outcomes of care (Brown and Scholle 2012). For example, many measures were simple counts of visits or assessed whether individuals received a combination of psychotherapy and medications after an initial diagnosis. Such measures most often are calculated using data from claims or medical records. Of the psychotherapy measures identified none are endorsed by the National Quality Forum (NQF) (Brown and Scholle 2012; NQF 2014). Although Medicaid is the single largest payer of mental health services in the United States, a recent search of the U.S. Department of Health and Human Services (HHS), Centers for Medicare and Medicaid Services (CMS) Measures Inventory ("measures used by CMS in various quality, reporting and payment programs") identified no "psychotherapy" measures currently in use by CMS (CMS 2014). Psychotherapy measures are needed to inform quality improvement initiatives and to enable stakeholders to assess whether the services purchased were delivered as intended and achieved desired outcomes (Schoenwald 2011c; Bond et al. 2011).

Several measures have been developed to assess the delivery of psychotherapy for the purposes of distinguishing it from other treatment in efficacy trials; some of these measures are also used for training or ongoing clinical supervision (Schoenwald et al. 2011a, 2011b, 2011c). Such measures have not been widely adapted for the purpose of holding providers, health plans, or state agencies accountable for care or broader quality improvement initiatives. Many of these measures require extensive collection of observational data, such as videotaped sessions coded by trained raters, which may not be feasible in large-scale quality monitoring and improvement efforts.

Due to reforms being implemented under the Affordable Care Act (ACA) there is an increased need for practical measures to monitor the delivery of and outcomes of care. In particular, the ACA encourages health care delivery system reforms that use budgeted payments coupled with performance measurement to disincentivize inefficient use of health care resources counterbalanced with accountability for providing access to good quality care. In this regard, health homes and Accountable Care Organizations (ACOs) may present opportunities to improve the accessibility of psychotherapy while pay-for-performance efforts could incentivize the delivery of psychotherapy that achieves positive outcomes (Bao et al. 2013). However, some of these same reforms create pressure to limit access to care in order to control costs. Furthermore, due to Medicaid coverage expansions authorized by the ACA and through the health insurance marketplaces, demand for mental health services is expected to increase, and we may expect that more people will begin to receive various forms of psychotherapy. Understanding the extent to which the delivery of psychotherapy changes in response to these and other reforms requires a strong set of measures.

Several national quality reporting initiatives may present opportunities to include psychotherapy measures. These include states' voluntary reporting of quality using the Medicaid Adult and Children's Core Set and the Medicare and Medicaid Electronic Health Record (EHR) Incentive Programs for adoption and "Meaningful Use" of EHRs. In addition, the National Behavioral Health Quality Framework developed by the HHS Substance Abuse and Mental Health Services Administration (SAMHSA) provides momentum to advance the development of psychotherapy measures that can be used for quality improvement and accountability. Specifically, the development of quality measures focused on psychotherapy is consistent with the first goal of the SAMHSA quality framework--to promote the most effective prevention, treatment, and recovery practices for behavioral health disorders.

B. Objectives

We intend for this paper to provide a foundation for future measure-development activities. We briefly describe some of the evidence supporting the most common psychotherapies and discuss three types of measures: (1) structure measures, which gauge the capacity of providers and health systems to deliver evidence-based psychotherapy; (2) process measures, which assess the delivery of psychotherapy, including whether the content and duration of psychotherapy resembles the treatment tested in efficacy trials; and (3) outcome measures, which assess improvements in symptoms and functioning among individuals receiving psychotherapy. For each type of measure, we discuss its potential benefits and the feasibility of using different data sources that could support it, focusing particularly on the extent to which each type of measure would inform quality improvement efforts for a broad group of stakeholders relative to its potential data collection and reporting burden (Teague et al. 2012). We then propose short-term, medium-term, and long-term opportunities for developing and implementing quality measures addressing psychotherapy.

We sought to identify the strengths and limitations of different measurement approaches. Although we concentrate on the delivery of psychotherapy for common mental disorders of varying severity (depression, anxiety, and PTSD), the framework and findings are likely to be relevant for other psychosocial interventions that may benefit different populations. In addition, although the paper is not a comprehensive systematic review of the strength of the evidence base for psychotherapy, we have described some of the research and clinical guidelines that support psychotherapy to provide context for our discussion of various measurement approaches. Finally, the perspectives of a diverse technical expert panel of consumers, performance measurement experts, providers, health plans, payers, and state officials informed the development of this paper.

View full report


"QualPsy.pdf" (pdf, 445.96Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®