Strategies for Measuring the Quality of Psychotherapy: A White Paper to Inform Measure Development and Implementation. B. Challenges


Lack of clarity on necessary structures. Defining structure measures that are clear but also flexible enough for implementation in different settings can be challenging. Although there is evidence to support the use of certain psychotherapies, research has not identified the structures that contribute to psychotherapy outcomes (for example, addressing the matter of whether there is an evidence base for the amount of training or supervision needed to implement the treatment with fidelity in typical treatment settings).

Limited resources for investing in training and infrastructure development. Building the capacity to deliver evidence-based psychotherapy may require significant investments in staff time as well as outlays for expert training, consultation, and supervision as therapists learn and implement new skills. This is especially a concern for proprietary therapies. Building data systems to support outcomes monitoring may also represent new expenses for mental health providers, particularly given the lower rates of EHR adoption in these settings (Druss and Dimitropoulos 2013).

Documentation burden. Documenting the steps involved in implementing evidence-based psychotherapies or use of structured processes for monitoring outcomes may impose an additional burden on clinics or organizations, and reviewing this information with an external auditor adds to costs. Managed care organizations and states now credential providers based on licensure, and they typically obtain self-reported information on expertise and training. New efforts to certify health homes offer an avenue for implementation, but auditing and documenting structures described here would be a new activity. Because of restrictions on access to behavioral health records in some locations, auditors may have to make special arrangements with provider organizations, health plans, or state agencies to access documentation for external review. It is also unclear whether the proprietary nature of some psychotherapies would limit the kinds of standards set for supervision or training.

Although audits could focus on protocols and reports, they might also include using independent raters to assess whether providers demonstrate competency in delivering evidence-based psychotherapy. Many studies have used trained, independent raters who either directly observe the delivery of therapy or review video or audio of therapy. Independent raters are considered the most valid reporters of fidelity to psychotherapy because they have been specifically trained to recognize the delivery of the therapeutic elements and, because their rating has no personal bearing for them, their responses are the least likely of all raters to be biased (Barber et al. 2007; Schoenwald et al. 2011a). Clinics or organizations could contract with independent raters as part of a system of evaluating the competency of their providers (although clinics would need to take special care to protect privacy of consumers).

Credentialing on the basis of reports from independent raters could be burdensome and costly. Nonetheless, some states are doing this. For example, beginning in fiscal year 2014, the Texas Department of State Health Services will require providers who offer psychotherapy to adults with depression to demonstrate competency in CBT, which will be determined by an independent rater who reviews and scores a videotaped therapy session (Robinson 2012). Providers who score below an established threshold will be prohibited from providing psychotherapy to adults with depression until their competence has been established. The state has given local mental health authorities a list of organizations qualified to provide independent competency ratings, which costs $150-$300 per videotape reviewed. As of this writing, we do not know how often providers must demonstrate competency, nor do we know the total cost to the Texas health system to obtain independent ratings.


View full report


"QualPsy.pdf" (pdf, 445.96Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®