Development of a Quality Measure for Adults with Post-Traumatic Stress Disorder. ABSTRACT

05/01/2019

Summary: Despite advances in the development of evidence-based treatment for adults with post-traumatic stress disorder (PTSD), the implementation of these treatments varies widely. To reduce this gap through wider dissemination of effective behavioral health treatment, the U.S. Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation and the HHS National Institute of Mental Health (NIMH) led a project that developed and pre-tested a quality measure of the delivery of psychotherapy for adults with PTSD that is concordant with evidence-based strategies.

Major findings: The project identified five measure constructs related to the delivery of evidence-based psychotherapy for PTSD: (1) structuring and conducting the therapy session; (2) psychoeducation and therapeutic techniques; (3) therapeutic alliance; (4) assessment; and (5) homework. The measure demonstrated fair to good reliability, but some items in the measure may be unnecessary or require refinement. Preliminary performance metrics were established that discriminate between clinicians who are high and low performers in the delivery of evidence-based psychotherapy. Stakeholders showed mixed support of the measure for quality improvement purposes; support for the measure's use in training and continuing education was strong.

Purpose: This project developed a measure of the delivery of evidence-based psychotherapy for adults with PTSD treated in ambulatory settings. The measure assesses care from three perspectives: the clinician, the clinical supervisor, and the client. The measure was pre-tested using quantitative and qualitative methods to assess attributes consistent with National Quality Forum endorsement criteria: importance, feasibility, usability, and scientific acceptability (reliability and validity).

Methods: This project first reviewed existing evidence and measures and gathered input from an advisory group to identify opportunities for new measures. Based on the evidence to support the measure concept, a survey was developed and pre-tested at six behavioral health organizations. Three parallel versions of the measure were developed and tested: clinician, supervisor, and client versions. Quantitative testing involved an examination of the measure's underlying constructs, estimation of its reliability, the creation of performance metrics, and calculation of the measure's sensitivity and specificity. Qualitative testing included focus groups with a range of stakeholders, as well as information gathering from test site coordinators, to obtain input on the measure's importance and face validity and to understand whether it could yield findings that could be used to inform quality improvement efforts.