Development of a Quality Measure for Adults with Post-Traumatic Stress Disorder. II. SELECTION OF MEASURE CONCEPTS

The selection of measure concepts involved several steps: (1) conducting an environmental scan of evidence-based treatments for adults with PTSD; (2) conducting a review of existing measures of PTSD care to identify measurement gaps; and (3) convening a TAG to provide input on measure concepts and the evidence supporting those concepts. This chapter briefly describes these steps and how they influenced the development of the measure.

• A. Environmental Scan of PTSD Treatments and Measures

  • After initial meetings with ASPE and NIMH to discuss priority measurement areas within the broad field of PTSD care and target populations for this quality measure development effort, we conducted a scan of research literature and clinical guidelines to identify evidence-based treatments for PTSD. The scan drew on systematic reviews (including meta-analyses), primary research studies, evidence-based clinical guidelines, and the recommendations of taskforces, including the Institute of Medicine's taskforce on the treatment of PTSD (Institute of Medicine 2008, 2012).

    TABLE II.1. Sources for Environmental Scan of PTSD Research Studies, Clinical Guidelines, and Quality Measures
    Source of Information	Data Sources	Selected Search Terms
    Research studies	PubMed Cochrane Database of Systematic Reviews PsychINFO National Center for the Study of PTSD website	PTSD, trauma, psychotherapy, medication, drugs, pharmacotherapy, treatment, care, services
    Clinical guidelines	National Guideline Clearinghouse Guidelines International Network PubMed Professional websites, for example, the American Psychiatric Association	Psychology, psychiatry, adult and trauma, anxiety disorders, stress
    Quality measures	CQAIMH's National Inventory of Mental Health Quality Measures database NQF Quality Positioning System PILOTS database NQMC Searches of behavioral health quality improvement initiatives, for example, the SAMHSA's National Outcomes Measures Conversations with PTSD experts	Mental, behavioral, psychiatry, psychology, PTSD, trauma, anxiety, depression, substance, and patient experience, diabetes, cardiovascular

    To identify relevant studies and guidelines, we developed search terms to guide this information gathering effort and identified data sources for the information (see Table II.1). We limited the scan to studies and guidelines in English and related to the treatment of PTSD in adults. We created detailed Excel spreadsheets with summaries of the treatment or intervention, the outcome measure(s), the results, and the study design and grading of the study design. We used this information to identify evidence-based treatments for adults with PTSD for which there was the strongest scientific evidence. Briefly, the results of the environmental scan identified strong evidence in support of the effectiveness of cognitive behavioral therapy (CBT), particularly exposure therapies, in the treatment of adults with PTSD. The scan also found clinical guideline support for -- but conflicting interpretations of -- the research on the effectiveness of selective serotonin reuptake inhibitors (SSRIs) and insufficient research evidence in adults with PTSD regarding the effectiveness of support services and care coordination (see Appendix A and the Institute of Medicine 2008, 2012).

• B. Scan of Measures

  • We first began our search for quality measures of PTSD care similarly, by defining search terms (see Table II.1). We then searched the three most widely used sources of quality measures: the National Quality Measure Clearinghouse (NQMC), the NQF, and the online inventory maintained by the Center for Quality Assessment in Mental Health (CQAIMH). Additionally, we searched the Published International Literature on Traumatic Stress (PILOTS) database, which includes a large inventory of measures that are primarily used in research, and the HHS Substance Abuse and Mental Health Services Administration's (SAMHSA's) National Outcomes Measures. Our search for quality measures included measures related to PTSD care as well as ones related to physical or behavioral health conditions that commonly co-occur with PTSD (see Appendix A). We again summarized the information in an Excel spreadsheet that included information on the measure developers, specifications and data sources, NQF endorsement status, and level of evidence to support the measure.

• C. Technical Advisory Group Review

  • The TAG was convened to provide input on the selection of measure concepts and available data sources to develop the measures. The group included research and clinical experts in the treatment of PTSD and behavioral health quality measurement. It also included a consumer representative as well as representatives from a health plan, the U.S. Department of Veterans Affairs (VA) health care system, and the community behavioral health system (see Appendix B for the list of TAG members).

    The TAG meeting was held in March 2012. We summarized the evidence for PTSD care, and, based on that evidence, presented measure concepts for consideration in five broad domains: (1) psychotherapy; (2) pharmacotherapy; (3) assessment, monitoring, and treatment of commonly co-occurring behavioral and physical health conditions; (4) care coordination; and (5) consumer experiences with care. The TAG provided feedback on these measure concepts, suggested additional concepts, and offered input on the feasibility of developing the measures, which rely on various data sources, including administrative data, electronic health records, medical record chart reviews, and survey data.

• D. Selection of Measure Concept

  • To further refine the list of potential measure concepts for consideration, the TAG completed a measure prioritization exercise in mid-March where each member was independently asked to rate each concept on a 1-9 rating scale, with 1-3 classified as low priority, 4-6 as moderate priority, and 7-9 as high priority for each of the four NQF criteria (importance, scientific acceptability, usability, and feasibility; see Section IV).[3] The TAG was asked to consider the availability of data, data collection burden, strength of the evidence supporting the concept, and saliency of the concept in prioritizing the concepts.

    The TAG rated eight concepts as being of high importance; these included measures of psychotherapy, pharmacotherapy, screening for risk of suicide, and patient outcomes (Table II.2). Of these eight concepts, six were rated moderate feasibility and two ("receive at least eight sessions of CBT" and "receive CBT that includes specific components") were rated low feasibility. None of the concepts was rated high feasibility. As noted in Table II.2, the TAG rated the other concepts to be of moderate importance.

    TABLE II.2. Summary of PTSD TAG Members' Prioritization of Measure Concepts
    Priority* Ranking	Concept	Importance Mean (range)	Scientific Acceptability Mean (range)	Usability Mean (range)	Feasibility Mean (range)
    High priority
    1	Screened for risk of suicide	7.75 (6-9)	6.86 (4-8)	8.00 (7-9)	6.00 (5-8)
    2	In psychotherapy and receive at least 8 sessions of CBT	7.50 (7-9)	5.00 (1-8)	8.14 (7-9)	3.71 (1-7)
    3	In psychotherapy and receive CBT that includes specific components	7.38 (4-9)	5.57 (1-8)	7.29 (7-9)	2.57 (1-5)
    4	Symptoms improve over a period of time	7.29 (4-9)	7.00 (4-9)	8.14 (4-9)	5.71 (4-8)
    5	Receive CBT or SSRIs	7.25 (5-9)	5.29 (3-8)	8.00 (6-9)	5.29 (3-8)
    6	Symptoms are assessed at routine intervals	7.13 (5-9)	7.29 (6-8)	7.43 (7-9)	5.57 (4-8)
    7	On medication and receive SSRIs	7.00 (5-9)	6.71 (2-9)	7.29 (5-9)	7.29 (5-9)
    8	On medication who receive a 4-month dosage of SSRIs	7.00 (4-9)	6.14 (2-9)	7.57 (6-9)	6.71 (4-9)
    Moderate Priority
    9	In psychotherapy who receive CBT	6.88 (6-9)	4.00 (1-6)	7.29 (4-9)	3.57 (1-6)
    10	Screened for depression	6.63 (4-9)	7.29 (6-8)	7.43 (5-9)	6.00 (4-7)
    11	Functioning improves over a period of time	6.63 (4-9)	6.00 (2-8)	7.29 (5-9)	5.29 (3-7)
    12	PTSD screened for substance abuse	6.50 (3-9)	6.86 (4-8)	7.86 (7-9)	5.00 (2-7)
    13	Quality of life improves over a period of time	6.50 (4-9)	5.71 (2-8)	6.86 (5-9)	5.14 (3-7)
    14	No improvement or a worsening of symptoms, and have a documented change in treatment approach	6.38 (2-9)	6.57 (3-8)	7.14 (4-9)	4.86 (4-7)
    15	Assessed for sleep problems	6.38 (3-9)	6.86 (6-8)	7.57 (6-9)	5.00 (2-7)
    16	Quality of life and functioning are assessed at routine intervals	6.38 (2-9)	5.71 (2-7)	7.14 (6-9)	4.71 (2-7)
    17	Treatment options such as psychotherapy, medications, or a combination discussed	6.25 (3-9)	5.71 (3-7)	6.86 (5-9)	4.14 (2-7)
    18	On medication and assessed regularly for medication side effects	6.13 (4-9)	6.57 (5-8)	7.14 (5-9)	3.71 (2-6)
    19	Adults with documented comorbidities who have a documented care management/ coordination plan	6.00 (4-9)	5.29 (4-7)	6.57 (3-9)	4.29 (2-7)
    20	Treatment preferences were considered	5.88 (3-9)	5.00 (3-7)	6.86 (5-9)	4.00 (2-6)
    21	Needs for support services have been assessed	5.88 (2-9)	5.29 (2-7)	6.71 (4-9)	4.00 (2-7)
    22	On medication who have a documented assessment of medication possession ratio (or other measure of medication adherence)	5.75 (1-9)	5.50 (3-7)	7.17 (6-9)	5.67 (2-8)
    23	On multiple medications who have documentation of an assessment for potential drug interactions	5.63 (3-9)	6.57 (5-9)	6.57 (5-9)	4.14 (2-7)
    24	Receive a referral and have documentation that the referral was followed up	5.63 (2-9)	5.57 (3-8)	6.71 (4-9)	4.43 (1-7)
    25	Screened for pain	5.63 (3-9)	6.00 (2-7)	6.14 (2-9)	5.14 (2-7)
    26	Receive care from more than 1 provider--that has been communicated to all providers	5.50 (2-9)	5.71 (2-7)	6.43 (3-9)	4.00 (1-7)
    27	Assessed to determine if care management/care coordination is needed	5.50 (2-9)	5.50 (4-7)	6.33 (3-9)	4.00 (2-7)
    28	Screened for glucose levels, lipids, high blood pressure	5.25 (3-7)	6.17 (2-9)	5.67 (2-8)	6.67 (5-8)
    29	Receive antipsychotic medication	3.75 (2-8)	5.86 (2-8)	5.29 (3-8)	6.86 (4-8)
    * Based upon the NQF "importance" criteria.

    The TAG's identification of the eight priority measure concepts provided valuable information to inform a discussion between Mathematica, ASPE, and NIMH regarding the subsequent direction and focus of the project. Together, we selected the "delivery of evidence-based psychotherapy" concept for development and specification. This decision was influenced by the strength of the evidence regarding CBTs as the recommended first line of treatment for adults with PTSD, as well as the limited national data on the quality of psychotherapy treatment. This gap provides ASPE and NIMH the opportunity to not only advance the state of knowledge regarding the quality of psychotherapy delivered to individuals with PTSD, but also inform the broader mental health and quality improvement fields in approaches to measuring quality of psychotherapy for other mental disorders.