Development of a Quality Measure for Adults with Post-Traumatic Stress Disorder. F. Approach to Creating a Measure Score

05/01/2019

In order for a measure to be useful for performance and accountability purposes, the measure must discriminate performance and there must be a mechanism for scoring it to identify individuals who delivery evidence-based psychotherapy. As an initial approach to developing a measure score, we created standardized factor scores for each of the five factors identified in the factor analyses. The scores were standardized to have a mean of zero and a standard deviation of one. A total standardized score was also created using the same method. As depicted in Figure V.1, the distribution in total scores varies for each of the three respondent types.

FIGURE V.1. Distribution of Total Standardized Score by Respondent Type
FIGURE V.1, Bar Chart:  As described in Section F, we created standardized factor scores for each of the 5 factors identified in the factor analyses. The scores range from -1.0 to greater than 1.5. The distribution in total standardized scores varies for each of the 3 respondent types. For clinicians, the distribution ranges from 6% of clinicians with a standardized score of 1-22% of clinicians with a standardized score of -0.5. For supervisors, the distribution ranges from 9 percent of supervisors with standardized scores of 0.5 and 1.5-19% of supervisors with scores of -0.5. For clients, the distribution ranges from 6% of clients with a standardized score of 1.5-26% of clients with a standardized score of 0.5.

Next we examined approaches to establishing measure thresholds that could be used to identify clinicians who deliver evidence-based psychotherapy. We examined four thresholds: the mean, median, mean plus one standard deviation, and the 75th percentile. We selected two thresholds -- the median and 75th percentile -- as more conservative and liberal estimates of measure performance for further investigation. In the subsequent section, we describe the measure's performance when using these thresholds.