Development of a Quality Measure for Adults with Post-Traumatic Stress Disorder. APPENDIX J: Summary of the Reliability Analyses

05/01/2019

TABLE J.1. Summary of the Reliability Analysis for Factor 1: Structuring and Conducting the Session
Item N Item Difficulty Item Variance Item-Rest Correlate
Supervisor Sample
2. agenda 66 0.62 0.24 0.77
5. expectations 66 0.61 0.24 0.69
6. goals 66 0.70 0.21 0.53
18. tx feedback 66 0.70 0.21 0.76
33.b.today outline 66 0.45 0.25 0.53
3. review agenda 66 0.59 0.24 0.81
16. directive 66 0.86 0.12 0.54
31.b. todayinstru NA NA NA NA
34.a. ever outline NA NA NA NA
Clinician Sample
1. agenda 80 0.56 0.25 0.63
4..expectations 80 0.58 0.24 0.49
5. goals 80 0.70 0.21 0.31
17. tx feedback 80 0.69 0.21 0.44
32.b.today outline 80 0.36 0.23 0.63
2. review agenda 80 0.51 0.25 0.65
15. directive 80 0.88 0.11 0.35
30.b. todayinstru 80 0.09 0.08 0.35
33.a. ever outline 80 0.86 0.12 0.27
Client Sample
1. agenda 57 0.89 0.09 0.41
2. expectations 57 0.86 0.12 0.67
3. goals 57 0.95 0.05 0.42
11. tx feedback 57 0.77 0.18 0.44
25.b. today outline 57 0.82 0.14 0.60
Question not on client survey NA* NA* NA* NA*
Question not on client survey NA* NA* NA* NA*
22.b. today instru 57 0.68 0.22 0.51
25.a. ever outline 57 0.95 0.05 0.37
NA = Question did not load onto this factor for the model indicated in the column header.
NA* = Question was not included in the client version of the survey.

 

TABLE J.2. Summary of the Reliability Analysis for Factor 2: Psychoeducation and Therapeutic Techniques
Item N Item Difficulty Item Variance Item-Rest Correlate
Supervisor Sample
13.a. imagine 49 0.16 0.14 0.34
12.b. write 49 0.29 0.20 0.46
13.c. other socrat 49 0.69 0.21 0.70
13.d. real 49 0.12 0.11 0.33
4. background 49 0.71 0.20 0.52
7. identify 49 0.80 0.16 0.53
8. cog restruc 49 0.82 0.15 0.08
9. socrat 49 0.76 0.18 0.52
11. Other identify 49 0.57 0.24 0.64
12. techniques 49 0.37 0.23 0.48
14. discuss 49 0.47 0.25 0.47
19. assign 49 0.45 0.25 0.60
20. Review instruc 49 0.45 0.25 0.48
31.a. ever symp edu NA NA NA NA
31.b. today symp edu 49 0.73 0.19 0.61
32.a. Ever trauma ed 49 0.82 0.15 0.58
32.b. today trauma ed 49 0.57 0.24 0.75
12. Overall techniques 49 0.73 0.19 0.72
Clinician Sample
12.a. imagine 80 0.16 0.14 0.44
13.write 80 0.18 0.14 0.52
12.c. other socrat 80 0.46 0.25 0.61
12.d. real 80 0.13 0.11 0.39
3. background 80 0.79 0.17 0.40
6. identify 80 0.75 0.19 0.49
7. cog restruc 80 0.65 0.23 0.36
8. socrat 80 0.61 0.24 0.46
10. other identify 80 0.60 0.24 0.31
11. techniques 80 0.36 0.23 0.37
13. discuss 80 0.45 0.25 0.43
18. assign NA NA NA NA
19. review instruc NA NA NA NA
30.a. Ever symp edu 80 0.94 0.06 0.41
30.b. today symp edu 80 0.59 0.24 0.46
31.a. Ever trauma ed NA NA NA NA
31.b. Today trauma ed 80 0.28 0.20 0.38
12. Overall techniques 80 0.54 0.25 0.66
Client Sample
Question not included in client survey NA* NA* NA* NA*
Question not included in client survey NA* NA* NA* NA*
Question not included in client survey NA* NA* NA* NA*
Question not included in client survey NA* NA* NA* NA*
Question not included in client survey NA* NA* NA* NA*
4. identify 51 0.92 0.07 0.23
Question not included in client survey NA* NA* NA* NA*
5. socrat NA NA NA NA
7. other identify 51 0.90 0.09 0.05
Question not included in client survey NA* NA* NA* NA*
9. discuss 51 0.88 0.10 0.61
13. assign NA NA NA NA
14. review instruc NA NA NA NA
23.a. ever symp edu 51 0.84 0.13 0.48
23.b. today symp edu 51 0.63 0.23 0.65
24.a. ever trauma ed 51 0.78 0.17 0.53
24.b. today trauma ed 51 0.63 0.23 0.62
8. Overall techniques 51 0.78 0.17 0.43
NA = Question did not load onto this factor for the model indicated in the column header.
NA* = Question was not included in the client version of the survey.

 

TABLE J.3. Summary of the Reliability Analysis for Factor 3, Factor 4, and Factor 5
Factor 3: Therapeutic Alliance
Item Obs Item-Domain Correlation Inter-Item Covariance Alpha If Deleted
Supervisor Sample
25. confident 98 0.84 0.44 0.89
26. likes 98 0.88 0.38 0.77
27. trust 98 0.92 0.33 0.71
Clinician Sample
24. confident 96 0.83 0.54 0.83
25. likes 96 0.87 0.49 0.71
26. trust 96 0.88 0.45 0.70
Client Sample
19. confident 77 0.88 0.78 0.81
20. likes 77 0.87 0.78 0.80
18. trust 77 0.89 0.79 0.73
Factor 4: Suicide Assessment
Item Obs Item-Domain Correlation Inter-Item Covariance Alpha If Deleted
Supervisor Sample
9. socrat NA NA NA NA
10. facilitate NA NA NA NA
18. Th feedback NA NA NA NA
15. struggle 78 0.12 0.10 0.19
18. th feedback NA NA NA NA
28.b. today suic 78 0.24 0.18 0.74
29.a. ever use suic NA NA NA NA
29.b. today use suic 78 0.22 0.17 0.61
Clinician Sample
8. socrat NA NA NA NA
9. facilitate 85 0.60 0.24 0.20
17.th feedback 85 0.31 0.21 0.25
14. struggle NA NA NA NA
17. th feedback NA NA NA NA
27.b. today suic 85 0.26 0.19 0.39
28.a. ever use suic 85 0.64 0.23 0.30
28.b. today use suic 85 0.27 0.20 0.53
Client Sample
5. socrat 56 0.91 0.08 0.40
6. facilitate 56 0.93 0.07 0.37
11. Th feedback NA NA NA NA
10. struggle 56 0.98 0.02 0.35
12. th feedback 56 0.57 0.24 0.59
21.b. today suic 56 0.43 0.24 0.40
Question not included in client survey NA* NA* NA* NA*
Question not included in client survey NA* NA* NA* NA*
Factor 5: Homework
Item Obs Item-Domain Correlation Inter-Item Covariance Alpha If Deleted
Supervisor Sample
19. assign 79 0.44 0.25 0.60
20. review instruc 79 0.46 0.25 0.64
21. address 79 0.15 0.13 0.61
22. solution 79 0.13 0.11 0.57
23. review hmwk 79 0.24 0.18 0.59
Clinician Sample
18. assign 82 0.61 0.24 0.64
19. review instruc 82 0.51 0.25 0.61
20. address 82 0.24 0.18 0.64
21. solution 82 0.27 0.20 0.50
22. review hmwk 82 0.28 0.20 0.57
Client Sample
13. assign 62 0.65 0.23 0.78
14. review instruc 62 0.68 0.22 0.81
Question not included in client survey NA* NA* NA* NA*
15. solution 62 0.69 0.21 0.66
16. review hmwk 62 0.66 0.22 0.77
NA = Question did not load onto this factor for the model indicated in the column header.
NA* = Question was not included in the client version of the survey.

 

TABLE J.4. Summary of Reliability Analysis by Measure Domain
Factor Statistical Test Supervisor Sample Clinician Sample Client Sample
Structuring and conducting the session Reliability (KR20) 0.88 0.78 0.77
Average difficulty 0.65 0.58 0.85
Average item-rest correlation 0.66 0.46 0.49
Psychoeducation and therapeutic techniques Reliability (KR20) 0.89 0.83 0.77
Average difficulty 0.56 0.50 0.80
Average item-rest correlation 0.52 0.45 0.45
Therapeutic alliance Reliability (Alpha) 0.85 0.82 0.84
Inter-Item covariance 0.39 0.50 0.78
Average item-rest correlation 0.73 0.67 0.72
Suicide assessment Reliability (KR20) 0.69 0.58 0.64
Average difficulty 0.19 0.41 0.76
Average item-rest correlation 0.52 0.33 0.42
Homework Reliability (KR20) 0.81 0.81 0.90
Average difficulty 0.28 0.38 0.67
Average item-rest correlation 0.60 0.59 0.76
NOTE: Domain difficulty represents the average item difficulty (percent facilities answering "Yes" to an item) across the domain and is only calculated for binary items. Inter-item covariance is the measure of the average covariance between the items and is only calculated for continuous items.