Once we finalized the common elements, we converted them into survey items (see Appendix E). Three items drawn from another instrument were measured on a Likert scale; the remaining items consisted of categorical "yes or no" response options, with options for "don't know" or "don't remember." For example, to assess the element "challenging the clients' problematic beliefs using the Socratic method," we developed the question:
Did you use a Socratic discussion method, that is, statements or questions designed for the client to examine their beliefs?
- How do you know this? Can you give me an example?
What are some other ways of viewing this? What are the pros and cons to your way of thinking about this?
How did you come to this conclusion? What evidence do you have to justify this?
We developed three parallel versions of the items to be completed by three different respondents within 24 hours of a sampled therapy session: clinicians, clinical supervisors, and clients (see Chapter IV for more information on the sampling design). The development of the three versions of the measure provides an opportunity to begin to assess which type of rater(s) results in the most credible and reliable measure. Table III.1 illustrates how the wording of the items differs based upon the rater.
|TABLE III.1. Examples of Clinician, Clinical Supervisor, and Client Survey Items|
|Common Element||Clinician||Clinical Supervisor||Client|
|Agenda setting||Did you set an agenda?||Did the therapist set an agenda?||Did you and your therapist discuss an agenda or plan for your session?|
|Socratic questioning||Did you use a Socratic discussion method, that is, statements or questions designed for the client to examine his/her beliefs?||Did the therapist use a Socratic discussion method, that is, statements or questions designed for the client to examine his/her beliefs?||Did your therapist ask you several direct questions to make you think critically about or examine your thoughts, feelings, or beliefs?|
|Risk assessment||Did you conduct a suicide risk assessment for this client?||Did the therapist conduct suicide risk assessment during this session?||During this session, did your therapist ask you if you had thoughts about committing suicide?|
In addition to modifications to the wording of the items, the client version also had a reduced number of items. Based upon recommendations from the TEP, we removed items that the TEP believed clients would have difficulty addressing (for example, the use of cognitive restructuring techniques) and combined related items (for example, setting an agenda and reviewing the agenda). The resulting client version included 25 items.