Strategies for Measuring the Quality of Psychotherapy: A White Paper to Inform Measure Development and Implementation. References


Aarons, G.A., C. Glisson, P.D. Green, K. Hoagwood, K.J. Kelleher, J.A. Landsverk, Research Network on Youth Mental Health, J.R. Weisz, B. Chorpita, R. Gibbons, C. Glisson, E.P. Green, K. Hoagwood, P.S. Jensen, K. Kelleher, J. Landsverk, S. Mayberg, J. Miranda, L. Palinkas, and S. Schoenwald. "The Organizational Social Context of Mental Health Services and Clinician Attitudes Toward Evidence-Based Practice: A United States National Study." Implementation Science, vol. 7, 2012, p. 56.

Aarons, G.A., M. Hurlburt, and S. M. Horwitz. "Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors."Administration and Policy in Mental Health, vol. 38, no. 1, 2011, pp. 4-23.

Aarons, G.A., R.S. Wells, K. Zagursky, D.L. Fettes, and L.A. Palinkas. "Implementing Evidence-Based Practice in Community Mental Health Agencies: A Multiple Stakeholder Analysis." American Journal of Public Health, vol. 99, no. 11, 2009, pp. 2087-2095.

Abbass, A., and E. Driessen. "The Efficacy of Short-Term Psychodynamic Psychotherapy for Depression: A Summary of Recent Findings." Acta Psychiatrica Scandinavica, vol. 121, no. 5, 2010, pp. 398.

Adair, C.E., G.M. McDougall, C.R. Mitton, A.S. Joyce, T.C. Wild, A. Gordon, N. Costigan, L. Kowalsky, G. Pasmeny, and A. Beckie. "Continuity of Care and Health Outcomes Among Persons with Severe Mental Illness." Psychiatric Services, vol. 56, no. 9, 2005, pp. 1061-1069.

Agency for Healthcare Research and Quality (AHRQ). "Selecting Structure Measures for Clinical Quality Measurement." Available at Measure.aspx. Accessed November 9, 2013a.

Agency for Healthcare Research and Quality. "Initiative Features Fixed Monthly Payments to Primary Care Clinics for Providing Depression Care Bundle, Allowing Many Patients to Achieve Good Outcomes." Available at Accessed November 11, 2013b.

Ahn, H., and B.E. Wampold. "Where Oh Where Are the Specific Ingredients? A Meta-Analysis of Component Studies in Counseling and Psychotherapy." Journal of Counseling Psychology, vol. 48, no. 3, 2001, pp. 251-257.

American Academy of Child and Adolescent Psychiatry (AACAP). "Practice Parameter for Cultural Competence in Child and Adolescent Psychiatric Practice." April 16, 2013. Available at Accessed February 15, 2014.

American Psychiatric Association. "Practice Guidelines for the Treatment of Major Depression." Washington, DC: American Psychiatric Press, Inc., 2006.

American Psychological Association. "2013 Psychotherapy CPT Codes for Psychologists." Accessed November 10, 2013.

Anastario M.P., H.P. Rodriguez, P.M. Gallagher, P.D. Cleary, D. Shaller, W.H. Rogers, K. Bogen, and D.G. Safran. "A Randomized Trial Comparing Mail Versus In-Office Distribution of the CAHPS Clinician and Group Survey." Health Services Research, vol. 45, 2010, pp.1345-1359.

Anker, M.G., B.L. Duncan, and J.A. Sparks. "Using Client Feedback to Improve Couple Therapy Outcomes: A Randomized Clinical Trial in a Naturalistic Setting."Journal of Consulting and Clinical Psychology, vol. 77, no. 4, 2009, pp. 693-704. Available at

Arean, P.A., P.J. Raue, J.A. Sirey, and M. Snowden. "Implementing Evidence-Based Psychotherapies in Settings Serving Older Adults: Challenges and Solutions."Psychiatric Services, vol. 63, no. 6, 2012, pp. 605-607.

Australian Centre for Posttraumatic Mental Health (ACPMH). "Australian Guidelines for the Treatment of Adults with Acute Stress Disorder and Posttraumatic Stress Disorder." Melbourne, Victoria: ACPMH, 2007.

Azocar, F., B. Cuffel, J. McCulloch, J.F. McCabe, S. Tani, and B.B. Brodey. "Monitoring Patient Improvement and Treatment Outcomes in Managed Behavioral Health." Journal for Healthcare Quality, vol. 29, no. 2, 2007, pp. 4-12.

Baldwin, S.A., A. Berkeljon, D.C. Atkins, J.A. Olsen, and S.L. Nielsen. "Rates of Change in Naturalistic Psychotherapy: Contrasting Dose-Effect and Good-Enough Level Models of Change." Journal of Consulting and Clinical Psychology, vol. 77, no. 2, 2009, pp. 203-211.

Bao, Y., L.P. Casalino, and H.A. Pincus. "Behavioral Health and Health Care Reform Models: Patient-Centered Medical Home, Health Home, and Accountable Care Organization." Journal of Behavioral Health Services and Research, vol. 40, no. 1, 2013, pp. 121-132.

Barber, J.P., E. Triffleman, and C. Marmar. "Considerations in Treatment Integrity: Implications and Recommendations for PTSD Research." Journal of Traumatic Stress, vol. 20, no. 5, 2007, pp. 793-805.

Baumann, B.L., D.J. Kolko, K. Collins, and A.D. Herschell. "Understanding Practitioners' Characteristics and Perspectives Prior to the Dissemination of an Evidence-Based Intervention." Child Abuse and Neglect, vol. 30, no. 7, 2006, pp. 771-787.

Beck, J.S. and A.T. Beck. "Cognitive Behavior Therapy: Basics and Beyond." New York, NY: Guilford Publication, 2011.

Bell, E.C., D.K. Marcus, and J.K. Goodlad. "Are the Parts as Good as the Whole? A Meta-Analysis of Component Treatment Studies." Journal of Consulting and Clinical Psychology, vol. 81, no. 4, 2013, pp. 722-736.

Berenson, R.A., P.J. Pronovost, and H.M. Krumholz. "Achieving the Potential for Health Care Performance Measures." May 2013. Robert Wood Johnson Foundation and Urban Institute. Available at Accessed July 12, 2013.

Bernal, G., M.I. Jiménez-Chafey, R. Domenech, and M. Melanie. "Cultural Adaptation of Treatments: A Resource for Considering Culture in Evidence-Based Practice." Professional Psychology: Research and Practice, vol. 40, no. 4, 2009, pp. 361-368.

Bickman, L., S.D. Kelley, C. Breda, A.R. de Andrade, and M. Riemer. "Effect of Routine Feedback to Clinicians on Mental Health Outcomes of Youths: Results of a Randomized Trial." Psychiatric Services, vol. 62, no. 12, 2011, pp. 1423-1429.

Bickman, L. "A Measurement Feedback System (MFS) Is Necessary to Improve Mental Health Outcomes." Journal of the American Academy of Child and Adolescent Psychiatry, vol. 47, no. 10, 2008, pp. 1114-1119.

Bickman, L. "A Common Factors Approach to Improving Mental Health Services." Mental Health Services Research, vol. 7, no. 1, 2005, pp. 1-4.

Blais, M., R.G. Frank, A.A. Nierenberg, and S.L. Rauch. "Measuring Outcomes Makes Good Sense." Psychiatric Services, vol. 60, 2009, p. 112.

Bond, G.R., D.R. Becker, and R.E. Drake. "Measurement of Fidelity of Implantation of Evidence-Based Practices: Case example of the IPS Fidelity Scale." Clinical Psychology: Science and Practice, vol. 18, 2011, pp. 125-140.

Bond, G.R., R.E. Drake, G.J. McHugo, A.E. Peterson, A.M. Jones, and J. Williams. "Long-Term Sustainability of Evidence-Based Practices in Community Mental Health Agencies." Administration and Policy in Mental Health, 2012, online ahead of print.

Boswell, J.F, D.R. Kraus, S.D., Miller, and M.J. Lambert. "Implementing Routine Outcome Monitoring in Clinical Practice: Benefits, Challenges, and Solutions."Psychotherapy Research, 2013, online ahead of print.

Brookman-Frazee, L., R.A. Haine, M. Baker-Ericzen, R. Zoffness, and A.F. Garland. "Factors Associated with Use of Evidence-Based Practice Strategies in Usual Care Youth Psychotherapy." Administration and Policy in Mental Health, vol. 37, no. 3, 2010, pp. 254-269.

Brookman-Frazee, L., R.A. Haine, E.N. Gabayan, and A.F. Garland. "Predicting Frequency of Treatment Visits in Community-Based Youth Psychotherapy."Psychological Services, vol. 5, no. 2, 2008, pp. 126-138.

Brown, J.D., A. Barrett, H.T. Ireys, E. Caffery, and K. Hourihan. "Evidence-Based Practices for Medicaid Beneficiaries with Schizophrenia and Bipolar Disorder." Report prepared for the U. S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation. Washington, DC: Mathematica Policy Research, April 2012. Available at Accessed July 12, 2013.

Brown, J.D., and S.H. Scholle. "Development and Testing of Behavioral Health Quality Measures: Recommendations for Areas of Measurement and Summary of Environmental Scan." Internal Project Memorandum prepared for the U. S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Contract Number HHSP23320095642WC, Task Order Number HHSP23320100019WI. Washington, DC: Mathematica Policy Research, January 20, 2012.

Brown, J.D., and L.S. Wissow. "Disagreement in Parent and Primary Care Provider Reports of Mental Health Counseling." Pediatrics, vol. 122, 2008, pp. 1204-1211.

Brown, J.D., L.S. Wissow, A. Gadomski, C. Zachary, E. Bartlett, and I. Horn. "Parent and Teacher Mental Health Ratings of Children Using Primary Care Services: Inter-Rater Agreement and Implications for Mental Health Screening." Ambulatory Pediatrics, vol. 6, 2007, pp. 347-351.

Brown, G.S., and E.R. Jones. "Implementation of a Feedback System in a Managed Care Environment: What Are Patients Teaching Us?" Journal of Clinical Psychology, vol. 61, no. 2, 2005, pp. 187-198.

Campbell, A., and S. Hemsley. "Outcome Rating Scale and Session Rating Scale in Psychological Practice: Clinical Utility of Ultra-Brief Measures." Clinical Psychologist, vol. 13, no. 1, 2009, pp. 1-9.

Campbell, B.K. "Fidelity in Public Health Clinical Trials: Considering Provider-Participant Relationship Factors in Community Treatment Settings." Journal of Public Health Dentistry, vol. 71, supp. 1, 2011, pp. S64-S65.

Cape, J., C. Whittington, M. Buszewicz, P. Wallace, and L. Underwood. "Brief Psychological Therapies for Anxiety and Depression in Primary Care: Meta-Analysis and Meta-Regression." BMC Medicine, vol. 8, no. 1, 2010, p. 38.

Chambless, D.L., and T.H. Ollendick. "Empirically Supported Psychological Interventions: Controversies and Evidence." Annual Review of Psychology, vol. 52, 2001, pp. 685-716.

Chapman, J.E., M.R. McCart, E.J. Letourneau, and A.J. Sheidow. "Comparison of Youth, Caregiver, Therapist, Trained, and Treatment Expert Raters of Therapist Adherence to a Substance Abuse Treatment Protocol." Journal of Consulting and Clinical Psychology, 2013, online ahead of print.

Chorpita, B.F., and E.L. Daleiden. "Mapping Evidence-Based Treatments for Children and Adolescents: Application of the Distillation and Matching Model to 615 Treatments from 322 Randomized Trials." Journal of Consulting and Clinical Psychology, vol. 77, no. 3, 2009, pp. 566-579.

Chorpita, B.F., and J. Regan. "Dissemination of Effective Mental Health Treatment Procedures: Maximizing the Return on a Significant Investment." Behaviour Research and Therapy, vol. 47, no. 11, 2009, pp. 990-993.

Centers for Medicare and Medicaid Services (CMS). "CMS Measures Inventory." Available at Accessed April 28, 2014 (downloaded database file of measures dated March 31, 2014).

Centers for Medicare and Medicaid Services. "ICD-10-PCS Official Guidelines for Coding and Reporting." Available at Accessed November 10, 2013.

Clark, M.S., K.L. Jansen, and J.A. Cloy. "Treatment of Childhood and Adolescent Depression." American Family Physician, vol. 86, no. 5, 2012, pp. 442.

Conway, P., H.F. Mostashari, and C. Clancy. "The Future of Quality Measurement for Improvement and Accountability." Journal of the American Medical Association, vol. 309, no. 21, 2013, pp. 2215-2216.

Cook, J.M., T. Biyanova, J. Elhai, P.P. Schnurr, and J.C. Coyne. "What Do Psychotherapists Really Do in Practice? An Internet Study of Over 2,000 Practitioners."Psychotherapy: Theory, Practice, Research, Training, vol. 47, no. 2, 2012, pp. 260-267.

Cuijpers, P., A.S. Geraedts, P. van Oppen, G. Andersson, J.C. Markowitz, and A. van Straten. "Interpersonal Psychotherapy for Depression: A Meta-Analysis."American Journal of Psychiatry, vol. 168, no. 6, 2011, pp. 581-592.

Czajkowski, S.M. "The Importance of Monitoring and Maximizing Treatment Fidelity in Public Health Research." Journal of Public Health Dentistry, vol. 71, supp. 1, 2011, pp. S67-S68.

Department of Health, National Health Service. "IAPT 3 Year Report: The First Million Patients. Available at Accessed February 20, 2014.

Department of Veteran Affairs and Department of Defense. "VA/DoD Clinical Practice Guidelines: Management of Post Traumatic Stress Disorder and Acute Stress Reaction." 2010. Available at Accessed June 17, 2013.

Donabedian, A. "The Quality of Care. How Can It Be Assessed?" Journal of the American Medical Association, vol. 260, no. 12, 1988, pp. 1743-1748.

Donabedian, A. "Exploration in Quality Assessment and Monitoring: the Definition of Quality and Approaches to its Assessment." Ann Arbor, MI: Health Administration Press, 1980.

Driessen, E., P. Cuijpers, S.C. de Maat, A.A. Abbass, F. de Jonghe, and J.J. Dekker. "The Efficacy of Short-Term Psychodynamic Psychotherapy for Depression: A Meta-Analysis." Clinical Psychology Review, vol. 30, no. 1, 2010, pp. 25-36.

Drapeau, M. "The Value of Tracking in Psychotherapy." Integrating Science and Practice, vol. 2, 2012, pp. 2-6.

Druss, B.G., and L. Dimitropoulos. "Advancing the Adoption, Integration and Testing of Technological Advancements within Existing Care Systems." General Hospital Psychiatry, vol. 35, no. 4, 2013, pp. 345-348.

Druss, B.G., P.S. Wang, N.A. Sampson, M. Olfson, H.A. Pincus, K.B. Wells, and R.C. Kessler. "Understanding Mental Health Treatment in Persons Without Mental Diagnoses: Results from the National Comorbidity Survey Replication." Archives of General Psychiatry, vol. 64, no. 10, 2007, pp. 1196-1203.

Eisen, S.V., J.A. Shaul, H.S. Leff, V. Stringfellow, B.R. Clarridge, and P.D. Cleary. "Toward a National Consumer Survey: Evaluation of the CABHS and MHSIP Instruments." Journal of Behavioral Health Services and Research, vol. 28, no. 3, 2001, pp. 347-369.

Ellsworth, J.R., M.J. Lambert, and J. Johnson. "A Comparison of the Outcome Questionnaire-45 and Outcome Questionnaire-30 in Classification and Prediction of Treatment Outcome." Clinical Psychology and Psychotherapy, vol. 13, 2006, pp. 380-391.

Foa, E. "Effective Treatment for PTSD, Second Edition." New York, NY: Guilford Press, 2009.

Forman-Hoffman, V., S. Knauer, J. McKeeman, A. Zolotor, R. Blanco, S. Lloyd, E. Tant, and M. Viswanathan. "Child and Adolescent Exposure to Trauma: Comparative Effectiveness of Interventions Addressing Trauma Other Than Maltreatment or Family Violence." Comparative Effectiveness Review No. 107. Report prepared by RTI International-University of North Carolina at Chapel Hill for the Agency for Healthcare Research and Quality (AHRQ). Rockville, MD: AHRQ. February 2013. Available at Accessed November 8, 2013.

Garland, A.F., L. Bickman, and B.F. Chorpita. "Change What? Identifying Quality Improvement Targets by Investigating Usual Mental Health Care." Administration and Policy in Mental Health, vol. 37, no. 1-2, 2010a, pp. 15-26.

Garland, A.F., L. Brookman-Frazee, M.S. Hurlburt, E.C. Accurso, R.J. Zoffness, R. Haine-Schlagel, and W. Ganger. "Mental Health Care for Children with Disruptive Behavior Problems: A View Inside Therapists' Offices." Psychiatric Services, vol. 61, no. 8, 2010b, pp. 788-795.

Garland, A.F., M.S. Hurlburt, L. Brookman-Frazee, R.M. Taylor, and E.C. Accurso. "Methodological Challenges of Characterizing Usual Care Psychotherapeutic Practice." Administration and Policy in Mental Health, vol. 37, no. 3, 2010c, pp. 208-220.

Garland, A.F., M. Kruse, and G.A. Aarons. "Clinicians and Outcome Measurement: What's the Use?" Journal of Behavioral Health Services and Research, vol. 30, no. 4, 2003, pp. 393-405.

Glasgow, R.E., R.M. Kaplan, J.K. Ockene, E.B. Fisher, and K.M. Emmons. "Patient-Reported Measures of Psychosocial Issues and Health Behavior Should Be Added to Electronic Health Records." Health Affairs, vol. 31, no. 3, 2012, pp. 497-504.

Hannan, C., M.J. Lambert, C. Harmon, S.L. Nielsen, D.W. Smart, K. Shimokawa, and S.W. Sutton. "A Lab Test and Algorithms for Identifying Clients at Risk for Treatment Failure." Journal of Clinical Psychology, vol. 61, no. 2, 2005, pp. 155-163.

Hansen, N.B., M.J. Lambert, and E.V. Forman. "The Psychotherapy Dose-Response Effect and Its Implications for Treatment Delivery Services." Clinical Psychology: Science and Practice, vol. 9, 2002, pp. 329-343.

Harding, K.J., A.J. Rush, M. Arbuckle, M.H. Trivedi, and H.A. Pincus. "Measurement-Based Care in Psychiatric Practice: A Policy Framework for Implementation."Journal of Clinical Psychiatry, vol. 72, no. 8, 2011, pp. 1136-1143.

Hepner, K.A., F. Azocar, G.L. Greenwood, J. Miranda, and M.A. Burnam. "Development of a Clinician Report Measure to Assess Psychotherapy for Depression in Usual Care Settings." Administration and Policy in Mental Health and Mental Health Services Research, vol. 37, no. 3, 2010, pp. 221-229.

Hoagwood, K.E., P.S. Jensen, M.C. Acri, S.S. Olin., R.E. Lewamdowski, and R.K. Herman. "Outcome Domains in Child Mental Health Research Since 1996: Have They Changed and Why Does It Matter?" Journal of the American Academy of Child and Adolescent Psychiatry, vol. 51, no. 12, 2012, 1241-1260.

Hofmann, S.G., A. Asnaani, I.J. Vonk, A.T. Sawyer, and A. Fang. "The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-Analyses." Cognitive Therapy and Research, vol. 36, no. 5, 2012, pp. 427-440.

Horvitz-Lennon, M., S.L. Normand, R.G. Frank, and H.H. Goldman. "'Usual Care' for Major Depression in the 1990s: Characteristics and Expert-Estimated Outcomes." American Journal of Psychiatry, vol. 160, no. 4, 2003, pp. 720-726.

Hurlburt, M.S., A.F. Garland, K. Nguyen, and L. Brookman-Frazee. "Child and Family Therapy Process: Concordance of Therapist and Observational Perspectives."Administration and Policy in Mental Health, vol. 37, no. 3, 2010, pp. 230-244.

Institute of Medicine. "Treatment for Posttraumatic Stress Disorder in Military and Veteran Populations: Initial Assessment." Washington, DC: National Academies Press, 2012.

Jakobsen, J.C., J.L. Hansen, E. Simonsen, and C. Gluud. "The Effect of Adding Psychodynamic Therapy to Antidepressants in Patients with Major Depressive Disorder. A Systematic Review of Randomized Clinical Trials with Meta-Analyses and Trial Sequential Analyses." Journal of Affective Disorders, vol. 137, no. 1, 2012, pp. 4-14.

Jonas, D.E., K. Cusack, C.A. Forneris, T.M. Wilkins, J. Sonis, J.C. Middleton, C. Feltner, D. Meredith, J. Cavanaugh, K.A. Brownley, K.R. Olmsted, A. Greenblatt, A. Weil, and B.N. Gaynes. "Psychological and Pharmacological Treatments for Adults with Posttraumatic Stress Disorder (PTSD)." Comparative Effectiveness Review No. 92. Report prepared by RTI International-University of North Carolina Evidence-based Practice Center. AHRQ Publication No. 13-EHC011-EF. Rockville, MD: Agency for Healthcare Research and Quality; April 2013. Available at Accessed November 8, 2013.

Kerr, E.A., M.A. Lucatorto, R. Holleman, N.M. Hogan, M.L. Klamerus, T.P. Hofer, and VA Diabetes Quality Enhancement Research Initiative (QUERI) Workgroup on Clinical Action Measures. "Monitoring Performance for Blood Pressure Management among Diabetic Patients: Too Much of a Good Thing?" Archives of Internal Medicine, vol. 25, no. 12, 2012, pp. 938-945. Available at

Kilbourne, A.M., D. Keyser, and H.A. Pincus. "Challenges and Opportunities in Measuring the Quality of Mental Health Care." Canadian Journal of Psychiatry, Revue Canadienne De Psychiatrie, vol. 55, no. 9, 2010, pp. 549-557.

Kiresuk, T.J., A. Smith, and J.E. Cardillo (eds.). "Goal Attainment Scaling: Applications, Theory, and Measurement." Hillsdale, NJ: Lawrence Erlbaum Associates, Inc., 1994.

Knaup, C., M. Koesters, D. Schoefer, T. Becker, and B. Puschner. "Effect of Feedback of Treatment Outcome in Specialist Mental Healthcare: Meta-Analysis."British Journal of Psychiatry: Journal of Mental Science, vol. 195, no. 1, 2009, pp. 15-22.

Kraus, D.R. "The Treatment Outcome Package (TOP)." Integrating Science and Practice, vol. 2, no. 2, 2012, pp. 43-45.

Lambert, M.J., C. Harmon, K. Slade, J.L. Whipple, and E.J. Hawkins. "Providing Feedback to Psychotherapists on Their Patients' Progress: Clinical Results and Practice Suggestions." Journal of Clinical Psychology, vol. 61, no. 2, 2005, pp. 165-174.

Lambert, M.J., and D.E. Barley. (2002). "Research Summary on the Therapeutic Relationship and Psychotherapy Outcome." In Psychotherapy Relationships that Work: Therapist Contributions and Responsiveness to Patients, edited by J.C. Norcross. Oxford: Oxford University Press.

Lambert, M.J., J.L. Whipple, M.J. Bishop, D.A. Vermeersch, G.V. Gray, and A.E. Finch. "Comparison of Empirically Derived and Rationally Derived Methods for Identifying Clients at Risk for Treatment Failure." Clinical Psychology and Psychotherapy, vol. 9, 2002, pp. 149-164.

Laska, K.M., A.S. Gurman, and B.E. Wampold. "Expanding the Lens of Evidence-Based Practice in Psychotherapy: A Common Factors Perspective."Psychotherapy, online ahead of print. doi: 10.1037/a0034332.

Leichsenring, F. "Are Psychodynamic and Psychoanalytic Therapies Effective?: A Review of Empirical Data." International Journal of Psychoanalysis, vol. 86, 2005, pp. 841-868.

Leichsenring, F., and E. Leibing. "Psychodynamic Psychotherapy: A Systematic Review of Techniques, Indications, and Empirical Evidence." Psychology and Psychotherapy, vol. 80, 2007, pp. 217-228.

Liptzin. B. "Quality Improvement, Pay for Performance, and Outcomes Measurement: What Makes Sense?" Psychiatric Services, vol. 60, 2009, pp. 108-111.

Long, M.E., A.L. Grubaugh, J.D. Elhai, K.J. Cusack, R. Knapp, and B.C. Frueh. "Therapist Fidelity with an Exposure-Based Treatment of PTSD in Adults with Schizophrenia or Schizoaffective Disorder." Journal of Clinical Psychology, vol. 66, no. 4, 2010, pp. 383-393.

Lu, W., P.T. Yanos, J.D. Gottlieb, S.M. Duva, S.M. Silverstein, H. Xie, S.D. Rosenberg, and K.T. Mueser. "Use of Fidelity Assessments to Train Clinicians in the CBT for PTSD Program for Clients with Serious Mental Illness." Psychiatric Services, vol. 63, no. 8, 2012, pp. 785-792.

Maalouf, F.T., and D.A. Brent. "Child and Adolescent Depression Intervention Overview: What Works, for Whom and How Well?" Child and Adolescent Psychiatric Clinics of North America, vol. 21, no. 2, 2012, pp. 299-312.

Markowitz, J.C., and M.M. Weissman. "Interpersonal Psychotherapy: Principles and Applications." World Psychiatry : Official Journal of the World Psychiatric Association, vol. 3, no. 3, 2004, pp. 136-139.

Martino, S., S. Ball, C. Nich, T.L. Frankforter, and K.M. Carroll. "Correspondence of Motivational Enhancement Treatment Integrity Ratings Among Therapists, Supervisors, and Observers." Psychotherapy Research, vol. 19, no. 2, 2009, pp. 181-193.

Miranda, J., K.A. Hepner, F. Azocar, G. Greenwood, V. Ngo, and M.A. Burnam. "Development of a Patient-Report Measure of Psychotherapy for Depression."Administration and Policy in Mental Health and Mental Health Services Research, vol. 37, no. 3, 2010, pp. 245-253.

McAleavey, A.A., S.S. Nordberg, D.R. Kraus, and L.G. Castonguay. "Errors in Treatment Outcome Monitoring: Implications of Multidimensional and General Measurements for Real-World Psychotherapy." Canadian Psychology, vol. 53, 2012, pp. 105-114.

McHugh, R.K., H.W. Murray, and D.H. Barlow. "Balancing Fidelity and Adaptation in the Dissemination of Empirically-Supported Treatments: The Promise of Transdiagnostic Interventions." Behaviour Research and Therapy, vol. 47, no. 11, 2009, pp. 946-953.

Mohr, D.C., M.N. Burns, S.M. Schueller, G. Clarke, and M. Klinkman. "Behavioral Intervention Technologies: Evidence Review and Recommendations for Future Research in Mental Health." General Hospital Psychiatry, vol. 35, no. 4, 2013, pp. 332-338.

Moss, S., F. Margison, and K. Godbert. "The Maintenance of Psychotherapy Skill Acquisition: A Two-Year Follow-Up." British Journal of Medical Psychology, vol. 64, 1991, pp. 233-236.

National Committee for Quality Assurance (NCQA). "Integrated Care for People with Medicare and Medicaid: A Roadmap for Quality." March 2013. Available at Accessed July 12, 2013.

National Institute for Health and Care Excellence. "Post-Traumatic Stress Disorder. The Management of PTSD in Adults and Children in Primary and Secondary Care." CG 26. London: National Institute for Health and Care Excellence, 2005.

National Institute for Health and Care Excellence. "Depression in Adults. The Treatment and Management of Depression in Adults." CG 90. London: National Institute for Health and Care Excellence, 2009.

National Institute for Health and Care Excellence. "Generalised Anxiety Disorder and Panic Disorder (with or Without Agoraphobia) in Adults. Management in Primary, Secondary and Community Care." CG 113. London: National Institute for Health and Care Excellence, 2011.

National Quality Forum. "NQF-Endorsed Measures (QPS)." Available at Accessed April 28, 2014.

National Quality Forum. "Patient Reported Outcomes (PROs) in Performance Measurement." January, 2013. Available at Accessed November 18, 2013.

National Quality Forum. "Measuring Healthcare Quality for the Dual Eligible Beneficiary Population." June 2012. Available at Accessed February 13, 2014.

Okiishi, J.C., M.J. Lambert, D. Eggett, L. Nielsen, D.D. Dayton, and D.A. Vermeersch. "An Analysis of Therapist Treatment Effects: Toward Providing Feedback to Individual Therapists on Their Clients' Psychotherapy Outcome." Journal of Clinical Psychology, vol. 62, no. 9, 2006, pp. 1157-1172.

Olfson, M., and S.C. Marcus. "National Trends in Outpatient Psychotherapy." American Journal of Psychiatry, vol. 167, no. 12, 2010, pp. 1456-1463.

Otte, C. "Cognitive Behavioral Therapy in Anxiety Disorders: Current State of the Evidence." Dialogues in Clinical Neuroscience, vol. 13, no. 4, 2011, p. 413.

Pfeiffer, N., J. Glass, K. Austin, M. Valenstein, J.F. McCarthy, and K. Zivin. "Impact of Distance and Facility of Initial Diagnosis on Depression Treatment." Health Services Research, vol. 46, no. 3, 2011, pp.768-786.

Ramsay, J.R. "Current Status of Cognitive-Behavioral Therapy as a Psychosocial Treatment for Adult Attention-Deficit/Hyperactivity Disorder." Current Psychiatry Reports, vol. 9, no. 5, 2007, pp. 427-433.

Reese, R.J., M.D. Toland, N.C. Slone, and L.A. Norsworthy. "Effect of Client Feedback on Couple Psychotherapy Outcomes." Psychotherapy: Theory, Research, Practice, Training, vol. 47, no. 4, 2010, pp. 616-630. Available at

Robinson, R. "Attachment O: Cognitive Behavioral Therapy Competency Policy." 2012. Available at Accessed July 10, 2013.

Rost, K., J. Fortney, and J. Coyne. "The Relationship of Depression Treatment Quality Indicators to Employee Absenteeism." Mental Health Services Research, vol. 7, no. 3, 2005, pp. 161-169.

SAMHSA-HRSA Center for Integrated Health Solutions. "Behavioral Health Homes for People with Mental Health and Substance Use Conditions: The Core Clinical Features." May 2012. Available at Accessed June 15, 2013.

Schoenwald, S.K., and A.F. Garland. "A Review of Treatment Adherence Measurement Methods." Psychological Assessment, vol. 25, no. 1, 2013, pp. 146-156.

Schoenwald, S.K., A.F. Garland, J.E. Chapman, S.L. Frazier, A.J. Sheidow, and M.A. Southam-Gerow. "Toward the Effective and Efficient Measurement of Implementation Fidelity." Administration and Policy in Mental Health, vol. 38, no. 1, 2011a, pp. 32-43.

Schoenwald, S.K., A.F. Garland, M.A. Southam-Gerow, B.F. Chorpita, and J.E. Chapman. "Adherence Measurement in Treatments for Disruptive Behavior Disorders: Pursuing Clear Vision Through Varied Lenses." Clinical Psychology, vol. 18, no. 4, 2011b, pp. 331-341.

Schoenwald, S.K. "It's a Bird, It's a Plane, It's...Fidelity Measurement in the Real World." Clinical Psychology: A Publication of the Division of Clinical Psychology of the American Psychological Association, vol. 18, no. 2, 2011c, pp. 142-147.

Schoenwald, S.K., J.E. Chapman, A.J. Sheidow, and R.E. Carter. "Long-Term Youth Criminal Outcomes in MST Transport: The Impact of Therapist Adherence and Organizational Climate and Structure." Journal of Clinical Child and Adolescent Psychology, vol. 38, no. 1, 2009, pp. 91-105.

Shedler, J. "The Efficacy of Psychodynamic Psychotherapy." American Psychologist, vol. 65, no. 2, 2010, pp. 98-109.

Shimokawa, K, M.J. Lambert, and D.W. Smart. "Enhancing Treatment Outcome of Patients at Risk for Treatment Failure: Meta-Analytic and Mega-Analytic Review of a Psychotherapy Quality Assurance System." Journal of Consulting and Clinical Psychology, vol. 78, no. 3, 2010, pp. 298-311.

Shiner, B., L.W. D'Avolio, T.M. Nguyen, M.H. Zayed, Y. Young-Xu, R.A. Desai, P.P. Schnurr, L.D. Fiore, and B.V. Watts. "Measuring Use of Evidence-Based Psychotherapy for Posttraumatic Stress Disorder." Administration and Policy in Mental Health, vol. 40, no. 4, 2013, pp. 311-318.

Simon, G.E., V. Ding, R. Hubbard, P. Fishman, E. Ludman, L. Morales, B. Operskalski, and J. Savarino. "Early Dropout from Psychotherapy for Depression with Group- and Network-Model Therapists." Administration and Policy in Mental Health, vol. 39, no. 6, 2012a, pp. 440-447.

Simon, G.E., Z.E. Imel, E.J. Ludman, and B.J. Steinfeld. "Is Dropout After a First Psychotherapy Visit Always a Bad Outcome?"Psychiatric Service, vol. 63, no. 7, 2012b, pp. 705-707.

Stiles, W.B., C. Leach, M. Barkham, M. Lucock, S. Iveson, D.A. Shapiro, M. Iveson, and G.E. Hardy. "Early Sudden Gains in Psychotherapy Under Routine Clinic Conditions: Practice-Based Evidence." Journal of Consulting and Clinical Psychology, vol. 71, no. 1, 2003, pp. 14-21.

Stirman, S.W., A. Gutierrez-Colina, K. Toder, G. Esposito, F. Barg, F. Castro, A.T. Beck, and P. Crits-Christoph. "Clinicians' Perspectives on Cognitive Therapy in Community Mental Health Settings: Implications for Training and Implementation." Administration and Policy in Mental Health, vol. 40, no. 4, 2013, pp. 274-285.

Sue, S., N. Zane, H.C. Nagayama, and L.K. Berger. "The Case for Cultural Competency in Psychotherapeutic Interventions." Annual Review of Psychology, vol. 60, 2009, pp. 525-548.

Teague, G.B., K.T. Mueser, and C.A. Rapp. "Advances in Fidelity Measurement for Mental Health Services Research: Four Measures." Psychiatric Services, vol. 63, no. 8, 2012, pp. 765-771.

Torda, P. "Measures Using Patient-Reported Outcomes." Presented at the PCORI Patient Reported Outcome Workshop, Atlanta, GA. November 19, 2013. Available at Accessed February 16, 2014.

van Hees, M.L., T. Rotter, T. Ellermann, and S.M. Evers. "The Effectiveness of Individual Interpersonal Psychotherapy as a Treatment for Major Depressive Disorder in Adult Outpatients: A Systematic Review." BMC Psychiatry, vol. 13, no. 1, 2013, p. 22.

Warren, J.S., P.L. Nelson, S.A. Mondragon, S.A. Baldwin, and G.M. Burlingame. "Youth Psychotherapy Change Trajectories and Outcomes in Usual Care: Community Mental Health Versus Managed Care Settings." Journal of Consulting and Clinical Psychology, vol. 78, 2012, pp. 144-155.

Watkins, K.E., M. Horvitz-Lennon, L.B. Caldarone, L.R. Shugarman, B. Smith, T.E. Mannle, D.R. Kivlahan, and H.A. Pincus. "Developing Medical Record-Based Performance Indicators to Measure the Quality of Mental Healthcare." Journal of Healthcare Quality, vol. 33, no.1, 2011, pp. 49-66.

Watkins, K.E., D.J. Keyser, B. Smith, T.E. Mannle, D.R. Kivlahan, S.M. Paddock, T. Mattox, M. Horvitz-Lennon, and H.A. Pincus. "Transforming Mental Healthcare in the Veterans Health Administration: A Model for Measuring Performance to Improve Access, Quality, and Outcomes." Journal for Healthcare Quality, vol. 32, no. 6, 2010, pp. 33-42.

Wei, W., U. Sambamoorthi, M. Olfson, J.T. Walkup, and S. Crystal. "Use of Psychotherapy for Depression in Older Adults." American Journal of Psychiatry, vol. 162, no. 4, 2005, pp. 711-717.

Weissman, M.M., H. Verdeli, M.J. Gameroff, S.E. Bledsoe, K. Betts, L. Mufson, H. Fitterling, and P. Wickramarantne. "National Survey of Psychotherapy Training in Psychiatry, Psychology, and Social Work." Archives of General Psychiatry, vol. 63, no. 8, 2006, pp. 925-934.

Weisz, J.R., A. Jensen-Doss, and K.M. Hawley. "Evidence-Based Youth Psychotherapies Versus Usual Clinical Care: A Meta-Analysis of Direct Comparisons."American Psychologist, vol. 61, no. 7, 2006, pp. 671-689.

Weisz, J.R., A.M. Ugueto, D.M. Cheron, and J. Herren. "Evidence-Based Youth Psychotherapy in the Mental Health Ecosystem." Journal of Clinical Child and Adolescent Psychology, vol. 42, no. 2, 2013, pp. 274-286.

Whipple, J.L., and M.J. Lambert. "Outcome Measures for Practice." Annual Review of Clinical Psychology, vol. 7, 2011, pp. 87-111.

Wissow, L.S., J.D. Brown, K. Fothergill, A. Gadomski, K. Hacker, P. Salmon, and R. Zelkowitz. "Framing and Discussing Universal Mental Health Screening in Pediatric Primary Care: A Systematic Review." Journal of the American Academy of Child and Adolescent Psychiatry, in press.

Youn, S.J., D.R. Kraus, and L.G. Castonguay. "The Treatment Outcome Package: Facilitating Practice and Clinically Relevant Research." Psychotherapy, vol. 49, 2012, pp. 115-122.

View full report


"QualPsy.pdf" (pdf, 445.96Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®