Needs Assessment Methodologies in Determining Treatment Capacity for Substance Use Disorders: Environmental Scan Final Report. INTRODUCTION


Project Overview

In September 2017, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) contracted with the Human Services Research Institute to evaluate needs assessment methodologies for substance use disorder (SUD) treatment capacity. The overarching aim of the project is to enhance the effectiveness of needs assessment to promote organization and system change, addressing gaps in the existing data and promoting the implementation of evidence-based practices (EBPs), peer supports, and innovative technologies.

Technical Advisory Group (TAG)

ASPE requested the project be advised by a TAG. The group is composed of nine non-government experts and three government experts. It was first convened during a conference call in February 2018, during which the group members provided valuable feedback on the project's research questions as well as the approach and design of the environmental scan. The TAG plays an important role in the project and will continue to provide guidance throughout the project, including guiding the development of recommended best practices in SUD system capacity needs assessment and identifying data improvements for future assessments. This guidance will help form the final report for the project.

The project will result in:

  • A reference library of SUD needs assessments from the peer reviewed and grey literature as well as needs assessments from other fields (e.g., physical health or other specialty health conditions) with direct relevance for SUD service needs assessments.

  • A summary of best practices in various aspects of needs assessment methodology, including specific data sources and data analytic techniques to project capacity needs (by level of treatment, staffing type, and staffing patterns) with a focus on EBPs, recovery supports, and telehealth.

  • An overview of current practices in SUD needs assessments--along with recommendations to address the limitations of existing methods.

Extent of the Problem

The need for an effective needs assessment method for SUD treatment is evident. According to the Substance Abuse and Mental Health Services Administration (SAMHSA, 2017), in 2016:

  • An estimated 28.6 million people (10.6% of the population aged 12 and older) had used an illicit drug in the month before the survey.

  • An estimated 16.3 million people aged 12 and over were heavy alcohol users.

  • An estimated 15.1 million people aged 12 or older met Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) criteria for alcohol dependence or abuse.

  • An estimated 21.0 million people aged 12 or older, about one in 13, needed substance use treatment.

  • Approximately 3.8 million people, 1.4% of all people aged 12 or over and 10.6% of those needing treatment, received any substance use treatment in the past year.

According to SAMHSA data for 2015, of the adults who needed substance use treatment but did not receive specialty treatment, only 4.8% (863,000) felt that they needed treatment for their use of alcohol or illicit drugs. About 1.7% of this group made an effort to obtain treatment, and 3.7% felt they needed treatment but did not make an effort to obtain it (SAMHSA, 2016).

It is evident from these statistics that increasing the number of people recognizing their need for treatment is a major public health issue with important implications for assessing system capacity. Increasing our understanding of how to conduct SUD treatment needs assessment may help communities, states, and the Federal Government ensure that we have adequate treatment capacity.

Research Questions

The project is guided by the following research questions regarding current methods and practices in the field of SUD treatment needs assessment, as specified in the Statement of Work (SOW), with several questions suggested by the project team.

  1. How have needs assessments with respect to SUD treatment capacity been conducted in the past? Which populations, workers, settings, and levels of care have been incorporated into the assessments? What data sources and methods have been used? What questions were the assessments meant to answer?

  2. Are there needs assessments from other provider types (for example, primary care physicians) that could be applied to the SUD treatment workforce? What are the differences between other workforces and the SUD workforce that might make such application difficult? How could these difficulties be overcome?

  3. Are there best practices with respect to these needs assessments? What data and methods most effectively gauge treatment needs and current gaps? What questions should be answered in conducting these needs assessments?

  4. How does American Society for Addiction Medicine (ASAM) Levels of Care relate to capacity needs (both in workforce and setting)? How can best staffing practices, telehealth policies, and peer supports be incorporated into needs assessments?

  5. What are the key pieces of missing data that would improve these needs assessments, including one-time data collection efforts and potential federal efforts?

  6. Have any of these needs assessments methods been validated? If not, how can these needs assessment methods be validated?

  7. How can needs assessment approaches be made more uniform while remaining adaptable to local conditions and stakeholder priorities?

  8. How can assessments become more effective at promoting organizational and system change to address, rather than simply identify, system gaps?

  9. How do states and managed care plans operationalize network adequacy standards?

Approach of the Environmental Scan

Steps in the environmental scan were:

  1. Developing a Workplan, reviewed by the Contracting Office Representative (COR) and TAG, specifying tasks and research questions (six from the SOW, with an additional two presented by our team and one added during the Post-Award meeting).

  2. Developing a Protocol, also reviewed by the COR and TAG, for retrieving and systematically reviewing a sample of SUD needs assessment reports based on the subset of research questions related to content and practices in the field.

  3. Reviewing recent literature on general needs assessment methodology and selected articles on needs assessment specific to SUD treatment capacity as background for reviewing SUD needs assessment reports.

  4. Formulating an operational definition of "SUD network capacity" based on the methodology literature and incorporating feedback from the TAG.

  5. Conducting a search of SUD needs assessment in the grey and published literature using the search strategy according to the protocol and described in in the Methods section of this report.

  6. Scanning needs assessment reports identified according to the search strategy to determine eligibility for inclusion and retrieving those meeting inclusion criteria to create a convenience sample for review.

  7. Sorting retrieved reports into two categories defined post-hoc based on the results of the search: "mandated" needs assessments (usually those produced in response to some funding requirement, which address SUD among other public health issues) and "locally initiated" needs assessments, typically one-time studies conducted in connection with a state or county SUD policy initiative.

  8. Conducting an overview of the reports and various sources of guidance within the sample of mandated needs assessments to identify typical features and methods.

  9. Conducting a fine-grained analysis of the sample of locally initiated needs assessments reports to identify content and quality, using the data extraction tool based on the research questions and presented in the Protocol.

  10. Synthesizing findings from the review of locally initiated reports in tabular and narrative form.

  11. Conducting a separate scan addressing the research question related to how states operationalize network capacity (added in the Post-Award meeting).

  12. Identifying examples to address the research question about how needs assessments from other provider types might be adapted to SUD treatment.

  13. Producing this report incorporating the above steps.

Organization of this Report

The report is generally organized according to the research questions from the SOW. These address a diverse array of topics that generally consist of two types: questions related to current practices in the field (e.g., "What data sources and methods have been used?") and questions that call for recommendations (e.g., "Are there needs assessments from other provider types that could be applied to the SUD treatment workforce?"). This report on the environmental scan focuses primarily on the research questions related to practices; however, it includes a preliminary discussion of recommendations. A more detailed discussion of recommendations will be provided in the final report.

The first section of this report consists of a background section with a review of the literature on needs assessment methodology in general and the literature specific to methodology of SUD needs assessment. This is not intended to be a systematic review of literature in the field but rather to clarify the concepts of "needs assessment" and "network capacity" as a framework for addressing the research questions.

The review of needs assessment methodology is followed by a review of a set of SUD treatment needs assessment reports retrieved from the published and grey literature utilizing modified methods of systematic literature reviews. Two types of reports are reviewed. The first, termed "mandated" reports, are those which are conducted by states as a requirement of various federal grant programs or by public health agencies for purposes of accreditation. These generally address SUD among a range of other public health issues. For this type, we provide a general overview of the content and methodology. The second type, termed "locally initiated," are needs assessments specific to SUD conducted on an ad hoc basis--usually in connection with some policy initiative. We review these in more detail, using a protocol for data extraction based on the research questions. Information thus obtained is presented in tabular and narrative form. The protocol is provided in Appendix E.

Next, we address the research question, "How do states and managed care operationalize network adequacy?" which was conducted as a separate scan using different information sources (primarily policy documents).

The next section addresses another single research question, about whether needs assessments for other provider types might be adapted to SUD treatment, by discussing several examples. This discussion will be expanded in the final report.

Finally, we provide a preliminary response to questions that call for recommendations, which will be expanded in the final report, with input from the TAG and the COR.