Needs Assessment Methodologies in Determining Treatment Capacity for Substance Use Disorders: Final Report

Needs Assessment Methodologies in Determining Treatment Capacity for Substance Use Disorders: Final Report

Human Services Research Institute

September 2019

Printer Friendly Version in PDF Format (49 PDF pages)

---

ABSTRACT

This project evaluated needs assessment methodologies for substance use disorder (SUD) treatment capacity, highlighting best practices and identifying gaps and opportunities for improvement. In 2017, 7.6% of the population age 12 or older needed treatment for an SUD treatment, as defined by the National Survey of Drug Use and Health; however only 1.5% of the population received any substance use treatment in the past year. Moreover, 94.3% of the individuals identified as needing substance use treatment did not perceive a need for treatment. A needs assessment can help close some of the treatment gap, by pinpointing service system and workforce needs that can help best address the need for treatment. An environmental scan conducted for the project found that SUD treatment system needs assessments are conducted widely, for purposes of state and local policy initiatives or in conjunction with federal grant programs. A review of a sample from diverse geographical areas indicated that a wide variety of topics such as specific substances, sub-populations, and system issues, are addressed. Data sources typically include a combination of federal and local sources of prevalence and utilization data along with qualitative information obtained from focus groups and stakeholder interviews. Needs assessment of SUD treatment capacity is faced with a number significant challenges including the complexity of the treatment system with multiple provider types, settings and treatment modalities, which lack standard definitions. Additionally, there is no standard definition or measurement of need for treatment. This study found that further research needs to be undertaken to uncover the relationship between prevalence of SUD and the need for specific types of services, as well as the staffing levels needed to support these services. While the American Society of Addiction Medicine Levels of Care Criteria can be used as a standard of a continuum of care, it is not yet in full use in SUD treatment capacity planning. In addition, while some federal data sources are available to answer pressing questions about the current capacity of the system, they are incomplete. This study concluded that more research will be necessary to standardize and validate needs assessment methodologies.

This report was prepared under contract #HHSP233201600015I between HHS's ASPE/DALTCP and the Human Services Research Institute. For additional information about this subject, you can visit the DALTCP home page at https://aspe.hhs.gov/office-disability-aging-and-long-term-care-policy-daltcp or contact the ASPE Project Officer, Judith Dey, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. Her e-mail address is: Judith.Dey@hhs.gov.

DISCLAIMER: The opinions and views expressed in this report are those of the authors. They do not reflect the views of the Department of Health and Human Services, the contractor or any other funding organization. This report was completed and submitted on December 2018.

---

 

TABLE OF CONTENTS

ACRONYMS

EXECUTIVE SUMMARY

INTRODUCTION AND BACKGROUND

• Project Overview
• Project Goals
• Technical Advisory Group
• Environmental Scan

CURRENT PRACTICES IN THE FIELD

• Questions and Topics Addressed by Substance Use Disorder Treatment System Needs Assessments
• Methods Used in Needs Assessments
• Data Sources Used in Substance Use Disorder Treatment System Needs Assessment
• Validation of Substance Use Disorder Needs Assessment Methods

OPPORTUNITIES FOR FURTHER RESEARCH AND ENHANCEMENT

• Data and Knowledge Gaps and Limitations
• Addressing Knowledge Gaps
• Enhancing the Utility of Substance Use Disorder Needs Assessments
• Improving Effectiveness at Promoting Organizational and System Change
• Conclusions

NOTES

APPENDICES

• APPENDIX A: References
• APPENDIX B: Technical Advisory Group Members
• APPENDIX C: Environmental Scan Report

 

ACRONYMS

The following acronyms are mentioned in this report and/or appendices.

ACA	Affordable Care Act
ACF	HHS Administration for Children and Families
AIDS	Acquired Immune Deficiency Syndrome
ASAM	American Society for Addiction Medicine
ASPE	HHS Office of the Assistant Secretary for Planning and Evaluation
BHSIS	Behavioral Health Services Information System
BLS	Bureau of Labor Statistics
BRFSS	Behavioral Risk Factor Surveillance System
CAPT	SAMHSA Center for the Application of Prevention Technologies
CAST	Calculating for an Adequate System Tool
CBHSQ	Center for Behavioral Health Statistics and Quality
CDC	HHS Centers for Disease Control and Prevention
CFR	Code of Federal Regulations
CHA	Community Health Assessment
CHI	Community Health Improvement
CHNA	Community Health Needs Assessment
CMS	HHS Centers for Medicare & Medicaid Services
CNA	Community Needs Assessment
COD	Co-Occurring Disorder
COR	Contracting Office Representative
CSHS	Colorado Social Health Survey
DAWN	Drug Abuse Warning Network
DHCS	California Department of Health Care Services
DHHS	North Carolina Department of Health and Human Services
DSM-IV	Diagnostic and Statistical Manual of Mental Disorders, 4th edition
DSRIP	Delivery System Reform Incentive Program
EBP	Evidence-Based Practice
FPL	Federal Poverty Level
FTE	Full-Time Equivalent
FY	Fiscal Year
GAO	U.S. Government Accountability Office
HHS	U.S. Department of Health and Human Services
HIV	Human Immunodeficiency Virus
HPSA	Health Professional Shortage Area
HRSA	HHS Health Resources and Services Administration
HUD	U.S. Department of Housing and Urban Development
ICF	International Classification of Functioning, Disability and Health
IMU	Index of Medical Underservice1
IOM	Institute of Medicine
IRB	Institutional Review Board
IRS	Internal Revenue Service
LGBT	Lesbian, Gay, Bisexual, Transgender
LGBTQ	Lesbian, Gay, Bisexual, Transgender, Queer and/or Questioning
LGBTQA	Lesbian, Gay, Bisexual, Transgender, Queer and/or Questioning, Asexual and/or Ally
LME-MCO	Local Management Entity-Managed Care Organization
MAPP	Mobilizing for Action through Planning and Partnerships
MAT	Medication-Assisted Treatment
MCO	Managed Care Organization
MDS	Minimum Data Set
MeSH	Medical Subject Headings
MHBG	Community Mental Health Services Block Grant
MIS	Management Information System
MUA/P	Medically Underserved Areas/Populations
N-MHSS	SAMHSA National Mental Health Services Survey
N-SSATS	SAMHSA National Survey of Substance Abuse Treatment Services
NACCHO	National Association of County and City Health Officials
NACT	Network Adequacy Certification Tool
NAMCS	CDC National Ambulatory Medical Care Survey
NAPHD	National Association of Public Health Directors
NCHS	HHS National Center for Health Statistics
NHCS	CDC National Hospital Care Survey
NIDA	HHS National Institute of Drug Abuse
NIMH	HHS National Institute of Mental Health
NPHAB	National Public Health Accreditation Board
NSDUH	HHS National Survey on Drug Use and Health
NY CNA	New York Community Needs Assessment
Opioid STR	State Targeted Response to the Opioid Crisis Grant
OTP	Opioid Treatment Program
PHAB	Public Health Accreditation Board
PPS	Performing Provider Systems
RSA	Rehabilitation Services Administration
SAMHSA	HHS Substance Abuse and Mental Health Services Administration
SAPTBG	Substance Abuse Prevention and Treatment Block Grant
SIG	State Incentive Grant
SOW	Statement of Work
SPF	Strategic Prevention Framework
SSR	State Sampling Region
SUD	Substance Use Disorder
TAG	Technical Advisory Group
TEDS	Treatment Episode Data Set
VA	U.S. Department of Veterans Affairs
WHO	World Health Organization
WICHE	Western Interstate Commission for Higher Education
YRBSS	Youth Risk Behavior Surveillance System

 

EXECUTIVE SUMMARY

Introduction and Background

In September 2017, the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation contracted with the Human Services Research Institute to evaluate needs assessment methodologies for substance use disorder (SUD) treatment capacity. The main goal identified in the Statement of Work is to highlight best practices for conducting needs assessments and to identify gaps and opportunities for improvement in the available data. The project had two main activities: (1) conducting an environmental scan to identify current practices in the SUD needs assessment field; and (2) convening a Technical Advisory Group (TAG) composed of nine non-government experts and three government experts to identify opportunities for advancing the field of SUD treatment needs assessment.

The Importance of Substance Use Disorder Treatment System Needs Assessment

Effective needs assessment methods help narrow SUD treatment capacity gaps. In 2017, 7.6% of the population age 12 or older had a need for substance use treatment in the past year (as defined by the National Survey on Drug Use and Health, or NSDUH); however, only 1.5% of the population received any substance use treatment in the past year (SAMHSA, 2018a). Moreover, 94.3% of the individuals identified as needing treatment based on the NSDUH criteria did not perceive a need for treatment (SAMHSA, 2018a). This large gap between the number of persons identified as having an SUD and the number who perceive a need for treatment is an important public health issue with major implications for assessing system capacity. Increasing our ability to explore these issues will help communities, states, and the Federal Government better plan for treatment capacity.

Research Questions

The research questions that guided the environmental scan are summarized as follows:

1. How have SUD treatment system needs assessments been conducted in the past?

2. Are there needs assessments from other provider types that could be applied to the SUD treatment workforce?

3. Are there best practices with respect to these needs assessments?

4. How does the American Society for Addiction Medicine (ASAM) Level of Care Criteria relate to capacity needs (both in workforce and setting)?

5. What are the key pieces of missing data that would improve these needs assessments?

6. Have any of these needs assessments methods been validated?

7. How can needs assessment approaches be made more uniform while remaining adaptable to local conditions and stakeholder priorities?

8. How can assessments become more effective at promoting organizational and system change to address, rather than simply identify, system gaps?

9. How do states and managed care plans operationalize network adequacy standards?

To address these, we reviewed a convenience sample of approximately 40 needs assessment reports, selected to represent a variety of issues addressed, geographical settings (e.g., rural or urban), populations and types of sponsoring organizations. Within this sample we distinguished between two general categories: the first, which we call "mandated" assessments, are conducted to fulfill various kinds of program requirements. The needs assessment that Substance Abuse and Mental Health Services Administration (SAMHSA) requires states to complete for their Substance Abuse Block Grant application is one example. The second type, which we call "locally initiated" assessments, are ad hoc studies conducted for specific policy purposes, such as consideration for increased funding.

Definition of Needs Assessment

To establish a conceptual framework for reviewing current practices in SUD needs assessment, we reviewed the recent needs assessment methodology literature. By general consensus, needs assessment is defined in the literature (for example, Altschuld & Watkins, 2014), as having three components:

1. Specification of a current condition ("what is"; optimally this is quantified--for example, the prevalence of SUD in a population or the number of deaths due to overdose in a year).

2. Specification of a desired state or result ("what should be"; for example, reduction in prevalence or deaths due to overdose, optimally by a specified amount).

3. Recommendations or strategies for closing the discrepancy between the "what is" and the "what should be" conditions.

"Need" in the context of methodology is understood to be the discrepancy between the current and desired states. The literature emphasizes the importance of distinguishing between needs and solutions, the latter of which properly belong in the recommendations component. For example, we may say "We need to reduce the number of deaths due to overdose by some target amount," and this would conform with the formal definition of a needs assessment; but if we say instead "We need to expand medication-assisted treatment (MAT) capacity," we are making a recommendation rather than identifying a need. The point is, there may be many possible solutions to a need--in this example, expanding MAT capacity and/or expanding prevention programs, distributing Naloxone, or implementing prescription registries, etc. The problem with confusing a solution with need is that it precludes consideration of other possible solutions, some of which may be more effective, efficient, or feasible.

Challenges in Conducting Substance Use Disorder Treatment Needs Assessments

SUD treatment presents two general challenges for conducting needs assessments. The first is the complexity of the treatment system and the second is gaps in the available data and knowledge. The boundaries of the treatment system are diffuse, requiring decisions about the scope of the assessment: will it include only specialty treatment, or also primary care, self-help groups, or faith-based programs? Additionally, patterns of substance use are variable, based on types of substances, geographical and demographic distribution, and the trajectories of individuals' SUD.

Gaps in the available data and knowledge are related to this complexity. There is no single source of information for all the locations of SUD treatment, nor are there standard classifications for the job categories in the SUD workforce; these gaps make it difficult for researchers to assess workforce capacity. Knowledge gaps include, most importantly, the lack of understanding of different types or degrees of need. Additionally, the tools for determining the most appropriate type of treatment for an individual have only recently become available in the form of the ASAM Criteria clinical decision support system, and these are still in the process of development for purposes of system-level needs assessment.

Conclusions

The following are the conclusions reached in the review of a sample of locally initiated and mandated SUD needs assessment reports and feedback from the TAG.

Methods for Measuring Need for Treatment

The environmental scan revealed both variability and challenges related to defining need for SUD treatment in a population. The NSDUH, which is widely used as a data source in both locally initiated and mandated needs assessments, defines need for treatment quite broadly, as a combination of a Diagnostic and Statistical Manual of Mental Disorders (4th edition) SUD diagnosis and use of SUD treatment services. This definition equates need for treatment with prevalence and service use. Using prevalence as a measure of need is problematic, however, because many people diagnosed with an SUD recover without treatment, and many do not perceive a need for treatment for various reasons (they prefer to obtain support elsewhere than the formal treatment system, their substance use does not cause a high level of distress, etc.). A planning model that does not take these factors into account would result in oversupply. Despite this shortcoming, many reports, both locally initiated and mandated, simply used prevalence as a measure of need for treatment in the population.

Methods for Measuring System Capacity

One seemingly simple method for measuring capacity would be to conduct an inventory of existing services. In practice, however, this is challenging due to the complexity of treatment systems, which makes it difficult to define system boundaries, service types, and treatment modalities. Because there is no single source of information for any of these features, it is usually necessary to query each program individually, making this approach excessively burdensome. Usually there is no single source of information for any of these features.

A formula that specifies the amount of various services required based on population size would be a way to assess system capacity. Such formulas have yet to be developed, however, and none of the reports reviewed in the environmental scan attempted this approach. There are a number of challenges for developing such formulas, including local variability in SUD patterns and existing assets, the complexity of the behavioral health system, increased privatization, and uncertainty over the relative effectiveness of different treatment modalities.

One potential approach for measuring capacity involves the network capacity standards required by the Centers for Medicare & Medicaid Services for Medicaid managed care organizations and other programs. Standards include time and distance to reach services and timely access. Yet, these standards have several limitations for the purpose of needs assessments; most importantly, they do not factor in research on the appropriateness of treatments for various conditions.

Another potential method is use of the ASAM Levels of Care Criteria (Mee-Lee, 2013). Developed as a clinical decision support tool designed to match patients to appropriate levels of care, the ASAM Levels of Care are increasingly recognized as a standard for defining a comprehensive continuum of care against which existing systems can be measured. It is likely that this will become more common in the near future.

Data Gaps

A primary goal of the project was to identify gaps in the currently available sources of information for conducting SUD needs assessments, particularly federal data sources. NSDUH and the Treatment Episode Data Set (TEDS) are primary sources of data on need and receipt of treatment, and the National Survey of Substance Abuse Treatment Services (N-SSATS) is a primary source on the number and type of programs; all have limitations. With NSDUH, the broad definition of need for treatment (diagnosis or use of services) does not provide for the kind of targeted program planning and policy making that is needed to use resources most effectively. TEDS and N-SSATS do not include the entire population; moreover, the quality of data, which is submitted by the states, is subject to limitations of state data systems.

Knowledge Gaps

In addition to improvements in the available data, SUD needs assessment as a field would benefit from further research in several areas. The most important of these are: (a) improvement in ways to capture the structure and capacity of the SUD workforce; (b) identification of subgroups in the SUD population on the basis of type of substance use, degree of impairment, and variations in treatment and recovery trajectories; (c) further research on the use of social indicators for estimating population-level need;[1] and (d) a fuller understanding of how SUD fits within the broader framework of population health, in order to align SUD needs assessment with current large-scale initiatives such as Healthy People 2020.

Enhancing the Utility of Needs Assessment for Effecting System Change

The most fundamental purpose of a needs assessment related to a SUD treatment system is to direct positive change in how SUDs are addressed. Given the key function of needs assessment to serve as a guide for planning and policy making, the recommendations section of reports should receive careful attention and ideally should include priorities and strategies for acting on those recommendations. These should incorporate principles from the rapidly growing field of implementation science to support action on recommendations. Though technically challenging, cost effectiveness analyses comparing different options are highly valued by policy makers.

Currently, most SUD needs assessments, unlike other forms of social research, are conducted autonomously with little oversight, little external review of methodology, and little assessment of reliability and validity. In fact, most of the efforts to validate needs assessment methodology were supported by federal agencies decades ago. The most extensive of these was conducted in the 1980s by researchers in Colorado, comparing several methods currently in use (Tweed, et al. 1992).

The quality and impact of SUD needs assessment could be enhanced by providing the field with the type of organizational infrastructure support that available for Community Health Needs Assessments and Community Health Assessments to promote best practices and standardize methodologies. This support might also consist of fostering of learning communities or increased participation in existing forums, such as the American Evaluation Association Needs Assessment Topical Interest Group.

 

INTRODUCTION AND BACKGROUND

Project Overview

In September 2017, the U.S. Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation (ASPE) contracted with the Human Services Research Institute and the Technical Assistance Collaborative to evaluate needs assessment methodologies for substance use disorder (SUD) treatment capacity under ASPE/Agency for Healthcare Research and Quality/Office of the National Coordinator of Health Information Technology Indefinite Delivery/Indefinite Quantity Request for Task Order Proposal (Solicitation) #17-233-SOL-00770.

Project Goals

The main goal identified in the Statement of Work (SOW) is to highlight best practices for conducting needs assessments, including staffing patterns, use of evidence-based practices and the incorporation of telehealth and peer supports, and to identify gaps and opportunities for improvement in the available data.

Technical Advisory Group

The project benefited from the advice of a Technical Advisory Group (TAG) composed of three experts from Federal Government agencies and nine experts from the field at large. It was first convened during a conference call in February 2018; at that time, the group members provided valuable feedback on the project's research questions as well as the approach and design of the environmental scan conducted as part of the project. The TAG continued to provide guidance throughout the project, including guiding the development of recommended best practices in SUD system capacity needs assessment and identifying data improvements for future assessments.

Environmental Scan

The first phase of the project involved an environmental scan consisting of: (1) reviewing the recent literature on needs assessment methodology; and (2) obtaining and reviewing a sample of SUD needs assessment reports from the published and gray literature. These reports were reviewed with reference to the following research questions. Questions 1-6 were specified in the SOW; Questions 7-9 were additionally proposed by the project team.

1. How have needs assessments with respect to SUD treatment capacity been conducted in the past? Which populations, workers, settings and levels of care have been incorporated into the assessments? What data sources and methods have been used? What questions were the assessments meant to answer?

2. Are there needs assessments from other provider types (for example, primary care physicians) that could be applied to the SUD treatment workforce? What are the differences between other workforces and the SUD workforce that might make such application difficult? How could these difficulties be overcome?

3. Are there best practices with respect to these needs assessments? What data and methods most effectively gauge treatment needs and current gaps? What questions should be answered in conducting these needs assessments?

4. How does American Society for Addiction Medicine (ASAM) level of care relate to capacity needs (both in workforce and setting)? How can best staffing practices, telehealth policies and peer supports be incorporated into needs assessments?

5. What are the key pieces of missing data that would improve these needs assessments, including one-time data collection efforts and potential federal efforts?

6. Have any of these needs assessments methods been validated? If not, how can these needs assessment methods be validated?

7. How can needs assessment approaches be made more uniform while remaining adaptable to local conditions and stakeholder priorities?

8. How can assessments become more effective at promoting organizational and system change to address, rather than simply identify, system gaps?

9. How do states and managed care plans operationalize network adequacy standards?

The questions generally fall into one of two categories: (1) questions about current practices in the field of SUD needs assessment; and (2) questions about the potential for further research and activities that would improve the practice of SUD needs assessment (addressing gaps in the available data, promoting more rigorous methodology, etc.). The "current practices" category questions inquire into the questions asked within the assessments, data sources and methods used, best practices identified, and how the ASAM Criteria relate to capacity assessment, among other elements. The "practice improvement" category examines validation of methods, whether approaches can be adopted from other provider types, and how the utility of the assessments can be improved through uniformity; it also calls for recommendations for improving the quality and utility of needs assessment.

To address the research questions, we obtained and reviewed a convenience sample of approximately 40 needs assessment reports. These were selected to represent diversity in several respects: geographical (region of the country, rural and urban), scale (local areas and entire states), authors (academic researchers and policy centers, government agencies and private consulting companies), publication type (peer reviewed journals, gray literature). We focused on reports produced more recently, though we included several important academic reports from the 1980s and 1990s. The reports were reviewed using a template to extract details related to the research questions. This information is presented in the environmental scan report (Appendix A) and summarized in this report.

Why is SUD Treatment System Needs Assessment Important?

Effective needs assessment methods help narrow SUD treatment gaps. This is particularly important due to the unprecedented levels of fatalities from SUD. In 2017, 7.6% of the population age 12 or older had a need for substance abuse treatment in the past year, as defined by the National Survey on Drug Use and Health (NSDUH); however, only 1.5% of the population received any substance abuse treatment in the past year (SAMHSA, 2018a). There are likely multiple reasons why so few people who are identified as needing treatment actually receive it, some of which are identified by NSDUH data: individuals' not perceiving a need for treatment, inability to afford services, and unavailability of services.

Increasing our ability to explore the reasons why more individuals with SUD do not receive treatment will help communities, states, and the Federal Government better plan for treatment capacity.

Definition of Terms

WHAT IS A NEEDS ASSESSMENT?

Needs assessment emerged as a distinct branch of program evaluation in the 1970s and has since evolved to address a broad scope of social issues. Despite this broad application, the field has generally settled on a fairly standard definition of what constitutes a needs assessment as distinct from other types of evaluation and research. This definition begins with what White & Altschuld (2012) describe as "the classic definition of need: the discrepancy between the normative or 'current' condition and the optimal or 'what should be' state"--a formulation that is often condensed to the gap between "what is" and "what should be."

Operational Definition Based on the literature review conducted in the environmental scan, a needs assessment was operationally defined for purposes of reviewing needs assessment reports as consisting of three components: A measurement of the current condition ("what is"). A specification of the optimal state ("what should be"). Recommendations, ideally prioritized, for actions to address the gap.

For the essential fourth component as stated by Altschuld & Watkins (2014), "Needs assessment also includes making judgments with regard to needs and putting them into prioritized order to guide decisions about what to do next."

This orientation to action is emphasized by the definition provided by Watkins, West Meiers, & Visser (2012; p. 19): "A needs assessment is simply a tool for making better decisions." It is this component of providing guidance for policy decisions that most sharply differentiates needs assessment from other forms of inquiry such as epidemiological studies and services research.

The essential point in this conceptual framework is that "need" is distinct from "want" or "solution." In everyday conversation I may say "I need a car." In the context of a needs assessment, however, that need may be defined as the gap between the time that I usually get to work (what is) and the time that I am expected to be at work (what should be)--and the statement "I need a car" would be a possible solution or recommendation. The distinction between need and solution is emphasized by the methodological literature. Offering a solution as the definition of need precludes the possibility that other, better solutions may be recommended--in this example, perhaps moving closer to work, taking the bus, carpooling, etc.

A Hypothetical Example A community has experienced a spike in the number of deaths due to heroin overdose and decides to apply for a grant to address the problem. The "what is" condition consists of the prevalence of heroin addiction in the community and the resources that are currently available to address this problem. The need in this case may be defined as "reduction in the number of people in the community who overdose by injecting heroin." The "what should be" condition is the desired degree of reduction in fatalities--for example, whatever reduction may be feasibly achieved, a 10% reduction, complete elimination, etc. The solutions/recommendations component could include a number of possibilities: prevention programs, increased access to medication-assisted treatment (MAT), naloxone, fentanyl testing strips, supervised injection sites, needle exchange programs, heightened law enforcement, etc. Optimally, the needs assessment will prioritize these options, suggest strategies for implementing them and perhaps costs, to be weighed in system planning and policy decisions. The needs assessment collected information about each of these options--the current availability, community opinion, experiences in other communities, etc. Feedback in community meetings indicated that supervised injection sites were unacceptable and testing strips were highly controversial. Needle exchange programs were rejected as they are excluded from block grant funding. The review of the current system demonstrated that the community already has a robust grant-funded prevention program and a nascent naloxone program, but there were long wait lists for MAT treatment. Considering these options, policy makers settled on allocating the largest part of available resources to expanding MAT treatment capacity, while investing a smaller part in expansion of the naloxone program. By weighing all possible options in the light of findings from the needs assessment, policy makers succeeded in investing resources where there was the greatest need and avoided both duplication of existing services and implementing programs that would be unacceptable to the community. NOTE: This is an example of a narrowly focused needs assessment; others may be much broader both in area and scope--for example, focusing on the state's entire SUD treatment capacity. The basic principles of this approach would be the same, however.

 

CURRENT PRACTICES IN THE FIELD

In the environmental scan, we identified two types of needs assessments related to SUD treatment capacity. The first type, which we call "mandated," consists of needs assessments conducted to fulfill program requirements by federal agencies or national organizations. Among these are the Substance Abuse and Mental Health Services Administration (SAMHSA) discretionary grants that require grantees to implement the Strategic Prevention Framework (SPF), which identifies needs assessment as the first step in prevention planning, and the Substance Abuse Block Grant (SABG) program. The other two programs address substance use as one of a number of public health issues. These are Community Health Needs Assessments (CHNAs), which the Internal Revenue Service (IRS) requires of all non-profit hospitals every three years, and Community Health Assessments (CHAs), which are conducted by state or county public health agencies as a requirement for accreditation by the National Public Health Accreditation Board (PHAB).

The second type of report, which we call "locally initiated," is typically the product of a standalone initiative conducted for a specific policy purpose, often by a county or state, or a single research project.

Questions and Topics Addressed by Substance Use Disorder Treatment System Needs Assessments

At the most general level, the two types of assessments addressed similar areas of focus: (1) What are the substance use treatment needs for a given population? (2) What services are available to meet those needs? (3) What needs are not being met because the appropriate services are not available? (4) What should be done to increase the availability of those services? However, within each of these areas there is great latitude in how these questions are framed and the details on which they focus.

Mandated assessments typically follow the specific guidance and requirements from authorizing organizations whereas locally initiated assessments showed greater diversity in the type and scope of questions they addressed and the way those questions were framed.

Questions Addressed by Mandated Needs Assessments

SAMHSA administers several grant programs that entail SUD treatment needs assessments. Two of the largest are the SABG program and the SPF State Incentive Grant program.

QUESTIONS ADDRESSED BY SABG NEEDS ASSESSMENTS

SAMHSA requires states applying for these grants to include a needs assessment as part of their application. The questions addressed by these needs assessments are determined by SAMHSA's specification (as directed by the authorizing legislation) of certain populations and service areas as targets for SABG funds: pregnant women and women with dependent children, intravenous drug users, tuberculosis services, early intervention services for HIV/AIDS, and primary prevention service.

Rather than using formulas for service system capacity needs, most of the mandated needs assessments reviewed in the environmental scan instead refer to different types of services without identifying the specific capacity. This makes quantifying the gap infeasible. The current (FY 2018-2019) SABG block grant application calls for applicants to "identify the unmet service needs and critical gaps within the current system" and asks that they use a "data-driven approach," including data from other state agencies that provide services. However, the block grant applications reviewed as part of this environmental scan provided only the most cursory information about unmet needs and service gaps. While the review was not exhaustive (see Appendix C for details), these perceptions were supported by the TAG as well.

The selected reports measured unmet needs primarily by comparing national prevalence and treatment rates with state rates for the target populations specified by SAMHSA (e.g., pregnant women with children, persons who inject drugs, etc.). While the applications typically mentioned SUD treatment workforce shortages as a challenge, and some identified planned activities to increase the workforce, they contained very little actual detail about the nature and distribution of the SUD workforce.

QUESTIONS ADDRESSED BY STRATEGIC PREVENTION FRAMEWORK NEEDS ASSESSMENTS

SPF implemented by multiple SAMHSA grant programs is a five-stage process that starts with needs assessment, followed by capacity building, strategic planning, implementation, and evaluation. As of January 2017, all 50 states, eight jurisdictions, 19 tribes, and hundreds of community-based organizations had received at least one SAMHSA grant that requires the implementation of the SPF process.

SAMHSA suggests that grantees implementing the SPF address the following questions in conducting their needs assessments:

• What substance use problems (for example, overdoses and alcohol poisoning) and related behaviors (for example, prescription drug misuse and underage drinking) are occurring in your community of focus?

• How often are these problems and related behaviors occurring?

• Where are these substance use problems and related behaviors occurring (for example, at home or in vacant lots; in small groups or during big parties)?

• Who is experiencing more of these substance use issues and related behaviors? For example, are they males, females, youth, adults, or members of certain cultural groups?

QUESTIONS ADDRESSED BY COMMUNITY HEALTH ASSESSMENTS AND COMMUNITY HEALTH NEEDS ASSESSMENTS

These mandated needs assessments conducted by non-profit hospitals/health care systems and public health agencies, respectively, address SUD as one among many health issues. Requirements for CHAs are established by the national PHAB, the development of which was initially sponsored by the Robert Wood Johnson Foundation. The requirements and guidelines for CHNAs were developed initially by the IRS and elaborated in the Affordable Care Act (ACA). In recent years the IRS and the PHAB have collaborated to provide a combined approach; as a result, increasing numbers of needs assessments are being conducted jointly by health systems and public health agencies.

Most CHAs are guided by a strategic planning framework recommended by the PHAB. The framework, known as Mobilizing for Action through Planning and Partnerships (MAPP), determines the type of questions that are addressed. The needs assessment component of MAPP combines four different areas of focus:

1. A Community Themes and Strengths Assessment that identifies themes that interest and engage the community, perceptions about quality of life, and community assets.

2. A Local Public Health System Assessment that measures the capacity of the local public health system to conduct essential public health services.

3. A Community Health Status Assessment that analyzes data about health status, quality of life, and risk factors in the community.

4. A Forces of Change Assessment that identifies forces that are occurring or will occur that will affect the community or the local public health system.

IRS requirements, as expanded by the ACA effective in 2015, are broader than the requirements for CHAs in terms of the types of questions that should be addressed. The ACA expanded the types of health needs that hospital facilities may address, from the IRS's initial focus on improving access to care. Facilities are now asked to address other public health objectives and social determinants of health, such as the need to "prevent illness, to ensure adequate nutrition, or to address social, behavioral, and environmental factors that influence health in the community". This broader conception of need encompasses many of the social factors correlated with the prevalence of SUD and has been incorporated into several of the reports reviewed for the environmental scan, for example a discussion of poverty as a contribution to behavioral health issues in Native American communities. It is also represented in some large-scale policy initiatives that are relevant to SUD such as Healthy People 2020.

Questions Addressed by Locally Initiated Needs Assessments

Locally initiated needs assessments, in contrast to those that are mandated, are generally conducted for more specific policy purposes, such as to advocate for additional funding or to prioritize the allocation of funding that has become available. Accordingly, the questions they ask are typically more narrowly focused, designed to address specific policy issues. For example, they may seek to identify the prevalence of opioid misuse in a specific community and the services available and needed to treat it, or they may be looking to identify rates of SUD in the LGBTQ population and the availability and need for specialized services.

At the same time, when viewed in totality, the local assessments address a wider range of policy and public health issues and include a wider variety of question types than the mandated assessments.

Among the most common types of questions they ask are:

• Questions about need and treatment capacity for particular population subgroups.

• Questions about need and treatment capacity for specific substances.

• Questions about need and treatment capacity of specific services (e.g., medication-assisted treatment).

• Questions related to workforce capacity alone (without any other capacity issues).

• Questions related to the potential impact of trends and policy initiatives on demand and system capacity (e.g., Medicaid expansion).

• Questions related to accessibility and quality, as well as capacity, of services.

Less frequently, the reports we reviewed assessed process and outcomes-related quality issues such as the use of evidence-based treatments, outcomes measurement, adherence, and patient satisfaction.

Some important topics related to SUD treatment received relatively limited attention. For example, only a handful of reports addressed network strength and asset topics (e.g., exceptional training programs, advanced data systems, etc.), as recommended in the recent methodology literature. Other topics that receive little attention in locally initiated needs assessments are treatment for individuals in the criminal justice system, disparities such as disproportionate unmet needs in minority populations, benefit design as a factor influencing capacity, peer involvement as an aspect of workforce capacity, and the use of technology and evidence-based practices.

Methods Used in Needs Assessments

In this section, we discuss the data and methods for addressing the "what is" and "what should be" components of a needs assessment.

Methods for Measuring Individuals' Need for Treatment

The way "need for treatment" is defined influences the entire process and the results of a needs assessment. SAMHSA's definition of "need for treatment" in the context of the NSDUH is one more commonly used. Respondents are classified as needing alcohol use treatment, illicit drug use treatment, or substance use treatment (i.e., treatment for an illicit drug or alcohol use problem) if they met the criteria of one of these conditions as defined in the Diagnostic and Statistical Manual of Mental Disorders, DSM-IV (4th edition; American Psychiatric Association, 1994) or they received treatment for one of these conditions at a specialty facility--that is, a drug and alcohol rehabilitation facility (inpatient or outpatient), hospital (inpatient only), or mental health center (CBHSQ, 2018).

NSDUH's definition of need has evolved from several workgroups and expert opinion over the years (CBHSQ, 2017). It starts with the diagnosis of an SUD. However, this definition was expanded because it was recognized that many individuals receive and need treatment even if they no longer meet the criteria for an SUD (for example, an individual may actively be in treatment and therefore their symptoms may have diminished). NSDUH also classifies those who have received treatment at a specialty facility as needing treatment, regardless of whether they met the criteria for an illicit drug use disorder in the past 12 months.

The challenge of defining need for SUD treatment was addressed at length by our TAG in a June 11, 2018, meeting. Key points included the following:

• Defining need based on diagnosis alone (such as the DSM criteria used in the NSDUH) is problematic. The DSM criteria for SUD consist of a certain number of symptoms (two out of a possible 11 in DSM-IV) displayed in the past 12 months; however, many people--perhaps as many as half, according to members of the TAG--who meet these criteria at some point in the 12 months, recover without receiving treatment, a view that is supported by a number of studies in the "self-change" literature (Klingerman, Sobell & Sobell, 2010). Using diagnosis alone would overestimate potential need and impact the service capacity required to address need.

• With regard to the limitations of diagnosis alone, many potential approaches were discussed--notably, supplementing diagnosis with some measure of clinical significance or functioning. Yet, this raises the challenge of determining appropriate cut-off points: At what point does substance use become clinically significant? How much and what kind of functional impairment indicates a need for treatment? The TAG suggested establishing some level of impaired functioning, along with diagnosis, as the criterion for need. In that the DSM diagnostic criteria includes several types of impaired functioning items (having serious problems at home, work, or school; being placed in physical danger; having problems with the law), NSDUH does allow for estimating the extent of different types of risk. These do not provide for measuring levels of global functioning, however, as represented for example by the International Classification of Functioning, Disability and Health (ICF), the framework developed by the World Health Organization (WHO) for measuring health and disability at both individual and population levels. As stated in an ICF manual (WHO, 2002):

  Studies show that diagnosis alone does not predict service needs, length of hospitalization, level of care or functional outcomes. Nor is the presence of a disease or disorder an accurate predictor of receipt of disability benefits, work performance, return to work potential, or likelihood of social integration. This means that if we use a medical classification of diagnoses alone we will not have the information we need for health planning and management purposes.

• An additional challenge in measuring need is that many individuals identified by some objective criteria--even a more restrictive definition that supplements diagnosis with clinical significance or functioning--do not themselves perceive a need or do not wish to receive treatment. NSDUH data indicates that 94.3% of the individuals identified as needing treatment based on the NSDUH's DSM-based criteria did not perceive a need for treatment (SAMHSA, 2018a). There are a number of possible explanations for this discrepancy: these individuals may be skeptical about the benefits of treatment, may find risks and burdens of SUD to be tolerable, or may prefer other ways of coping (Klingemann, Sobell, & Sobell, 2010). A planning model that fails to account for motivation would therefore result in excess demand and affect capacity needed. Nor has adequate research been conducted to understand the extent to which, with appropriate intervention, treatment-seeking can be increased among this population.

• Finally, there is the fact that some number of people receive services even though they do not meet SUD diagnostic criteria; this may be because they are in recovery but rely on services for continuing support or to manage current stressors, etc.

Methods for Estimating Prevalence

Although assessing the need for SUD treatment should not rely only on prevalence rates, estimating prevalence is a necessary first step in measuring need of treatment. Prevalence is generally estimated using one of two approaches, direct or indirect estimation.

Direct and Indirect Estimation. As defined by Schaible (1993), "A direct estimator uses values of the variable of interest only from the time period of interest and from units in the domain of interest. An indirect estimator uses values of the variable of interest from a time other than that of interest and/or from a domain other than that of interest." A national household survey is an example of direct estimation in that it uses information (for example about substance use) acquired from a defined population in a defined area (the nation) during a specified time period in order to produce prevalence estimates. These surveys typically employ multi-stage probability sampling as it would not be feasible to survey every household. Thus, a survey of a nationally representative sample that directly asks respondents whether or not they engage in a given behavior (say, past-month opioid use) would provide appropriate data for directly estimating the prevalence of that behavior in the nation.

Indirect estimation methods are typically (though not exclusively) used to develop estimates of prevalence in smaller areas, and for that reason are often referred to as "small area" analysis. There are many varieties of indirect estimation methods; however, all are based on evidence of a correlation between prevalence and certain population characteristics, such as age, race, etc. Indirect estimates utilize information from other sources such as national surveys, which is adjusted statistically to be applicable to the local population. Thus, the estimates of prevalence at the national level reported by SAMHSA from NSDUH data are examples of direct estimation; however, applying these estimates to a local level requires statistical adjustments to accommodate differences between the national and local population. If, hypothetically, adolescents constitute 10% of the national sample, but 40% of the local area population, it is necessary to use statistical methods to adapt national prevalence rates to the local area. For this reason, another frequent term for indirect estimation is "model-based."

SAMHSA has recently produced small area (406 sub-state areas) estimates for 15 substance abuse and mental health measures from the combined 2014-2016 NSDUH surveys (SAMHSA, 2018). Though too recent to have appeared in any of the reports reviewed for the environmental scan, it may be expected that this resource will address one of the major limitations of NSDUH data for purposes of local needs assessments. Moreover, the regions were defined by government officials from each of the 50 states and the District of Columbia, typically based on the substance abuse treatment planning regions specified by the states in their applications for the SABG; consequently, there is opportunity for the needs assessments provided for those applications to be more rigorous and detailed in the future.

One of the most extensive explorations of indirect methodology in behavioral health was an National Institute of Mental Health (NIMH) funded study in the 1980s that compared several different indirect methods with a direct estimation method used by the Colorado Department of Mental Health (Ciarlo & Tweed, 1992).

Hierarchical (multi-level) Models. A more complex variation of model-based estimation is hierarchical modelling. This approach addresses a potential limitation of the methods described above. These assume that population demographics are correlated with SUD consistently across different environments--that is, that there is no effect of the environment on SUD patterns independent of demographics. Hierarchical models account for the possible effect of environmental characteristics as well as demographics and also the possible interaction between these two levels.

An example of studies related to SUD are one by Gillespie (2005) that involved a survey of inmates in a number of prisons and also collected information about the prisons in addition to their population, based on the hypothesis that patterns of drug use in prisons are affected both by characteristics of the prison population and by characteristics of the prison such as policy, staffing, etc. In this model, prisoners were the Level 1 unit of analysis and prisons were the Level 2 unit of analysis.

As another example, Twigg, Moon, & Jones (2000) and Twigg & Moon (2002) employed a hierarchical model to examine smoking and alcohol use in different health districts of the United Kingdom) that takes into account simultaneously the individual and contextual nature of influences on health-related behavior. The example offered by the authors is that a person's social class may influence their decision to smoke and, equally, areas of higher social status may provide a cultural influence on smoking prevalence. Moreover, these two levels (individual and area) may interact--thus, for example, the likelihood of an individual from a lower socioeconomic class being a smoker may be higher if that individual lives in a lower social status area. The results of the analysis showed that the relative effects of individual and contextual factors differed between smoking and alcohol, including the interesting finding that the affluence of an area equates with a reduced likelihood of smoking but an increased likelihood of problem drinking.

CAPTURE-RECAPTURE ESTIMATION

Another indirect estimate approach is "capture-recapture," a statistical method for estimating the size of a population when direct estimation is not feasible, used, for example, in measuring the number of mentally ill homeless persons in an area (Fisher, et al., 1994), drug use (Frischer, et al., 1991; Gemmell, Millar, & Hay, 2004), and epidemiological studies (Hook & Regal, 1995). This method was employed by Larson, Stevens, & Wardlaw (1994) to estimate the population of heroin users within an Australian community--a "hidden population" not easily identified through surveys because of social desirability response bias. The capture-recapture approach involves taking two separate samples from the same population and using the number of individuals present in both samples versus the number of individuals present in only one or the other to compute an estimate for the total population.

SOCIAL INDICATORS FOR ESTIMATION--THE CAST MODEL

A recently developed, promising framework that uses social indicators to estimate substance abuse treatment need in a population is the Calculating for an Adequate System Tool or CAST (Green, et al., 2016). This methodology provides a framework for estimating needs at the local level and, based on these estimates, calculating community-specific recommendations at the service level for components of the continuum of care (promotion, prevention, referral, treatment, and recovery) by using social indicators to modify estimates of the population's needs. According to the authors, testing of the approach is underway.

Methods for Measuring System Capacity

A seemingly straightforward method of assessing current capacity would be a direct inventory of existing services. In fact, however, this is relatively uncommon in SUD needs assessments, most likely because this approach is more challenging than it might initially appear. First, there is the challenge of defining the boundaries of the SUD treatment system. When substance abuse services were funded primarily by states and counties, either by providing services directly or through contracts with vendors, it was relatively straightforward to enumerate programs and the numbers of people served by them, using administrative data generated for service-rate contracts and other purposes. Substance abuse treatment systems of today, however, have diffuse boundaries that make it difficult to determine even what services should be included, let alone their capacity. For example, the needs of many people with SUD are met through self-help groups and, increasingly, SUD treatment services are provided in primary care settings--both of which are extremely difficult to measure.

An additional challenge is that "capacity" is a hierarchical concept that includes many different levels--for example, the number of provider agencies, the number of programs such as clinics operated by agencies, the number and type of services provided by programs, the number of slots in the various services, and the number and capacity of the staff providing the services. These are obviously related but independent. A system-level assessment of capacity must measure each of these levels, which is challenging in practice, primarily because the information--if available at all--must be acquired from different sources; and if it is not available, it must be collected directly. Allen, LeMaster, & Deters (2004) describe some of these challenges encountered in the course of an evaluation of a Circles of Care system, for which grantees were required to provide a detailed description of the service system. A template was provided by the technical assistance center to guide grantees' data collection efforts in describing their current service system. Grantees soon found, however, that the template required extensive modification to fit local contexts, community norms, and community acceptance, and had generally poor response rates. In most cases, the grantees found that in-person interviews, either face-to-face or via phone, compared to the template resulted in a better response rate and more complete information. Other sources of data included focus groups and surveys with providers, traditional healers, community and family members, and youth, as well as town and village meetings. According to the authors, "This process was quite labor-intensive; one grantee site employed a team of eight graduate students who systematically interviewed agencies and tribal governments within the region over one summer."

Methods for Identifying Service System Gaps

As discussed in the introduction, the definition of "need" in a needs assessment is the discrepancy between the current and the desired status or results. For example, need may be defined as the difference in the number of people who currently receive treatment and the number that optimally should receive, or the difference in the number of people who use a particular substance and a goal of a smaller number. How need is defined will determine the range of possible recommendations for how to address the need.

It might be expected that at least some SUD needs assessments would quantify service system gaps according to some formula--such as a certain number of residential programs, inpatient beds, etc. per population prevalence estimates. However, in the environmental scan, we identified no reports in either the published or the gray literature that did so. Based on input from the TAG and our experience in the field, we can suggest several reasons why this might be the case.

In the first place, there are no empirically based formulas to determine the appropriate number of any form of treatment slots for any form of SUD. This issue is epitomized by the ongoing controversy over whether there is a shortage of inpatient psychiatric beds in the nation. Some advocates assert that the drastic reduction in beds through deinstitutionalization has gone too far, and patients are being harmed by the shortage of this service, while the alternative point of view holds that the solution is not more beds but more and better community-based services. Although there is a generally accepted conviction that a continuum of care is desirable, as yet there is not adequate research to compare the effectiveness of inpatient versus outpatient services for various types of patients and to determine how resources should be allocated along the continuum; consequently, the debate continues to be waged on the basis of values and opinion.

A few needs assessments that are more research-oriented use statistical models to measure the type of services that are needed. For example, Gunn et al. (2018) used statistical models for comparative analysis of county heroin and opioid deaths for purposes of resource allocation. These models identify correlations between heroin deaths and county demographic characteristics and other covariates (e.g., rurality, number of prescriptions) to classify counties based on different trajectories of death rates. Comparing these rates with the services currently available at the county level provides policy makers with information about areas with system gaps and the type of services needed, such as various evidence-based addiction treatments.

Likewise, the previously referenced 1991 study of substance use in Rhode Island by McAuliffe et al. used statistical modeling to identify areas of greatest need for different types of services based on area variations of drug use patterns. For example, they recommended increases across the continuum of care but proportionately greater increases for methadone treatment because half of the abusers in the survey who said they wanted treatment reported heroin addiction, and because there was a two-year waiting period at the state's only methadone center. Then, to decide where to locate the new methadone treatment services, they performed a regression analysis using hospital discharge data to determine the area of greatest need. This study was also unusual in that it prescribed specific numbers of treatment slots for different services based on a combination of need and the likelihood of available funding.

Mandated assessments for the block grant applications tend to provide narrative (qualitative) descriptions of service system gaps, for which block grant funds will be used. Few mandated needs assessments reviewed or provided details about the nature and distribution of the SUD workforce.

Locally initiated assessments typically define service system gaps quite specifically, based on the purpose of the assessment. For example, they may focus on gaps in services for a particular population, services that target specific substances, and/or services or treatment modalities (e.g., MAT). However, even locally initiated assessments seldom address needs at specific levels of care (number of residential beds, outpatient counseling slots, etc.). A few referred to the ASAM Criteria as a standard for a continuum of care (the "what should be" condition of a needs assessment) against which the existing service system could be measured.

Network capacity standards were reviewed in the environmental scan as an alternative approach to identify service system gaps. States are required by the Centers for Medicare & Medicaid Services (CMS) to develop standards for Medicaid managed care organizations' (MCOs') capacity for certain specialized services, including behavioral health. These requirements are relevant to SUD needs assessment in that they represent one way of specifying the "what should be" component of service needs and determining whether there is adequate capacity to meet these standards. CMS requires three types of standards: time and distance (limits on how long and how far beneficiaries must travel to receive services), and timely access (how long before an appointment is available). States are required to develop time and distance standards for adult and pediatric behavioral health providers and to demonstrate that the range of services and number of providers is adequate to meet the needs of the population. For purposes of needs assessment, however, network capacity standards have several limitations described in the "Opportunities for Further Research" section of this report.

Data Sources Used in Substance Use Disorder Treatment System Needs Assessment

Data Used for Measuring Need for Treatment

Estimates of need for treatment in early SUD needs assessments relied on locally conducted household surveys (e.g., McAuliffe, 1991) or extrapolation from national epidemiological studies such as the National Co-Morbidity Survey. More recently, in both locally conducted and mandated needs assessments, NSDUH is the most common source of data for estimates of need for treatment; however, it is often used in simplistic ways, providing prevalence figures to represent the extent of treatment need in the population.

Some needs assessments use publicly-available vital statistics, such as deaths due to opioid overdose, to estimate need for treatment, especially to identify trends.

Data for Measuring Service System Capacity and Gaps

Two SAMHSA resources for assessing system capacity are the National Survey of Substance Abuse Treatment Services (N-SSATS) and the Treatment Episode Data Set (TEDS) (SAMHSA, 2017a). N-SSATS is an annual survey of all known public and private substance abuse treatment facilities in the United States. N-SSATS collects information on the characteristics of individual facilities (e.g., types of treatment provided, public or private ownership, special groups served), on the point-in-time count of clients served, and on licensure, certification, etc. (SAMHSA, 2018b). N-SSATS also identifies the number of facilities that provide MAT and the number of clients that receive these medications and collects information on hospital and residential capacity. N-SSATS does not cover private practices, or care that occurs within primary care, so it may not be complete. The environmental scan indicated that, despite this wealth of information about local service systems, the NSSATS is not being used to its full potential.

TEDS (SAMHSA, 2017a) is another federal data source with considerable potential for SUD needs assessment that also might be used more widely, both for estimating need at the population level as well as gaps at the service system level. TEDS collects data on the demographic and substance abuse characteristics of admissions to and discharges from substance abuse treatment. Data are reported by approximately 10,000 facilities, programs, or administrative units in the 50 states, the District of Columbia, and Puerto Rico. Admission and discharge data are collected for persons aged 12 and older.

N-SSATS and TEDS have several limitations for purposes of measuring service system capacity that are discussed under "Data and Knowledge Gaps and Limitations," in the next section of this report.

Another federal data source which was suggested by the TAG is the Centers for Disease Control and Prevention's (CDC's) National Ambulatory Medical Care Survey (NAMCS), a potential source of information about services provided in primary care. It would likely require more specific information about SUD to serve this purpose, however. CDC is also developing methodology for estimating emergency department visits related to substance use with the National Hospital Care Survey (NHCS). This survey would supplement SAMHSA's Drug Abuse Warning Network (DAWN), which was discontinued in 2011 (Brown, et al., 2018) and restarted for 2019 (SAMHSA, 2019).

DATA USED IN "MIXED METHODS" FOR IDENTIFYING SERVICE SYSTEM CAPACITY AND GAPS

Most needs assessments, including those conducted for block grant applications as well as those that are locally initiated, used mixed methods approaches, combining quantitative (direct or indirect) estimates with qualitative data. Most locally initiated and many mandated needs assessment efforts supplement quantitative data such as survey data with qualitative information obtained through community meetings, key informant interviews, focus groups, and similar activities--in which stakeholders are asked questions such as whether they were able to obtain certain services, and if not, why. Qualitative information is also often obtained from previously published documents such as reports and news articles.

DATA USED FOR MEASURING WORKFORCE CAPACITY AND GAPS

The challenges of assessing current capacity include not only program capacity but also workforce capacity. Some locally initiated needs assessment reports reviewed in the environmental scan attempted to measure the workforce, primarily relying on state licensure databases. However, these sources provide very limited information--for example, of the licensed clinical social workers, information is lacking for whether they provide SUD treatment services, their caseload sizes, their programs versus private practice, etc. Additionally, state licensing boards vary in how they classify and define the job functions of the behavioral health workforce.

Moreover, only a few needs assessment reports address peers as an aspect of workforce capacity, which is somewhat surprising given the role that individuals in recovery play in the SUD treatment system. Based on input from the TAG, one explanation is that, unlike in the mental health field, the term "peer" in the SUD treatment field is generally not applied to a person who is in recovery working in a position such as substance abuse counselor, and the role of peers in the SUD system is not as clearly established.

POPULATION HEALTH DATA AS AN INDICATOR OF NEED

Health care policy initiatives, for example Healthy People 2020, increasingly focus on population health as an indicator of how well the health care system is performing. The CDC defines population health as "as referring broadly to the distribution of health outcomes within a population, the range of personal, social, economic, and environmental factors that influence the distribution of health outcomes, and the policies and interventions that affect those factors."[2]

Consistent with the definition of need presented in the introduction, the discrepancy between the current population health status and that which is desired or sought may be construed as a need. The approach of CHAs and CHNAs to identifying need is to identify a set of priority "community health needs" and to provide an action plan for how the institution will address them. CHAs and CHNAs are both required to conduct extensive efforts to engage the community in identifying these population health needs, and in the reports we reviewed SUD was consistently among the issues of greatest concern. The action plans that CHAs and CHNAs are required to develop are the equivalent of the recommendations component of the needs assessment model, and typically include various types of services and programs that the organization will support to address these problems. In broadening the assessment to include "social determinants of health" such as poverty and housing (which are known to be correlated with SUD), these approaches reflect this increasing emphasis in health care as exemplified, for example, by Healthy People 2020 (Secretary's Advisory Committee, 2010). This orientation is also consistent with the broad community-based attention to "environmental strategies" in the SAMHSA Strategic Prevention Framework--Partnerships for Success grant program.

LOCALLY INITIATED NEEDS ASSESSMENTS APPROACH TO IDENTIFYING GAPS

Because locally initiated needs assessments, in contrast to mandated assessments, are typically more focused on specific issues, service system gaps are usually defined quite specifically--consistent with the purpose of the assessment. For example, they may focus on gaps in services for a particular population (e.g., Native Americans in tribal areas, LGBTQ populations in a particular city); that is, they address disparities, in which case the optimal system is equitable distribution, perhaps with cultural enhancement.

Another common focus is on specific substances. These studies identify gaps in services and policies specific to these substances. One area of SUD treatment that has recently become an increased focus of needs assessment, as well as policy initiatives at the federal and state levels, is MAT, prompted by a combination of increased use of and deaths due to opioids, the demonstration of the effectiveness of MAT, and the recognition of various barriers limiting access to MAT. This combination of circumstances has led to federal initiatives to collect data on MAT capacity that exceed any previous efforts related to other SUD programs and workforce components. Currently there are 56,154 physicians certified by the SAMHSA Drug Addiction Treatment Act of 2000 (known as DATA 2000) waiver program; the majority (72%) are certified for the 30-patient limit, while 8% are certified for the maximum 275-patient limit.[3]

Given their narrower focus compared to mandated assessments, it is easier for locally initiated needs assessments to quantify the amount and types of services that are needed; for example, an assessment of the needs of a Native American community in Arizona, because of the relatively small and concentrated population and the limited number of providers, was able to obtain very detailed information about needs and barriers from multiple sources including reviews of medical records, which would not be feasible in a larger-scale study (Chester, Mahalish, et al., 2008).

Nonetheless, these studies seldom address needs at specific levels of care (for example, how many residential beds, outpatient counseling slots, etc.).

One locally initiated needs assessment used a simulated patient approach to identify service system gaps by calling agencies as though a potential client to inquire about availability of special services for special populations (LGBTQ) that were identified in N-SSATS. In this case, the method demonstrated inaccuracies in the self-report information represented in N-SSATS.

Secret shoppers are also one method for complying with CMS requirements that MCOs monitor network adequacy standards. According to a report from the Kaiser Family Foundation on survey of manage (Garfield, et al. 2018), this approach is used by more than half of MCO surveyed.

ASAM LEVELS OF CARE IN RELATION TO NEEDS ASSESSMENT

For planning purposes, needs assessments must go beyond identifying a general need for treatment to assessing the need for different types or levels within an overall treatment system. The ASAM Criteria has been adopted for this purpose by many state Medicaid programs as a form of "medical necessity" determination. Additionally, CMS is encouraging states to adopt the ASAM Criteria as a standard for a continuum of care in their 1115 Waiver programs (CMS, 2015).

We reviewed how levels of care were defined as an aspect of service capacity--that is, what kind of services are required to meet the individual-level needs of a population. Reports varied with respect to this important feature of needs assessment, with many simply providing data for utilization of existing services without discussion of service gaps. Only a few specifically referenced the ASAM Criteria; none attempted to characterize the current or optimal service system according to ASAM Criteria.

Data and Methods for Incorporating Populations, Workforce Categories, Geographical Settings and Levels of Care

In addition to measuring overall needs, treatment capacity, and gaps, needs assessments may also focus in on certain populations, workforce categories, and geographical locations.

POPULATIONS

Needs assessments differ in the way they define the populations of focus, reflecting differences in the purposes they serve. B y definition, CHNAs and CHAs include the entire population of a geographically defined community, though they break out certain subgroups, especially racial and ethnic populations on the basis of these groups having different or more extensive needs. Subgroups may include certain vulnerable populations; a county needs assessment, for example, examined U.S. Census Bureau American Community Survey data to enumerate four categories of "vulnerable and at-risk" populations: economically disadvantaged (below poverty level, unemployed, homeless, single parents, less than high school education); limited language competence; physical, cognitive or sensory disability; and age vulnerable (foster care children, nursing home residents). Each category is stratified by age, town, and racial group.

Other mandated needs assessments, such as those required for block grants, focus on defined populations for whom services are to be funded by the grant. SABG needs assessments provide prevalence estimates for the entire state population, but there are several target populations that are required to be addressed: persons who inject drugs, pregnant women and women with dependent children, parents with substance use and/or mental disorders who have dependent children, individuals with tuberculosis, persons at risk for HIV/AIDS, and individuals in need of primary SUD prevention. In addition to these required target populations, states are encouraged to address a variety of other groups including individuals who are homeless, rural populations, persons with disabilities, and older adults--and many SABG needs assessments do present data on these groups as well.

Locally initiated needs assessments conducted in earlier years generally included entire communities as well. Like CHAs and CHNAs, these also broke out subgroups such as ethnic and racial populations but differed from the community assessments in that they used this information as covariates in synthetic estimation models. One of the hallmarks of more recent needs assessments, however, is that they drill down to examine availability and accessibility of services for a wide range of specific subgroups, such as demographic subpopulations including age, gender, racial/ethnic and sexual orientation, and persons in a particular status (such as homelessness and incarceration). More recent needs assessments also tend to focus on users and service capacity of specific substances--particularly opioids. As a general explanation for this discrepancy, needs assessments focusing on specific subpopulations generally do not draw comparisons among subgroups but focus only on the capacity issues related to the group of interest.

WORKFORCE CATEGORIES

This area is one of the most significant shortcomings of most reports reviewed, testifying to the challenge of definition and measurement. Most do address workforce issues to some extent but usually in a very general way, such as quoting from some national report on behavioral health workforce shortages or narrative reports from stakeholders about problems with recruitment and retention. Several needs assessments that focus specifically on workforce capacity provide somewhat more detail. For example, a fairly typical workforce assessment, in this case from a largely rural state, a member of the Western Interstate Commission for Higher Education (WICHE), cited statistics for various occupational categories from the state licensure board, numbers graduating from the state university social work programs, Bureau of Labor (BLS) statistics for various occupational categories, some statistics collected five years prior in the course of a SAMHSA grant project, and rankings compared to other states in the WICHE network. They also presented estimates of prevalence (using Epidemiologic Catchment Area data from 14 years earlier) and estimates of population and workforce projected growth from the state's labor bureau. They did not, however, provide any quantified estimate of the current capacity of the workforce, such as numbers served, caseload sizes, etc., nor did they provide any definition of what an optimal level would be.

GEOGRAPHICAL SETTINGS

System gaps may be related not only to what services are provided but also to where they are located. It is important therefore, that a needs assessment have a clear definition of the area that a system is expected or required to serve. For publicly funded services this is usually straightforward, as the service area is generally defined by some jurisdictional boundary. With the increasing privatization of SUD treatment, however, this becomes more challenging. Hospitals, for example, generally conceive of their service areas in terms of markets, which frequently cross jurisdictional boundaries.

In contrast, locally initiated needs assessments tend to define the target area in terms of jurisdictional boundaries, primarily due to the source of their funding, such as a state or county behavioral health agency. The administrative boundary may not correspond with patterns of SUD, and some number of people may cross these borders to obtain services in adjacent localities. This may have been less of an issue in the past, when SUD services were mainly publicly financed and the catchment areas of a service system corresponded to administrative boundaries. Information about available services and data on utilization was then relatively easy to obtain. In today's more complex systems, however, with diverse payment sources and types of provider organizations, service areas are affected by other factors in addition to jurisdictional boundaries.

A model for defining service areas in needs assessment is the requirement for CHNAs not only to identify their service area but also to describe the methodology for defining their community according to their service area and function. For example, a CHNA report for an Ohio hospital defined its community according to a combination of factors: 19 ZIP codes that account for 66% of the hospital's inpatient volumes and its target population based on its principal function as a long-term acute care facility.

Geographical areas addressed by CHAs, on the other hand, reflect their administration by either county or state public health agencies. Needs assessments conducted for block grant programs vary in the areas they include; typically, they address needs on a state-wide basis, but they may identify certain sub-state areas, such as rural or tribal areas, with respect to certain gaps.

It is important that a needs assessment define setting or geographic target in a way that is clearly understandable and appropriate (that conforms to the target population that is or will be served by the system). The setting will influence decisions about the optimal scope and distribution of the service system. For example, the optimal configuration of the system will differ depending on whether the area is primarily urban or rural. In an urban setting, primary considerations may be location in ethnic neighborhoods or proximity to public transportation; in rural settings, important considerations may be delivery models such as co-location in Federally Qualified Health Centers and telepsychiatry. In large states with both rural and urban regions, such as California and Texas, a state-wide needs assessment would need to consider both kinds of service requirements.

A second reason for defining the area appropriately is that patterns of substance use differ from one locale to another (McAuliffe, et al., 1991). Proper definition of the area, therefore, will insure that the appropriate services are identified.

Validation of Substance Use Disorder Needs Assessment Methods

While some of the epidemiological data sources used in SUD needs assessments, such as the NSDUH, have been validated, validation of overall needs assessment methodology is very limited. In fact, the environmental scan identified only two comprehensive validation studies; these were published within a year of each other, almost 30 years ago. The Rhode Island study by McAuliffe, described previously, was one; it was funded in part by the National Institute of Drug Abuse (NIDA) and published in 1991. The NIMH funded the other study, the Colorado Social Health Survey (CSHS), a household-based probability sample of 4,745 adults in Colorado, conducted in 1984-85, described in a series of articles in a special issue of the journal Evaluation and Program Planning in 1992.From the reports on these, it is evident why validation studies are so limited: the resource burden associated with direct estimation methods, which are necessary for validation. Both of these validation studies were supported by grants from federal agencies in the 1980s--one which focused exclusively on drug use (NIDA) and the other (NIMH) which included both mental health and substance use.

The needs assessment by McAuliffe et al. (1991) used a telephone survey to assess the need for drug treatment in Rhode Island. The reliability and validity of the methodology was tested in several ways. First, the use of telephone surveys as a method of data collection had already been validated in previous studies. Second, the researchers used a method known as randomized response to conduct surveys on sensitive issues (Rosenfeld, Imai, & Shapiro, 2016). This involves asking a subset of respondents to first toss a coin and then to answer truthfully or not depending on the outcome without telling the researcher the outcome. This introduces randomization into the response patterns which is compared against the total sample to identify possible bias. Additionally, for a set of 100 consecutive interviews conducted by several different interviewers, researchers compared the answers recorded by a supervisor listening on a remote monitor with the answers recorded by the interviewers. To estimate the potential bias of respondents who refused to cooperate, the researchers compared the responses of individuals who initially refused to respond but agreed to be interviewed when called again to the responses of individuals who never refused. To estimate the potential bias of households that failed to answer the telephone, the researchers called all numbers not reached after seven attempts, and then compared the responses of hard-to-reach individuals to those of respondents who were reached with seven or fewer calls. They also compared the sample demographics with census data. Finally, they analyzed established drug abuse indicators to validate the telephone survey.

Validity of the methodology for estimating the amount of needed additional services was also demonstrated, as the state did add services as recommended, and utilization of these services (for example, proportionate decreases in MAT waiting times) were consistent with predictions.

Whereas the study by McAuliffe et al. (1991) was to validate a method of direct estimation, the validation study based on the CSHS tested six different models of indirect estimation including both social indicator and synthetic estimation types that have been used in substance use and mental health needs assessments (Tweed, Ciaslo, Kirkpatrick, & Shern, 1992).

The models tested included one developed by NIMH, known as the "rank-by-race" model, which employed area-level social indicators to predict relative need without estimating prevalence. Four additional models measured prevalence in various ways by means of social indicators, and the researchers also used one synthetic estimation model. Results of these initial tests showed that none of the models as constructed were accurate predictors of surveyed need for SUD and mental health services in Colorado but did show potential. The researchers then "optimized" the model equations by adjusting their parameters to best fit the CSHS data, with the result that all models except one were considerably more accurate predictors than a flat-rate model based on the assumption of equal need across all subareas.

Members of the TAG suggested several other possibilities for validating SUD needs assessment methodology. For example, some states maintain registries that could be used to validate estimates of population and service system needs; Virginia, for instance, tracks opioid-related health care use and maintains dashboards with information such as emergency department visits related to opioids, overdoses, and deaths. Indirect estimation methods could be compared to this information to assess predictive effectiveness.

 

OPPORTUNITIES FOR FURTHER RESEARCH AND ENHANCEMENT OF SUBSTANCE USE DISORDER NEEDS ASSESSMENT

Opportunities for improving the methodology and utility of needs assessment are discussed below. Some of these are suggested by best or exemplary practices in the field, many of which are associated with academic research studies supported with grants from federal agencies. As such, the benefit of these must be weighed against the increased burden they impose. An important opportunity for further research is how to make these tools available to smaller-scale, less-resource-intensive needs assessments. Additionally, however, the environmental scan indicated that SUD needs assessments often do not take advantage of tools and data sources that are available to them, such as the NSDUH small area analyses. More technical assistance and guidance would increase the use of these resources.

Data and Knowledge Gaps and Limitations

Data gaps are information that is lacking in existing data sources--such as federal survey data--that, if available (by adding questions, smaller area sampling, etc.), would enhance the utility of SUD treatment system needs assessment for planning purposes. Knowledge gaps are aspects of SUDs and their treatment where further research would likely produce results that allow for greater precision in needs assessment--for example, by identifying factors that affect how individuals respond to treatment or understanding small area variation in the social indicators that are correlated with SUD.

Measuring Need for Treatment: NSDUH Limitations

NSDUH uses a rigorous methodology for estimating prevalence; yet, it has two limitations that somewhat limit its utility for SUD treatment system needs assessment, especially at the local level. First, the broad definition of need for treatment (diagnosis or use of services) does not provide for the kind of targeted program planning and policy making that is needed to use resources most effectively. Second, it lacks the small area granularity necessary to capture the local variability in substance use patterns and need for treatment that is often required for targeted systems planning.

Measuring Capacity: N-SSATS and TEDS Limitations

As measures of system capacity, N-SSATS and TEDS both have some limitations. NSSATS is a voluntary survey with no adjustment for non-response, which was about 7% for the nation as a whole and nearly 15% for some states; it focuses primarily on facilities licensed by state substance abuse agencies, excluding some solo practitioners and self-help groups; it also excludes correctional facilities; and it provides point-in-time prevalence rather than annual totals (SAMHSA, 2018b).

TEDS is an admissions-based system--that is, it provides a count of number of admissions rather than an unduplicated count of individuals treated, so there is some duplication at the individual level (a person with two admissions in a year is counted twice), which could be of importance for systems planning both at the local and at the national level. It is possible, however, to use unique identifiers to calculate number of individuals.

An additional limitation of TEDS is that it relies on state data systems, which vary considerably in quality and structure, thereby limiting the reliability of state-to-state comparisons (SAMHSA, 2017b). Reporting facilities are primarily those that receive state alcohol and/or drug agency funding. U.S. Department of Veterans Affairs and military facilities are excluded, as are private for-profit facilities. Some states, but not others, report data for correctional facilities, and some report only on admissions to publicly funded treatment. Inclusion also varies among the states due to differences in licensure, certification, accreditation, and disbursement of public funds.

Measuring Capacity: Workforce Data Gaps

Many reports described the challenges in measuring workforce capacity mainly due to the lack of standard job titles and inconsistency with the national workforce databases maintained by BLS.[4] The BLS databases do include several job categories relevant to SUD treatment, including the specialty "addictions psychiatrist" under the psychiatrist category and a category called "substance abuse, behavioral disorder, and mental health counselors" with information about numbers and employment projections, including metropolitan and non-metropolitan statistical areas. Although these statistics would provide for a very general estimate of workforce size and trends, they do not differentiate between workers in SUD treatment versus other behavioral health workers. A second limitation is that there are several occupation categories in the SUD treatment workforce, notably social workers, that are not formally identified as substance abuse counselors.

One solution for improving measurement of workforce capacity would be to implement the Minimum Data Set (MDS) provided by the Behavioral Health Workforce Research Center (Beck, 2016) which provides a standardized set of occupational categories for the behavioral health workforce. The MDS characterizes the behavioral health workforce according to a number of uniform job characteristics such as demographics; licensure and certification; education and training background; occupation and area of practice; and practice characteristics and practice settings.

Another approach to standardizing labor force category definitions would be to refine the BLS labor force categories to correspond more closely to the way behavioral health job categories are defined in practice, although that would require that uniform job titles be adopted by licensing boards and other state data sources.

Addressing Knowledge Gaps

In addition to supplementing existing data sources, there are areas where SUD treatment system needs assessment would benefit from further research in the nature of SUDs, characteristics of the population of people with SUDs, and effective treatment for SUDs. Based on the review of needs assessments conducted in the environmental scan and related literature and on input from the TAG, the following points are offered as ways in which the methodology and utility of needs assessment in the field of SUD treatment may be improved.

Refining Measurement of Need for Treatment

As discussed in the "Introduction" section of this report, there are few operational definitions of need used in the field, other than simple prevalence. Further development of ways to measure need more precisely and guidance in how to use them would significantly enhance the utility of SUD needs assessment. For example, federal grant funding programs that require a needs assessment would be able to provide an operational definition of need that is more useful for planning purposes than what many now offer (simple untreated prevalence). In addition, research is needed to differentiate the different degrees and types of needs among this broad population.

Too often, needs assessments rely solely on prevalence of a SUD as a measure of a need for treatment. Further research to classify and measure subgroups of the population of persons with SUD on the basis of type of substance use, degree of impairment, and variations in treatment and recovery trajectories would greatly enhance the utility of needs assessment for SUD treatment system planning.

In addition to estimating the need for treatment in the population, system planning must also consider how many people will actually use any new or expanded service. One way to develop estimates of potential utilization suggested by the TAG would be to add questions to NSDUH or other surveys: besides simply asking people whether they received services, these instruments could inquire about the amount of services received and whether the respondent felt these were adequate and appropriate.

Further research on the use of social indicators for estimating population-level need would be another valuable contribution. The testing of the CAST methodology that is currently underway will provide validation of the social indicators that are represented in the model.

The CAST methodology also holds promise for the recommendation in the needs assessment methodology literature that assessments should identify not only unmet need and gaps in the service system but also redundant or ineffective services. While this is important in SUD needs assessment to maximize the benefit of limited resources, the challenge is that evidence for the effectiveness of various SUD services is not yet well-enough developed to provide firm guidance for these decisions. It will be important to track research, such as that supported by ASAM, to examine the evidence for various treatment modalities that are now common practice but with uncertain evidence for their effectiveness and to incorporate this into the practice of needs assessment as it becomes available. Locally conducted needs assessments that seek to identify unproductive components of a system should explore the possibility of using performance measurement data--from state or county contracting requirements, for example--when available.

Refining Measurement of Capacity

The biggest challenge in defining capacity is to set the boundaries: should capacity include only the specialty SUD system or should it also include self-help groups, primary care, solo practitioners, etc.? Furthermore, as mentioned previously, workforce capacity, both in the current and the desired system, is difficult to define and consequently to measure. What are the functions and caseloads of various occupational categories--for example, if it can be determined that a program has a certain number of nurses, how many of these are providing direct service versus management and supervision, and of those providing direct services, how many patients do they carry in a caseload? Development of templates to standardize measurement of system components, treatment modalities, and workforce categories and functions--especially in the non-specialty sector--would enhance the utility of SUD needs assessment for policy making purposes.

Another potential source of information for assessing capacity is the NAMCS, which identifies services provided in primary care. It would likely require more specific information about SUD to serve this purpose, however. The CDC is also developing methodology for estimating emergency department visits related to substance use with the NHCS as a supplement for SAMHSA's DAWN, which was discontinued in 2011 (Brown, et al., 2018) and restarted for 2019 (SAMHSA, 2019).

As noted on the measurement of patient needs, the ASAM Criteria also define the levels of a full continuum of care for SUD treatment. The ASAM Criteria specifies the competencies required for each level of care and amount of service required for each level (e.g., hours of counseling). Further research to refine these prescriptions and test their validity will advance the use of the Criteria for assessing capacity adequacy.

Similarly, further research is required to match need for treatment with the staff levels required to adequately meet the need. This type of research would allow for the development of standards similar to those used for Health Professional Shortage Areas.

Refining Identification of Service System Gaps

The following are current challenges in identifying treatment system gaps that would benefit from further research and methodology development:

• Lack of empirically based formulas to determine the appropriate number of any form of treatment slots for any form of SUD.

• Local variability in SUD patterns: It is a well-established fact that patterns of SUD vary from one locality to another--and sometimes within very small areas, such as one neighborhood to another (for example, where adolescent use of marijuana may be the predominant problem in the suburbs, heroin use may be predominant in the inner city).

• Local variability in existing assets: Some areas may have very well-developed prevention programs, in which case they would have less need for more intensive services such as inpatient treatment, MAT, etc. A service capacity formula would need to assess the impact of all these existing assets, which would be very difficult to quantify.

• Complexity of the behavioral health treatment system: In the past, when substance use treatment services were financed primarily by state agencies, it was relatively simpler (though by no means simple) to identify service gaps and allocate resources accordingly. As payer sources have expanded, especially through parity legislation and Medicaid expansion, it becomes increasingly difficult for policy makers, such as state behavioral health agencies, to direct resources to one or another type of service.

• Increased privatization: Related to the above, the increased privatization of the SUD treatment system introduces market dynamics into the equation--in particular, the issue of supplier-induced demand. The capacity of services and demand for them in a particular community, especially for inpatient treatment, may change very rapidly depending on the market-based decisions of providers, both for-profit and non-profit.

• Uncertainty over the relative effectiveness of different treatment modalities: The lack of an evidence base for different treatment modalities, especially comparative effectiveness research, makes it difficult if not impossible to specify the appropriate number of slots in each treatment modality. As a hypothetical example, should a system have 100 inpatient beds and 50 residential beds or 50 inpatient beds and 100 residential beds?

As discussed previously, identifying service system gaps requires some standard--the "what should be" condition--against which the existing system can be measured. The ASAM Criteria is a potential framework for this purpose. As it has evolved, the levels identified by the criteria are increasingly identified as a definition of a full continuum of care for SUD treatment. The ASAM Criteria establishes a standard for a continuum of care and specification of workforce requirements for each level that allow for identification of service gaps and redundancies, and it provides a definition of services based on treatment needs rather than length of stay (e.g., 3o-day residential programs). Since it is an evolving framework, there are likely to be many additional developments that would offer even more uses.

Network adequacy standards also offer potential for comparisons with existing systems to identify gaps. While states have made substantial and innovative progress in operationalizing network adequacy as required by CMS, based on time and distance standards, the TAG identified several areas in which there is a need for refinement.

• There may need to be some refinement in wait-time standards, for example by differentiating among types of services. A wait period that might be acceptable for treatment of alcoholism is perhaps too long for treatment for opioid disorders. Wait periods for methadone treatment are excessive in many areas. Information about the number of people who enter treatment versus those who fail to enter treatment after waiting periods would inform these policies.

• Members of the TAG recommend support for innovations linking data such as that for certification of network capacity with that collected for other purposes, such as claims, to build models for measuring service need and utilization and for monitoring client engagement and initiation of treatment.

• The use of simulated shoppers was suggested as an effective method for testing compliance with network adequacy standards.

• Network adequacy standards with provisions for monitoring compliance offer an approach that may be an alternative to needs assessment entirely. Several factors limit this potential, however. The main issue is whether network adequacy standards alone have the flexibility to address the local variations in prevalence, treatment demand, and supply that are characteristic of SUD. In short there is some circularity involved in the assumption that network adequacy standards might supplant needs assessment, in that an evidence-based determination of what constitutes an adequate standard requires needs assessment. Another reason for the continuing use of SUD needs assessment is that determining network adequacy by MCOs is only one of many purposes of needs assessment--other uses include, among others, determining how to allocate resources, assessing the implications of some policy initiative, and identifying the need for services to address emerging issues such as the opioid crisis.

Enhancing the Utility of Substance Use Disorder Needs Assessments

In addition to improving the quality of measurement of need by more detailed data and a broader knowledge base, there are opportunities for strengthening the additional component of needs assessment, to serve as a guide for planning and policy making.

The large number of needs assessments produced each year represents an untapped resource for research on issues such as area differences and trends over time. However, this potential is limited to some extent by the fact that SUD needs assessments, especially those that are locally initiated, are conducted by small, local consulting groups or local academic policy centers and vary considerably in quality and methodology. Needs assessment as a field lacks the infrastructure to support quality and dissemination that is available in the form of peer-reviewed journals for academic research. While it will always be important for needs assessments to be adapted to local conditions and concerns, there are some opportunities for enhancing quality and consistency. To establish more uniform approaches by locally initiated needs assessments, it would be beneficial to draw upon the example of the manuals and other forms of guidance provided for the development of CHNAs, CHAs and other population health needs assessments. For example, the National Association of County and City Health Officials has developed a framework for comprehensive planning known as MAPP, in which needs assessment is one phase in a broader process of planning for improvement of population health. The NACCHO website provides a wealth of resources for conducting community-based needs assessment.[5] Another resource along the same line is the webpage maintained by CDC called Assessment and Planning Models, Frameworks and Tools that provides a wealth of information on how to conduct a CHNA.[6]

Enhancing the Utility of Mandated Needs Assessment

Mandated needs assessment would benefit from advancing the science through further research, as federal agencies could then call for more rigorous approaches than those currently required for block grant applications and the like. At the same time, any call for more comprehensive and rigorous needs assessment must recognize the importance of minimizing additional burden on the states. It should be noted, however, that the return on investment of a more rigorous and comprehensive needs assessment could be positive if the result were enhanced utility and support from other planning initiatives.

Establishing a means of providing technical assistance for localities that are considering a needs assessment would enhance the quality and utility of these efforts. National organizations could consider providing technical assistance for members. Professional organizations (e.g., the American Evaluation Association) might be approached for developing guidance on conducting SUD needs assessments. Other suggestions for supporting SUD needs assessments include commissioning a white paper and publishing a special issue of a publication, such as the CDC's Preventing Chronic Disease, on the subject of conducting SUD treatment capacity needs assessments, and exploring opportunities for developing professional learning communities where, for example, states can learn from those that are models in conducting needs assessments or from experiences with applying for relevant grants.

Improving Effectiveness at Promoting Organizational and System Change

Achieving system change and improvement is the ultimate goal of all SUD needs assessment, and there are various ways in which this potential could be further developed. For example, those conducing needs assessments should strive for clarity in representing the purpose of the needs assessment, such as whether it is primarily for policy planning or for advocacy and political purposes or a combination of the two. Additionally, they should insure that recommendations are actionable to the greatest possible extent and incorporate the principles of implementation science to support action on the recommendations. Agencies and departments within agencies that support the conduct of needs assessment or that use the results might maximize collaboration and dissemination of needs assessment reports.

Conclusions

A number of the research questions addressed in this report flag areas of potential opportunities for advancing the field of SUD needs assessment. Key aspects of these are summarized below.

Calculating Service System Capacity Requirements. Several of the research questions are related to the possibility of developing a formula that might be applied to produce a measure of the types and amounts of services that are needed in a community, for example adapting the Health Resources and Services Administration methodology for designating Health Professional Shortage Areas. Any such formula, however, would require considerably more research on the appropriate amounts of each type of service and how these could be adapted to address local variation. The network adequacy standards employed by CMS for Medicaid MCOs offer another angle on this approach; however, these would likewise require more research to establish an empirical basis for the standards (e.g., appropriate wait-times for various conditions).

ASAM Levels of Care Criteria. The current and potential use of the ASAM Criteria was a question addressed in the environmental scan. We found that current use is limited, but we suggest that there is considerable potential for the ASAM Criteria to advance SUD needs assessment, the most important being to refine the "what should be" condition: the optimal continuum of care adapted to local conditions.

Data Gaps. A primary goal of the project was to identify gaps in the currently available sources of information for conducting SUD needs assessments. NSDUH is the most rigorous and extensive source of prevalence data but is limited in some respects for the planning functions of needs assessments: it provides limited capability for differentiating among degrees of impairment (i.e., levels of need for treatment); it omits individuals who are in institutions or prisons, serving in the military, or are unsheltered homeless. TEDS and N-SSATS are the primary sources of data for assessing capacity at the program level, with TEDS providing information on volume and N-SSATS on programs. Both have some limitations, however. TEDS data is provided by the states, whose data systems vary considerably in structure and quality, and reporting facilities are primarily only those that receive state funding. Also, TEDS data reflect episodes rather than individuals receiving treatment. NSSATS also includes primarily facilities that are licensed by state substance abuse agencies, thereby excluding solo practitioners and self-help groups, with the further limitation that it provides point-in-time counts rather than annual counts. Measuring capacity at the workforce level is particularly challenging, mainly because of the diversity of the SUD workforce and variability in classifying practitioner types. The most common sources of information about workforce capacity are state registries and BLS statistics, but neither of these matches very precisely the actual structure and functions of the SUD treatment workforce.

Knowledge Gaps. In addition to improvements in the available data, SUD needs assessment as a field would benefit from further research in several areas. The most important of these are:

1. Improvement in ways to capture the structure and capacity of the SUD workforce.

2. Identification of subgroups in the SUD population on the basis of type of substance use, degree of impairment, variations in treatment and recovery trajectories.

3. Further research on the use of social indicators for estimating population-level need (the recent development of the CAST methodology by Green, Lyerla, Stroup et al. (2016) is a promising effort in this area).

4. A fuller understanding of how SUD fits within the broader framework of population health, in order to align SUD needs assessment with current large-scale initiatives such as Healthy People 2020.

Validation Studies. Most of the efforts to validate needs assessment methodology were supported by federal agencies decades ago. The most extensive of these was a test of several methods of indirect estimation conducted in the 1980s by researchers in Colorado (Tweed, et al., 1992).

Enhancing the Utility of Needs Assessment for Effecting System Change. The most fundamental purpose of a needs assessment related to a SUD treatment system is to direct positive change in how SUDs are addressed. Given the key function of needs assessment to serve as a guide for planning and policy making, the recommendations section of reports should receive careful attention and ideally should include priorities and strategies for acting on those recommendations. These should incorporate principles from the rapidly growing field of implementation science to support action on recommendations. Though technically challenging, cost effectiveness analyses comparing different options are highly valued by policy makers.

Currently, most SUD needs assessments, unlike other forms of social research, are conducted autonomously with little oversight, little external review of methodology, and little assessment of reliablity and validity.

The quality and impact of SUD needs assessment could be enhanced by providing the field with the type of organizational infrastructure support that available for CHNAs and CHAs to promote best practices and standardize methodologies. This support might include fostering of learning communities or increased participation in existing forums such as the American Evaluation Association Needs Assessment Topical Interest Group.

As a field, SUD needs assessment offers unexploited opportunities for advancing our understanding of the nature of SUD treatment needs and the requirements for systems to address them effectively. The additional research and initiatives to address data gaps would allow for more rigorous needs assessments to be conducted with less demand on resources. Increasing resources to support the more rigorous implementation and broader dissemination of SUD treatment needs assessment would yield benefits beyond the specific policy purposes served by individual needs assessments, bringing the field into line with other fields of social science.

 

APPENDIX A. REFERENCES

Allen, J., P.L. LeMaster, and P.B. Deters (2004). "Mapping pathways to services: description of local service systems for American Indian and Alaska Native children by Circles of Care." Am Indian Alsk Native Ment Health Res, 11(2): 65-87.

Altschuld, J.W., and R. Watkins (2014). "A primer on needs assessment: More than 40 years of research and practice." New Directions for Evaluation, (144): 5-18.

Beck, A.J., P.M. Singer, J. Buche, R.W. Manderscheid, P. Buerhaus, C.M. Tuohy, and M.L. Boulton (2016). A Minimum Data Set for the Behavioral Health Workforce. Behavioral Health Workforce Research Center.

Brown, A.M., C. DeFrances, E. Crane, R. Cai, and S. Naeger (2018). Identification of Substance-Involved Emergency Department Visits Using Data From the National Hospital Care Survey. Centers for Disease Control and Prevention, National Center for Health Statistics.

Center for Behavioral Health Statistics and Quality (CBHSQ) (2018). 2017 National Survey on Drug Use and Health: Methodological Summary and Definitions. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Center for Behavioral Health Statistics and Quality (CBHSQ) (2017). 2016 National Survey on Drug Use and Health: Methodological Summary and Definitions. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Centers for Medicare & Medicaid Services (2015). New Service Delivery Opportunities for Individuals with a Substance Use Disorder. https://www.medicaid.gov/federal-policy-guidance/downloads/smd15003.pdf.

Centers for Medicare & Medicaid Services (2010). Marketing Surveillance Industry Summary Report. https://www.cms.gov/Medicare/Health-Plans/ManagedCareMarketing/Downloads/SurveillanceReport081010.pdf.

Chester, B., P. Mahalish, and J. Davis (2008). "Mental health needs assessment of off-reservation American Indian People in Northern Arizona." American Indian & Alaska Native Mental Health Research, 8(3): 25-40.

Ciarlo, J.A., and D.L. Tweed (1992). "Implementing indirect needs-assessment models for planning state mental health and substance abuse services." Evaluation and Program Planning, 15(2): 195-210.

Druss, B.G., P.S. Wang, N.A. Sampson, M. Olfson, H.A. Pincus, K.B. Wells, and R.C. Kessler (2007). "Understanding mental health treatment in persons without mental disorders." Arch Gen Psychiatry, 64: 1196-203.

Fisher, N., S.W. Turner, R. Pugh, and C. Taylor (1994). "Estimating numbers of homeless and homeless mentally ill people in north east Westminster by using capture-recapture analysis." Br Med J, 308: 27-30.

Frischer, M., M.J. Bloor, A. Finlay, D.J. Goldberg, S. Green, S. Haw, N. McKeganey, and S. Platt (1991). "A new method of estimating prevalence of injecting drug use in an urban population: results from a Scottish city." Int J Epidemiol, 20: 997-1000.

Garfield, R., E. Hinton, E. Cornachione, and C. Hall (2018). Medicaid Managed Care Plans and Access to Care: Results from the Kaiser Family Foundation 2017 Survey of Medicaid Managed Care Plans.

Gemmell, I., T. Millar, and G. Hay (2004). "Capture-recapture estimates of problem drug use and the use of simulation based confidence intervals in a stratified analysis." J Epidemiol Community Health, 58(9): 758-65.

Gerstoff, J., and S. Gibson (2016). Medicaid Encounter Data: The Next National Data Set. Milliman Insight, Milliman.

Gillespie, W. (2005). "A multilevel model of drug abuse inside prison." Prison Journal, 85(2): 223-246

Green B., R. Lyerla, D.F. Stroup, A. Azofeifa, and P.M. High (2016). "A tool for assessing a community's capacity for substance abuse care." Prev Chronic Dis, 13: 1-10.

Gunn, A.H., B. Bartlett, G. Feng, M. Gayed, K. Kanter, E. Onuoha, M. Thornton, A. Muzyk, and N. Schramm-Sapyta (2018). "Running the numbers: County level dynamics of heroin mortality in North Carolina." NC Med J, 79(3): 195-200.

Henderson, S., A. Korten, and J. Medway. "Non-disabled cases in a national survey." Psychol Med, 31: 769-77.

Hook, E.B., and R.R. Regal (1995). "Capture-recapture methods in epidemiology: Methods and limitations." Epidemiol Rev, 17: 243-64.

Klingemann, H., M.B. Sobell, and L.C. Sobell (2010). "Continuities and changes in self-change research." Addiction, 105(9): 1510-1518.

Larson, A., A. Stevens, and G. Wardlaw (1994). "Indirect estimates of 'hidden' populations: Capture-recapture methods to estimate the numbers of heroin users in the Australian capital territory." Social Science & Medicine, 39(6): 823-831.

Lin, E., P.N. Goering, A. Lesage, and D.L. Streiner (1997). "Epidemiologic assessment of overmet need in mental health care." Soc Psychiatry Psychiatr Epidemiol, 32: 355-62.

McAuliffe, W.E., P. Breer, N.W. Ahmadifar, and C. Spino (1991). "Assessment of drug abuser treatment needs in Rhode Island." American Journal of Public Health, 81(3): 365-371.

Mee-Lee, D. (Ed.) (2013). The ASAM Criteria: Treatment Criteria for Addictive, Substance-Related, and Co-Occurring Conditions.

Office of Management and Budget Statistical Policy, Office of Information and Regulatory Affairs (1993). Indirect Estimators in Federal Programs. Working Paper 21.

Rosen, B.M., and H.F. Goldsmith (1981). "The health demographic profile system: Current and longitudinal data base for social area analysis." Evaluation & Program Planning, 4(1): 57-73.

Rosenfeld, B., K. Imai, and J.N. Shapiro (2016). "An empirical validation study of popular survey methodologies for sensitive questions." American Journal of Political Science, 60(3): 783-802.

Schaible, W. (1993). Indirect Estimators: Definition, Characteristics, and Recommendations. American Statistical Association Survey Research Section Proceedings.

Secretary's Advisory Committee on National Health Promotion and Disease Prevention Objectives for 2020 (2010). Healthy People 2020: An Opportunity to Address Societal Determinants of Health in the United States. https://www.healthypeople.gov/2010/hp2020/advisory/societaldeterminantshealth.htm.

Substance Abuse and Mental Health Services Administration (n.d.1). FFY 2018-2019 Block Grant Application Community Mental Health Services Block Grant (MHBG) Plan and Report Substance Abuse Prevention and Treatment Block Grant (SABG) Plan and Report. https://www.samhsa.gov/sites/default/files/grants/fy18-19-block-grant-application.pdf.

Substance Abuse and Mental Health Services Administration (n.d.2). Strategic Prevention Framework--Partnerships for Success (SPF-PFS). Initial Announcement, Funding Opportunity Announcement (FOA) No. SP-18-008. Catalogue of Federal Domestic Assistance (CFDA), No. 93.243. https://www.samhsa.gov/grants/grant-announcements/sp-18-008.

Substance Abuse and Mental Health Services Administration (2017a). Key Substance Use and Mental Health Indicators in the United States: Results from the 2016 National Survey on Drug Use and Health. Rockville, MD: Center for Behavioral Health Statistics and Quality. https://www.samhsa.gov/data/.

Substance Abuse and Mental Health Services Administration (2017b). Treatment Episode Data Set (TEDS): 2005-2015. National Admissions to Substance Abuse Treatment Services. BHSIS Series S-91, HHS Publication No. (SMA) 17-5037. Rockville, MD: Center for Behavioral Health Statistics and Quality.

Substance Abuse and Mental Health Services Administration (2018). 2012-2014 National Surveys on Drug Use and Health: Guide to Substate Tables and Summary of Small Area Estimation Methodology. https://www.samhsa.gov/data/sites/default/files/NSDUHsubstateMethodology2014/NSDUHsubstateMethodology2014.pdf.

Substance Abuse and Mental Health Services Administration (2018a). Key Substance Use and Mental Health Indicators in the United States: Results from the 2017 National Survey on Drug Use and Health. HHS Publication No. (SMA) 18-5068, NSDUH Series H-53. Rockville, MD: Center for Behavioral Health Statistics and Quality. https://www.samhsa.gov/data/.

Substance Abuse and Mental Health Services Administration (2018b). 2017 National Survey of Substance Abuse Treatment Services (N-SSATS): 2017. Data on Substance Abuse Treatment Facilities. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Substance Abuse and Mental Health Services Administration (2019). Drug Abuse Warning Network. Rockville, MD: Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/data-we-collect/dawn-drug-abuse-warning-network.

Tweed, D.L., J.A. Ciarlo, L.A. Kirkpatrick, and D.L. Shern (1992). "V. Empirical validity of indirect mental health needs-assessment models in Colorado." Evaluation & Program Planning, 15(2): 181-194.

Twigg, L., and G. Moon (2002). "Predicting small area health-related behaviour: A comparison of multilevel synthetic estimation and local survey data." Social Science & Medicine, 54(6): 931-937.

Twigg, L., G. Moon, and K. Jones (2000). "Predicting small-area health-related behaviour: A comparison of smoking and drinking indicators." Social Science & Medicine, 50(7): 1109-1120.

Üstün, T.B., N. Kostanjsek, S. Chatterji, and J. Rehm (Eds.) (2010). Measuring Health and Disability Manual for WHO Disability Assessment Schedule. WHODAS 2.0. Malta: World Health Organization.

Watkins, R., and J. Kavale (2014). "Needs: Defining what you are assessing." New Directions for Evaluation, 144: 19-31.

Watkins, R., M. West Meiers, Y.L. Visser (2012). A Guide to Assessing Needs: Essential Tools for Collecting Information, Making Decisions, and Achieving Development Results. World Bank. https://openknowledge.worldbank.org/handle/10986/2231.

White, J.L., and J.W. Altschuld (2012). "Understanding the 'what should be condition' in needs assessment data." Evaluation & Program Planning, 35(1): 124-132.

World Health Organization (2002). Towards a Common Language for Functioning, Disability and Health: ICF. Geneva.

 

APPENDIX B. DETAILED SURVEY SUMMARIES

Non-Federal Members

Jay Butler, MD
Alaska Department of Health and Social Services

Peter Cunningham, PhD
Virginia Commonwealth University

Tom Hill, MSW
National Council for Behavioral Health

Ron Manderscheid, PhD
National Association of County Behavioral Health and Developmental Disability Directors

Tami L. Mark, PhD
RTI International

Dennis McCarty, PhD
Oregon Health and Science University

William McAuliffe, PhD
Harvard Medical School at Cambridge Health Alliance and the North Charles Foundation

Alexander Shekhdar, JD
Medicaid Health Plans of America

Sarah A. Wattenberg, MSW
National Association of Psychiatric Health Systems

Federal Members

Anne Herron, MS
Substance Abuse and Mental Health Services

Tyler Sadwith
Centers for Medicare & Medicaid Services

George Zangaro, PhD, RN
Health Resources and Services Administration

Michelle M. Washko, PhD
Health Resources and Services Administration

Hayden O. Kepley, PhD
Health Resources and Services Administration

 

APPENDIX C. ENVIRONMENTAL SCAN REPORT

Available separately at:

https://aspe.hhs.gov/report/needs-assessment-methodologies-determining-treatment-capacity-substance-use-disorders-environmental-scan-final-report

 

NOTES

1. The recent development of the Calculating for an Adequate System Tool (CAST) methodology by Green, Lyerla, Stroup et al. (2016) is a promising effort in this area.

2. See https://www.cdc.gov/pophealthtraining/whatis.html.

3. See https://www.samhsa.gov/programs-campaigns/medication-assisted-treatment/training-materials-resources/physician-program-data

4. "Databases," Tables and Calculators by Subject. https://www.bls.gov/data/.

5. See http://www.naccho.org/programs/public-health-infrastructure/performance-improvement/community-health-assessment/mapp.

6. See http://www.cdc.gov/stltpublichealth/cha/assessment.html.

---

Network Capacity for Substance Use Disorder Treatment

This report was prepared under contract #HHSP233201600015I between HHS's ASPE/DALTCP and the Human Services Research Institute. For additional information about this subject, you can visit the DALTCP home page at https://aspe.hhs.gov/office-disability-aging-and-long-term-care-policy-daltcp or contact the ASPE Project Officer, Judith Dey, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. Her e-mail address is: Judith.Dey@hhs.gov.

Reports Available

Needs Assessment Methodologies in Determining Treatment Capacity for Substance Use Disorders: Final Report

• HTML   https://aspe.hhs.gov/basic-report/needs-assessment-methodologies-determining-treatment-capacity-substance-use-disorders-final-report
• PDF   https://aspe.hhs.gov/pdf-report/needs-assessment-methodologies-determining-treatment-capacity-substance-use-disorders-final-report

Needs Assessment Methodologies in Determining Treatment Capacity for Substance Use Disorders: Environmental Scan Final Report

• HTML   https://aspe.hhs.gov/report/needs-assessment-methodologies-determining-treatment-capacity-substance-use-disorders-environmental-scan-final-report
• PDF   https://aspe.hhs.gov/pdf-report/needs-assessment-methodologies-determining-treatment-capacity-substance-use-disorders-environmental-scan-final-report