Introduction and Background
In September 2017, the Office of the Assistant Secretary for Planning and Evaluation contracted with the Human Services Research Institute to evaluate needs assessment methodologies for substance use disorder (SUD) treatment capacity. The overarching aim of the project is to enhance the effectiveness of needs assessment to promote organization and system change, addressing gaps in the existing data and promoting the implementation of evidence-based practices (EBPs), peer supports, and innovative technologies. The project was advised by a Technical Advisory Group (TAG) composed of nine non-government experts and three government experts.
Need for Effective SUD Treatment Capacity Needs Assessment
According to the Substance Abuse and Mental Health Services Administration (SAMHSA, 2017), in 2016:
An estimated 28.6 million people (10.6% of the population aged 12 and older) had used an illicit drug in the month before the survey.
An estimated 16.3 million people aged 12 and over were heavy alcohol users.
An estimated 15.1 million people aged 12 or older met Diagnostic and Statistical Manual of Mental Disorders (4th edition) criteria for alcohol dependence or abuse.
An estimated 21.0 million people aged 12 or older, about one in 13, needed substance use treatment.
Approximately 3.8 million people, 1.4% of all people aged 12 or over and 10.6% of those needing treatment, received any substance use treatment in the past year.
According to SAMHSA data for 2015, of the adults who needed substance use treatment but did not receive specialty treatment, only 4.8% (863,000) felt that they needed treatment for their use of alcohol or illicit drugs. About 1.7% of this group made an effort to obtain treatment, and 3.7% felt they needed treatment but did not make an effort to obtain it (SAMHSA, 2016).
Approach and Methods
We reviewed literature on needs assessment specific to SUD treatment capacity and recent literature on needs assessment methodology generally to formulate an operational definition of SUD "treatment system needs." We then developed a protocol for retrieving and systematically reviewing a sample of SUD needs assessment reports to examine:
Workforce categories addressed.
Levels of care incorporated.
Data sources/methods used.
Best practices identified.
Method validation (if any).
The protocol also included criteria for rating the quality of needs assessments, based on the approach of systematic literature reviews.
We conducted a systematic search of SUD needs assessment in the grey and published literature, resulting in a convenience sample for review, consisting of two categories: "mandated" needs assessments (usually those produced in response to some funding requirement, which address SUD among other public health issues) and "locally initiated" needs assessments, typically one-time studies conducted in connection with a state or county SUD policy initiative. We reviewed these two categories separately, with a summary overview of mandated reports and a more fine-grained analysis of the locally initiated reports. Two additional research questions were addressed by separate scans, examining how states operationalize network capacity and whether there are needs assessments of other provider types that might be adapted to SUD.
Definition of Needs Assessment. Based on the literature review, needs assessment was operationally defined for purposes of the search and analysis as consisting of four components: (1) a measurement of the current condition ("what is"), (2) a specification of the optimal state ("what should be"); (3) a measurement of the gap between the current and optimal states; and (4) recommendations, ideally prioritized, for actions to address the gap.
The search produced a convenience sample of 40 reports, seven mandated and 33 locally initiated, selected to be diverse in geographical area, type of substance abuse, and populations.
Mandated Needs Assessments
There are at least 14 types of nationally required needs assessments that typically address SUD among other public health issues topics addressed. Compared to locally initiated needs assessments, these are fairly generic in their approach because they are required to follow specific guidelines; we therefore subjected these to a more general overview rather than the detailed data extraction applied to locally initiated assessments, focusing on three types: (1) those required by SAMHSA including the Substance Abuse Block Grant and Mental Health Services Block Grant programs, Strategic Prevention Framework State Incentive Grants, and State Targeted Response to the Opioid Crisis Grants; (2) Community Health Needs Assessments (CHNAs), which are required of all non-profit hospitals; and (3) Community Health Assessments (CHAs), which are conducted by local public health agencies for accreditation by the National Association of Public Health Directors (NAPHD).
SAMHSA Grant Programs. Requirements for SAMHSA needs assessments mainly consist of a set of questions to be addressed, with little guidance or requirements related to methodology. The major strengths of the SAMHSA-mandated reports are in addressing the "what is" component of a needs assessment: describing the existing service system, utilization patterns, and consumer characteristics. The greatest limitations are in the approach to measuring need, which typically consists of no more than prevalence figures from national data sources such as National Survey on Drug Use and Health (NSDUH) compared to utilization statistics.
Community Health Needs Assessments. CHNAs benefit from requirements added by the Affordable Care Act to the original Internal Revenue Service requirements that include broad community representation and an accompanying Community Health Improvement Plan detailing activities to be conducted by the hospital to promote public health and address issues identified in the CHNA--the "what should be" component of a needs assessment.
Community Health Assessments. CHAs benefit from initial support provided by the Robert Wood Johnson Foundation and very detailed requirements by the NAPHD for areas and topics to be addressed. Methodology is enhanced by the availability of a framework for conducting the needs assessment known as Mobilizing for Action through Planning and Partnerships, which was developed jointly by the National Association of County and City Health Officials and the Centers for Disease Control and Prevention. Also, many state public health agencies provide additional guidance and support for local agencies.
Locally Initiated Needs Assessments
The following is a summary of results of the review of locally initiated needs assessment, organized according to the research questions.
What questions are addressed?
Approximately 40 different issues and topics were identified, with the largest number, as expected, related to prevalence, utilization, and unmet need. Estimated demand (services likely to be requested in response to certain initiatives) was addressed by 17 of the 33 reports.
Some important topics received relatively limited attention; for example, only a handful of reports addressed network assets and strengths, a topic recommended in the recent methodology literature. Only one report addressed law enforcement, disparities, benefit design, peer involvement, technology, and EBPs, despite the importance of these topics in the field of SUD treatment.
Aspects of quality addressed most frequently were access and availability of services; relatively few reports examined the use of evidence-based treatments, outcomes measurement, and patient satisfaction.
What populations are addressed?
The older reports we reviewed tended to assess need for the general population, whereas more recent reports drilled down to examine availability and accessibility of services for a wide range of specific subgroups, such as demographic subpopulations and persons in a particular status (such as homelessness and incarceration), and users of specific substances.
What workers are incorporated?
Overall, the reports incorporated a wide variety of workforce categories, although individual reports tended to address only a few general categories, and few addressed peers as an aspect of workforce capacity.
What settings (geographic areas/regions) are addressed?
Most reports focused on a specific state, county, or city; only a few focused on urban, rural, or frontier regions/boundaries of the state or county. One focused on a tribal community within the northern portion of a state.
What levels of care are addressed?
We reviewed how the needs assessments addressed levels of care in connection with service capacity--that is, what kind of services are required to meet the individual-level needs of a population, with specific reference to the use of the American Society for Addiction Medicine (ASAM) Levels of Care Criteria. This important feature of needs assessment was relatively neglected, with many reports simply providing data for utilization of existing services without discussion of service gaps and few referencing ASAM Criteria.
What data sources and methods are used?
Most reports combined qualitative and quantitative data. Qualitative data included stakeholder interviews, surveys, focus groups, and public meetings, as well as review of documents. Quantitative usually consisted of national data sets (mostly NSDUH) and local administrative data such as service utilization; a few (mainly academic studies) used social indicators. While a number of the mandated needs assessments (e.g., SAMHSA block grant applications) used the National Survey of Substance Abuse Treatment Services and Treatment Episode Data Set, few of the locally initiated reports drew upon these sources. Notably, none used data from Healthy People 2020 and the National Directory of Drug and Alcohol Treatment Facilities. Need was usually estimated simplistically as the gap between treated and untreated prevalence. Recommendations were seldom data-based; instead, they were presented as expert opinion.
Are there best practices identified?
Our assessment is that the field of SUD treatment needs assessment is not yet sufficiently developed to identify formal best practices, but we were able to identify a number of exemplary and promising approaches, such as techniques to improve the precision of estimates and methods for enhancing the utility of results for planning.
Have any methods been validated?
Only three of the reports described validation of any aspects of methodology.
Quality of SUD Needs Assessment Reports
We rated the quality of the reports using an approach adapted from systematic literature reviews, whereby the quality of studies are rated based on clarity and transparency of the information presented about various aspects of the methodology, in this case the methodology related to the research questions and to needs assessment generally.
Quality ratings for the reports as a whole varied considerably depending on the measure. For example, most reports rated highly on measures related to settings, populations, data methods and limitations, treatment needs and recommendations. On the other hand, few clearly defined the selection criteria for workers and only about half provided clear definitions of levels of care. Few offered more than a brief explanation of need or described validations of any aspects of methodology. About half referenced a model of "what should be," such as the ASAM Level of Care or the SAMHSA Good and Modern Behavioral Health System.
Needs Assessment from Other Provider Types
A separate environmental scan was conducted to identify needs assessments for other provider types that may be adapted for SUD. We examined Health Professional Shortage Areas (HPSA) and Delivery System Reform Incentive Program (DSRIP).
HPSA assessments, which are conducted by primary care clinics applying for Health Resources and Services Administration funding, address provider shortages in primary care, dental care, and mental health. Determination of eligibility for funding is based on a scoring algorithm that is specific to each of the three provider types. Features relevant to SUD needs assessment are: determination of need based on three types of shortage (geographic, population, facilities), scoring based on weights for a set of social indicators, and detailed definitions of components. As a model for SUD treatment capacity needs assessments, the HPSA approach has one primary limitation: it is a method for allocating resources to localities where the shortages are greatest but is not a method for planning at the system level (i.e., allocating resources within areas, such as how many psychiatrists, nurses, social workers, etc., should be provided in a particular area).
DSRIP is a type of Centers for Medicare & Medicaid Services (CMS) Section 1115 Waiver that allows Medicaid funding to be used to create incentives for providers to pursue a wide range of system reforms. States require eligible entities (e.g., health care systems) to submit a plan describing projects and metrics they intend to implement, features of which could provide models for smaller-scale SUD needs assessments. We reviewed one example of a DSRIP report, the Capital Region, Mohawk and Hudson Valley DSRIP Community Needs Assessment (CNA) conducted by the Albany Medical Center and Ellis Performing Provider Systems (Albany Medical Center PPS & Ellis PPS, 2014). Some features of this report that offer models for SUD treatment needs assessments include: engagement with a diverse group of organizations in developing the CNA, diverse methods of data collection designed to fill gaps in any one source, detailed information about providers from the state's Provider Network Data System, distribution of behavioral health providers in local areas (square miles by neighborhood), and summary of system assets and resources.
How States Operationalize Network Adequacy Standards
We also reviewed procedures and policies related to network adequacy standards in selected states (California and North Carolina). Network adequacy standards established by the states reflect the "what should be" condition in a needs assessment as defined (at least implicitly) at the level of state policy making.
CMS regulations require states to develop and assess standards for certain specialty services including behavioral health. These changes aimed to align Medicaid managed care regulations with requirements of other major sources of coverage. There are three parts of the Managed Care Rule that comprise the majority of network adequacy standards:
Time and Distance
States are required to develop time and distance standards for adult and pediatric behavioral health providers (that is, the number of minutes it takes a beneficiary to travel from their residence to the nearest provider site, and the number of miles a beneficiary must travel from their residence to the nearest provider site). Plans are required to meet the standards for time or distance.
Plans are required to meet state standards for timely access to care and services, taking into account the urgency of the need for services. Timely access standards refer to the number of days in which a plan must make an appointment available.
Plans are required to demonstrate that they offer an appropriate range of services that is adequate for the anticipated number of beneficiaries for the service area and that they maintain a network of providers adequate to meet the needs of the anticipated number of beneficiaries.
Some states are using newer approaches to determine if they have sufficient capacity to meet the needs of beneficiaries with SUD.
The California Department of Health Care Services (DHCS) has developed a Network Adequacy Certification Tool (NACT) for plans to collect information on location, current capacity, and projected capacity of SUD providers. Using the information from the NACT, the Plan must submit to DHCS geo-access maps of all the network providers in the Plan's service area, which plot the time and distance for all SUD network providers, stratified by service type and geographic location.
The North Carolina Department of Health and Human Services (DHHS) has developed network adequacy standards for plans, but requests that the plans use this information to identify the gaps in network adequacy and accessibility as well as strategies to address the identified gaps. DHHS has developed a tool that set forth requirements for Managed Care Organizations (MCOs) to conduct a Community Behavioral Health Service Needs, Providers and Gaps Analysis and requires MCOs to develop a strategic plan to address gaps for Medicaid and non-Medicaid beneficiaries.
Recommendations and Conclusions
The final report will focus on the research questions that call for recommendations, drawing upon input from the TAG and issues identified in the reviews of literature and needs assessments reports described above. The following are areas identified in the environmental scan that call for recommendations in the final report:
Data and methods for gauging treatment needs and gaps.
Questions that should be answered in needs assessments.
Incorporating best staffing practices, telehealth policies and peer supports into needs assessments.
Supplying key pieces of missing data.
Validating needs assessment methods.
Approaches for making needs assessments more uniform.
Increasing the effectiveness of needs assessment in promoting organizational and system change.
The SUD-specific assessments that we reviewed mostly share certain features such as mixed methods of data collection (surveys, key informant interviews, focus groups, services utilization data, etc.) but with considerable variability in the scope and level of detail provided. Some areas of variability include:
Extent to which recommendation priorities, feasibility, and strategies are presented.
Extent to which resources/assets/strengths are identified.
- Extent to which system capacity needs are broken out by levels of care.
Extent to which system redundancy/efficiency is addressed.
Other important aspects of treatment capacity were addressed by only a few of the needs assessments. These included peer involvement, the functions of law enforcement, estimates of changes in demand resulting from system reforms, treatment completion/adherence as an aspect of utilization, and the extent of EBPs.
It is evident from our review that SUD treatment presents some unique challenges for needs assessment, mainly in the considerable variability in how SUD treatment systems are structured and in the relative lack of uniformity in defining aspects of systems (such as workforce designations, service categories, provider types and levels of care) compared to health care and even mental health.