To determine the factors that contribute to health plan success in engaging plan members in SUD treatment and to identify a more effective strategy to measure engagement, we used a sequential, explanatory mixed methods study design where quantitative data were analyzed before qualitative data were collected to help explain results observed in the quantitative analyses. We initially conducted an environmental scan to provide background on: (1) the epidemiology of substance use, SUDs, and treatment; (2) factors associated with treatment initiation and engagement; (3) interventions designed to improve initiation and engagement; and (4) the development and use of the IET measure. The results of the scan informed both the quantitative and qualitative research that followed. The quantitative analyses relied on data from a large commercial health plan data set linked to geographic state-level market variables. Guided by the results of the quantitative analyses and the preparatory scan, we then undertook a series of qualitative case studies of health plans that were selected based on their performance on the IET measure and other critical characteristics (e.g., geographic distribution). This tripartite study design is depicted in Figure 5.
|FIGURE 5. Tripartite Mixed Methods Study Design|
The following research questions guided the quantitative analyses and include questions related to health plan and environmental factors regarding SUD treatment and OUD treatment.
Question 1: Do any of the following health plan factors affect initiation and/or engagement in alcohol or other SUD treatment?
Coverage of care continuum for SUD.
Out-of-pocket expenses for SUD treatment.
Reimbursement for outpatient and inpatient SUD treatment.
Percentage of population with an SUD.
Type of plan.
Plan beneficiary characteristics (age, sex, use of emergency department).
Number of beneficiaries.
Question 2: Do any of the following environmental factors affect a health plan's performance on initiation and/or engagement in alcohol or other SUD treatment?
State demographic profile (race/ethnicity, poverty).
Prevalence of alcohol or SUDs.
Attitudes toward SUD treatment.
Policies to reduce opioid prescriptions for pain.
Question 3: Do any of the following health plan factors affect the initiation and/or engagement in SUD treatment for individuals with OUDs?
Coverage of care continuum for SUD.
Out-of-pocket expenses for MAT.
Reimbursement for outpatient and inpatient SUD treatment and for MAT.
Receipt of MAT for OUDs.
Percentage of beneficiaries with OUD.
Type of plan.
Plan beneficiary characteristics (age, sex, use of emergency department).
Number of beneficiaries.
Question 4: Do any of the following environmental factors affect a health plan's performance on initiation and/or engagement in SUD treatment for individuals with OUDs?
Availability of MAT providers.
State demographic profile (race/ethnicity, poverty).
Prevalence of OUDs.
State attitudes toward MAT as reflected in Medicaid coverage for all three medications for OUDs and SSA funding.
Policies to reduce opioid prescriptions for pain.
For the quantitative analysis of factors associated with initiation and engagement among adults, we used the 2013-2014 Truven Health MarketScan® Commercial Claims and Encounters (CCAE) Research Database, linked to geographic information that provided state-level market characteristics. The MarketScan database is created from two sources: self-insured employers and regional health plans. For this study, we limited the data to those submitted by self-insured employers because this information includes all behavioral health claims.
We linked several state-level variables to the health plan data to provide information about the environmental context in which the health plan functions. We linked the data using the state where most plan beneficiaries live. Our primary goal with these variables was to assess the following environmental characteristics: (1) provider availability; (2) underlying prevalence of alcohol and other substance disorders; (3) attitudes toward SUD treatment; and (4) state demographic characteristics. We considered several publicly available data sources to provide the environmental variables. The databases we selected to incorporate these characteristics are included in our description of covariates in Table 9.
The unit of analysis for this study is the employer health plan. Table 7 describes the number of employer health plans with more than 10, 20, 30, and 50 beneficiaries that met the denominator criteria for the IET measure. It also includes the number that met those denominator criteria if restricted to beneficiaries with OUDs. Because comparatively few plans had sufficient eligible beneficiaries with OUDs to allow meaningful analysis, we implemented a requirement for 20 or more cases (as is done in some CMS reporting practices) and used data from both 2013 and 2014. This allowed us to include 321 health plans in our larger analysis of the rates for SUD treatment, with a mean of 50,585 beneficiaries, and 82 plans for the analysis of the rates for OUD treatment, with a mean of 92,521 beneficiaries. Because the IET measure requires that an Index Episode Start Date (IESD) be established with a 60 day "clean period" immediately preceding the IESD, we also used data from the end of 2012 to assure a clean period for those with episodes beginning early in 2013. More information on the IESD and related clean period is provided in the methods summary of outcome variables.
|TABLE 7. Number of Employer Health Plans with more than 10, 20, 30, and 50 Beneficiaries Meeting the Denominator Criteria for the IET Measure, Overall and Limited to Those with OUDs, 2013-2014|
|Measure||More than 10 Beneficiaries||More than 20 Beneficiaries||More than 30 Beneficiaries||More than 50 Beneficiaries|
|Limited to OUDs|
|SOURCE: Truven Health MarketScan CCAE Research Database.|
The beneficiary population for the health plans studied included all adults (aged 18 years and older) with an alcohol or other drug (AOD) diagnosis who were enrollees in the employer health plans that contributed data to the 2013-2014 MarketScan CCAE data. As discussed in greater detail below regarding the outcome variables, an IESD was established for that group; any adults who did not have a 60 day "clean period" prior to the IESD were excluded from the beneficiary population used for rate calculation. For the analyses that focused only on those with OUD disorders, the beneficiary population was further reduced to include only that subset of the larger group.
Four outcome variables were included in these analyses (Table 8). The outcome variables were derived from the commonly used measure of IET, discussed in greater detail below.
|TABLE 8. Outcome Variables|
|Variable Name||Data Source||Description|
|Initiation SUD||MarketScan CCAE Database||As specified in the IET measure for AOD dependence treatment initiationa|
|Engagement SUD||MarketScan CCAE Database||As specified in the IET measure for AOD dependence treatment engagement|
|Initiation OUD||MarketScan CCAE Database||Amended the specifications from the IET measure for AOD dependence treatment initiation so that it only included the services with OUD diagnoses for identification|
|Engagement OUD||MarketScan CCAE Database||Amended the specifications from the IET measure for AOD dependence treatment engagement so that it only included the services with OUD diagnoses for identification|
The IET quality measure contains separate rates for initiation and engagement, with engagement measured among those who have initiated treatment. The IET quality measure defines initiation and engagement as follows:
Initiation: the percentage of members who initiate treatment through an inpatient AOD admission, outpatient visit, IOP encounter, or partial hospitalization within 14 days of diagnosis.
Engagement: the percentage of members with a diagnosis of AOD dependence who initiated treatment and had two or more additional services with an AOD diagnosis within 30 days of the initiation visit.
|FIGURE 6. Initiation Rate Calculation Process|
The initiation rate is the Initiation Numerator divided by the IET Denominator.a
In these analyses, each rate was used twice--first, to measure initiation and engagement in AOD dependence treatment and second, to measure initiation and engagement in OUD treatment for those health plan members with OUD diagnoses. Appendix B provides the list of diagnoses included for OUD and other SUD analyses.
The IET specifications used to measure initiation and engagement in this study are those that are published by NCQA as HEDIS measures, which may be obtained through the NCQA website. The NCQA specifications are similar to those used for the Medicaid Adult Core Set, although the Adult Core Set only applies to Medicaid enrollees and there are differences in the age ranges covered. The 2016 CMS Medicaid Adult Core Set is publicly available on the CMS website. The value sets that include the codes needed to calculate the measure also are located on the CMS website, or they may be acquired with the NCQA specification. Copyright considerations preclude including the NCQA specifications or the NCQA and Adult Core Set value sets as an appendix to this document. Figure 6 and Figure 7, however, depict the process of rate calculation for the IET measure.
|FIGURE 7. Engagement Rate Calculation Process|
The engagement rate is the Engagement Numerator divided by the IET Denominator.a
IET Denominator: Same as Denominator in Figure 6.
We included covariates in our quantitative analyses that addressed five types of potential influences that were based on the results of our environmental scan, in which we examined the literature on factors that may promote or inhibit initiation and engagement in SUD treatment. These included: (1) health plan structure; (2) reimbursement factors; (3) benefit design; (4) plan beneficiary characteristics; and (5) state-level market and environmental characteristics. Table 9 describes in detail the variables within each category.
|TABLE 9. Covariates|
|Variable Name||Data Source||Description|
|Health Plan Structure|
|Plan type||MarketScan CCAE Database||Categorical variable indicating health plan structure as: (1) PPO; (2) HMO or capitated; or (3) high deductible health plan.a For regressions, PPO was the reference group.|
|MAT OOP||MarketScan CCAE Database||Median OOP cost per user for MAT medications for OUDs (paid by enrollee).b|
|Outpatient OOP||MarketScan CCAE Database||Median OOP cost per user for SUD outpatient services, including IOP treatment and partial hospitalization services (paid by enrollee).|
|Inpatient OOP||MarketScan CCAE Database||Median OOP cost per user for SUD inpatient and residential services within health plan (paid by enrollee).|
|MAT reimbursement||MarketScan CCAE Database||Median reimbursement per user to pharmacy for MAT medications for OUDs (paid by enrollee and insurer).b|
|Outpatient reimbursement||MarketScan CCAE Database||Median reimbursement per user to outpatient providers, including for IOP and partial hospitalization services (paid by insurer).|
|Inpatient reimbursement||MarketScan CCAE Database||Median reimbursement per user to inpatient and residential providers for SUD services (paid by insurer).|
|MAT use||MarketScan CCAE Database||Categorical variable indicating percentage of beneficiaries with OUD who received: (1) no MAT; (2) buprenorphine/naloxone, extended-release naltrexone, or methadone (if coded in the claims) for 14 days or less (assumed to be for detox); or (3) buprenorphine/naloxone, XR naltrexone, or methadone for more than 14 days (assumed to be for maintenance).b|
|Residential||MarketScan CCAE Database||Boolean indicator that a health plan covers residential SUD services as evidenced by having at least one or more claims for each category of service.|
|Intensive outpatient (IOP) or partial hospitalization services||MarketScan CCAE Database||Benefit availability of IOP or partial hospitalization SUD services defined as the number of IOP or partial hospitalization services per plan beneficiary.|
|Outpatient services||MarketScan CCAE Database||Benefit availability of outpatient SUD services defined as the number of outpatient SUD services per plan beneficiary.|
|Health Plan Beneficiary Characteristics|
|SUD beneficiaries||MarketScan CCAE Database||Percentage of beneficiaries/member-years within health plan with either an identified SUD diagnosis on a medical claim (non-laboratory and non-radiology) or receipt of MAT prescription. See Appendix C for the algorithm used to ascertain SUD beneficiaries.c|
|OUD beneficiaries||MarketScan CCAE Database||Percentage of beneficiaries/member-years within health plan with either an identified OUD diagnosis on a medical claim (non-laboratory and non-radiology) or receipt of a MAT prescription.b|
|Ages 18-44 years||MarketScan CCAE Database||Percentage of beneficiaries in a plan who were within the age group of 18-44 years.|
|Female||MarketScan CCAE Database||Percentage of beneficiaries in a plan who were female.|
|ED use||MarketScan CCAE Database||Percentage of beneficiaries in a plan with an SUD (or OUD) who had: (1) 0 treat-and-release ED visits with any diagnosis; (2) 1 treat-and-release ED visit with any diagnosis; or (3) 2+ treat-and-release ED visits with any diagnosis.|
|State-Level Market and Environmental Characteristicsd|
|SUD prevalence||2013-2014 NSDUH||State-level data on the prevalence of alcohol and illicit drug abuse and dependence. Because health plans may have members in multiple states, the variable was the mean prevalence of the adult population with past-year dependence on or abuse of illicit drugs or alcohol in states where health plan members reside.c|
|Opioid prescriptions||2012 CDC infographic: Number of opioid pain prescriptions per 100 people by state||The infographic provided the number of opioid pain prescriptions by state per 100 adults. Because health plans may have members in multiple states, the variable was the mean number of opioid pain prescriptions per 100 adults in states where health plan members reside.b|
|SUD capacity||2013 N-SSATS||The data provided the number of adults receiving treatment in SUD treatment facilities by state per 100,000 adults. Because health plans may have members in multiple states, the variable was the mean number of individuals receiving SUD specialty treatment per 100,000 adults in states where health plan members reside.c|
|OTP capacity||2013 N-SSATS for methadone number||The data provided the number of OTP spaces available for methadone treatment by state per 100,000 adults. Because health plans may have members in multiple states, the variable was the mean number of OTP spaces available per 100,000 adults in states where health plan members reside.b|
|Buprenorphine prescribers||SAMHSA Buprenorphine Physician Locator US Census data for population aged 18 and over||The buprenorphine locator identified the number of buprenorphine prescribers per state. The census data provided population aged 18 or older by state. Because health plans may have members in multiple states, the variable was the mean number of prescribers per 100,000 adults in states where health plan members reside.b|
|Six MAT medications||Medicaid Coverage and Financing of Medications to Treat Alcohol and OUDs (SAMHSA publication)||The publication provided information on state Medicaid coverage of MAT medications as of 2011-2012. The variable was the percentage of health plan members who live in a state that covers all 6 medications for alcohol and OUDs: disulfiram, acamprosate, methadone, buprenorphine/naloxone, naltrexone, and extended-release naltrexone.c|
|Three MAT medications||Medicaid Coverage and Financing of Medications to Treat Alcohol and OUDs (SAMHSA publication)||The publication provided information on state Medicaid coverage of MAT medications as of 2011-2012. The variable was the percentage of health plan members who live in a state that covers all 3 medications for OUDs: methadone, buprenorphine/naloxone, and naltrexone or extended-release naltrexone.b|
|Non-Hispanic White||US Census data||The census data provided the percentage of the population in each state who identified as non-minority (i.e., non-Hispanic White) versus minority. Because health plans may have members in multiple states, the variable was the mean percentage of the adult population that was non-Hispanic White in states where health plan members reside.|
|Poverty||US Census data||The census data provided the percentage of the population in each state who were below the 100 percent Federal Poverty Threshold. Because health plans may have members in multiple states, the variable was the mean percentage of the adult population that was below the poverty threshold in states where health plan members reside.|
|Private insurance||US Census data||The census data provided the percentage of the population in each state who have private, commercial insurance. Because health plans may have members in multiple states, the variable was the mean percentage of the adult population that has private insurance in states where health plan members reside.|
|PDMP||2015 Annual Review of PDMPs by the National Alliance for Model State Drug Laws||State-level data that describe which states require prescribers and/or dispensers to access the PDMP in certain circumstances. The variable was the percentage of health plan members who live in one of the 24 states that required prescribers or dispensers to access the PDMP in certain circumstances in 2014.|
|Single state authority (SSA) spending||SAMHSA Publication: Funding and Characteristics of Single State Agencies for Substance Abuse Services and State Mental Health Agencies, 2013||The data provide the total amount of SSA spending on SUDs by state per 100,000 adult population. Because health plans may have members in multiple states, the variable was the mean amount of SSA spending per 1,000 adults in states where health plan members reside.|
We calculated descriptive statistics separately, focusing on SUDs and OUDs. We examined characteristics of the health plan cohorts used for the overall SUD analysis and the more specific OUD analysis. We calculated mean initiation and engagement rates on the basis of health plan characteristics. For continuous variables, we calculated separate mean initiation and engagement rates for health plans that performed at or above the health plan mean and those that performed below the health plan mean. We examined plan characteristics by performance. For the SUD initiation and engagement measures, we divided the plans into tertiles on the basis of performance. For the OUD initiation and engagement measures, we divided the plans into those performing above the median (i.e., high performing plans) and those performing below the median (i.e., low performing plans). We then completed four multivariate regressions using general linear model analysis, examining the relationship between the covariates described above and the initiation and engagement measure outcomes. The separate regressions addressed initiation and engagement measure outcomes for SUD treatment and for OUD treatment alone.
Three primary research questions guided the qualitative analyses to help us determine how successful health plans improve enrollee initiation and engagement in SUD and OUD treatment.
Question 1: Which types of health plan characteristics and strategies are demonstrated by plans with higher performance or greater improvement in IET in SUD and OUD treatment?
Question 2: What other factors (e.g., patient, setting, provider, state, and local market characteristics) do health plans identify as affecting rates of initiation and engagement in SUD and OUD treatment?
Question 3: What do health plan representatives believe are significant barriers and facilitators to initiating and engaging beneficiaries in SUD treatment?
Protocol and Discussion Guide Development
To guide the health plan interviews, we developed a combined site visit protocol and discussion guide document. The protocol component of this document contains logistical information to prepare the research team for specific site visits. Protocol elements included a list of all interviewees and their health plan division, position titles, and contact information. The document also included a site visit agenda, which indicated how interviewees were grouped and the key focus of their specific interview session. The final pages of the protocol included useful background information for the research team conducting the site visit. This background provided an overview of the health plan, relevant news articles related to the plan, and the plan's marketplace activity. This section also included information from documents sent by the plan prior to the site visit. The background information is described in further detail under Preliminary Data Collection.
We developed the discussion guide component of the document to identify facilitators and barriers to initiating or engaging health plan beneficiaries in SUD treatment. Major topic areas included health plan descriptive characteristics such as governance structure, benefit design, care models, reimbursement procedures and rates, network adequacy, and quality improvement methods; community and market characteristics such as state and Medicaid policies; patient factors such as use of MAT and attitudes and beliefs toward SUD treatment; provider characteristics such as billing proficiency, stigma toward SUD, outreach efforts, and use of evidence-based practices; and facilitators and barriers to initiation and engagement in treatment.
The guide includes semi-structured, open-ended interview questions to promote discussion and elicit the full range of potential responses from interviewees. Each semi-structured interview question includes a short list of optional probes to facilitate clarification and expand on the main question. Interview questions were designed to be flexible enough to allow the interviewer to stray from the guide if participant answers raise themes that the site visit lead deemed of value to the research questions. Following the first site visit, the research team made minor revisions to a few questions to promote a more natural language flow throughout the interview questions. The site visit protocol and discussion guide is attached as Appendix D.
We selected health plans for site visits on the basis of their performance across various behavioral health measures reported in NCQA data. Health plan performance was based on the most recent data available, which differed for commercial and Medicaid plans. The study analyzed NQCA Quality Compass data for commercial plan performance between January 1, 2015, and December 31, 2015, and for Medicaid plan performance between January 1, 2014, and December 31, 2014. The NCQA HEDIS measure data captures at least 75 percent of health plans in the United States.
Health plans initially were assessed on the basis of their HEDIS IET measure performance. The research team stratified the 396 commercial plans to identify plans performing in the 95th percentile on either the initiation (n=20) or engagement (n=20) measures. Among the 40 commercial plans identified, 17 were selected for extremely high performance either on one of the two measures or on both. Only seven plans performed in the 95th percentile on both measures. We repeated the stratification process on the 182 Medicaid plans to identify top performers on initiation (n=10) and engagement (n=10) measures. A total of ten Medicaid plans were selected for high performance, although only four Medicaid plans were identified as top performers on both measures.
|HEALTH PLAN SELECTION|
|We selected health plans for site visits on the basis of their performance across various behavioral health measures reported in NCQA data.|
Top-performing commercial and Medicaid plans also were assessed for their performance across several other HEDIS behavioral health measures of interest, including: (1) antidepressant medication management for the effective acute and continuation phases of treatment; (2) 7-day and 30-day follow-up after hospitalization for mental illness; and (3) mental health utilization of any IOP or partial hospitalization, or inpatient services. Additionally, we assessed two composite measures from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) measurement set, including customer service and getting needed care.
We prioritized for site visit selection any commercial and Medicaid plans identified as performing in the top 5 percent for both components of the IET. We also prioritized for site visit selection health plans that performed well on only one component of the IET and also performed well across the other behavioral health measures. High performance on the antidepressant medication management acute phase and 7-day follow-up after hospitalization for mental illness measures was favored over high performance on the other behavioral health measures because of the immediacy of measured outreach. To ensure that the health plans selected for site visits reflected geographic and socioeconomic diversity, as well as differences in state health policy, the research team also considered the state in which the health plan was located--including whether it was operating in a Medicaid expansion state--and its total enrollment size. We identified for secondary outreach an additional group of commercial and Medicaid plans with slightly lower IET, behavioral health measure performance, or both, should the prioritized plans not agree to participate in the site visits. However, this group of plans was not outreached because the researchers were able to schedule visits with the prioritized plans.
There was one exception to the aforementioned site selection process. The researchers chose to include a Northwest Medicaid plan that was not included in the NQCA data but had a history of high IET performance. Additionally, the researchers were familiar with this plan's organizational and incentive structure, which varied substantially from other Medicaid plans. We included the plan in an initial list of 12 plans (seven commercial, five Medicaid) prioritized for outreach. See Figure 8 for a visual depiction of the site selection process.
|FIGURE 8. Health Plan Site Visit Selection Process|
|* One Medicaid health plan that did not appear in the NCQA data was included in the site visits due to their history of high performance and unique organizational and incentive structure.|
Initial outreach to health plans was via email. A standard, brief description of the study was included in each initial email message. We sent multiple follow-up emails to initially unresponsive plans. When contact was established, we made a follow-up phone call to explain the study and site visit request. In several cases, the initial health plan representative with whom we spoke requested to hold multiple follow-up calls with us and other health plan staff members to gain site visit approval. Many health plans never responded to our outreach emails, and some with whom we spoke ultimately declined to participate in the study (Figure 8). When a health plan did agree to participate in a site visit, we sent site visit agendas to the health plan contact describing which health plan representatives should be included in each group interview, and we used subsequent phone calls to clarify scheduling and participation. The researchers aimed to conduct six health plan site visits that included a mixture of commercial and Medicaid health plans.
We planned site visits to include multiple natural group interviews including health plan executives (e.g., Chief Executive Officer, Chief Operating Officer, Chief Marketing Officer), managers (e.g., Director of Quality, Director of Behavioral Health, Director of Contracting, Director of Care Management), and front-line providers (e.g., care managers). The researchers aimed to conduct separate group interviews for health plan executives and leadership, behavioral health teams, quality improvement teams, and care management teams in an effort to reproduce the natural social and team dynamics in which health plan staff operate. Researchers anticipated that interviewee groupings would be based on the organizational structures of individual health plans. The researchers planned to design interview groupings in ways that would allow for: (1) duplicative questioning across different staffing groups; and (2) groups with common characteristics, such as health plan responsibilities or frequency of contact with plan beneficiaries, which would allow us to hone in on specific areas of interest.
Preliminary research. Researchers conducted preliminary research on health plans that were prioritized for site visits. Preliminary research included a limited Internet search of each health plan to identify basic plan information, including individuals in key leadership roles who should be included in outreach efforts. Once plans agreed to participate in preliminary calls to discuss the potential of study participation, the research team engaged in additional preliminary data collection.
Preliminary calls. Preliminary calls with health plans expressing interest in participating in a site visit provided valuable insight on health plan organization and an overview of quality improvement initiatives. The researchers inquired about the governance structure of the health plans to better identify key informants for the site visit interviews. Additionally, the researchers asked health plan contacts to provide a brief, general overview of health plan quality improvement efforts that occurred during the performance period or were ongoing at the time of the call. We used this information to inform the interview group schedule and to frame researcher's semi-structured questions during the interviews.
Plan-specific environmental scans. Prior to conducting the site visit, the research team performed a brief environmental scan of each health plan. These brief scans included reviews of the health plan website, plan publications including annual reports, and patient advisory reports. Such reviews yielded insight on health plan organizational structure, beneficiary population, and marketplace characteristics, including changes in Medicaid policy over time. The researchers also reviewed any recent news publications relevant to each plan.
Data Collection During Site Visits
Two research team members jointly conducted qualitative, semi-structured group interviews in person with health plan affiliates. Each interview lasted approximately 90 minutes, and there were multiple 90-minute interviews for each plan. Each interview included a group of health plan personnel or related stakeholders, with interview groupings determined on the basis of area of expertise and inquiry. One of the research team members recorded detailed descriptive notes during the interview, while the other team member conducted the interview using the prepared interview protocol and guide. Both the notetaker and the interviewer reviewed notes from each interview for accuracy. The data presented are from notes and may not reflect verbatim quotes from the interviewees.
We imported all interview notes into NVivo 11 (QSR International), a qualitative data management program. The qualitative project lead analyzed interviews using a thematic framework analysis approach in combination with more inductive strategies of grounded theory to enable novel themes to emerge within the analysis. To develop initial parent-code and subcode categories, we used constructs included in the theoretical model to explain participation in SUD treatment (Figure 1). The model describes four categories of factors hypothesized to affect initiation and engagement: (1) health plan factors; (2) market/environmental factors; (3) individual patient-level factors; and (4) provider-level factors. Each of these four model components represented an original parent-code, populated with related subcodes. Using a partial grounded theory approach enabled the researchers to analyze the data inductively and to generate new theories about additional facilitators and barriers to initiation and engagement in SUD treatment that were not previously hypothesized in the conceptual framework. Through analysis, we identified multiple subcodes within the primary framework. We added one emergent parent-code to categorize comments specific to issues concerning IET measurement, including HEDIS criteria and costs associated with tracking and reporting on the HEDIS measures.
Consensus of the research team formed the basis for development of the initial codebook. One researcher used the codebook to systematically code interview notes. A senior researcher reviewed the coding. All coding discrepancies were reviewed by the research team and resolved by consensus decision. The inclusion of new codes and subcodes that emerged through the grounded approach also was conferred on and resolved by consensus within the research team. Appendix E provides the coding scheme that was developed.