DISCLAIMER: The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.
Despite improvements in behavioral health treatments, gaps remain between evidence-based care and the care provided to millions of individuals living with mental health problems (Institute of Medicine 2006). As part of its National Quality Strategy, CMS is committed to reducing this gap by developing and implementing measures that can be used for quality improvement within inpatient psychiatric facilities (IPFs). To further the implementation of such measures, and as mandated in Section 3401, Subsection 10322 of the Patient Protection and Affordable Care Act of 2010, CMS developed the Inpatient Psychiatric Facility Quality Reporting (IPFQR) program, a pay-for-reporting program that went into effect for fiscal year 2014. Under this program, IPFs must report their performance on a set of quality measures or face a 2 percentage point reduction to the update of their Medicare standard federal rate for that year.
Over 1,800 IPFs (both freestanding psychiatric hospitals and psychiatric units of general hospitals) reported their performance on several measures in the first year of the IPFQR program. These measures include six chart-based process measures that address patient safety, care coordination, and medication use.1 Although the six measures currently included in the IPFQR program provide a strong foundation for improving the quality of inpatient behavioral health care, gaps in measurement persist.2
In September 2012, the Office of the Assistant Secretary for Planning and Evaluation, with support from CMS, modified an existing contract with Mathematica Policy Research and its subcontractor -- the National Committee for Quality Assurance (NCQA) -- to develop measures for the IPFQR program. The goal of this new component of the project was to develop and test four chart-based measures that assess screening for risk of suicide, risk of violence, substance use, and metabolic conditions, and one claims-based measure that assesses whether Medicare beneficiaries receive follow-up care after IPF hospitalization.
The first phase of work under this contract involved conducting a targeted review of evidence to support the selected measure concepts; this review was completed in late 2012. Next, the team held several meetings with IPF staff and other subject matter experts to obtain input and guidance on the technical specifications of these measures. In September 2013, the team presented draft specifications for the five measures to a technical expert panel (TEP), and the TEP provided the team with useful feedback on ways to further refine and strengthen the specifications prior to measure testing.
In early 2014, the team pilot tested the chart-based measures at six IPFs and began testing the claims-based measure using Medicare claims data. Starting in April 2014, Mathematica and NCQA staff also gathered qualitative feedback on the performance and usability of the measures through debriefing sessions with IPFs that participated in testing, as well as focus groups with state policymakers, consumer and advocacy groups, measure experts, IPFQR program vendors, and additional IPF staff. The results of quantitative measure testing are summarized in Table ES.1.
| TABLE ES.1. Measures Tested, Performance | |||
|---|---|---|---|
| Measure | Variation in Measure Performance Across IPFs (number of IPFs)1 | Mean Measure Performance1 | Reliability2 |
| Screening for risk of suicide | 67.6-99.4% (6 IPFs) | 93.4% | 0.65 |
| Screening for risk of violence | 47.7-99.1% (6 IPFs) | 89.0% | 0.63 |
| Screening for substance use | 51.4-96.4% (6 IPFs) | 85.8% | 0.49 |
| Metabolic screening | 6.2-98.6% (6 IPFs) | 41.5% | 0.93 |
| Follow-Up after IPF hospitalization (30 days)3 | 0-100% (1,669 IPFs) 25th percentile: 42.3 75th percentile: 67.3 | 53.5% | 0.93 |
NOTES:
| |||
Measure Testing Results
Admission Screening Measures. The three admission screening measures -- screening for risk of suicide, risk of violence, and substance use -- require that IPF staff collect information on core screening elements within one day of patient admission. Performance was quite high across IPFs on these measures, with average performance on the measures ranging from 86 percent (in the case of substance use) to 93 percent (in the case of suicide). Reliability was moderate for the substance use measure and substantial for the suicide and violence measures. Stakeholders were generally supportive of the measures and thought they represented an improvement over existing screening measures used in an inpatient psychiatric setting, including HBIPS-1: Admission Screening for Violence Risk, Substance Use, Psychological Trauma History and Patient Strengths, a TJC measure reported by a large portion of IPFs throughout the country.
Regarding changes to measure specifications, stakeholders generally recommended that the final specification of the substance use, violence, and suicide screening measures use a three-day time frame to allow for complete and accurate screenings. Obstacles to performing accurate screenings within one day of admission include staff shortages, patient uncooperativeness, and lack of patient lucidity. Some stakeholders noted that the suicide and violence measures should be conducted within a one-day time frame, given the clinical importance of obtaining that information quickly. Based on this feedback, the research team recommends changing the time frame for the substance use screening measure from one day to three days, and keeping the suicide and violence screening specifications at one day (as specified prior to testing). The additional two days for the substance use measure will facilitate the capture of complete and accurate information regarding patients' alcohol and drug use, without compromising the need to capture important information on suicide and violence risk in the first day of admission.
Metabolic Screening Measure. The metabolic screening measure requires that the following four screenings are documented in the patient record for all individuals discharged on antipsychotic medications: (1) body mass index (BMI); (2) blood pressure; (3) glucose or glycated hemoglobin (HbA1c); and (4) a full lipid panel. Performance on the metabolic screening measure was low, on average, across the six IPFs. The measure's average performance rate of 42 percentage points highlights a sizable performance gap on the measure. The metabolic screening measure also demonstrated non-trivial variation in performance among IPFs as well as by patient characteristics. In addition, it demonstrated near-perfect agreement between chart abstractors (kappa of 0.93 for the measure numerator).
Overall, stakeholders found the metabolic screening measure to be important for addressing a notable gap in psychiatric care. However, focus group participants and TEP members were divided over whether to keep the requirement of a full lipid panel, as some felt that blood pressure, BMI, and glucose/HbA1c tests were sufficient screening requirements. In particular, three of nine TEP members expressed concern that the measure might inadvertently encourage IPFs and other clinicians to conduct unnecessary tests -- namely a full lipid panel in instances in which there is no clinical need. However, given the preponderance of clinical evidence supporting a full lipid panel on an annual basis for patients taking regularly prescribed antipsychotic medications, we suggest that the full lipid panel remain a screening element in the metabolic screening measure.
Follow-Up Measure. The claims-based follow-up measure calculates the proportion of patients that had an outpatient visit with a mental health practitioner within seven and 30 days following IPF hospitalization. The measure demonstrated strong quantitative performance; there was good variation in measure performance across IPFs and among demographic subgroups. In addition, IPFs' low average performance on the measure on a national scale highlights room for improvement. The measure also had very good reliability (beta-binomial statistic of 0.93 for the 30-day measure).
Stakeholder support for the follow-up measure was mixed. Three of the six IPFs involved in testing, and at least 11 of the 28 focus group participants expressed concern that the measure may inappropriately hold IPFs solely accountable for follow-up care, despite the range of community-level factors that may influence performance on the measures. However, at least five focus group participants -- primarily policymakers and measurement experts -- noted that this measure could help to drive innovative partnerships between facilities, community mental health agencies, health plans, and providers to improve follow-up care for IPF patients. Likewise, TEP members were divided in their support for the follow-up measure. Two TEP members were concerned that the measure would unfairly hold IPFs accountable for factors outside of their control, whereas two other TEP members expressed strong support for the follow-up measure, arguing that it could identify opportunities for quality improvement among facilities with low rates of follow-up care.
NOTES
-
These are the Hospital-Based Inpatient Psychiatric Services (HBIPS) measures 2-7, developed by The Joint Commission (TJC) and endorsed by the National Quality Forum.
-
Another TJC measure, Alcohol Use Screening (SUB-1) will be incorporated into the IPFQR in 2015, along with the follow-up measure presented in this report.