Performance Measurement in the Hospital Outpatient Setting. Study Limitations and Considerations for Future Analysis and Measurement Development


This study constitutes an initial assessment of the hospital outpatient measurement landscape. We identify several limitations that could be addressed by additional analytic work to flesh out the best opportunities for performance measurement in the hospital outpatient setting:

  • The analytic files made available contained service-line information rather than information aggregated at the level of an encounter or an episode of care. As such, we were not able to describe the spectrum of individual services a Medicare beneficiary receives during a single visit. The data used in this analysis only permitted us to examine separate pieces of encounters without being able to account for the full set of services provided to a patient during an encounter, for example a patient with diabetes. Subsequent analyses conducted using encounter-level data, or possibly even the level of an episode, would provide a more complete picture of the quality of care Medicare beneficiaries receive for a given condition.
  • The analysis of International Classification of Disease Version 9 (ICD-9-CM) diagnostic and procedure codes was restricted to the first four digits. Analyses conducted at the level of the fifth digit could assist, in some cases, in further elucidating distinctions that may not have been apparent based on our more aggregated analyses. These distinctions may have important implications for measures development for some diagnoses and services/procedures that are specified in greater detail than was examined in our analyses. For example, the fifth digit of diabetes codes (250.xx) is used to distinguish between both type I and type II diabetes, and whether the diabetes is controlled (i.e., whether the current treatment regimen keeps the blood sugar level of a patient within acceptable levels) or uncontrolled.41
  • The analysis compared reasons for visits to the Hospital Outpatient Setting (HOPS) to existing measures. For the measures mapping segment of our analysis, we focused on the HOPS because the majority of existing measures correspond to conditions and diagnoses that most commonly occur in the HOPS, rather than the ED. We acknowledge that some conditions and services/procedures occur more frequently in the ED setting. Therefore, a separate synthesis that focuses on mapping measures to the care provided in the ED, in order to capture encounters in the ED that are distinct from the HOPS and do not result in an acute admission, merits consideration for future analyses.
  • The groupings used to classify reasons for visits were informed by the judgment of two clinicians and not an interdisciplinary panel of physicians and measurement experts. RAND developed diagnosis groupings to ensure that the most common diagnoses that have multiple diagnosis codes at the four-digit level were aggregated, thereby accurately reflecting their collective frequency and costs. While these groupings were established by two RAND physicians-researchers, it may be useful to convene a larger, interdisciplinary group to revisit them to ensure they have clinical face validity.
  • The analysis examined only the five most common diagnoses within an Ambulatory Payment Classification (APC) code. While this level of analysis provides a useful overview, a more detailed examination of APCs may shed light on additional issues related to services/procedures than were captured in our review.
  • The findings regarding drugs/biologicals were not aggregated by drug or drug class. For our analyses, we had information only for drugs paid via APCs and we had incomplete information even for those drugs with APC payments. Because of the data incompleteness, we did not go the next step and aggregate drugs/biologicals by particular drugs or drug classes, as our estimates would have been inaccurate. Future analyses could examine a complete set of drug/biologics information and consider opportunities for measure development.
  • The analysis did not consider changes in cost over time. To better understand the key drivers of cost growth in the Medicare hospital outpatient setting, future analyses could include an analysis of changes in costs over time by various conditions, services/procedures, and drugs/biologicals.
  • The analysis did not assess the density of services/procedures or conditions within each hospital that bills under OPPS, to determine whether there are likely to be a sufficient number of scoreable events. In addition to these limitations, we identified a number of issues that could be taken into account during the measures development process. These include the following:
  • Care and services delivered in the hospital outpatient setting are not homogenous across hospitals or populations served. The types of services/procedures delivered in the hospital outpatient setting vary hospital to hospital, making it challenging to develop a set of hospital outpatient measures that can be applied to all institutions because of differences in service mix and patient populations. As one discussant noted, outpatient care is “a hodgepodge of ambulatory and inpatient-like care.” Consequently, CMS may wish to consider having a stable of measures, and hospitals would be required to submit data on all measures applicable to their service mix and patient population. The variations will create unevenness in the burden of data collection and reporting across institutions.
  • Some existing ambulatory care measures may need to be modified for use in the hospital outpatient setting. The majority of existing clinical effectiveness measures has been developed to assess ambulatory care (applied at the physician level), and many of them likely could be directly applied with minimal changes to ensure the administrative codes or other data sources used to populate the measure are correct. Existing measures should be carefully reviewed by measurement and clinical content experts to determine whether and how adjustments to the measures specifications are required in order for them to be applied to the hospital outpatient setting.
  • To assess appropriate use of services/procedures, it is necessary to distinguish between the provider ordering a service and the one delivering it. Most ambulatory care measures do not explicitly distinguish the fact that the physician ordering a service (e.g., an ED physician ordering an MRI) may differ from the one providing it (e.g., the radiologist reading the MRI results). This is often true for services provided by consulting physicians (e.g., radiologists, pathologists, cardiologists), and is of significant interest given the frequent use and cost of such services. Our analysis found that radiological imaging services were among the top 20 most common and most costly for both the HOPS and ED — underscoring the need to have measures that assess appropriate ordering of these services by providers. Measures that take this issue into account could assist CMS in better understanding the drivers of use, and in assessing whether that use is appropriate.

View full report


"outpatient.pdf" (pdf, 1001.94Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®