Performance Measurement in the Hospital Outpatient Setting. Challenges in Performance Measurement


In addition to identifying the need for additional measures, our synthesis and gap analysis underscored several challenges related to the operational aspects of measurement from the vantage point of the hospital, including:

  • Sample Size: Although the 2005 dataset used in our analyses contains many millions of claims, the number of visits and services/procedures that occur in any given hospital outpatient setting for a specific condition may be small. As measures are developed, it will be important to examine the data to assess whether there are adequate patient volumes at the hospital-level to ensure stable estimates of HOPS or ED performance. Additionally, the fact that additional sample size would be required to stratify data by such factors as race/ethnicity, socio-economic status, and/or gender to assess equity of care also merits consideration.
  • Lack of Provider Engagement: Most physicians are not employed directly by hospitals, making it difficult for hospitals to force adherence to hospital protocols  – in either the inpatient or outpatient setting. Discussants encouraged gain-sharing as a means to encourage joint accountability for physicians and hospitals. As one said, “In order to get traction, everyone (i.e., hospitals and physicians) has to have something at risk.” Appropriately and fairly assigning accountability for specific actions is another important way to engage providers; however, it is not always clear how to do so. For example, some patients have several physicians, raising the question of who should be held responsible in such situations.
  • Lack of Staff: Although the HOPS and ED may be able to draw staff from their associated hospitals (which generally have quality departments with analysts experienced in data abstraction), hospital outpatient settings have not traditionally been staffed or budgeted to provide the medical record abstraction and analytic services needed for performance measurement activities.
  • Lack of Adequate Detail in Existing Claims Data to Support Quality Measurement: Because claims data are generated for billing and not quality of care purposes, data elements needed for quality of care assessment are often lacking, particularly those data required for risk adjustment.
    • E&M codes broadly address the level of service provided, but do not capture specific service details and organ systems addressed. For example, it would not be possible, using standard claims data, to ascertain whether a patient’s blood pressure was taken as part of an office visit
    • Claims data also do not include specific laboratory, radiographic, or clinical values, so it is not possible to use claims data to score quality of care measures that require these data elements. For example, laboratory data indicating the need for altered diabetic therapy would be dependent on the HbA1c level. Claims data will reflect only that an HbA1c was ordered, not whether the result was such that action should be undertaken. Similarly, knowing that a hematocrit was performed is insufficient to determine whether a patient met criteria for receipt of erythropoietin. Capturing medication prescribing information related to specific conditions is notably difficult, although increasingly data may start to become available with the Medicare Prescription Drug benefit.
    • While the date of a service is captured, the actual timing of the care provided is often not evident from claims data; thus, it could not support measures that examine timing in hours, such as many measures addressing AMI care.

Development of new CPT Category II performance measurement codes as well as Medicare G codes (HCPCS level II codes) is underway, which, if used, will shed more light on the care provided during visits. In the meantime, however, this information is often not available given that it is beyond what is currently required for claims submission. Claims submissions forms may require modification to collect the necessary data elements to produce a performance measure, if administrative data sources will be used to construct measures.

Although some discussants expressed hope that electronic health records (EHRs) will be able to provide easily retrievable data, they underscored that the implementation of EHR systems ready for use in clinical performance measurement is still “a long way off.” In the near term, implementation of measures will likely entail manual chart abstraction or changes in billing codes. Registries were mentioned as a potential source of data for performance measures by representatives of at least two organizations. They indicated they were placing a higher priority on registries, as opposed to developing specific performance measures. One group believes that utilizing registries is a more effective way to improve health care quality. Both groups encouraged CMS to do more to develop and encourage national registries in a broad spectrum of clinical areas. It should be noted that TRHCA requires that, as part of rulemaking for 2008 measures, CMS address a mechanism for providing data on quality measures through an appropriate medical registry. As such, CMS is currently exploring the possibility of drawing on existing databases and registries maintained by a variety of organizations (e.g., medical professional societies, medical boards, medical group management organizations), with the goal of decreasing the burden of quality reporting for all involved while increasing the quality and usefulness of the data (Kuhn, 2007).

View full report


"outpatient.pdf" (pdf, 1001.94Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®