Performance Improvement 1997. Oregon Consumer Scorecard (OCS) Project



The final report on the Oregon Consumer Scorecard (OCS) Project describes the development, pilot testing, and revision of a user-friendly guidebook, "A Consumer Guide to Selecting a Health Plan," by the Oregon Consumer Scorecard Consortium. The prototype scorecard is intended to help consumers choose a health plan consistent with their individual needs and health care service delivery preferences. The model also may help purchasers and State policymakers develop their own comparative reports about health plan performance.


A contract from the Agency for Health Care Policy and Research was awarded in January 1995 to the Oregon Health Policy Institute through the University of Washington for this Federal-State collaboration in policy making and knowledge-building. There were five project objectives:

  1. Identify information consumers want and need to make better choices.
  2. Identify different information needs, based on geographic location and health conditions.
  3. Identify relevant health plan performance measures.
  4. Pilot test various formats for displaying quality, access, and satisfaction measures to consumers.
  5. Make recommendations for designing a broader statewide demonstration to evaluate a preferred scorecard format for statewide implementation.

The model scorecard was intended to aid consumer decision making in two primary ways. First, it was to provide comparative health plan descriptions based on the expressed information preferences of Oregon consumers. Second, it was designed to educate consumers about health plan function, management, and performance. The project was guided by the presumption that people will make more prudent choices among health plans if they have access to objective, reliable measures of technical quality, health plan performance, and consumer satisfaction.


Oregon has one of the most extensive managed care markets in the United States. After Oregon received approval for a Federal Medicaid waiver, the State legislature enacted a series of reforms known as the Oregon Health Plan (OHP). Since its implementation, OHP has extended coverage to approximately 130,000 individuals, including many previously without health insurance coverage who must now navigate an unfamiliar health care delivery system.

These reforms led to the establishment of the Oregon Office of the Health Plan Administrator (OHPA), with responsibilities that included "to assist OHP consumers in selecting a health care provider or a health care plan" (SB 5530, Oregon Statutes of 1993). The OHPA facilitated the creation of the Oregon Consumer Scorecard Consortium, a public-private partnership committed to developing a high-quality, user-friendly health plan guide.


A series of activities was undertaken to develop and pilot test stimulus materials that would subsequently inform the design of a prototype consumer scorecard to be evaluated on a statewide basis. The first step was the formation of the Oregon Consumer Scorecard Consortium, comprising more than 50 stakeholders from the public and private sectors that may benefit from or be affected by the scorecard.

Second, a work plan was established for reviewing and synthesizing policy-related information, conducting focus groups, and reporting on all other Consortium-sponsored data collection activities. The information review and synthesis included analyzing the Oregon Office of Medical Assistance Programs' annual consumer satisfaction survey, designed during the project year in consultation with the OCS Project staff and the Consortium's Technical Committee; pilot testing measures from Medicaid HEDIS (Health Plan Employer Data and Information Set), a widely used health plan performance measurement tool; and gathering objective, comparative, and descriptive navigational information to help consumers understand how a health plan really works.

Through a subcontract to Oregon Health Decisions, Inc., two rounds of four focus groups were conducted with OHP-Medicaid consumers. Targeted at consumers living in rural areas and persons with chronic health care conditions and disabilities, the first-round focus groups were designed to gain understanding of consumers' expressed information needs and preferences for selecting a health plan. In the second round, consumers who had a choice between two or more competing health plans in their counties were given a model guidebook and a set of alternative formats and media presentations to use and critique.

Third, the first set of prototype scorecard materials was produced. Finally, using real health plan data but fictitious health plan names, the guidebook was presented to Oregon consumers in a series of focus groups for their review and assessment.


Consumers expressed a desire for more accountability from their health plans. They wanted objective information on satisfaction and health plan performanceþinformation that is understandable, reliable, meaningful, and geographically sensitive. There is a need for unambiguous descriptive data, free of marketing bias, that show real differences in how health plans manage care.

Materials developed during this project were well received. Consumers were interested in multimedia presentations of choice information, using interactive computer kiosks, videos, and telephone advice. The availability of a trained, live individual was seen as an important backup to the scorecard.

The report noted that consumers and health professionals think differently about quality. Plan-level information was less important to consumers than provider-level information. Consumers preferred information that was relevant to their individual experience, including how they personally use health care services. Condition-specific indicators in population-based performance measures (such as HEDIS) were found to be less well understood, and therefore not as useful to consumers in selecting a health plan.

Despite the advantages of this voluntary collaboration, producing a reliable and uniform consumer scorecard in a timely fashion proved difficult. The report recommended the establishment of uniform standards and data specifications that all health plans and purchasers agree to abide by, with an independent "audit" to ensure compliance.

The report also noted that the political nature and the potential economic consequences of the comparative health plan information made it critically important to have an objective, disinterested third party responsible for producing a scorecard. In addition, because scorecard development is politically and methodologically complex, the report advised that future efforts should proceed in stages, with more modest goals and an examination of the considerable costs of producing a scorecard.

Use of Results

This project provides a unique model for integrating major types of health plan performance information into a consumer-oriented guidebook and adds to the emerging body of knowledge on consumer information needs and preference modeling. This contribution to the science base for statewide quality assessment and reporting can be used by other States in their efforts to evaluate quality of care.

Major policy and methodological issues to pursue in the future are addressed in the report, including the need to address differences in how professionals and consumers think about quality; the need to balance the focus on health plans as the unit of comparison for scorecard purposes with the interest of most consumers in the attributes and performance of individual clinicians and the facilities in which they practice; the need to reconcile population-based measures and personalized, consumer-relevant information; the need to tailor information to consumers with special concerns; and the need to critically evaluate the costs and benefits of various forms of presentation. The report recommends as a top public policy goal the development of a real and meaningful quality feedback loop that extends from the State to health plans to consumers and back to the State.

Agency sponsor:

Office of Planning and Evaluation

Federal contact:

Sandra Robinson
PIC ID: 5960

Performer organization:

Oregon Health Policy Institute, Portland, OR