Performance Improvement 1996. Agency for Health Care Policy and Research

02/01/1996

Contents

Develop, Apply, and Evaluate Medical Review Criteria and Educational Outreach Based Upon Clinical Practice Guidelines: Acute Postoperative Pain and Urinary Incontinence

Developing Quality and Utilization Review Criteria for Management of Cataracts in Adults: Phase II Final Report

Developing Quality and Utilization Review Criteria for Pressure Ulcers in Adults: Phase II Final Report

Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

Design of a Survey to Monitor Consumers' Access to Care, Use of Health Services, Health Outcomes, and Patient Satisfaction: Final Report

Overcoming Barriers to Implementation and Integration of Clinical Information Management Systems: Feasibility Study

Practice Patterns and Expenditures in Managed Care and Indemnity Plans: A Condition-Specific Analysis

Directory of Minority Health and Human Services Data Resources

Development of Designs for Evaluation of the Process of Clinical Guideline Development

Users' Perceptions of Four Clinical Practice Guidelines and Their Development Process

Setting Priorities for Clinical Practice Guidelines

AHCPR Clinical Practice Guideline Program: Report to Congress

TITLE: Develop, Apply, and Evaluate Medical Review Criteria and Educational Outreach Based Upon Clinical Practice Guidelines: Acute Postoperative Pain and Urinary Incontinence

ABSTRACT NUMBER: 025

ABSTRACT: This project, a contract with the Center for Clinical Quality Evaluation in Washington, DC, was designed to translate three AHCPR-supported clinical practice guidelines into tools that could be used to evaluate quality of care. Using the methodology developed in "Using Clinical Practice Guidelines to Evaluate Quality of Care: Issues and Methods" (AHCPR Publication Numbers 95-0045 and 95-0046), the project developed quality/utilization review criteria and clinical performance measures based on three guidelines: urinary incontinence; acute postoperative pain; and benign prostatic hyperplasia, or BPH. The project used five Medicare Peer Review Organizations to develop and test the criteria and performance measures. These criteria and measures were used to assess the care provided to a sample of randomly selected cases of Medicare patients. The peer review organizations also played an integral role in developing, implementing, and evaluating alternative educational outreach strategies based on the BPH guideline.

The project found that it is possible to develop valid and reliable guideline-based performance measures from AHCPR-supported guidelines. The structure of guideline recommendations can strongly influence the development of clinical performance measures. The project also found that targeted educational efforts based on the BPH-supported guidelines resulted in improved performance on specified measures. (Final report 107 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5973

PERFORMER ORGANIZATION: Center for Clinical Quality Evaluation, Washington, DC

TITLE: Developing Quality and Utilization Review Criteria for Management of Cataracts in Adults: Phase II Final Report

ABSTRACT NUMBER: 026

ABSTRACT: This project developed and tested quality and utilization review criteria and review systems based on AHCPR-supported clinical practice guidelines on cataracts in adults. The project included two phases. In phase I, project teams developed and tested initial drafts of review criteria and review systems. These drafts were then refined and key components tested again in phase II. Conclusions from phase I are (1) guideline-based review criteria and systems can be developed and used in reviewing quality and utilization in a range of health care delivery settings; (2) review criteria derived from a guideline can be organized into many types of review systems with different goals and purposes; (3) guidelines must contain several critical elements to be useful for clinical decisionmaking and for development of review criteria and review systems; (4) operationalization and measurement are key challenges in review system development. Phase II conclusions are (1) the medical record-based review system designed to assess adherence to AHCPR-supported cataract guidelines is feasible and useful; (2) guidelines must be more specific and have more comprehensive information to optimize their development; and (3) use of a review system may improve documentation practices and adherence to recommendations.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5633

PERFORMER ORGANIZATION: Rand Corporation, Santa Monica, CA

TITLE: Developing Quality and Utilization Review Criteria for Pressure Ulcers in Adults: Phase II Final Report

ABSTRACT NUMBER: 027

ABSTRACT: This project developed guideline-based quality and utilization review criteria that can be used to assess clinical care to prevent pressure ulcers. These criteria can be used to assess conformance with recommendations in clinical practice guidelines and to target areas for clinical quality improvement. The project developed and tested measures that can be used to assess aspects of care that may be related to the development of pressure ulcers. The measures included the development of care plans and nutritional assessments, as well as care behavior related to positioning patients, alleviating pressure, and cleansing and moving patients. The focus of the review system is on patients with limited mobility, specifically patients with stroke or hip fracture. However, the review system can be adapted for use with other patient groups. The review system based on the pressure ulcer guideline was designed in collaboration with the Department of Veterans Affairs (VA) and focused on testing the review criteria for use in both VA and non-VA hospitals. The criteria will be used as part of future collaborative research between AHCPR and the VA. AHCPR and the VA are involved in a VA-wide effort to disseminate the guideline to 45 hospitals.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5634

PERFORMER ORGANIZATION: Rand Corporation, Santa Monica, CA

TITLE: Understanding and Choosing Clinical Performance Measures for Quality Improvement: Development of a Typology

ABSTRACT NUMBER: 028

ABSTRACT: Clinical performance measures are instruments that estimate the extent to which a health care provider (1) delivers appropriate clinical services in a safe, competent, and timely manner; and (2) achieves desired outcomes in terms of patient health and satisfaction. This project formulates a typology for analyzing the properties and uses of clinical performance measures that have been developed by government and nongovernment organizations, including AHCPR. The report stresses that the typology developed does not classify measures in terms of good and bad, but rather according to the setting for which they were developed, the purpose for which they were developed, and the properties that relate to the measures' potential suitability for its original or other purpose. The report also discusses using performance measures for quality improvement and consumer education. It discusses the use of performance measure sets, ranging in size from less than 10 measures to hundreds. Furthermore, the report examines the desired properties of measures, including (1) detailed specifications to ensure uniformity of application; (2) testing for reliability and validity as measures of performance; (3) allowance for patient differences in making comparisons; and (4) well-planned graphical displays of comparisons. Finally, the report discusses making the measures affordable and offers possible uses for them. (Final report variously paginated, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5630

NTIS ACCESSION NUMBER: PB 96-144639

PERFORMER ORGANIZATION: Center for Health Policy Studies, Columbia, MD

TITLE: Design of a Survey to Monitor Consumers' Access to Care, Use of Health Services, Health Outcomes, and Patient Satisfaction: Final Report

ABSTRACT NUMBER: 029

ABSTRACT: This project developed survey instruments to be used by health plans, employers, health alliances, States, accrediting agencies, the Federal Government, and others. The surveys measure consumers' assessments of their health insurance plans, in order to provide meaningful and useful information to other consumers choosing a health insurance plan. The report presents a set of prototype instruments containing an initial set of questions and procedures to assess a range of consumer issues. The report also contains an overview of the conceptual development process and the rationale for the current survey design. The methods and results of cognitive testing are described, as is the logic and evolution of the items contained in the prototype survey. The report concludes with a discussion of the prototype survey's strengths and limitations and a brief overview of recommendations for additional development and testing. (Final report 85 pages, plus appendixes.)

AGENCY SPONSOR: Center for Quality Measurement and Improvement

FEDERAL CONTACT: Sandra Robinson

PHONE NUMBER: 301/594-1352

PIC ID: 6011

PERFORMER ORGANIZATION: Research Triangle Institute, Research Triangle Park, NC

TITLE: Overcoming Barriers to Implementation and Integration of Clinical Information Management Systems: Feasibility Study

ABSTRACT NUMBER: 030

ABSTRACT: Clinical information systems capture and retrieve information at the point of health care delivery, display clinically meaningful results from multiple services, display chronological histories of patients' medical interventions, compare data on patients with similar characteristics, and share information between sites of care. Although such systems are commercially available, hospitals have been slow to implement them. This report examines the sociological, organizational, technological, and financial barriers to the implementation and optimal use of integrated clinical information systems. The report provides details on the feasibility of a study of 8 to 16 sites representing 4 to 8 major vendor products. It also provides details on site selection, research design, and a data collection plan. The report finds that the study should use focused case studies to carry out four main objectives: (1) identify target systems and provider site characteristics; (2) identify and recruit provider sites; (3) identify and solicit vendor participation; and (4) design an evaluation plan for the study, including a process and structure evaluation and an outcomes study. The report also gives a schedule for the implementation of the study. (Final report variously paginated, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5039

PERFORMER ORGANIZATION: Abt Associates, Inc., Cambridge, MA

TITLE: Practice Patterns and Expenditures in Managed Care and Indemnity Plans: A Condition-Specific Analysis

ABSTRACT NUMBER: 031

ABSTRACT: This project studied overall costs and utilization in managed care settings and how differences in practice patterns affect these variables. The project focused, in particular, on differences in practice patterns in managed care settings. The project assessed the use of selected tests and procedures for patients with heart attacks, congestive heart failure, delivery, back pain, or asthma. One of the conclusions of the project was that although the volume of claims studied was enormous, the coding used on the claim forms appeared to allow the selection of specific target procedures and tests. Also, the prevalence of such interventions was enough to detect important differences in practice patterns. In general, the use of claims to assess differences in practice patterns seems promising. However, during the project, problems continued in the receipt of usable data, causing one of the three participating carriers to withdraw from the project. This withdrawal terminated the project because confidentiality could no longer be guaranteed. General lessons from these experiences relate to project design and data management. For example, projects of this sort should be segmented, with initial data acquisition and cleaning coming first. Thus, carriers will be participating in a funded project, but large amounts of resources will not be expended until all the data are acquired.

AGENCY SPONSOR: Center for Primary Care Research

FEDERAL CONTACT: Linda A. Siegenthaler

PHONE NUMBER: 301/594-1357

PIC ID: 5664

PERFORMER ORGANIZATION: University of California, San Francisco, San Francisco CA

TITLE: Directory of Minority Health and Human Services Data Resources

ABSTRACT NUMBER: 032

ABSTRACT: This directory is intended for use by policymakers, researchers, and the public as a reference to resources within the Department of Health and Human Services that contain race and ethnicity data. Databases with broad utility, from ongoing departmental data projects to program administrative or evaluation activities, are included in the directory. The directory includes information from the Office of the Secretary, the Administration for Children and Families, the Health Care Financing Administration, the nonindependent Social Security Administration, and the Public Health Service. Each entry includes the following information: (1) the title of the data source; (2) the sponsoring office; (3) an abstract summarizing key characteristics of the database; (4) the race and ethnicities categorized in the database; (5) data limitations; (6) other data; (7) the program supported by the data project or system; (8) the general purpose of the database; (9) the status of the database; (10) the start date, start date period, or start/end dates; (11) the availability of the database; (12) constraints on use of the database; (13) the data media; (14) the distributor of public use files; (15) the agency contact person; and (16) the available data products. (Final report 278 pages.)

AGENCY SPONSOR: Center for Information Technology

FEDERAL CONTACT: Harvey Schwartz

PHONE NUMBER: 301/594-1483

PIC ID: 5911

PERFORMER ORGANIZATION: Moshman Associates, Inc., Bethesda, MD

TITLE: Development of Designs for Evaluation of the Process of Clinical Guideline Development

ABSTRACT NUMBER: 033

ABSTRACT: This project was a response to AHCPR Reauthorization Act of 1992. The project focused on the development and testing of designs to evaluate AHCPR's process for guideline development. The project identified key dimensions of the process of clinical practice guideline development and compared these dimensions among AHCPR and non-AHCPR-funded guidelines. The key dimensions selected were chosen because they varied across guideline development efforts; and they were viewed by experts as likely to affect the quality of guidelines and their acceptance by clinicians and other targeted audiences. The report recommends that AHCPR (1) consider pursuing its legislative mandate to review and assess guidelines developed by other organizations using appropriate methodologies; (2) examine mechanisms to streamline the organizational arrangements of expert panels (one of two types of panels used by AHCPR); (3) consider the development of standing panels, each addressing a broad class of conditions or technologies; (4) consider obtaining some front-end external review during the guideline development process to obtain information that could enhance the usefulness of the guideline to clinicians; (5) consider identifying and training a small pool of consumer representatives who could serve on multiple panels; and (6) undertake research to determine how the development process relates to the use of guidelines by key end users.

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5629

PERFORMER ORGANIZATION: George Washington University Medical Center, Washington, DC

TITLE: Users' Perceptions of Four Clinical Practice Guidelines and Their Development Process

ABSTRACT NUMBER: 034

ABSTRACT: Clinical practice guidelines assist practitioners, health care providers, and consumers in making decisions about health care interventions for specific clinical circumstances. This report determines if aspects of various guideline development processes are associated with typical users' perceptions about the guideline's validity, reliability, cost, and utility. The report finds that (1) clinical applicability and ease of use are two major criteria by which participants assess the potential value of clinical guidelines and upon which they base their decisions to adopt and use them; (2) most participants want guidelines that help them solve problems in a clinical setting by updating their knowledge, outlining the latest developments, or providing guidance in areas where the participants' experience is limited; (3) panel selection, topic selection, and literature review are critical to a guideline's validity; (4) participants believe that recommendations made in the clinical guidelines must be verified by their own judgment and experience; and (5) some participants also feel that inappropriate implementation of clinical guidelines can have a negative impact on practice. The report recommends that (1) AHCPR continues to develop--under rigorous conditions--guidelines that reflect changes in the field of medicine. (Final report 110 pages, plus appendixes.)

AGENCY SPONSOR: Office of Planning and Evaluation

FEDERAL CONTACT: Irma Arispe

PHONE NUMBER: 301/594-1455

PIC ID: 5956

PERFORMER ORGANIZATION: American Institutes for Research, Washington, DC

TITLE: Setting Priorities for Clinical Practice Guidelines

ABSTRACT NUMBER: 035

ABSTRACT: This project described and evaluated possible methods for setting priorities for guideline topics. The report concludes that the current priority-setting approach used by AHCPR is relatively open, fairly explicit, and generally defensible. The report recommends that six general criteria be applied in considering topics for guidelines and technology assessment: prevalence of the problem; burden of illness imposed by the problem; cost; variability in practice; potential of a guideline or assessment to improve health outcomes; and the potential of a guideline or assessment to reduce costs. The report recognizes that the data used to score particular topics on these six criteria will often be incomplete. The report recommends possible ways to improve the procedures used in setting priorities for clinical practice guidelines. These recommendations focus on (1) methods for obtaining expert judgment and developing consensus positions; and (2) the desirability of a basic procedure manual. The report also recommends that AHCPR more precisely and narrowly define the topics selected for guideline development. The report concludes that AHCPR could play a useful role as a clearinghouse that collects and disseminates guidelines developed by other organizations. It also concludes that AHCPR needs an explicit, open, and defensible process for assessing the soundness of guidelines developed by other organizations.

AGENCY SPONSOR: Center for Primary Care Research

FEDERAL CONTACT: David Lanier, M.D.

PHONE NUMBER: 301/594-1357

PIC ID: 5631

PERFORMER ORGANIZATION: Institute of Medicine, Washington, DC

TITLE: AHCPR Clinical Practice Guideline Program: Report to Congress

ABSTRACT NUMBER: 036

ABSTRACT: The AHCPR Reauthorization Act of 1992 (Public Law 102-410) required that AHCPR report to Congress on three aspects of its clinical practice guideline program: methods for selecting guideline topics, methods for developing guidelines, and methods for assessing guideline quality. This report summarizes AHCPR's clinical practice guideline program to date in areas including guideline development, research and evaluation activities, and analyses of the guideline program conducted by organizations such as the Physician Payment Review Commission and the Office of Technology Assessment. It also examines potential directions for the future. (Final report 39 pages, plus appendixes.)

AGENCY SPONSOR: Office of the Forum for Quality and Effectiveness in Health Care

FEDERAL CONTACT: Kathryn Rickard

PHONE NUMBER: 301/594-1455

PIC ID: 5949

PERFORMER ORGANIZATION: Agency for Health Care Policy and Research, Rockville, MD