Performance Improvement 2003. Summary of Findings

An Assessment of the Centers for Education and Research on Therapeutics (CERTS) Initiative

This study provided an assessment of the progress CERTs has made in meeting its programmatic objectives. The purpose of this study was to assess the capability of the current CERTs programmatic structure to adequately address all of the objectives stipulated in the authorizing legislation.

Recommendations include the following: (1) CERTs need an effective mechanism to develop communication channels with state and local public health agencies; (2) long term communication strategy should include identification of key target audiences and the most effective means to reach those audiences; (3) additional professional editorial resources should be provided through the Coordinating Center; (4) existing public relations resources should be utilized to reach its various publics; and (5) the design and implementation of public affairs campaigns should be assessed.

FEDERAL CONTACT: Lynn Bosco, 301-427-1490 PIC ID: 7686

PERFORMER: Washington Consulting Group, Bethesda, MD

Feasibility Study for Developing a Web-based Redesigned and Enhanced CONQUEST and Integrating it with an Enhanced National Guideline Clearinghouse

This project sought to provide advice and recommendations to AHRQ about the design of a Web-based database of clinical performance measures that could be appropriately and functionally integrated with an enhanced National Guideline Clearinghouse database. CONQUEST, the second of a suite of Web-based quality improvement tools to be developed by AHRQ, is operated as a clearinghouse for clinical performance measurement protocols. The contractor evaluated the features that a clearinghouse for performance measures should embody and explored alternative approaches to CONQUEST’s design and management. The study found that the construction of a compendium of clinical performance measures was feasible and highly valuable. There was strong evidence that quality cannot be improved unless effects and outcomes are measured, and there was also evidence that measurement tends to influence the priorities of health care institutions and individual providers. The study also found that by putting these measures on the Web, AHRQ could very disseminate information about measures and maintain them so that they remain up-to-date in this rapidly changing scientific and clinical environment.

AHRQ Market File: Feasibility Project Report

This project examined the feasibility of developing and maintaining a data system that could be used by researchers to study healthcare markets. The research consultant conducted an evaluation of data sources used in research on healthcare markets. That initial effort resulted in a report that identified and described over 80 different data sources that could potentially be used to conduct research on the characteristics of healthcare markets. The report described a healthcare market data system and described the feasibility of establishing such a system. The report included five chapters. Chapter 1 outlines the tasks required for system development. Chapter 2 presents staffing requirements and estimated total ‘loaded’ personnel costs for the first year of operation. Chapter 4 contains a discussion of possible methods for addressing data issues that will need to be resolved if AHRQ decides to move forward and develop the healthcare market data system. The final chapter, Chapter 5, is a discussion of recommendations regarding the development and implementation of a healthcare market data system. Recommendations included: having ready access to some subset of data; the need for wide coverage in the types of data sets that can be accessed through the system; achieving high quality data set documentation and related data resources; and assuring an intuitive and easy to use web site, a streamlined process of ordering data files, and responsive technical assistance.

FEDERAL CONTACT: Michael Hagan, 301-427-1409 PIC ID: 7484

PERFORMER: Center for Health Policy Studies, Columbia, MD

Validating the Healthcare Cost and Utilization Project Quality Indicators

In 1994, the Agency for Healthcare Research and Quality developed the Healthcare Cost and Utilization Project (HCUP) Quality Indicators (QIs) in response to the increasing demand for information regarding the quality of health care. These measures, based on discharge data, were intended to flag potential quality problems in hospitals or regions. The purpose of this project was to refine the original set of HCUP QIs (HCUP I) and recommend a revised indicator set (HCUP II). Specifically, this project aimed to: (1) identify quality indicators reported in the literature and in use by health care organizations, (2) evaluate HCUP I and other indicators using literature review and novel empirical methods, and (3) make recommendations for the HCUP II set and further research. The project deferred evaluation of indicators of complications to a separate study and report. This project identified 45 indicators that were promising for use as quality screens, demonstrating through literature review and empirical analyses that useful information regarding quality of health care can be gleaned from routinely collected administrative data. However, these indicators had important limitations and could benefit from further research. Techniques such as risk adjustment and multivariate smoothing could reduce the impact of some of these limitations, but other limitations remained. There were two major recommendations for further action and research:

(1)  the need to improve HCUP data and subsequently the HCUP QIs to address some of the noted limitations, and (2) the need for further research into quality measurement and the reality of these limitations. The HCUP QIs could benefit from the inclusion of additional data, some of which can be routinely obtained in some states.

http://aspe.hhs.gov/pic/pdf/7671.pdf

FEDERAL CONTACT: Anne Elixhauser, 301-427-1411 PIC ID: 7671

PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Vision Rehabilitation Within Models of Care and Benefit Plans

AHRQ undertook this study to: (1) assess the scientific evidence on the effectiveness of services to improve functioning of persons with vision impairment, (2) survey and analyze how rehabilitation services are included in health delivery systems and health benefit plans, and (3) identify opportunities for AHRQ, to contribute to advancement of knowledge, practices, and policy deliberations with regard to vision rehabilitation services. This report was based on an extensive literature synthesis supplemented by interviews with clinical experts, provider organizations and researchers in the field. The preliminary results were critiqued in conference discussions with researchers, service delivery leaders, and leaders of consumer organizations. The report provides some background on the etiologies of low vision, some discussion of the apparent under use of clinical preventive services and treatments to halt or slow progression of disease. The main focus was on rehabilitation services that were poorly coordinated with clinical providers and relied on scattered sources of financing. The limited likelihood for cure or rehabilitation for many case of vision deterioration led the study team to suggest that stronger prevention guidelines be discussed by the Preventive Services Task force, and that AHRQ encourage greater patient awareness of benefits of early detection of eye disease, which in many cases could benefit from early intervention. However, for the rapidly growing incidence of age-related macular degeneration more progress in knowledge about prevention and intervention is needed.

Evidence Report/Technology Assessment: Criteria for Determining Disability in Speech-Language Disorders

The study included a systematic review of the literature to address two key questions about evaluating and diagnosing speech and language disorders in adults and children of concern to the Social Security Administration in making disability eligibility determinations: (1) What instruments have demonstrated reliability, validity, and normative data? (2) Do these instruments have predictive validity for an individual’s communicative impairment, performance? Approximately 42 million Americans have some type of communication disorder, which annually costs the nation $30 billion to $154 billion for lost productivity, special education, and medical care. The quality of the numerous evaluation procedures and instruments for clinical decision-making about language, speech, or voice disorders influences decisions about access to services and funding (e.g., special education services, Social Security disability income). The study found the following: (1) Reliability and validity data for the majority of instruments rarely came from peer-reviewed literature; instrument manuals yielded most such data. (2) Some manuals provided comprehensive data from well-conducted standardization studies; most did not. (3) Because normative data were usually not derived from nationally representative samples, generalizing results beyond the populations studied was difficult. (4) Sample size and representativeness problems limited the predictive validity studies. (5) Evidence about diagnostic or predictive properties of instruments addressing language, speech, and voice disorders is weak and incomplete at this time. The sparse evidence base suggested an outline and need for advancing a substantial methodological, clinical, and policymaking research agenda.

Assessing the Information Technology Infrastructure in Integrated Delivery Systems

There is evidence of increased investment in information technology (IT) by the health care industry, but little is known about how integrated delivery systems (IDSs) acquire, implement, use, and evaluate IT. How extensive is the use of IT in IDSs, and what capabilities do IDSs have to integrate data from different IT system components? How are the special needs of vulnerable populations addressed using IT systems? To address these questions and others, the Agency for Healthcare Research and Quality funded the University of North Carolina’s IDS Research Network. Information was obtained about the IT infrastructures of the six IDSs in the RTI-UNC IDSRN through a replicated series of in-depth case studies that involved a series of six two-day site visits. The study found that decisions to adopt, and subsequent implementation of, these systems in IDS settings are extremely complex processes. The financial investment and cultural implications for implementing this type of innovative clinical process are different from those typically encountered with traditional IT systems developed for billing or registration  purposes. Key observations from the analysis of case study data suggest that: (1) with experience, IDSs moved toward centralized decision making, (2) IDSs took a phased approach to implementation that involved sequential installation of IT systems across care delivery sites and maintenance of parallel data entry, storage, and retrieval systems, and (3) there was a necessity for organizational infrastructure (i.e., sufficient staffing, dedicated IT department) to support implementation and maintenance of IT solutions.

FEDERAL CONTACT: Kelly Morgan, 301-427-1570

PIC ID: 7674

PERFORMER: Research Triangle Institute, Research Triangle Park, NC

Evaluating AHRQ’s Low-Income Research Portfolio, 1989-2000: Final Report

The purpose of this report was to provide an understanding of the breadth and depth of AHRQ’s low- income populations research portfolio and to identify potential directions for continued research in this area for the next decade. This report was the final deliverable for a project representing the first phase in low-phase evaluation of AHRQ’s low-income research portfolio. This phase of the evaluation answered the following two questions: (1) what contributions has AHRQ’s sponsored research made to knowledge about health care for low-income populations and (2) comparing AHRQ’s research portfolio to the body of literature in the field, in what areas has the agency made the greatest contributions and in what areas have agency investments been low? This report presented findings from the first phase of the evaluation. Studies of access to prenatal and obstetric care found that it is possible to improve provider participation in obstetrics for Medicaid patients with a comprehensive Medicaid expansion program. Also, low income is associated with significantly elevated risks of untimely prenatal care, but the increased risk of untimely care is not confined to women in absolute poverty or to those lacking private insurance. Studies that focused on pregnancy outcomes found that density of obstetricians appeared to have no impact on pregnancy outcomes. Also, heightened levels of stress were not associated with earlier deliveries. Studies focused on prenatal quality of care found that patients’ ideas of what constitutes quality in prenatal care did not appear to differ by ethnicity. In another study, satisfaction also did not appear to vary according to personal characteristics.