This study identified existing and potential barriers to the implementation of performance measurement activities in the public health arena. To conduct the research, the project team combined a literature review with structured interviews in city, county, and regional health jurisdictions in Illinois, Oregon, and Washington. A range of perspectives and experiences were sought from private sector health providers and Federal agencies and programs, as well as from local public health agencies. The findings from the literature review were in accord with the information obtained through the structured interviews. That is, barriers do exist in implementing performance activities, and they are consistent across all health settings and public programs. As a result, this study enabled the research team to identify a set of common barriers and to offer seven recommendations to the Department of Health and Human Services (HHS) on how to overcome these barriers. Several aspects of the findings can be linked to the collaborative ongoing effort by the Robert Wood Johnson Foundation and the W.K. Kellogg Foundation's project, "Turning Point: Collaborating for a New Century in Public Health." This ongoing study is based in 14 States and 41 communities, and seeks to improve the infrastructure of health programs by developing effective performance measurement systems in States and local public health jurisdictions. The project team for this study intends to disseminate the findings from this study to the Turning Point project.
At the heart of the controversy surrounding performance measurement, including why it works and what it takes to make it work, is a lack of understanding and fear of the process. The central aim of this study is to provide a clear understanding of performance measurement by delineating some common barriers to implementation, recommending strategies to overcome these barriers and evaluating models that can be used to implement these strategies.
Over the past ten years, the HHS has committed to assisting State and local public health agencies in developing ways to improve and promote health. To assess their work, these agencies have been urged to transition from a process of counting resources used and services provided, to a method that measures and accounts for the effectiveness of these activities. Many of these agencies have done so, but in the process have met with internal and external barriers that fall into the following three categories: (1) designing an efficient performance measurement system, (2) gathering the appropriate data needed to inform such a system, and (3) reporting reliable performance measurement results. The HHS selected the Northwest Prevention Effectiveness Center and the Health Policy Analysis Program at the University of Washington's School of Public Health and Community Medicine to look for means by which the HHS could assist its components and partner agencies move toward results-oriented management. The identification of barriers and suggestions for strategies to overcome these barriers were important aspects of the study. Another important aspect was the identification of barriers that are unique to the public health field.
Two methods were used to implement this study: a review of the literature and structured interviews. The literature search included electronic database searches and searches of a broad range of published print materials from the public and private health sectors. Also included was a review of government and business literature from non-health related organizations. The literature review was followed by key informant interviews conducted with public health and academic leaders in Illinois, Oregon and Washington. These States were selected because they have developed measures and processes for assessing the performance of their public health systems. A semi-structured interview protocol was first developed and then used with the key informants to obtain specific information about barriers to implementing performance activities. Interviewees included State department directors, upper and middle-level managers, line staff and directors, and managers within associated organizations. Analysis and summary of the reviewed literature significantly contributes to the body of knowledge about performance measurement. This analysis and summary, combined with the one-on-one interviews, allowed the project team to isolate a set of 12 common barriers to performance measurement implementation and to provide the HHS with seven strategies that could be used to overcome these barriers.
The literature review consistently supported the research team's hypothesis that barriers to performance measurement and strategies to overcome them have no relation to setting. In looking at public and private programs, upper, line and lower level managers all reported some common barriers to performance measurement. Overall, the dominant barrier to performance measurement, as discovered in the literature review and from key informant interviews, was the negative perception of the process. Many could not see the value of the process and how or whether it could enhance their work. The project team viewed leadership support as critical to successful implementation of performance measurement and management systems, and found the following key techniques for communicating the value and utility of performance measurement in an organization: (1) committing adequate resources to assist staff in performance measurement activities, (2) training and involving managers as a form of encouragement, and (3) creating and using positive incentives.
Key informant interviews revealed barriers that consisted of: (1) fear that performance measurement activities would be used to judge performance, (2) fear that a program's funding would be cut, and (3) the fear of losing control over a program.
The team offered seven recommendations to facilitate performance measurement activity in State and local public health jurisdictions: (1) develop clear ideas and goals for performance measurement, (2) develop standard performance measurement terminology, (3) identify major aspects of performance measurement systems as well as a way that State and local health agencies can use them to meet their needs, (4) disseminate performance measurement tools and train others to use them, (5) develop collaboration for training and technical assistance in staff development, and (6) offer financial incentives to agencies to encourage participation in performance measurement activities.
Use of Results
Although many commonalities were identified as barriers to implementing performance measurement in the public and private health-related sectors, gaps continue to exist in the knowledge base. For example, this study was not able to note with any specificity, the particular barriers by health field. The literature provided sparse information about specific health specialities and the key informant interviews suggested the need for further analysis of barriers to performance measurement in the areas of mental health and environmental health.
The Turning Point project relates to the findings from this study in that it too suggests reshaping local and State health systems by: (1) involving key players, (2) strengthening the relationship between clinical health care and public health systems, and (3) stressing the need for more collaboration between local and State agencies in the planning and implementation of an improved public health infrastructure. At the heart of the activities should be the development of effective performance measurement systems. The findings strongly suggest that this study and the activities of the Turning Point project should be integrated.
AGENCY SPONSOR: Office of Public Health Services, Office of Disease Prevention and Health Promotion
FEDERAL CONTACT: Linda Bailey
PHONE NUMBER: 202-205-4872
PERFORMER ORGANIZATION: University of Washington, School of Public Health and Community Medicine, Seattle, WA