Performance Improvement 1999. Foreword


Performance Improvement 1999: Evaluation Activities of the U.S. Department of Health and Human Services is the fifth annual report to Congress summarizing previous fiscal year evaluation efforts. The purpose of this report is to provide Congress with evaluative information on the Department's programs, policies, activities, and strategies. It contains brief summaries of evaluation results and provides a federal agency contact name for obtaining more detailed information. The report is useful to health and human service researchers, stakeholders and practitioners who use the information to assist their work.

In the era of results-oriented management, evaluations are playing an increasingly important role in program improvement. To this end, the Department of Health and Human Services (HHS) is committed to ensuring its evaluations yield valuable knowledge, and that knowledge is used to improve program performance. This is a consistent theme of our annual reports. Evaluations in the report can be characterized by the following improvement themes: evaluating program effectiveness, performance measurement, assessing environmental impacts on health and human services, and improving program management.

Program effectiveness provides a way to determine the impact of HHS programs on achieving intended goals and objectives. For example, a major departmental and national goal is to reduce drug abuse. Treatment programs are a key element in this effort. The Substance Abuse and Mental Health Services Administration, at the request of the Office of National Drug Control Policy, completed the Services Research Outcomes Study, the first national drug abuse treatment outcome study based on a nationally representative sample of treatment providers. The principal finding was that both drug use and criminal behavior are reduced following inpatient, outpatient and residential treatment for drug abuse, thus confirming that treatment programs work.

Performance measurement is the primary mechanism used to monitor annual progress in achieving departmental strategic and performance goals. To support performance measurement, we are investing evaluation funds to develop and improve performance measurement systems and the quality of the data that supports those systems. For example, the Office of Public Health and Science conducted a project to identify barriers to implementing performance measurement in State and local public health programs. Questions concerning the utility of measurement for decision making, uncertainty about linking performance to resource allocation, and accountability for outcomes beyond the program's span of control were among some of the barriers identified. These results are now being used by the Department, in collaboration with private foundations, to foster improvements in performance measurement of State and local health programs, particularly those supported through health and human services block grants.

Environmental assessment is the way we understand the forces of change in the health and human services environment that will influence the success of our programs and the achievement of our goals and objectives. In turn, this understanding allows us to adjust our strategies and continue to deliver effective health and human services. For example, the emergence of managed care and welfare reform may have a profound impact on a wide range of HHS programs. Therefore, we are investing in studies to understand the impact of these and other innovations on our programs. In this report, we are beginning to provide some of those results. For example, the Office of the Assistant Secretary for Planning and Evaluation assessed the impact of expanding managed care organizations and increased hospital and commercial laboratory consolidations on the viability of public health laboratories. In light of the observed impact, the study further identified ways in which these laboratories must adapt their services to changes in the health care marketplace in order to maintain their viability as part of the Nation's public health infrastructure.

Program management reflects the need of program managers to obtain information or data helpful for effectively designing and managing a program. These evaluations generally focus on developmental or operational aspects of program activities and provide understanding of services delivered and populations served. For example, the Administration for Children and Families completed an interim impact evaluation of the Parents' Fair Share demonstration project. The goal of this project was to increase employment and earnings of non-custodial fathers in order to raise support payments to children receiving welfare. The evaluation results helped program managers to learn which services could possibly increase the amount of child support collected from fathers. With these services, the number of fathers who paid child support could be increased, but with no corresponding increase in the fathers' employment duration and earnings.

Performance Improvement 1999 contains information on HHS evaluation projects completed and in progress during fiscal year (FY) 1998. The report is organized into three chapters. Chapter I presents the results of selected HHS agency evaluations that focus on measurement of program effectiveness. Chapter II highlights the results from eight FY 1998 evaluations, selected by an outside review panel as outstanding for their potential use by the larger health and human services community. Chapter III presents summaries of all FY 1998 evaluations completed and in progress by the 11 HHS agencies and the Office of the Secretary. A description of the HHS Evaluation Program, its activities, resources, planning, quality control and dissemination is provided in appendix A. The criteria used by the Evaluation Review Panel to select the reports highlighted in chapter II are in appendix B.

We hope that you will find this report useful and informative.

Donna E. Shalala Margaret A. Hamburg, M.D.
U.S. Department of Health
Assistant Secretary for  Planning and Evaluation