Continuous improvement in government is a central theme of the Clinton Administration. As we move toward a balanced Federal budget, funds available for all Departments will be increasingly constrained. All programs must become more efficient and must demonstrate their effectiveness in improving the lives of all Americans.
The Department of Health and Human Services (DHHS) is committed to examining and redirecting its programs to ensure that its customers--the American people--are being well served. In the spring of 1993, shortly after Vice President Gore initiated the National Performance Review, the Continuous Improvement Program (CIP) was established in DHHS. The CIP calls on all DHHS employees to work together as agents of change to create a more customer-oriented, cost-efficient, and effective government. It is important to ensure that decisions for change are based on sound information about programs and policies. A solid evaluation program can provide the information needed to guide constructive change, including programmatic, budgetary, and policy decisions.
The U.S. Public Health Service (PHS) recognizes the importance of evaluation in improving the health and quality of life for all Americans. PHS supports and encourages evaluation efforts that--
- determine whether programs and services achieve their desired effect and how to improve them;
- measure the success of programs in reaching and helping target populations or communities;
- communicate knowledge about program successes and other lessons learned to the public health community; and
- develop or refine effective evaluation tools.
PHS Evaluation Program studies have had a major impact on planning, budgeting, and legislative development. For example, a 1989 National Academy of Sciences evaluation of the training and supply of biomedical and behavioral scientists helped the National Institutes of Health (NIH) determine the number of positions to be supported by National Research Service Awards (NRSAs) and justify NIH training budget requests to meet future needs for biomedical and behavioral scientists. In 1993, the Centers for Disease Control and Prevention (CDC) evaluated the impact of an influenza vaccination program for the elderly, documenting the cost-effectiveness of a vaccination promotion campaign for reducing incidence of secondary acute respiratory bacterial and viral infections. The results influenced the Department's decision to make influenza vaccine reimbursable by Medicare as a cost-effective clinical preventive service.
In addition, the PHS Evaluation Program has played a role in supporting development of evaluation tools for use by the larger public health community. In 1991, the National Research Council's Committee on AIDS Research and the Behavioral, Social, and Statistical Sciences produced a series of reports on strategies for evaluating CDC's AIDS prevention programs. These evaluation methodology reports first helped provide the CDC with a blueprint for designing more effective health communication strategies for AIDS prevention. The reports also made these evaluation tools available to public health practitioners, enabling them to assess the effectiveness of AIDS prevention interventions.
In keeping with the PHS commitment to produce high-quality, useful evaluations, we are pleased to present Performance Improvement 1995: Evaluation Activities of the Public Health Service. This report is the first in a series of annual reports documenting evaluation efforts across all PHS Agencies and Offices. The report has three audiences: decisionmakers who need information on program results across the broad spectrum of public health activities; program managers who want to know how other public health programs are operating and how performance can be improved; and the community of researchers, advocates, and practitioners who will use the program information and the evaluation tools.
The report contains information on projects completed in fiscal year (FY) 1994 and their applications. For example,
- Research on adverse events associated with childhood vaccines is being used to educate parents about vaccine-associated risks and to guide policies related to childhood immunization and victim compensation.
- An Institute of Medicine report on the impact of nonindustrial indoor allergens spurred the creation of a new research program and numerous recommendations for patient education, treatment, and methods to improve indoor environments.
- A school health program evaluation produced designs that school systems can use to evaluate school health interventions and their impact on student academic performance and health.
This report samples the diverse array of evaluation studies conducted by PHS in 1994 and provides information on evaluations in progress. The report presents a comprehensive picture of the PHS Evaluation Program--its past, present, and expected future activities.
Future studies will place even greater emphasis on outcome or impact evaluations, increased methodological and scientific rigor, and projects that are crosscutting and provide widely applicable results. Projects also must meet information needs outside the PHS. They must contribute to our national understanding of how public health and preventive approaches improve the health of all Americans, reduce the risks to health, and enhance the accessibility and availability of care.
Program Performance 1995 is organized into three chapters. Chapter I describes the PHS Evaluation Program--its activities, funding, planning and management, and future directions. Chapter II highlights the results from 13 FY 1994 evaluations selected for their potential application by the public health community. Chapter III presents the evaluation activities of the seven PHS Agencies and the Office of the Assistant Secretary for Health (OASH), including information on their evaluation programs, evaluations completed in FY 1994, evaluations in progress, and future directions. A complete inventory of the 71 PHS evaluation projects completed in FY 1994 is provided in appendix A, and PHS Agency projects in progress are listed in appendix B. Review criteria used by the special panel of senior editorial advisors for assessing program evaluations can be found in appendix C.
We hope that you will find this report useful and informative.
Donna E. Shalala
U.S. Department of Health and Human Services
Philip R. Lee, M.D.
Assistant Secretary for Health
U.S. Public Health Service