Performance Improvement 1997. Office of Public Health and Science


MISSION: To provide advice on public health and science to the Secretary of Health and Human Services, to provide executive direction to program offices within OPHS, and, at the direction of the Secretary, to coordinate cross-cutting Public Health and Science initiatives in the Department.

Evaluation Program

The Office of Public Health and Science (OPHS) within the Department of Health and Human Services (HHS) provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary. OPHS provides advice to ensure that HHS conducts broad-based public health assessments designed to better define public health problems and solutions to those problems. It assists other components of HHS in anticipating future public health issues and problems and provides assistance to ensure that HHS designs and implements appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides a focus for leadership on matters including recommendations for policy on population-based public health and science and, at the Secretary's direction, leads or coordinates initiatives that cut across agencies and operating divisions. In addition, it communicates and interacts, on behalf of the Secretary, with national and international professional and constituency organizations on matters of public health and science. It also provides guidance and coordination across the Department on international results and domestic refugee health issues, and serves as the focal point for senior staff support in the governance of health agencies of the United Nations system.

In keeping with its newly defined role within the Department, OPHS has specifically sought to develop an evaluation plan that avoids duplication of efforts that might more appropriately and effectively be undertaken by the operating divisions of HHS or by the Office of the Assistant Secretary for Planning and Evaluation (ASPE). Thus, the direction of the FY 1997 evaluation strategy for OPHS is toward public health and science issues that cut across multiple interests of the operating divisions and that require a coordinated approach to achieve the most effective results. In addition, OPHS commits itself to seeking to carry out every project proposed as part of this year's strategy in collaboration with relevant operating divisions.

As to its specific responsibilities, OPHS conducts evaluations requested of the Secretary by Congress that are most appropriately managed by staff with medical or public health sciences credentials, and that cannot be assigned to an operating division. Also, evaluations are conducted to support the Assistant Secretary for Health in his or her role as senior advisor for public health and science to the Secretary. This role includes the responsibility to "assist the Secretary in developing a policy agenda for the Department to address major population-based public health, prevention and science issues" and to provide "leadership and a focus for coordination of population-based health, clinical preventive services and science initiatives that cut across operating divisions." In addition, OPHS conducts evaluations specific to the needs of the program offices located within OPHS, such as Women's Health, Minority Health, Disease Prevention and Health Promotion, International and Refugee Health, and Emergency Preparedness.

Summary of Fiscal Year 1996 Evaluations

In FY 1996, OPHS completed four evaluations in an effort to better inform policy decisions throughout the Public Health Service (PHS). Studies completed this year focus on performance and accountability of the public health and science communities and include such topics as improved methodologies for cost-effectiveness analysis and quantification of State public health expenditures, an examination of refugee health care issues, and the impact of research misconduct investigations. As illustrated below, OPHS efforts cut across a variety of PHS program areas.

In an effort to study and learn from past experiences, the cost-effectiveness analysis (CEA) report provides an assessment of the current state-of-the-science of CEA methodology and makes recommendations about ways to improve the quality, comparability, and utility of CEA's. CEA is a powerful tool for evaluating which strategies best serve programmatic and financial objectives. Unresolved methodological issues are discussed in the context of factors that limit the policy relevance of existing CEA's. The CEA report is described in more detail in Chapter II.

A second methodological report, Measuring Expenditures for Essential Public Health Services, was completed on quantification of State public health expenditures. Working with State and local public health, mental health, substance abuse, and environmental agencies in nine States and with Federal PHS agencies, a methodology was developed and tested for estimating investments in 10 essential public health service areas. For all nine participating States, the total expenditure for essential public health services was $8.8 billion in 1994-95, of which $2.7 billion is population based. Estimates of public health expenditures are provided for agencies within the States and by service areas. In addition to estimating the relative investment in public health in the United States, the report lays the groundwork for a system to track these expenditures over time, across agencies, and across levels of Government. The resulting financial data can be related public health outcomes, infrastructure, and workforce measurements. An article about this study was published in Morbidity and Mortality Weekly Report on February 21, 1997.

A report on the health needs of newly arriving refugees evaluated the health assessment and follow-up care available to refugees, Cuban and Haitian entrants, and Amerasian immigrants from Vietnam during the first 8 months following their arrival in the United States. During this 8-month period, the Federal Government provides funding for health care coverage to refugees to prevent the spread of health conditions that could affect the public health and to identify health conditions that could impede effective refugee resettlement. The findings indicate areas of success, as well as areas needing improvement. For example, the flexibility of Federal funding has enabled States to fill gaps in their refugee health programs. In addition, statewide infrastructure for refugee health care has been developed in States that contract with local providers to deliver health-screening services. Concerns are raised about the accuracy and completeness of screening conducted outside of refugee-specific clinics, the reporting of refugee health screening results to the Centers for Disease Control and Prevention, and the availability of follow-up care to refugees.

Scientific misconduct is a topic of great interest to the public health and science communities. In its report on the consequences of being accused of research misconduct, the Office of Research Integrity (ORI) examined the impact of misconduct investigations on exonerated researchers. About 70 percent of all cases of alleged scientific misconduct that come to the attention of ORI result in exoneration. Yet little was known about the adverse consequences for the accused and the extent to which institutions comply with confidentiality requirements. The findings raise questions about the confidentiality of ongoing investigations. In a survey of 54 individuals who were accused of scientific misconduct and exonerated (prior to 1995), 60 percent reported one or more negative consequences of being accused of scientific misconduct. Of these, 90 percent indicated that the negative actions began during the inquiry or investigation and 65 percent reported that these actions continued after they had been exonerated. Yet only 39 percent viewed the impact of accusation on their careers as negative; 39 percent believed there was a continuing stigma attached to having been accused of misconduct. A similar 38 percent of the respondents were dissatisfied with the efforts of their institutions to restore their reputations.

Evaluations in Progress

In reviewing evaluation projects for FY 1996, OPHS considered the following priority areas: congressional mandates, cross-departmental initiatives, OPHS strategic plan-related initiatives, program improvements, and the Government Performance and Results Act (GPRA). OPHS selected 17 program and policy evaluations for funding this year, which are described below.

The Commission on Dietary Supplement Labels, appointed by the President, is evaluating factors relevant to Food and Drug Administration regulation and possible legislation of label claims and statements for dietary supplements, including health claims as requested by Congress. A second nutritional study, undertaken by the National Academy of Sciences, is evaluating human nutrient requirements. The study will provide reference intakes, guidance about these values for clinical and public health use, and a listing of research needs.

An evaluation of the Cooperative Agreements for Demonstration Projects for Capacity Building at Historically Black Colleges and Universities (HBCU's) is under way. The setting will examine the extent to which the overall program and individual projects have resulted in the establishment of offices of sponsored programs and the adoption of uniform processes in participating schools. The evaluation will also assess how well the program has achieved increased funding for health-related research and training at the institutions and how effectively it has enhanced current research, training, and services.

The OPHS evaluation priorities related to strategic planning initiatives will focus on several areas. First is the organization of the Healthy People 2000 objectives. To promote an integrated public health message, OPHS is consulting a broad range of members of the Healthy People 2000 Consortium and using focus groups to evaluate the basic framework of national health goals and objectives in preparing for the Healthy People 2010 Plan. The study will collect insights on the current functions of the objectives framework and explore opportunities for making them more relevant to the Healthy People 2000 Consortium members, State and local agencies, managed care industry representatives, and Fortune 500 major purchasers of health care plans.

School health programs, both comprehensive and categorical, continue to be developed and implemented throughout the Nation. OPHS has funded a study to identify evaluations of school health programs and make available an updated compilation of methodologically sound studies and their findings to assist school boards, administrators, health personnel, and health educators trying to maximize limited resources for effective health programs.

OPHS will continue to expand efforts to improve nationally available data on the public health infrastructure, focusing on local public health capacity and workforce issues. The results will provide local health officials and planners with a validated instrument for estimating the level of support for essential public health functions. This local study of expenditures represents the community analog to the State study funded in FY 1996.

In the area of health data and the environment and in continuance of support for public health infrastructure revitalization, OPHS funded development of two data-tracking systems, one related to sentinel public health indicators, health outcomes, levels of health risks, and the health protection infrastructure; and the second on State and local environmental health data needs and sources of State and local environmental information. OPHS also funded an assessment of health plan involvement in community-based initiatives and a survey of clinicians to assess the level of provision of and the importance placed on preventive clinical services by primary care providers,

OPHS supports several projects to assist in improving program operations and management. These include an evaluation of a new model of coordinating and integrating HIV-prevention messages and primary care in high risk populations, an evaluation of the Minority Health Resource Center, and the assessment of the Adolescent Family Life Program.

The implementation of the GPRA of 1992 is a priority for OPHS. The development of performance measures for public health programs will be crucial for program planning, budgeting, and legislative development. OPHS, in collaboration with the Office of the Assistant Secretary for Planning and Evaluation, has targeted efforts in three program areas to develop these measures. First, it is supporting completion of the consultation begun in FY1995 with State and local health officials concerning the development of performance measurement systems that can be used to improve accountability for expenditure of Federal funds. OPHS will assist States and local governments in determining the impact Federal programs are having on improving the health status of Americans. OPHS will also discern the extent to which States and selected counties and cities have current data available to measure their own State year 2000 objectives and selected national objectives.

Two communication studies are also under way. The first focuses on the design and evaluation of interactive communication applications for consumer health information. A second study examines the dissemination of public health information from PHS agencies.

Finally, the Office of Minority Health is examining the efficiency and effectiveness of its Bilingual/Bicultural Service Demonstration Grant Program, which is intended to build the capacity of community-based organizations to address access to health services for limited-English-speaking minority populations.

New Directions for Evaluation

The next few years will continue to be a time of enormous transformation for this Nation's health care system. The need to manage skyrocketing health care costs and rein in the Federal budget deficit--while trying to address the health needs of over 40 million uninsured Americans and a general population that is becoming increasingly aged as well as linguistically and culturally diverse--presents enormous challenges to HHS, OPHS, and our partners in the public health community.

The shifting emphasis on managed care presents new opportunities coupled with new dilemmas regarding how best to ensure that, in the quest to manage the cost of care, overall quality of care is not unduly compromised. It raises new questions of how to ensure a workforce that is properly trained and fully capable of performing the essential services of public health to increasingly diverse populations in this dynamic health care environment. Furthermore, transformations in the health care and public health arena necessitate the monitoring and assessment (and the data systems to do so) of the effects and impacts of these changes on the financing, organization, and availability of population- and clinic-based preventive services and medical treatment.

OPHS has given highest priority in its FY 1997 strategic plan and evaluation efforts to addressing these issues. The FY 1997 OPHS strategic plan emphasizes the need for a strong foundation for public health in the 21st century. The priorities focus attention on (1) the information systems and workforce that comprise the Nation's public health infrastructure and that are necessary to effectively provide the essential services of public health; and (2) the impact of managed care arrangements on the resources available for this infrastructure and on the health of all Americans, especially those most vulnerable.

Future OPHS evaluation planning efforts will reflect the OPHS strategic plan, as well as the broader HHS programmatic priorities related to improving the health of all Americans and assessing the effects of health-sector transformation. Planned investigations in support of the public health infrastructure include an assessment of response to health and medical consequences of chemical and biological agent incidents; an evaluation of summary measures of health status; an evaluation of the electronic gateway to consumer health information; analyses of international year 2000 plans to inform the United States effort and the WHO Renewing Health for All strategy; and an examination of outreach efforts and potential uses of Healthy People Objectives for the Year 2010.

Other planned evaluation activities will focus on addressing specific health needs of our most vulnerable populations. These include evaluations of strategies and approaches for addressing the health of racial and ethnic minorities in the United States as compared with the United Kingdom, cultural competence in health care to address issues of measurement and linkages to health and patient outcomes, an inventory of State data on women's health, an inventory and assessment of linguistically and culturally appropriate services in managed care outreach organizations and an assessment of minority consumer experiences with State outreach activities related to Medicaid managed care.

Last, in response to the increased need to measure the performance and effects of public health programs and activities, OPHS will support efforts to enable performance measurement activities among local, State, and Federal health agencies.