MISSION: To provide advice on public health and science to the Secretary of HHS, executive direction to program offices within OPHS, and, at the direction of the Secretary, coordinate crosscutting public health and science initiatives in the Department.
OPHS Evaluation Program
OPHS provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary of HHS. OPHS provides advice to ensure that the Department conducts broad-based public health assessments designed to better define public health problems and solutions to those problems. It assists other components of HHS in anticipating future public health issues and problems and provides assistance to ensure that HHS designs and implements appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides a focus for leadership on matters including recommendations for policy on population-based public health and science and, at the Secretary's direction, leads and coordinates initiatives that cut across agencies and operating divisions. In addition, OPHS communicates and interacts, on behalf of the Secretary, with national professional and constituency organizations on matters of public health and science.
In keeping with its newly defined role within HHS, OPHS has specifically sought to develop an evaluation plan that avoids duplication of efforts that might more appropriately and effectively be undertaken by the operating divisions of the Department or by the Office of the Assistant Secretary for Planning and Evaluation. Thus, the direction of the fiscal 1996 evaluation strategy for OPHS is toward public health and science issues that cut across multiple interests of the operating divisions and that require a coordinated approach to achieve the most effective results. In addition, OPHS commits itself to carrying out every project proposed as part of this year's strategy in collaboration with relevant operating divisions.
OPHS specifically conducts evaluations requested of the Secretary by Congress that are most appropriately managed by staff with medical or health sciences credentials and that cannot be assigned to an operating division. Also, evaluations are conducted to support the Assistant Secretary for Health's role as senior advisor to the Secretary on matters of public health and science. This role includes the responsibility to "assist the Secretary in developing a policy agenda for the Department to address major population-based public health, prevention and science issues" and to provide "leadership and a focus for coordination of population-based health, clinical preventive services and science initiatives that cut across operating divisions." In addition, OPHS conducts evaluations specific to the needs of the program offices located within OPHS, such as Women's Health, Minority Health, Disease Prevention/Health Promotion, International Health, and Emergency Preparedness.
Summary of Fiscal 1995 OPHS Evaluations
In fiscal 1995, OPHS's predecessor, the Office of the Assistant Secretary for Health (OASH), completed 12 evaluations in an effort to better inform policy decisions throughout the Public Health Service (PHS) as the health care sector went through a period of rapid change. Studies were in diverse areas of public health in an effort to strengthen the public health infrastructure, including primary health care, HIV/AIDS, work force training, women's health, and substance abuse. OASH efforts focused on issues that cut across PHS program areas, as follows.
In an effort to study and learn from past experiences, one study, performed by the Institute of Medicine (IoM), analyzed crisis decisionmaking by the Food and Drug Administration, the Centers for Disease Control and Prevention, and the National Institutes of Health in response to a threat to the blood supply that emerged in the early 1980s--HIV. These Federal entities, as well as the plasma fractionation industry, community blood banks, and other groups, have responsibilities to protect the supply of blood from communicable disease. The report concludes that the system did not deal well with blood safety issues, that strong leadership is required to counteract legal and competitive concerns that may inhibit effective agency action when a crisis is not well defined, and that agencies need to formulate a systematic approach to advisory committees and should not rely on the entities they regulate for data analysis or modeling. These lessons are being used by the Assistant Secretary for Health in a cross-departmental review on blood safety.
Another project, also performed by the IoM, examined current HHS standards for methadone programs and evaluated the effects of Federal regulations on the provision of methadone treatment services. Unlike other controlled substances, methadone use is controlled by a multitiered system of regulations, and Federal oversight of methadone programs is provided by FDA, the Drug Enforcement Administration, the National Institute on Drug Abuse, and the Substance Abuse and Mental Health Services Administration. The IoM report examines the underlying ideology of methadone regulations that societal risks of methadone outweigh its benefits to such an extent that its use must be extraordinarily regulated. The IoM committee concludes such an approach is not valid in the current environment, and the scope of Federal regulation should be scaled back to untie clinicians' hands in determining the best course of a patient's treatment.
A third report examined the clinical background for trials involving the experimental drug fialuridine (FIAU) and its parent drug fiacitabine to determine whether any rules governing the trial process should be changed, and if so, what burdens or costs these changes might place on future clinical trials. The study followed up on an NIH clinical drug trial in which several hepatitis B virus (HBV) outpatients who had been administered FIAU were hospitalized with liver failure; five patients died and two were saved only by liver transplantation. The report finds that the entire set of trials reviewed was an ethically sound clinical research project designed and carried out by highly competent investigators who frequently exceeded regulation requirements imposed by institutional review boards. The report discusses conclusions and recommendations issued by FDA and concurred with them without ascribing blame for the tragedy to any party.
Among projects in the Office of Disease Prevention and Health Promotion (ODPHP), one report obtained baseline data on the disease-prevention activities of 1,310 nurse practitioners as primary care providers and compared the findings to target goals from 17 objectives of Healthy People 2000: National Health Promotion and Disease Prevention Objectives. The 892 respondent reported spending 50 percent or more of their time providing primary care. In the areas of assessment services, screening and immunizations, cancer-screening services, obstetrics/gynecology, and family medicine, nurse practitioners generally met or exceeded the target percentages of the Healthy People 2000 objectives for selected services.
In an effort to document the relative investment in public health in the United States, nine States participated in another ODPHP project to develop a tool for collecting Federal, State, and local public health expenditures using the essential services of public health terminology. Data-collection instruments were distributed to State health officials who, in turn, coordinated the data collection for State and local public health, substance abuse, and environmental agencies. Mental health data collection was coordinated through the National Association of State Mental Health Program Directors. The report is intended to lay important groundwork for tracking these expenditures over time and across various agencies and levels of government.
An evaluation by the Office of Minority Health (OMH) was designed to document, describe, and assess the effectiveness of community- based projects that address HIV/AIDS prevention within minority communities. The study compares various community-based approaches to disease prevention in nine projects and will be used by OMH and other agencies to identify successful practices and common themes to apply as innovative community-based risk- reduction strategies when targeting minority communities and at- risk populations. The study found that viable linkages with community-based providers and social service agencies are critical to the provision of effective, coordinated, and comprehensive HIV/AIDS services in the community; that projects that build upon existing AIDS activities are better able to influence the local health care and social service agencies and are more likely to continue after Federal funding ends; and that projects must conduct and document internal formative/process evaluations.
Four OASH evaluation projects were concerned with the operation of programs in the PHS regions. One project evaluated the impact of factors that impede the implementation of breast and reproductive cancer prevention efforts at nine Community and Migrant Health Centers (C/MHCs) located in California. In reviewing screening, referral, diagnosis, treatment, and case management protocols, the report finds that the majority of C/MHCs have weaknesses in the areas of community education and outreach; designation of staff to perform specific duties; attention to ethnic or culturally specific barriers to health care; and overall lack of understanding about the importance of protocols. To correct these weaknesses, a Primary Care Effectiveness Review has been scheduled for each center.
After identifying a gap in reproductive health care information within the C/MHC setting, an interagency task force in PHS Region IX began an evaluation project to strengthen the regional public health information infrastructure. The study focused on a representative sample of women 20 to 44 years of age who received reproductive health care services from 1 of 36 study sites. Using a 220-item data-collection instrument designed to collect and analyze the frequency and extent of services provided, the study gives regional programs an information base that will help improve quality of care by disclosing demographic and utilization patterns of the typical client. The system provides immediate feedback on quality of care through documentation analysis; for example, preventive health screenings are highly documented, while services related to the report of abuse and violence are the most poorly documented. Also, prenatal documentation tends to be complete, and documentation of health promotion education and counseling appears low.
Copies of the PHS Region IX report were sent to all C/MHCs involved in the study and to executive and clinical staff in the region. Technical assistance in the areas of violence and abuse, reproductive health, and depression is being made available to the centers as part of a continuous quality-improvement plan. In addition, forms are being redesigned to improve staff use. A domestic violence task force has been organized in the region to address the issues raised in the report.
A report from Region IX assesses HBV education and outreach efforts among primary care centers serving Asians and Pacific Islanders, who constitute more than 50 percent of the infectious HBV carriers in the United States. The report summarizes programs at five centers, which completed and returned a self-assessment tool. It finds that program staffing is the strongest component of HBV education and outreach programs, and media-based outreach is the weakest. The report recommends sensitizing staff to cultural beliefs and attitudes impacting on HBV, using translated health education materials, integrating HBV activities with standard patient care protocols, using ethnic media, and developing linkages with school-based health programs and other community-based programs that help high-risk Asian/Pacific Islanders overcome the barriers identified by the study. The centers are now working to incorporate these findings into ongoing staff training and development.
From surveys mailed to nine C/MHCs in Region VI, another project assessed the level of interest in primary care, practice-based research. The report finds that significant barriers to research include lack of research skills and expertise, too many other clinical responsibilities, and lack of access to research consultation. The report recommended further evaluation of PHS and non-PHS clinic sites to determine topics of interest; encouragement of collaborative arrangements with academic researchers; and, where possible, the use of electronic forums for clinicians. As a result, the Region VI office convened a primary care research meeting in San Antonio to provide clinicians with an opportunity to discuss research interests and provided technical assistance to clinicians in accessing electronic bulletin boards as a means of acquiring research information.
Two studies were completed demonstrating the impact of the rapidly changing health care delivery system. One project evaluated responses to a Federal Register notice soliciting comments on the extent to which competitive health plans contract with academic medical centers (AMCs). The objective was to obtain insight into the potential impact of further growth in managed care organizations (MCOs) on the viability of AMCs and their teaching hospitals and to determine what additional research would be useful to further explore these issues. The study finds that the growth of managed care enrollment will change the mix of patients admitted to AMCs, thus affecting training opportunities for resident physicians and other health professionals. Both AMCs and MCOs suggest that the costs of teaching and research be separated from patient care costs and funded through separate mechanisms.
Another project analyzed issues raised by proposals for improving the supply, training, and distribution of primary care providers. Many proposals considered by Congress included provisions that would have altered the mode of funding graduate medical education, thus changing the numbers and types of new physicians. The report includes a chart book on the supply, training, and distribution of physicians; background on the need for an increased number of generalist physicians; and estimates of the number of advanced practice nurses and physicians' assistants that would be needed if the number of physician residents were reduced.
OPHS Evaluations in Progress
In reviewing evaluation projects for 1996, OPHS considered the following priority areas: congressional mandates, cross- departmental initiatives, OPHS strategic plan-related initiatives, program improvements, and the Government Performance and Results Act. OPHS selected 20 program and policy evaluations for funding for 1996, which are described below.
The Commission on Dietary Supplement Labels, appointed by the President, is evaluating factors relevant to possible FDA regulation of labels for dietary supplement regarding health claims. This project was requested by Congress.
An evaluation of the Cooperative Agreements for Demonstration Projects for Capacity Building at Historically Black Colleges and Universities (HBCUs) is under way. This cross-departmental initiative will examine the extent to which the overall program and individual projects have resulted in the establishment of offices of sponsored programs and the adoption of uniform processes. The evaluation will also assess how well the program has achieved increased funding for health-related research and training at the institutions and how effectively it has enhanced current research, training, and services.
The OPHS evaluation priorities related to strategic planning initiatives will focus on several areas. First, the organization of the Healthy People 2000 objectives reflects categorical funding streams and the concern of special interests in the field of health. To promote an integrated public health message, OPHS will reconsider the basic framework of national health goals and objectives in preparing for the Healthy People 2010 plan. The study will collect insights on the current objectives framework from the Healthy People 2000 consortium members, State health agencies, managed care industry representatives, and major Fortune 500 purchasers of health care plans.
School health programs, both comprehensive and categorical, continue to be developed and implemented throughout the Nation. OPHS will fund a study to identify evaluations of school health programs and make available an updated compilation of methodologically sound studies and their findings to assist school boards, administrators, health personnel, and health educators trying to maximize limited resources for effective health programs.
OPHS will continue to expand efforts to improve nationally available data on public health infrastructure, especially focusing on local public health capacity. In addition, funds will be used to support a similar effort conducted by several pilot States to categorize funding for population-based health activities, as distinct from funding of personal clinical services. The results will provide local health officials and planners with a validated instrument for estimating the level of support for essential public health functions.
In the area of health data and the environment and in continuing support for public health infrastructure revitalization, OPHS will fund development of two data-tracking systems: one related to sentinel public health indicators, related health outcomes, levels of health risks, and the health protection infrastructure, and the second on national and State-level data concerning environmental health outcomes and risks. OPHS will also fund an assessment of health plan involvement in community-based initiatives and a survey of clinicians to assess the level of provision of and the importance placed on preventive clinical services by primary care providers.
OPHS will support several projects to help improve program operations and management. These include a study of management alternatives for emergency preparedness and response, an evaluation of a new model of coordinating and integrating HIV- prevention messages and primary care in high-risk populations, an evaluation of the Minority Health Resource Center, and the assessment of the Adolescent Family Life Program.
OPHS will continue support for the work of the Panel on Cost- Effectiveness in Health and Medicine, a non-Federal expert panel appointed in 1993, which will publish its report this year. The report will be discussed at a conference to explore the panel's recommendations and conduct workshops on the application of cost- effectiveness analysis for specific medical and public health applications. OPHS will also provide support through the National Academy of Sciences for a critical reassessment and revision of dietary reference intakes (recommended dietary allowances).
The implementation of GPRA is a priority for OPHS. The development of performance measures for public health programs will be crucial for program planning, budgeting, and legislative development. OPHS will target efforts in three program areas to develop these measures. First, it will support completion of the consultation begun in fiscal 1995 with State and local health officials concerning the development of performance measurement systems that can be used to improve accountability for expenditure of Federal funds. OPHS will help States and local governments determine the impact Federal programs are having on improving the health status of Americans. OPHS will also discern the extent to which States and major American cities have current data available to measure their own year 2000 objectives and selected national objectives. Finally, the Office of Minority Health will examine its Bilingual/Bicultural Service Demonstration Grant Program, which intends to build the capacity of community-based organizations to address access to health services for limited English-speaking minority populations.
New Directions for OPHS Evaluation
The next few years will be a time of enormous transformation for this Nation's health care system. The need to manage skyrocketing health care costs and rein in the Federal budget deficit--while trying to address the health needs of more than 40 million uninsured Americans and a general population that is becoming older as well as more linguistically and culturally diverse-- presents enormous challenges to HHS, OPHS, and their partners in the public health community.
The shifting emphasis on managed care presents new opportunities coupled with new dilemmas about how best to ensure that, in the quest to manage the cost of care, overall quality of care is not unduly compromised. It raises new questions of how to ensure a work force that is properly trained and fully capable of performing the essential services of public health for increasingly diverse populations in this increasingly dynamic health care environment. Furthermore, transformations in the health care and public health arena necessitate monitoring and assessing (and the data systems to do so) effects and impacts of these changes on the financing, organization, and availability of population- and clinic-based preventive services and medical treatment.
OPHS has given highest priority in its fiscal 1997 strategic plan and evaluation efforts to addressing these issues. The plan emphasizes the need for a strong foundation for public health in the 21st century. The priorities focus attention on (1) the information systems and work force that constitute the Nation's public health infrastructure and that are necessary to effectively provide the essential services of public health; and (2) the impact of managed care arrangements on the resources available for this infrastructure and on the health of all Americans, especially those most vulnerable.
Future OPHS evaluation planning efforts will reflect the OPHS strategic plan as well as the broader HHS programmatic priorities related to improving the health of all Americans and assessing the effects of health sector transformation. Planned investigations in support of the public health infrastructure include analyses of existing and needed data systems and data to adequately monitor health status and address potential threats to the public's health before they become actual threats; better assessments of public health expenditures at the Federal, State, and local level; and examinations of policies and programs that impact population-based services in the States and communities. Efforts related to managed care include evaluations of service delivery to Medicaid managed care beneficiaries; identification of deficiencies in the "safety net" (e.g., the uninsured) as a result of managed care; an assessment of outreach efforts to racial/ethnic minority populations in the implementation of Medicaid managed care; and collection of baseline data on the extent to which managed care organizations and State and local health agencies with linguistically and culturally diverse populations in their service areas are able to provide services.
Other planned activities will focus on addressing specific health needs of the most vulnerable populations. These include evaluations of successful models for overcoming system barriers to better integration of prevention and treatment services for populations at high risk of HIV infection and those with HIV disease; the effectiveness of demonstration projects intended to promote abstinence among teens who are not sexually active and a project to improve parenting skills among pregnant or parenting teens; guidelines and tools to enhance the development and assessment of health information and materials for linguistically and culturally diverse audiences; the impact of HBCUs consortium activities on reducing family and domestic violence; and the research base for post-traumatic stress disorders in women, children, and minorities.
Finally, in response to the increased need to measure the performance and effects of public health programs and activities, OPHS will support efforts to integrate national Healthy People 2000 objectives and performance measures in grants with program and department performance plans; develop and implement measures that assess the capacity of State and local health agencies to provide essential public health services and to meet OPHS goals; and ensure expansion of the public health knowledge base through scientific investigations in the behavioral and social sciences, preventive medicine, public health practice, nutrition, environmental and occupational health, and health systems and services.