MISSION: To provide public health advice to the Secretary of Health and Human Services and executive direction to the PHS Agencies through coordination and oversight of their programs.
OASH Evaluation Program The primary role of the Office of the Assistant Secretary for Health (OASH) is to coordinate and develop evaluations across the entire Public Health Service. The Assistant Secretary for Health (ASH) issues guidance each fiscal year to the PHS Agencies for preparation of their plans outlining evaluation priorities and projects proposed for implementation in that fiscal year (as mentioned in chapter I). OASH reviews the Agency plans to identify gaps in evaluation data needed for program or policy development. OASH also identifies potential program areas that could benefit from a collaborative evaluation.
In addition to its PHS-wide coordination function, OASH uses the 1 percent set-aside funds to initiate its own evaluations or policy studies, usually in consultation with relevant Agency program offices. The projects conducted by OASH are generally short, quick-turnaround policy studies to support initiatives of the ASH. Occasionally, OASH will initiate or participate in large-scale evaluations, particularly those that cut across PHS program areas.
The OASH review and approval of evaluation projects is performed by the Evaluation Policy Review Committee (EPRC), which makes recommendations to the ASH on OASH evaluation priorities for each fiscal year; reviews project proposals for relevance to PHS strategic planning objectives; and provides advice to the ASH in determining the OASH use of 1-percent evaluation funds and allocation of those funds to approved projects. Among the criteria used by the EPRC to review proposed projects are relevance to ASH priorities for policy, legislative, budget, program development, or crosscutting PHS issues; relevance to departmental and PHS strategic planning objectives and related evaluation strategies; coordination with other PHS and departmental offices; and relevance to the Healthy People 2000 objectives. OASH evaluation staff review all EPRC-approved projects for technical quality and feasibility, providing feedback to OASH program offices on project proposals and statement of work for contract procurements. In addition, the staff serves as a resource group for individual technical assistance and consultation to project officers in developing evaluation proposals.
Summary of FY 1994 Evaluations In FY 1994, OASH completed 16 evaluations in diverse areas of public health: health care reform, immunizations, adolescent and school health, nutrition, primary care, emergency preparedness, international health, and other smaller regional studies of health services delivery. Most of these evaluations focused on policy analysis to support initiatives in health care reform. Despite the failure of health care reform legislation, these studies are relevant because health care reform proposals are still being considered by Congress and because market-based reforms are occurring in the absence of legislation.
One study on the returns on investment in public health examined the contributions of public health programs in reducing the incidence of preventable illnesses and injuries. This work summarized the effectiveness of public health strategies from a cost-benefit analysis perspective to underscore the importance of the public health infrastructure in realizing the two primary goals of health care reform: to expand access to care and to control costs.
Two related studies examined the implications of health care reform on the health care workforce. The study on a proposed Graduate Nursing Education Account, which is highlighted in chapter II, documented the projected scarcity of advance practice nurses such as nurse practitioners, certified nurse-midwives, certified registered anesthetists, and clinical nurse specialists. It analyzed several options for channeling funds to support advance practice nursing education to meet future needs. Another study examined the supply, training, and distribution of primary care providers as part of health care reform efforts. It examined the rationale for new proposals to control the supply of new physicians, proposals for an all-payer pool for graduate medical education, and estimates of the number of advanced practice nurses that would be needed as substitutes in the case of a reduction in the number of physicians.
In recent years, OASH has given priority to evaluations of PHS immunization policies and programs. One study, also highlighted in chapter II, examined the economic underpinnings of the vaccine supply. A major finding was that States that supply vaccines at low prices to physician offices and encourage parents to have their children vaccinated have higher rates of immunization. In another study, OASH developed a design to assess the functional capabilities of the Vaccine Adverse Event Reporting System (VAERS). This design project will help PHS officials evaluate VAERS to determine whether it is providing reliable and accurate information, and how health care providers, manufacturers, and consumers perceive the effectiveness of the system.
Support for nutrition policy studies has been another OASH evaluation priority. In FY 1994, OASH completed a project with the American Institute of Nutrition to examine state-of-the-art techniques for measuring nutritional status and to make recommendations for a core set of nutritional status indicators for low-income populations. Development of these indicators will be important for the development of national and State nutrition monitoring systems, particularly in expanding the survey coverage of low-income populations.
Adolescent health, including school heath education programs, is another OASH program evaluation priority area. Two projects in this area, both described in chapter II, were completed in FY 1994. The first study sought ways to evaluate the academic benefits of school health programs. It developed a general framework, with alternative research designs, for assessing the effects of school health interventions on students' school performance. The second study was a focus group study of the knowledge, attitudes, behaviors, beliefs, and environments of youth engaged in high-risk behaviors. The results of this study will be used to develop appropriate health messages to youth about high-risk behavior.
Evaluations in Progress OASH has 14 program and policy evaluations under way in two major areas: health care reform and population-based services. In addition, OASH is supporting several smaller projects investigating crosscutting PHS issues and several evaluations specific to OASH program offices. Some of the evaluations that are likely to be completed in FY 1995 are described below.
With respect to health care reform, OASH is continuing to examine two major issues: the role of Federal, State, and local health information systems in achieving public health objectives, and the impact of health care reform on the health care workforce and the role of academic health centers (AHCs).
Health information systems at all levels of government are the focus of three projects. One is developing sets of indicators that communities can use to assess performance and to foster closer collaboration between public health and personal health care systems and employers in achieving specific public health goals. Another is reviewing current State laws, policies, practices, and environments governing public health reporting systems, health care information systems, health data confidentiality, and access and linkage policies. This project will identify aspects of existing laws, policies, or practices that create barriers to standardization, integration, data linkage, and access relating to health information systems within and across States. Third, OASH is supporting the National Committee on Vital and Health Statistics in a project to conduct analyses for the development of uniform health data sets for enrollment and encounter information from health care providers in reformed health care systems. Data derived from the uniform data sets will provide a foundation for monitoring and evaluating public health programs as well as systems of care in the framework of national health information networks.
Health care reform's impact on the health care workforce and on the AHCs, where many health professionals are trained, is the subject of two projects. First, OASH initiated a project to study how the growth of competitive health care financing and service delivery systems based on managed care affects the financial support available to AHCs and graduate medical education. The project focuses on AHCs in three cities: San Diego, Minneapolis-St. Paul, and Washington, D.C. Another project is examining the extent to which teaching hospitals associated with the AHCs are reorganizing and/or reducing services and other activities in areas in which competitive health plans hold a substantial share of the market. The project will assess the impact of such changes on hospital training programs for physicians.
In addition to these analytic studies, OASH will support a series of roundtable discussions among leading national experts in government, academic, clinical, and research settings regarding the role of AHCs in an era of health care reform. The purpose of these discussions is to exchange views and reach a national consensus on questions of the potential impact of health care reform on AHCs. Many questions are being raised about their mission, financial capability, and contribution to patient care and public health; how they can best participate in a national shift from training specialist physicians to training primary care practitioners; how health care reform affects clinical research; and what strategies may best protect the quality of clinical research and its contributions to public health.
The OASH evaluation program has also given priority to evaluations in the area of clinical preventive services. The Office of Disease Prevention and Health Promotion (ODPHP) will soon complete the design for the "Put Prevention Into Practice" initiative, a package of resource materials for health personnel in clinical settings to promote more effective use of clinical preventive services for their patients. The design will be implemented in late FY 1995. The results of this evaluation will help national primary care provider organizations determine the optimal use of these materials in their respective clinical settings.
ODPHP is also supporting a major effort to improve methodologies for evaluating the cost-effectiveness of clinical preventive services. A group of specialists in cost-effectiveness analysis, both inside and outside government, is working to develop guidelines to apply cost-effectiveness methods to clinical preventive services. The group will address current controversies about these methods and propose alternatives for improving the comparability of cost-effectiveness analyses.
As mentioned above, OASH often initiates evaluations of programs or policy issues that cut across the PHS Agencies. Several OASH projects in progress illustrate collaborative evaluations. Since 1993, OASH has worked with the Food and Drug Administration (FDA), the National Institutes of Health (NIH), and the Substance Abuse and Mental Health Services Administration to evaluate the departmental methadone regulations, a project expected to be completed in FY 1995. The study is examining the impact of Federal regulations on the provision of methadone treatment services and on the development of new anti-addictive medications. The final report will contain recommendations for improving the current methadone regulations.
In another project, OASH is working with the Centers for Disease Control and Prevention, the FDA, and NIH to study HIV transmission to hemophiliacs through blood products. In the early period of the AIDS epidemic, before the invention of rapid laboratory tests for identifying HIV infection, blood transfusion services for hemophiliacs operated without effective HIV screening or production methods to eliminate the virus. The Secretary of the Department of Health and Human Services, Donna Shalala, launched an investigation in 1993 to get a more complete understanding of the events that occurred in those early years. The results, expected in FY 1995, will be helpful for ensuring the safety of the Nation's blood supply against future challenges.
OASH also uses evaluation funds to support projects that measure the effectiveness of program activities managed by the various OASH Offices. For example, the Office of Minority Health presently is evaluating its HIV/AIDS Education/Prevention Grant Program. The grants program is providing assistance to community-based organizations in developing innovative approaches for reaching persons living in minority communities who are at risk for HIV infection. The evaluation is now analyzing data on disease prevention intervention models that have a likelihood of successful replication in other minority communities.
Another example of OASH evaluation activity is the efforts of the OASH Office of Research Integrity (ORI) to examine the consequences of being accused of research misconduct, as well as the consequences for the whistleblower. The results of these two projects will be useful to ORI for monitoring institutional compliance with PHS regulations on research misconduct, planning educational activities, and making improvements in PHS policies and procedures for responding to allegations of research misconduct.
New Directions for Evaluation The PHS FY 1995 Strategic Plan has a goal to strengthen the public health infrastructure. In recognition of the danger faced by health departments as a result of years of shrinking budgets, the PHS Agencies and OASH are striving to augment the capability of the Federal Government, tribal governments, States, and communities to identify and address high-priority health problems for their populations. One approach is to expand the public health knowledge base by supporting an appropriate balance of focused and multidisciplinary research in the areas of basic biological science, clinical medicine, public health practice, behavioral and social sciences, epidemiology, health systems and services, nutrition, and occupational and environmental health. In coming years, OASH will support the development and implementation of measures that will assess the capacity of State and local public health agencies to perform population-based functions and the extent to which States meet PHS program goals.
Managed care is changing the way health care is paid for and delivered and shifting responsibilities for care. This has serious implications not only for vulnerable populations but also for public health "safety net" providers, academic health centers, and public health agencies. This year, OASH will participate with the PHS Agencies and the Health Care Financing Administration to examine the impact of managed care on public health systems, primarily through demonstrations of the Medicaid Waiver (Section 1115) in five States. The Medicaid Waiver allows States to experiment with alternative approaches to improving the access and quality of health care services for the uninsured and reducing the costs of care. The analysis is expected to focus on the number of eligible participants, the costs to providers, the quality and types of care, and the access for special populations--such as American Indians and Native Alaskans, children with special health care needs, people in underserved areas, substance abusers, and the chronically mentally ill.