MISSION: To protect and promote public health through food, drug, medical device, and cosmetic regulation.
The evaluation program of the Food and Drug Administration (FDA) continues to be aligned with the Agency's policy, its program and strategic planning initiatives, and the goals of the Government Performance and Results Act (GPRA). Since GPRA and the reorientation of all government managers toward performance management, the responsibility for program evaluation has shifted from traditional specialized staff offices and contractor studies to managers of all FDA programs. In addition, FDA managers are identifying further opportunities to involve their customers in the design and testing of alternative ways of doing business. Both performance-based goals and customer participation also impact FDA's response to the needs of small businesses and enhance the participation and performance of small businesses.
One element of FDA's strategic framework is called external leveraging. The goal here is to augment the ability of external stakeholders to manage FDA-regulated risk. FDA's evaluation program illustrates the implementation of this goal with a number of studies under way or completed during FY 1997 that illustrate the strategies that FDA is using to accomplish this goal. One strategy is the fostering of an industry quality assurance program, where FDA will establish protocols to guide firms, certify their approaches, and audit industry practices. A current example of this quality assurance is the Mammography Quality and Standards Act (MQSA), which monitors the administration of mammography practices. The following reports are based on evaluations of mammography activities: the Mammography Quality Standards Act: Second Annual Government Entity Declaration Program Audit, and the Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992. In addition, the Center for Safety and Applied Nutrition (CFSAN) has implemented the Survey of Current Hazard Assessment Quality Control Points (HAACP) Practices of the Food Industry. While excluding seafood, this survey originated as a result of the Seafood HAACP Program, another FDA-fostered quality assurance program.
Other CFSAN evaluations, both completed and in progress, are clearly in line with FDA's objective of communicating risk associated with FDA-regulated products. These studies are directly linked to the President's National Food Safety Initiative, which will involve a collaboration between FDA, the U.S. Department of Agriculture (USDA), the Environmental Protection Agency (EPA), and the Centers for Disease Control and Prevention (CDC) in order to assure safety and quality of our food supply. Special emphasis will be placed on microbial hazards, which account for 90 percent of the confirmed food-borne outbreaks, 6.5 to 33 million illnesses, and 9,000 deaths in the U.S. annually. Since food-borne chemical hazards often present chronic rather than acute health threats, specific estimates of their impact on health and the economy are not readily available. Finally, data collected from the FDA's Safe Alert Survey will demonstrate how well the Agency is communicating risk associated with medical devices to the medical community.
Summary of Fiscal Year 1997 Evaluations
Prescription Drug User Fee Act
The Prescription Drug User Fee Act (PDUFA) has demonstrated that the Agency and industry, by combining their resources and working together, can achieve the levels of performance the American people deserve. The fifth and final report completed in November 1997 demonstrates that the successes reported in FY 1996 have been confirmed and surpassed. PDUFA has resulted in more and better applications filed which can be accepted immediately, reviewed, and approved more quickly (6079.2). Nearly all applications now go directly into the review process, with "applications refused" approaching the vanishing point. There are substantially more approvals on first review, which demonstrates that submission quality is a key factor in achieving timely approvals. Another indication of improved submission quality is the dramatic increase in the percentage of submissions that are ultimately approved faster. By working to meet the PDUFA's stringent review goals, new products are getting on the market at an unprecedented pace without compromising safety.
Product Review Performance Report
In FY 1997, FDA completed its first annual report analyzing performance in reviewing applications for human drugs, biological products, medical devices, food and color additives, and veterinary drugs (6731). Responding to legislation (such as PDUFA) as well as to requirements under the Federal Food, Drug, and Cosmetic Act, FDA has developed clear performance goals for reviewing new product applications. Setting these goals has provided a valuable management tool for identifying performance expectations, assessing achievements, and meeting the requirements of GPRA. The report finds that (1) for most categories of products, FDA performance has improved since FY 1993; (2) the greatest improvements in performance have occurred for product reviews covered by PDUFA; (3) significant progress has been made in reviewing applications for medical devices; and (4) on-time performance for reviews of non-PDUFA applications should be improved.
Assessment of the Mammography Quality Standards Act of 1992 (MQSA)
Under this statute, FDA is charged with ensuring that accessibility to quality mammography services is maintained for all facilities performing mammographies. This program is FDA's second major user-fee, performance-oriented, GPRA-style initiative. During FY 1997, Eastern Research Group, Inc. completed an economic analysis of FDA's final rule under MQSA working under contract to the Office of Planning and Evaluation (OPE), Economics Staff. The purpose of the cost-benefit analysis was to predict the impact of the proposed standards on the costs of providing mammography. The results of their analysis are presented in the report entitled, Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992 (6080). The results of this analysis will be used by FDA's managers to develop a performance-oriented regulatory policy that will be coordinated with existing customers. OPE's Economics Staff developed a model for estimating health outcomes to a patient population at various levels of mammography quality.
Quality Standards Act: Second Annual Government Entity Declaration Program Audit and the Economic Impact Analysis of Regulations Under the Mammography Quality Standards Act of 1992
FDA's Division of Mammography Quality and Radiation Programs requested an audit of the MQSA Government Entity (GE) Declaration Program (6730). The audit was undertaken by the Division of Planning, Analysis, and Finance. The goals of the audit were to notify facilities that they will be held accountable for their self-certification of fee exemption and to determine the rate of compliance with the program. Based on the results, the auditors concluded that the GE Declaration Program appears to be quite effective. Several recommendations were made based on the findings of the audit. First, the results should be published to allow the impact of the audit to reach most of the facilities. This would discourage any future misrepresentations and demonstrate that FDA will exercise its authority. Second, the sample size of future audits should be smaller and should contain a larger portion of those facilities claiming fee exemption based solely on funding from the CDC. Finally, the MQSA contractor should continue to examine each GE Declaration form to ensure that it identifies the body operating the facility.
Evaluation of Device Performance in the Determination of Substantial Equivalence
Many diagnostic medical devices that reach the market are based on diagnostic agreement with some device already on the market that targets the same attribute. Agreement is often assessed by side-by-side testing of the same subjects or specimens, even if the true disease status is unknown. If each new test is compared with the most recent predicate, what would be the effect on device performance over time? Would it deteriorate, remain the same, or improve? FDA produced an analysis that addressed these questions using statistical measures of sensitivity, specificity, and predictive values (6734). When a new test was found to be equivalent to the current predicate, it became the new predicate. After 50 new predicates were found, a forecast procedure was applied to the sequence of 50 performance parameters in order to evaluate the long-term trend.
Center for Food Safety and Applied Nutrition (CFSAN) Evaluation
During FY 1997, three evaluations relating to food issues were completed by the Economics Branch in CFSAN's Division of Market Studies. The first, Preliminary Investigation into the Morbidity and Mortality Associated with the Consumption of Fruit and Vegetable Juices (6858), assessed the population risk from the consumption of fruit and vegetable juices. The main risks studied were microbial pathogens. Two studies evaluated regulation of the seafood industry: Cost of Restrictions on Gulf Oyster Harvesting for Control of Vibrio vulnificus-Caused Disease (6856), an economic impact analysis of alternative remedies for controlling Vibrio vulnificus-related illnesses in the Gulf States oyster industry, and Health Transfers: An Application of Health-Health Analysis to Assess Food Safety Regulation (6857), an evaluation tool based on the first study's methodology that can be used to measure economic versus health costs of public health decisions.
Evaluations in Progress
FDA currently has a broad range of evaluations in progress which, as in previous years, are based upon the two driving forces relating to GPRA goals: performance management and customer participation (with particular focus on small businesses). The following are examples of FDA evaluations in progress during FY 1997.
Although PDUFA ends in FY 1997, FDA and the pharmaceutical and biological prescription drug industry have negotiated re-enactment beyond FY 1997. Reauthorization of the collection of user fees under the Food and Drug Administration Modernization Act (enacted in November 1997) will continue to enhance the review process of new human drugs and biological products (6079.3). The PDUFA program serves as a model for reinventing government, with the Congress, FDA, the industry, and consumer groups working together to provide the necessary resources, set performance goals, and hold the Agency accountable. Revenues generated from the fees paid by the pharmaceutical and biological prescription drug industries will continue to be dedicated to improving and expediting the prescription and application review and approval process. The legislation reauthorizing PDUFA for FY 1998 was critical in ensuring uninterrupted service to U.S. customers.
Evaluation of Influenza Virus Vaccines in High-Risk Patient Categories (4355) is intended to ensure public health by investigating the reactogenicity and immunogenicity of influenza virus vaccines. The information will be critical to making accurate recommendations for the selection of new strains of the influenza virus vaccine to be used in the United States.
Nationwide Evaluation of X-Ray Trends (NEXT) (4984) is a collaborative program conducted jointly between FDA and State radiation control agencies. The study estimates the radiation dosage from diagnostic x-ray examinations using data collected from previous NEXT surveys of different types of x-ray examinations. The data are used by public health and professional organizations in setting policy.
Food Labeling and Package Survey (FLAPS) (5711) was implemented by the FDA in 1977 to produce a data base containing label and package information obtained from a sample of food products from the processed packaged food industry. FLAPS data enable FDA personnel to keep abreast of market responses to food labeling rules via changes in product package labels.
FDA Penalty Reduction Program for Small Businesses (6461) reviews pre-existing and revised policies related to monetary penalties for noncompliance with FDA regulations, evaluates the scope of small businesses that qualified (or failed to qualify) for the program, and assesses the total amount of penalty reductions or waivers. The Small Business Regulatory Enforcement Fairness Act (SBREFA) required FDA to create a program or policy that specifies the circumstances under which the Agency may reduce or waive the penalties levied against small business entities for noncompliance. The final report for this project is anticipated in March 1998.
FDA Inquiries Assistance Program for Small Businesses (6462) is FDA's response to the SBREFA requirement to establish a program to respond to inquiries and to help small businesses and other small entities apply FDA rules. The report on this project is anticipated in March 1998.
During FY 1998, FDA will continue to implement the requirements of the Mammography Quality Standards Act of 1992 (MQSA) (6080.1). FDA requested an increase in MQSA authorized inspection fees over the FY 1997 level. Continuing implementation of the MQSA requirements is essential to ensure that women receive quality mammography from facilities that maintain a high standard of safety and accuracy. The primary emphasis of this program for FY 1998 will be to continue to ensure that all facilities meet the quality standards and that deficiencies previously identified through annual inspections are corrected. In FY 1998, Federal and State personnel will continue to conduct annual inspections of 10,000 facilities and certifications of 6,000 facilities, and to provide training for new inspectors. The fees will be used to pay for the costs of the inspections.
Review of Therapeutic Agents as an Antimicrobial Adjunct in Treatment of Sepsis (6736) is a technique used in the drug approval process. No therapeutic agent has yet been identified that shows either a clinical benefit or an alternative that is superior to standard care. FDA faces many challenges in designing clinical trials that demonstrate efficacy. These are due to the heterogeneity of patient populations, poor understanding of the pathophysiology of acute sepsis, and no uniformly-accepted standard of care in these international trials. Intensive efforts have been undertaken by FDA to propose the optimal trial design and prospective analysis plans. This analysis of failed sepsis trials is being conducted by FDA to achieve better clinical trial results for the evaluation of adjunctive sepsis therapy in the future.
The Real-Time PMA Supplement Program Evaluation (6732), implemented by the Office of Device Evaluation (ODE), has evolved as a result of the Medical Device Review of the Premarket Approval (PMA) Applications program. FDA has reduced review times and backlogs for medical device applications. But, in order to achieve mandated goals, continued improvement is needed. FDA requested an increase in funds for FY 1998 in order to improve the quality and timeliness of its review process for Class III PMA applications and PMA supplements. These applications involve the highest-risk devices that require full clinical investigations of safety and efficacy. An increase in funding will facilitate FDA in allocating more resources to the PMA process without sacrificing performance gains made in the 510(k) program. It is projected that FY 1998 will be the first year of this enhancement effort, and FDA expects a 15 percent increase in PMA workload during FY 1997 and FY 1998 due to requests for reclassifications and submissions of required preamendment PMAs. Given these additional resources, FDA will attempt to limit the increase in pending PMAs at the end of FY 1998 to 5 percent. FDA has defined specific review performance goals for Class III applications that should be achieved by the end of FY 1998 with the additional resources.
The Safe Alert Satisfaction Survey (6733) will determine the impact of the safety alert and public health advisory on the behavior and knowledge of the public. The collection of these data is an important step in determining how effectively FDA is communicating risk to the public.
Survey of Current Hazard Analysis and Critical Control Point (HACCP) Practices of the Food Industry (6866) is a two-part survey that will yield information that will be used to estimate the HACCP-related practices of the food industry (except seafood, meat, and poultry), and a model that will be used to estimate the marginal costs of firms to comply with HACCP.
Economic Characterization of the Dietary Supplement Industry (6867) will provide a detailed analysis of the dietary supplement industry, including a description of the types of firms producing and marketing dietary supplements, and will create a relational data base that will link firms to products.
Estimating the Benefits of Various Kinds of Food Standards (6865) will categorize food standards by type of food and function and to estimate consumers' valuations of those functions.
Harmonizing Nutritional Goals and Food Processing (6864) is a paper to be given at the "Towards Convergence on the Dietary Guidelines, Research and Policy Needs in the Nutrition, Health, and Agricultural Sectors" conference on April 30, 1998.
The Impact of Tobacco Initiative on Youth (6890) is a project in which Office of Planning and Evaluation staff are currently designing a sample questionnaire to facilitate data collection on tobacco use. The survey responder will determine whether the tobacco initiative has had any positive impact on the propensity of youth to use these products. In August 1996, President Clinton approved FDA's final rule for regulating tobacco products. The President's Tobacco Initiative demonstrates a collaborative effort to thwart major public health risk with the objective of decreasing tobacco use over the next 7 years. FDA is working with the States to conduct unannounced compliance checks of retail establishments that sell tobacco products. FDA will rely on Federal and State agencies to monitor retailer compliance with the new regulations and track the degree of success in decreasing tobacco use by youth.