MISSION: To protect and promote public health through food, drug, medical device, and cosmetic regulation.
Systemic changes in the government management environment are strongly influencing the setting, conduct, and utilization of evaluation activities in the Food and Drug Administration (FDA). Two forces related to the goals of the Government Performance and Results Acts (GPRA) are driving and reshaping the evaluation function within the FDA.
The reorientation of all government managers toward performance management has shifted the responsibility for program evaluation from specialized staff offices and contractor studies to the line managers. Picking appropriate program goals, establishing a valid measure of these goals, and collecting management information to record the measured progress toward these goals are now an integral part of the manager's responsibility. Relearning the role of management with regard to these shifted responsibilities is a key priority.
Most of the FDA's management measures of its own performance are also measures of the performance of the regulated industries. Thus, while the beneficiary of FDA's performance is ultimately the general public, the FDA supplies industry with an essential component of commercial success. The FDA's approval of a new drug, for example, not only satisfies a legal requirement but also assures the public of the safety and efficacy of the drug. Collaboration between the FDA and its regulated customers regarding the design and coordination of the joint responsibilities to ensure effective high-quality products has been a revolutionary concept, but it is becoming the new norm under the customer-conscious GPRA directives.
An extension of both performance-based goals and customer participation also affects our response to small businesses.
Responding to the Needs of Small Business
A high proportion of industries regulated by the FDA are small businesses. In order to improve responsiveness to industry needs without compromising the protection of the public health, the FDA is emphasizing two strategies. One is to enhance industry input during the rulemaking process and to increase the rigor of analysis of possible regulatory strategies. The second is to improve compliance, where possible, through better education and remedial technical assistance. Both strategies extend the existing priority of customer participation and performance-based goals to focus on enhancing the participation and performance of small businesses.
In sum, the FDA's evaluation efforts are driven by the mandates of GPRA and its corollaries, are carried out by line managers rather than specialized evaluation staffs, and are focused in the areas of performance management and customer participation, with additional attention to the needs of small businesses. Because the FDA receives its funds from agriculture appropriations rather than Public Health Service Act appropriations, the FDA does not manage a 1-percent evaluation set-aside.
Summary of Fiscal Year 1996 Evaluations
FY 1996 marked the third year under the new evaluation paradigm. The following example illustrates the manner in which managers have integrated evaluation into their line responsibilities and conducted evaluation in cooperation with the affected customer.
Implementation of the Prescription Drug User Fee Act
FY 1996 marked the fourth year of the 5-year implementation of the Prescription Drug User Fee Act (PDUFA) of 1992, which authorized the collection of fees from the pharmaceutical industry in order to facilitate the FDA's timely review of human drug applications. The act mandates that the FDA prepare annual reports for each fiscal year in which fees are collected. The Fourth Annual Performance Report (covering FY 1996) was prepared in response to that mandate and summarizes findings on how well the FDA has accomplished its performance goals for FY 1996. These performance-related goals were jointly established between the FDA and the pharmaceutical industry and fall into three main categories: eliminating overdue backlogs, building excellence into the review process, and achieving high performance. Eighteen of the goals have already been accomplished and reported in previous years' performance reports. Four additional goals were in effect for FY1996, while seven more will be in effect for FY 1997.
The success of the fourth year of PDUFA is reported in the Fourth Annual Performance Report. Despite an increasing workload, the agency has exceeded every PDUFA performance goal, resulting in a record number of new product approvals and shorter approval times. Under PDUFA, during FY 1996, the FDA approved 131 new drug applications and product license applications, an increase of 56 percent over FY 1995's total (84) and a 96-percent increase over FY 1994's total (67). PDUFA has enhanced the working relationship between the FDA and its sponsors, resulting in higher quality applications that can be accepted immediately and reviewed more quickly. Ultimately, new products are reaching the market faster.
Consumer Use of Food Labels
The Nutrition and Labeling and Education Act (NLEA), passed by Congress in November 1990, led to the most far-reaching changes in food labeling since the Food, Drug, and Cosmetic Act of 1934. The new regulations, governing the content and format of food labels on nearly all FDA-regulated products, went into effect in 1994. The NLEA had multiple objectives that will have multiple short- and long-term impacts on consumers and the marketplace. To monitor some of these impacts, the FDA is tracking consumer awareness, attitudes, and reported behaviors through nationally representative sample consumer surveys, is examining changes in the marketplace by looking at sales trends for selected product categories, and is assessing compliance with NLEA regulations. Some preliminary findings are presented in The Impact of the NLEA on Consumers: Recent Findings From FDA's Food Label and Nutrition Tracking System, which was completed during FY 1996. This report offers some insights into the impact of the new food label regulations that occurred in the period shortly after the regulations were implemented:
- The data support the popular perception that Americans have embraced the new food label as an effective tool to enable them to make healthier food choices.
- Consumer confidence in nutrient content and health claim information also showed improvements.
- Information on food labels appears to be having a growing impact on consumer decision making.
For a small number of product categories in which fat-modified products are available, there has been an impressive, simultaneous increase in new product introductions and percentage of market share for these fat-modified products. While each product category has a unique history, these data mirror recent news reports that suggest both consumers and food manufacturers have responded to the impact of the nutrition facts label on food products. These preliminary market trend analyses are consistent with the views that increased availability of nutrition information, as mandated by the NLEA regulations, has contributed to a growing number of new fat-modified products in the marketplace and that these products are garnering an increasing share of their respective markets.
Food Labeling and Package Survey
The report, Status of Nutrition Labeling of Processed Foods--1995, completed in FY 1996, reviews the status of processed, packaged food labels subsequent to publication of regulations promulgated in response to the NLEA of 1990. The findings were based on data from the Food Label and Package Survey (FLAPS), started in 1977. The purpose of FLAPS is to produce a data base containing label and package information obtained from a sample of food products from the processed packaged food industry. The primary information gathered consists of the prevalence of nutrition and sodium labeling. Label and package information are obtained from the food labels of approximately 1,300 brands of food. The sampling frame for FLAPS is based on sales data obtained from the A.C. Nielson Company, from which sales-weighted estimates are subsequently derived. The major findings from this review include the following:
- Nutrition-labeled products account for an estimated 96.1 percent of the annual sales of processed, packaged food across 58 major product groups.
- There was a 27.2-percent increase in the proportion of nutrition-labeled products between 1993 and 1995.
- For processed meat products, 100 percent are now nutrition labeled, an increase of 47.9 percent over the rate in 1993.
- In three product groups, less than 90 percent of the products sold have nutrition labels--tea, salt (including seasoning and spices), and coffee.
The current study focuses only on the percentage of packaged food products sold annually that bear quantitative nutrition labeling, as well as data on health claims, nutrient content claims, and ingredients.
Evaluations in Progress
The evaluation agenda of managers during FY 1997 and in the future will be influenced by two objectives, performance management and customer participation, which reflect the forces shaping their evaluation role. Managers of all FDA programs are engaged in the process of evaluating their performance measures in light of the GPRA standards. Broad-based training of managers to enable them to implement the performance design of their responsibilities is under way. This multiyear process toward a new standard of performance measure will end its first phase with the formulation of the FDA's FY 1997 budget. FDA managers are also identifying further opportunities to involve their customers in the design and testing of alternative ways of doing business. The design and implementation of the PDUFA is the notable example of the successful pursuit of this objective. FY 1997 will mark the fifth annual performance evaluation report on this successful program.
The following projects are currently under way.
Implementation of the Prescription Drug User Fee Act
FY 1997 marks the fifth year of the PDUFA program. Seven goals have been established, delineating the most ambitious standards of this program. The effects of additional resources were evident in the FY 1996 program where, despite an increasing workload, the agency exceeded every performance goal, resulting in a record number of new product approvals and shorter approval times. There is also evidence that the quality of applications has improved. Based on last year's performance, it is anticipated that the FY 1997 phase of this program will not only meet the established goals, but substantially exceed those goals.
Assessment of the Mammography Quality Standards Act of 1992 (MQSA)
This program is the FDA's second major user fee, performance-oriented, GPRA-style initiative. Under this statute, the FDA is charged with ensuring that accessibility to quality mammography services is maintained for all facilities performing mammographies. The agency is required to certify and annually inspect more than 10,000 mammography facilities. Prior to passage of the MQSA, the FDA's oversight responsibility was limited to approving new mammographic processors, x-rays, and tubing. However, MQSA requires that facilities within its jurisdiction meet standards developed by the agency in order to ensure that a reasonable level of quality is maintained to achieve certification by the agency.
The FDA is concerned that recent trends by third-party payers to capitate payments for mammography services have severely limited the ability of facilities to absorb any cost increases caused by the quality standards. Studies have indicated that excess capacity exists within the industry. Therefore, more efficient provision of mammography services may result in low-volume facilities leaving the industry. The agency needs to assess these economic implications in order to ensure that patients will continue to have access to affordable, high-quality mammography services. This study addresses these issues in two ways. The first involves measuring the effect of standards on patient access to mammography, particularly in rural areas. Findings for this analysis were reported in the document Effects of Facility Closures on Access to Mammography. The second component is a cost-benefit analysis that will be used to predict the impact of proposed standards on the costs of providing mammography. These issues are addressed using data collected from facilities that provide mammography services. A draft report of the results of the cost-benefit analysis was presented in the spring of 1996, and comments are currently being reviewed. A final version will be available in the fall of 1997. Agency managers are using these results to develop a performance-oriented regulatory policy that will be coordinated with existing customers.
Food Labeling and Package Survey (FLAPS)
This is an ongoing study conducted by the FDA's Center for Food Safety and Applied Nutrition; it will continue through December 1999. In order to monitor labeling practices of domestic food producers, the FDA implemented FLAPS in 1977. The purpose of FLAPS was to produce a data base containing label and package information obtained from a sample of food products from the processed, packaged food industry. The primary information gathered consists of the prevalence of nutrition and sodium labeling. Label and package information is obtained from the food labels of approximately 1,300 brands of food. The sampling frame for FLAPS is based on sales data obtained from the A.C. Nielson Company, from which sales-weighted estimates are subsequently derived. The FDA uses these data to quantify the prevalence of nutrition and sodium labeling in the food supply, the extent of quantitative labeling for cholesterol and fatty acid content, the extent of use of contents product ingredient lists, and the extent and types of nutrition claims appearing on food products. FLAPS data enable FDA personnel to keep abreast of market responses to food labeling rules via changes in product package labels.
FDA Inquiries Assistance Program for Small Business
Under the Small Business Regulatory Enforcement Fairness Act, the FDA must establish a program for answering inquiries and providing assistance to small businesses and other small entities, regarding how to comply with FDA rules. The FDA currently provides assistance and advice to small businesses in a multitude of ways, including Center and regional small business representatives, inspectors, public affairs staff, and others. This project will assess the level of success achieved in coordinating these many sources of advice and usefulness and appropriateness of the assistance. Elements to be evaluated include the frequency, subjects, sources, and recipients of inquiries; the type and timeliness of response; and the number and characteristics of complaints or requests for second opinions. The evaluation will also consider the training and materials available to FDA staff to help them answer inquiries, the barriers to providing timely and useful responses, and issues related to tracking inquiries and responses. The report on this project evaluation is due to be completed by March 1998.
FDA Penalty-Reduction Program for Small Business
The Small Business Regulatory Enforcement Fairness Act requires the FDA to create a program or policy that specifies the circumstances under which the agency may reduce or waive the penalties levied against small business entities for noncompliance. The program must establish the defined circumstances under which alternative penalties can be assessed. This project involves assessing preexisting and revised policies related to monetary penalties and evaluating the scope of the small business penalty-reduction program, the number of enforcement actions against small businesses that qualified (or failed to qualify) for the program, and the total amount of penalty reductions or waivers. The report on this program will be completed by March 1998.
Influenza Virus Vaccines in High-Risk Patient Categories
This study was initiated in FY 1995 and will be ongoing through FY 1998. No reports or publications will be produced from the study. The findings from this project will be used to safeguard public health by investigating the reactogenicity and immunogenicity of influenza virus vaccines. The information derived from this study will also be used to facilitate the scientific review at the FDA's Center for Biologics Evaluation and Research of current influenza virus vaccines used in the United States. Serum samples collected from these studies provide data on antibody responses to the influenza viruses circulating in human populations, indicating whether current vaccines are likely to fail to protect against emerging influenza virus vaccines. This information is critical for making accurate recommendations for the selection of new strains to be used for influenza virus vaccines that are manufactured domestically, guaranteeing the annual availability of serological data to the Public Health Service, data which are unavailable from any other source.
Nationwide Evaluation of X-Ray Trends (NEXT)
Another ongoing domestic evaluation, this survey is sponsored through a cooperative agreement jointly conducted between the FDA and State regulation control agencies. No reports are produced from this study, and this program will continue indefinitely. The study estimates the radiation dosage from diagnostic x-ray examinations, using data collected from previous NEXT surveys of different types of x-ray examinations. Data are available on image quality, photographic processing quality, and types of practice. Public health and professional organizations use this information in setting policy. Some of the products generated in the past have included the National Surveys of Mammographic Facilities conducted during the 1980's and 1990's, a study dealing with poor mammographic quality associated with photographic processing, assessment of skin entrance kerma in the United States, and automatic film-processing analysis of 9 years of data.
New Directions for Evaluation
Changes in government management have created a systemic change in the FDA's evaluation function. The new paradigm--driven by the line manager's performance responsibilities, by the imperatives for involving customers, and by directives for analytically rigorous rulemaking--has replaced the traditional practice of evaluation guided and managed by centralized, specialized evaluation staffs.