HHS Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated to the Public. VII. Influential Scientific, Financial, and Statistical Information

10/01/2002

As illustrated by the number and types of information we disseminate and the variety of methods we use to disseminate them, it is clear that we strive for a high degree of transparency with regard to all of our information dissemination activities. The OMB Guidelines, however, apply special quality standards to the dissemination of information that is considered influential. Such information must meet high standards of transparency of the data and methods used to facilitate the reproducibility of such information by third parties. As defined below, influential scientific, financial and statistical information that FDA disseminates will meet the high standards in the OMB Guidelines for such information.

  1. Definition of the Term Influential

    The term influential information, when used in the OMB Guideline in the phrase "influential scientific, financial, or statistical information," applies when the agency can "reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions" (67 FR 8452; February 22, 2002). However, because each agency is different, and there are vast differences in the types of information they disseminate, each agency has been asked to elaborate on the definition of influential in the context of their missions and duties, "with due consideration of the nature of the information they disseminate." As stated in the OMB Guideline (V.9), "[e]ach agency is authorized to define 'influential' in ways appropriate for it given the nature and multiplicity of issues for which the agency is responsible."

    For purposes of this guidance, influential information is defined as disseminated information that results from or is used in support of agency actions that are expected to have an annual effect on the economy of $100 million or more or will adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or tribal governments or communities. It should be noted that the definition applies to "information" itself, not to decisions that the information may support. Even if a decision or action by FDA is itself very important, a particular piece of information supporting it may or may not be "influential."

    Two examples of what FDA considers to be influential information follow.

    • Quality Mammography Standards

      On October 28, 1997, we issued a final rule (62 FR 55852) amending our regulations governing mammography to provide increased assurance of adequate and consistent evaluation of mammography facilities on a nationwide level and compliance of the facilities with quality standards. Costs of the regulation include replacing below standard mammography units and film processors, providing written results of tests to patients, providing telephone results of tests to referring physicians, and conducting required weekly image quality tests. Because this rulemaking was expected to have an annual effect on the economy of more than $100 million, the information FDA disseminated in support of this rulemaking would be considered influential information.

    • Hazard Analysis and Critical Control Point (HACCP); Procedures for the Safe and Sanitary Processing and Importing of Juice

      On January 19, 2001, we adopted a final rule (66 FR 6138) to ensure the safe and sanitary processing of fruit and vegetable juices. The regulations mandate the application of HACCP principles to the processing of these foods. HACCP is a preventive system of hazards control. FDA adopted this rule in response to a number of food hazards associated with juice products and because preventive control measures are the most effective and efficient way to ensure that these products are safe. The final regulation involves costs to the manufacturers and processors of juice products for implementing procedures consistent with the regulation that exceeded $100 million. Therefore, the information disseminated in connection with this rulemaking would be considered influential information.

  2. Transparency

    If information that meets the criteria for influential information is disseminated, the OMB Guidelines provide that it must meet certain higher standards of transparency and methods to facilitate the reproducibility of information by qualified third parties. When we disseminate information, but particularly in those cases involving influential information, we strive to ensure that the information is accurate and unbiased, as well as substantially reproducible and replicable. This goal is accomplished by using reliable data sources and sound analytical techniques, and by employing a high degree of transparency about the data, methods, measures, assumptions and limitations used to develop the information to facilitate reproducibility by third parties. Our goal is to provide a clear explanation of the assumptions and data upon which we base our conclusions, the criteria used to determine the suitability of the data for use, the methods used in our analysis, and the conclusions we have drawn.

    Because of legal obligations to maintain the confidentiality of data supplied by third parties, there may be instances when original or supporting data may not be available to the public. In such cases, we will disclose the specific data sources used and the specific quantitative methods and assumptions employed. We will also conduct especially rigorous robustness checks of any models used in the analysis and the analytic results so that there will be a high degree of confidence in the results.

    Biases, if any, will be revealed. All assumptions used in the analysis, the scientific rationale, and data used to estimate the impact of the various factors influencing the analysis should be clearly stated. This ensures that biases will be eliminated or minimized and that any introduced biases will be clearly identified.

    Clarity includes ensuring the information disseminated is clear and understandable. When detailed technical information is needed to provide sufficient information so that a qualified third party could reproduce the analysis, the resulting document may be lengthy and difficult the public to understand. One approach that can provide additional transparency in such cases is to develop an interpretative summary document as a companion to the technical analysis. The summary document can provide a non-technical explanation of the data, process, results, and conclusions in a manner that the public can understand. As discussed under "Objectivity," we have a strong commitment to writing all our new documents in plain English. As we revise and update existing documents, we will ensure that they are written in plain English. Our goal is to make our written communications more understandable.

    A participatory process should be used. The process for generating information defined as influential should be transparent. One approach is to invite public comment on the information to be disseminated and encourage stakeholders to submit scientific data and information that can be used in preparing the information. As appropriate, we will solicit advice and opinions of advisory committees as well as peer review from experts within and outside of the agency. To the extent practicable under confidentiality laws, we will strive to make supporting data and analyses available to the public for technical review and comment. This can be accomplished by posting the information on our web pages and providing printed copies as requested.

  3. Risk Assessment

    Some of the influential information that we disseminate is based on an analysis of the risks to the public of certain actions or exposures to hazardous substances. For purposes of this guidance, we are defining risk as the likelihood that injury or damage is or can be caused by a substance, technology, or activity. We use risk analysis (the integration of risk assessment with risk management and risk communication) as a tool to enhance the scientific basis for all of our regulatory decisions.

    The OMB Guidelines provide that in addition to the ordinary standards for utility, objectivity, and integrity that apply to dissemination of information, special considerations must be taken into account in certain risk assessments, i.e., those that provide the basis for the dissemination of influential information. The Guidelines state that "With regard to analysis of risks to human health, safety, and the environment maintained or disseminated by the agencies, agencies shall either adopt or adapt the quality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B)).

    The SDWA risk assessment principles are as follows:

    1. To the degree that the agency action is based on science, the agency shall use

      1. the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
    2. In the dissemination of public information about risks, the agency shall ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.

    3. In a document made available to the public in support of a regulation, the agency shall specify, to the extent practicable

      1. Each population addressed by any estimate of applicable risk effects
      2. The expected risk or central estimate of risk for the specific populations affected
      3. Each appropriate upper-bound or lower-bound estimate of risk
      4. Each significant uncertainty identified in the process of the assessment of risk effects and the studies that would assist in resolving the uncertainty and
      5. Peer-reviewed studies known to the agency that support, are directly relevant to, or fail to support any estimate of risk effects and the methodology used to reconcile the inconsistencies in the scientific data

    Many of our actions are based on scientific experts' judgments using available data, are essentially qualitative, and are generally carried out for non-cancer-causing hazards. Such assessments provide useful answers in most instances that are sufficient for regulatory purposes, and much more elaborate, quantitative estimates extrapolating beyond the data are unnecessary. For example, we may issue regulations on submission requirements for product approval applications, electronic submission of product labeling, or periodic reporting by manufacturers of adverse events from drugs; devices; and biologics, including blood, vaccines, and tissues. Regulations like these do not always lend themselves to the types of quantitative risk assessments contemplated by the Safe Drinking Water Act principles.

    Other actions are based on research and supporting data that are generated outside FDA. For example, most product approval actions are based on scientific studies conducted by sponsors seeking marketing approval in accordance with our regulations and guidance documents. Our regulations and guidance documents describe sound scientific practices for conducting human and animal studies of medical products and analyzing the resulting data. Most information in these studies is considered confidential commercial information and is closely held by the sponsors. As a result, formal peer-review of the data is rare. However, for certain drug approval applications, the safety and/or effectiveness information is presented to scientific advisory committees for recommendations. Evaluations of food safety and nutritional data are also presented to scientific advisory committees.

    As a result, we have adapted the general principles for risk assessments from the SDWA to fit these situations. The principles we intend to apply to risk assessments involving the dissemination of influential information affecting product approval actions or regulations that do not lend themselves to quantitative risk assessment are as follows:

    1. The Agency will use

      1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer reviewed science and supporting studies when available
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
    2. In the dissemination of public information about risks, the Agency will ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.

    In situations requiring a quantitative risk assessment, we generally follow basic risk assessment principles in the NAS paradigm of 1983. Our needs for quantitative risk assessments range over a wide variety of hazards including physical hazards encountered during use of a medical device, food chemical residues, and antimicrobial resistance genes in bacteria. Thus, we also ascribe to the statement from NAS when it revisited the risk assessment process in 1994 (Science and Judgment in Risk Assessment, NAS 1994): "Risk assessment is not a single process, but a systematic approach to organizing and analyzing scientific knowledge and information. "In each of the areas we regulate, we apply risk assessment practices to the specific task that are widely accepted among relevant domestic and international public health agencies.

    For quantitative risk assessments in support of the dissemination of influential information, FDA intends to apply the following principles:

    1. The agency will use-

      1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer reviewed science and supporting studies when available;
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justifies use of the data)
    2. In the dissemination of public information about health risks, the agency shall ensure that the presentation of information is comprehensive, informative, and understandable, within the context of its intended purpose.

    3. In a risk assessment document made available to the public, the agency shall specify, to the extent practicable-

      1. Each population addressed by any estimate of applicable effects;
      2. The expected or central estimate of risk for the specific populations affected;
      3. Each appropriate upper-bound and/or lower-bound risk estimate and the methodology used to reconcile the inconsistencies in the scientific data;
      4. Data gaps and other significant uncertainties identified in the process of the risk assessment and the studies that would assist in characterizing the uncertainties; and
      5. Additional studies not used to produce the risk estimate that support or fail to support the findings of the assessment, and the rationale of why they were not used.